Repository navigation
Issues with Numpy 1.11 release candidate #5950
Description
I tried running the test suite for 1.5.1 on numpy 1.11 beta 2 and I am seeing a single failure that we should investigate.
Python 3.5 OSX
======================================================================
FAIL: matplotlib.tests.test_ticker.test_AutoMinorLocator
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 118, in wrapped_function
func(*args, **kwargs)
File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/tests/test_ticker.py", line 50, in test_AutoMinorLocator
assert_almost_equal(ax.xaxis.get_ticklocs(minor=True), test_value)
File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/numpy/testing/utils.py", line 517, in assert_almost_equal
return assert_array_almost_equal(actual, desired, decimal, err_msg)
File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/numpy/testing/utils.py", line 912, in assert_array_almost_equal
precision=decimal)
File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/numpy/testing/utils.py", line 688, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals
(shapes (21,), (22,) mismatch)
x: array([ 0.05, 0.1 , 0.15, 0.25, 0.3 , 0.35, 0.45, 0.5 , 0.55,
0.65, 0.7 , 0.75, 0.85, 0.9 , 0.95, 1.05, 1.1 , 1.15,
1.25, 1.3 , 1.35])
y: array([ 0.05, 0.1 , 0.15, 0.25, 0.3 , 0.35, 0.45, 0.5 , 0.55,
0.65, 0.7 , 0.75, 0.85, 0.9 , 0.95, 1. , 1.05, 1.1 ,
1.15, 1.25, 1.3 , 1.35])
----------------------------------------------------------------------
Activity
jenshnielsen commented on Jan 31, 2016
Furthermore it raises 2 new future warnings very frequently:
and
We should probably suppress them somehow.
jenshnielsen commented on Jan 31, 2016
I have not tested 2.x or master yet
efiring commented on Jan 31, 2016
Both the "expected" test values and those that are obtained are wrong. The algorithm is intended to remove minor ticks that coincide with major ticks, but the expected values in the test include 1, which is a major tick. The test is now failing because floating point arithmetic is causing the erroneous algorithm to break in a different place.
@anntzer, would you like to look at this? I think you are the current master of dealing with such floating point problems.
Longer term, we might consider how minor ticks should be handled in a larger context. If they should be prevented from landing on major ticks, then this should be handled systematically and uniformly, not as a special case in AutoMinorLocator.
anntzer commented on Jan 31, 2016
I would say that the old test was actually written in a way to accomodate a bug in matplotlib (probably itself stemming from a bug in numpy... numpy/numpy#7022 is the obvious suspect here): if you only plot minor ticks
xlim(0, 1.39) gca().minorticks_on() gca().tick_params("x", length=0)you see that there is a spurious minor tick at x=1 (but not at the other major tick positions). I don't know exactly how to handle this in the tests while maintaining backcompatibility but from a correctness POV it is pretty clear that the correct test_value should not include 1 (and the actually returned values are now correct).
seberg commented on Feb 5, 2016
If you have time, maybe you could check how much numpy/numpy#7187 helps with the future warning spam? If it is almost gone, you can maybe work around it (it is a bit ugly in any case).
Frankly, if one downstream package like matplotlib has to silence the warning globally, I see basically no point in even having it in the first place.
jenshnielsen commented on Feb 5, 2016
@seberg Thanks I will try to have a look over the weekend
anntzer commented on Feb 10, 2016
wrt. the test failure I'd suggest switching between testing for the new value (which is after all the correct one...) and the old value depending on the version of numpy.
jenshnielsen commented on Feb 11, 2016
Numpy 10.11 beta 3 is now released on pypi so our allow failures job with python 3.6 and --pre is now picking it up i.e. https://travis-ci.org/matplotlib/matplotlib/jobs/107907120
jenshnielsen commented on Mar 20, 2016
The issues with the numpy release candidate have changed a bit and I now see:
====================================================================== FAIL: matplotlib.tests.test_axes.test_formatter_large_small.test ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer result = f(*args, **kwargs) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test '(RMS %(rms).3f)'%err) matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small.png vs. /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small-expected.png (RMS 19.050) ====================================================================== FAIL: matplotlib.tests.test_axes.test_formatter_large_small.test ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer result = f(*args, **kwargs) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test '(RMS %(rms).3f)'%err) matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small_pdf.png vs. /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small-expected_pdf.png (RMS 27.854) ====================================================================== FAIL: matplotlib.tests.test_axes.test_formatter_large_small.test ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer result = f(*args, **kwargs) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test '(RMS %(rms).3f)'%err) matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small_svg.png vs. /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small-expected_svg.png (RMS 18.371) ====================================================================== FAIL: matplotlib.tests.test_bbox_tight.test_bbox_inches_tight_raster.test ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer result = f(*args, **kwargs) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test '(RMS %(rms).3f)'%err) matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_bbox_tight/bbox_inches_tight_raster.png vs. /Users/jhn/src/python/matplotlib/result_images/test_bbox_tight/bbox_inches_tight_raster-expected.png (RMS 14.603) ====================================================================== FAIL: matplotlib.tests.test_bbox_tight.test_bbox_inches_tight_raster.test ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer result = f(*args, **kwargs) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test '(RMS %(rms).3f)'%err) matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_bbox_tight/bbox_inches_tight_raster_pdf.png vs. /Users/jhn/src/python/matplotlib/result_images/test_bbox_tight/bbox_inches_tight_raster-expected_pdf.png (RMS 14.634) ====================================================================== FAIL: matplotlib.tests.test_bbox_tight.test_bbox_inches_tight_raster.test ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer result = f(*args, **kwargs) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 210, in do_test self._tol, in_decorator=True) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/compare.py", line 331, in compare_images rms = calculate_rms(expectedImage, actualImage) File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/compare.py", line 248, in calculate_rms "actual size {1}".format(expectedImage.shape, actualImage.shape)) matplotlib.testing.exceptions.ImageComparisonFailure: image sizes do not match expected size: (449, 575, 3) actual size (450, 575, 3) ----------------------------------------------------------------------jenshnielsen commented on Mar 20, 2016
The changes are in auto limits in these 2 tests (duplicated in png, svg and pdf)
This seems to mainly be due to a change in the implementation of divmod in numpy.
For example:
Numpy 1.10
Numpy 1.11rc2
divmod is used among other places within ticker.MaxNLocator.bin_boundaries which is responsible for calculating the min and max boundaries in autoscaling.
jenshnielsen commented on Mar 20, 2016
The numpy 1.10 result is obviously wrong in this case since 5.0 * 0.2 + 0.19999999999999996 = 1.2 and not 1 as expected.
anntzer commented on Mar 20, 2016
Pretty sure this is at least partially handled by #5768 (which includes a local fix to divmod, and led me to comment with that specific example on numpy's bug tracker).
jenshnielsen commented on Mar 20, 2016
@anntzer Looks good. I somehow missed that PR, I think we should make a push for getting this in.
1 remaining item
jenshnielsen commented on Mar 21, 2016
The remaining issues with test_formatter_large_small is apprently due to change in floor division which is used within the new _raw_ticks method.
Numpy 1.10
Numpy 1.11rc2
QuLogic commented on Apr 2, 2016
There seemed to have been a few bugs in the betas; have you tried again with final 1.11?
tacaswell commented on Apr 2, 2016
I get the second behavior with 1.11 from conda.
TST: skip broken test
TST: skip broken test
tacaswell commented on Apr 2, 2016
This is another fallout from numpy/numpy#6127 which #6192 was working around. In those cases the incorrect rounding was hurting us, in this case it was helping us.
This is the situation for np 1.10
so we are clearly hitting the edges of 64bit float precision and the cpython / np1.11 values are clearly correct.
TST: skip broken test
tacaswell commented on May 23, 2016
have we fixed these issues?
jenshnielsen commented on May 24, 2016
AFAIK we are skipping the relevant tests on 2.x and master i.e. 26cba4b
On 1.5.x I think there are a few additional tests we need to skip. (they are probably failing in the nightly build on travis which installs directly from pypi and skips the wheelhouse.
anntzer commented on Jul 25, 2017
Closing as (AFAICT) the only remaining thing to do is to remove the version check on numpy once we only support numpy 1.11+ (but there will be plenty of these).