Prepare for ragged array warnings in NumPy 1.19 by QuLogic · Pull Request #17289 · matplotlib/matplotlib · GitHub

PR Summary

NumPy 1.19 will issue a deprecation warning about ragged arrays numpy/numpy#15119, which causes our tests to fail as we fail on warning. This fixes the easy things, but there are 4 more classes of failures:

cbook._reshape_2D's test checking that a ragged array is turned into an object array. Fixes BUG: VisibleDeprecationWarning in boxplot #16353
hist casting ragged x input to a 2d array (most of the failures are this one)
- this should be fixed by the first one
test_scatter_marker passes an array of color-likes, which goes through _combine_masks which tries to cast as an array
- We could tell _combine_masks to use object array, or pass a flag to allow it? I'm not sure if it should always try to use an object array.
test_bad_plot_args passes some ridiculous values, and I don't know if they should be checked earlier or just allow it to fail when NumPy turns this into an error

I haven't just changed these to explicitly ask for object arrays because I haven't yet figured out what the implications are in NumPy and why they started warning.

PR Checklist

Has Pytest style unit tests
Code is Flake 8 compliant
[N/A] New features are documented, with examples if plot related
[N/A] Documentation is sphinx and numpydoc compliant
[N/A] Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
[N/A] Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

Do you want to get these is as-is and address the rest later or hold these to fix them all in one shot?

Ah, telling _reshape_2D to use object dtype breaks violinplot instead...

This is mostly done, except for test_bad_plot_args, and I'm not sure what to do there. That test does:

plt.plot((np.arange(5).reshape((1, -1)), np.arange(5).reshape(-1, 1)))

which is of course, non-sensical. It currently checks that it raises a ValueError. Unlike the other inputs, which raise in _plot_args, this one actually works fine, and raises somewhere deep in Line2D basically trying to draw. I think this case should have been caught in _plot_args, but I'm not sure how to check it safely.

Not sure if that's the best last commit, but that should fix all the warnings.

I pushed a commit to fix test_violinplot_pandas_series[png] that was failing for me locally.

Do we want to backport this to 3.2.x?

So, what happens after the deprecation period?

I assume it'll be a ValueError, but it wasn't clear in the NEP.

The `segments` parameter is a list of lines (a line being a 2D array or list of 2-tuples), but every line is not required to be the same length. This would cause NumPy to produce an object array, but it will start to warn about ragged arrays in 1.19. An array isn't really needed as `Line3DCollection._segments3d` is only iterated over.

These inputs may be ragged, but we only just need to know the length of the items. There's no need to make an array to check that.

This is a raising a deprecation warning in NumPy 1.19, may go away some time later, and is not strictly necessary for the implementation.

This causes a NumPy deprecation warning if the list is ragged.

This function accepts almost anything list-like, so we really do want conversion to object arrays if the input is ragged. For example, this might occur for `scatter()` if passed different types of `colors`, like in the `test_scatter_marker` test.

Oh, hmmm, didn't notice that had a bunch of commits. Sorry if that was meant to be squashed...

Prepare for ragged array warnings in NumPy 1.19 Conflicts: lib/matplotlib/axes/_axes.py - implicitly backported changes to wording in error messages

Do we want to backport this to 3.2.2? I tried cherry-picking and there is 12 failures on 3.2.x with this backported and numpy master.

I guess our choices are :

backport nothing
backport, but don't clean up the other failures (most of them look like they are in scatter's input munging)
backport and clean up all the other warnings.

I'm between 1 and 3. Doing it half way seems like a poor life choice and I am not sure that it is worth the effort for 3.

At least for Fedora, this would not be a problem as they would not show up at the same time. We can possibly just say they aren't supported on other systems (mainly conda-forge, I'd assume).

I tried the backport and there are about 11 extra failures. I believe that most of them are fixed by #15834, as that removes the np.array call, but I have not tested it out, as I'm not sure we want to actually backport that.

Given that this caused a regression I am glad that we opted not to backport.

QuLogic added this to the v3.3.0 milestone May 1, 2020

QuLogic force-pushed the np119 branch from 00ab15a to c06ba77 Compare May 1, 2020 05:08

tacaswell reviewed May 1, 2020

View reviewed changes

Comment thread lib/mpl_toolkits/mplot3d/art3d.py

QuLogic force-pushed the np119 branch from 73ab7fa to e5c48d6 Compare May 6, 2020 04:23

QuLogic marked this pull request as ready for review May 6, 2020 04:58

QuLogic added the Maintenance label May 7, 2020

QuLogic mentioned this pull request May 14, 2020

Possible problem with ragged-array as object deprecation numpy/numpy#15041

Closed

tacaswell reviewed May 16, 2020

View reviewed changes

Comment thread lib/matplotlib/cbook/__init__.py Outdated

tacaswell reviewed May 16, 2020

View reviewed changes

Comment thread lib/matplotlib/cbook/__init__.py Outdated

tacaswell approved these changes May 16, 2020

View reviewed changes

tacaswell added the Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions. label May 16, 2020

timhoffm reviewed May 18, 2020

View reviewed changes

Comment thread lib/matplotlib/cbook/__init__.py Outdated

Comment thread lib/matplotlib/axes/_axes.py Outdated

QuLogic force-pushed the np119 branch from 87aafd8 to 41bb917 Compare May 19, 2020 23:15

jklymak reviewed May 22, 2020

View reviewed changes

Comment thread lib/matplotlib/cbook/__init__.py Outdated

QuLogic and others added 7 commits May 21, 2020 23:08

Avoid unnecessary conversions to arrays.

2e57d5b

These inputs may be ragged, but we only just need to know the length of the items. There's no need to make an array to check that.

Rewrite _reshape_2D to not use ragged ndarrays.

ac018af

This is a raising a deprecation warning in NumPy 1.19, may go away some time later, and is not strictly necessary for the implementation.

Change hist empty check to not use 2D arrays.

b8b1733

This causes a NumPy deprecation warning if the list is ragged.

Error out of index_of if input is not a NumPy array.

98f22a9

MNT: avoid casting to numpy in violin_stats

00deb04

QuLogic force-pushed the np119 branch from 41bb917 to 00deb04 Compare May 22, 2020 03:43

jklymak approved these changes May 22, 2020

View reviewed changes

jklymak merged commit 5ab1352 into matplotlib:master May 22, 2020

QuLogic deleted the np119 branch May 22, 2020 05:07

TomAugspurger mentioned this pull request Jul 16, 2020

pin numpy<1.19 in doc build pandas-dev/pandas#35312

Merged

jklymak mentioned this pull request Aug 13, 2020

_reshape_2D function behavior changed, breaks hist for some cases in 3.3.0 #18234

Closed

This was referenced Aug 28, 2020

Plotting a pandas DataFrame with string MultiIndex #18371

Closed

FIX: make _reshape_2D accept pandas df with string indices #18374

Merged

eldariont mentioned this pull request Feb 19, 2021

SVIM tries to use deprecated Numpy feature eldariont/svim#45

Closed

-                          x = np.asanyarray(x)
+                          try:
+                              x = np.asanyarray(x)
+                          except np.VisibleDeprecationWarning:

Prepare for ragged array warnings in NumPy 1.19 by QuLogic · Pull Request #17289 · matplotlib/matplotlib · GitHub

Repository navigation

Prepare for ragged array warnings in NumPy 1.19 - #17289

Prepare for ragged array warnings in NumPy 1.19#17289jklymak merged 7 commits intomatplotlib:mastermatplotlib/matplotlib:masterfrom QuLogic:np119QuLogic/matplotlib:np119Copy head branch name to clipboard

Conversation

QuLogic commented May 1, 2020 • edited by tacaswell Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklist

Uh oh!

tacaswell commented May 1, 2020

Uh oh!

Uh oh!

QuLogic commented May 5, 2020

Uh oh!

QuLogic commented May 6, 2020

Uh oh!

QuLogic commented May 16, 2020

Uh oh!

Uh oh!

Uh oh!

tacaswell commented May 16, 2020

Uh oh!

Uh oh!

Uh oh!

tacaswell commented May 22, 2020

Uh oh!

jklymak May 22, 2020

Choose a reason for hiding this comment

Uh oh!

QuLogic May 22, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jklymak commented May 22, 2020

Uh oh!

tacaswell commented Jun 10, 2020

Uh oh!

QuLogic commented Jun 11, 2020

Uh oh!

tacaswell commented Jul 3, 2020

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Prepare for ragged array warnings in NumPy 1.19#17289
jklymak merged 7 commits into
matplotlib:masterfrom
QuLogic:np119

QuLogic commented May 1, 2020 •
edited by tacaswell
Loading