Repository navigation
_reshape_2D function behavior changed, breaks hist for some cases in 3.3.0 #18234
Description
Bug report
Bug summary
In version 3.2.2, plt.hist(df['class']) will produce a correct histogram where df['class'] is a series of strings. In version 3.3.0, _reshape_2D gives different output in this case, leading to slow computation and a graph with too many bars.
Code for reproduction
import seaborn as sns
import matplotlib.pyplot as plt
df = sns.load_dataset('titanic')
plt.hist(df['class'])Actual outcome
This is under matplotlib 3.3.0:
This is _reshape_2D under 3.3.0:
Expected outcome
This is under matplotlib 3.2.2:
This is _reshape_2D under 3.2.2:
Can produce the same output for 3.3.0 by putting the series in a list but ideally I think the behavior should remain the same since this unexpectedly also breaks other function behavior relying on this (pandas.DataFrame.hist(), plt.hist())
Matplotlib version
- Operating system: macOS Mojave 10.14.6
- Matplotlib version: 3.3.0
- Matplotlib backend (print(matplotlib.get_backend())): module://ipykernel.pylab.backend_inline
- Python version: 3.7.4
- Jupyter version (if applicable):
- Other libraries: numpy: 1.18.1, pandas: 1.1.0, seaborn: 0.9.0
matplot lib installed via pip
Activity
tacaswell commented on Aug 12, 2020
Thanks for reporting this.
In the future please always post code and text output as text rather than screen shots.
tacaswell commented on Aug 12, 2020
Do you have this problem with all pandas series or just categorical?
tacaswell commented on Aug 12, 2020
As a workaround,
will get back the old plotting behavior. Something is not quite right with the way we are trying to duck-type pandas catarogical series. I suspect the right fix is for pandas to register a converter on their side that tells us how to interpret them...
jklymak commented on Aug 13, 2020
This bisects to #17821 (which was fixing part of #17289) where we messed with _reshape_2D to keep bumpy happy.
jklymak commented on Aug 13, 2020
I do think that if we first-class pandas like this, we need to have far more comprehensive pandas testing to catch stuff like this.
RyanBaten commented on Aug 13, 2020
To address @tacaswell 's question:
Found this to not just be the case for categorical.
>>> series = pd.Series(['a', 'b', 'b', 'c', 'd']) >>> plt.hist(series) (array([[1., 0., 0., 0., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 1., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 1., 0., 0., 0., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 1., 0., 0., 0.], [0., 0., 0., 0., 0., 0., 0., 0., 0., 1.]]), array([0. , 0.3, 0.6, 0.9, 1.2, 1.5, 1.8, 2.1, 2.4, 2.7, 3. ]), <a list of 5 BarContainer objects>) >>> plt.show() >>> series.dtype dtype('O')The workaround
and
both produce expected output as it was under the older version.
For series that are 'int64' dtype, the output is as expected (although the bars are not centered in both 3.3.0 and 3.2.2).
>>> series = pd.Series([1,2,3,4,5,4,3,2,3,4,3,2]) >>> series.dtype dtype('int64') >>> plt.hist(series) (array([1., 0., 3., 0., 0., 4., 0., 3., 0., 1.]), array([1. , 1.4, 1.8, 2.2, 2.6, 3. , 3.4, 3.8, 4.2, 4.6, 5. ]), <BarContainer object of 10 artists>) >>> plt.show()jklymak commented on Aug 13, 2020
I'm confused. Categoricals has the problems, and integers do not, right?
RyanBaten commented on Aug 13, 2020
Yes, problem exists for 'category' and 'object' but not integers.
jklymak commented on Aug 13, 2020
The problem is that if isinstance(xi, collections.abc.Iterable): returns True for a string, but we really want it to be false...
jklymak commented on Aug 13, 2020
Basically categoricals never got a test with hist at all.
fails... The fix is pretty easy, but we need to fix the tests as well...