As you can see in the comments here, and various other questions about how to use matplotlib, the way the functions have been programmed, and the mind-boggling way in which various nuances have to be attended to in order to plot some simple figures, matplotlib really needs a usability makeover.
Proposed solution
In the course of a few years, please take some time to think-through how you can simplify the usage of matplotlib and make functions and parameters more intuitive. Also, about how you can ensure that various functionalities are handled automatically, instead of requiring the user to explicitly specify various parameters. I'm not mentioning specifics, since every single aspect of matplotlib needs to be improved. You could retain the existing functions for backward compatibility, and create a wrapper that'd expose the new functions.
The specific linked issue is already discussed at #19463.
Closing as the rest is non-actionable; feel free to propose more specific suggestions instead.
Thank you anntzer, but as I mentioned initially, the issue is not just with subplots. There are various other aspects of matplotlib that need to be improved to make it usable. You can see the various "how to" questions asked on StackOverflow.
Hundreds of thousands of people use Matplotlib across thousands of publications. Clearly Matplotlib is "usable". Better usability can always be discussed. If you have specific usability issues, please open a specific feature request.
Hi @nav9 if you'd like to discuss this further in the general, discourse.matplotlib.org is our discussion forum.
The major reason that 'usable' is not actionable for us is because Matplotlib aims to be a general purpose visualization library that any domain can build on top of for their own needs. These fields differ on what should be handled automatically and what default parameters should be and when possible we prefer to keep the functions as domain neutral as possible.
You can see the various "how to" questions asked on StackOverflow.
If you have the time, it would be a lot of help if you could go through some of those questions and surface some of the most common frustrations and asks. They definitely raise some good points, but our mostly volunteer team is currently spread a bit too thin to do that work. @paniterka has been doing some work in this space too.
Thanks @story645 for the CC and I agree with your points. From my experience, which visualization library is perceived by the users as "intuitive" and for which reasons, really boils down to the user's mental model of how coding for visualization should work, and this bases on their previous experience (mostly which dataviz library they have used in the past). This is why MATLAB people find the PLT-API so usable :) This goes both ways though: once a user acquires a mental model for how MPL works, they also start seeing things as "intuitive", and this is also what we have seen for MPL in particular in the developer survey. Another point of tension is that MPL is a library with a very wide scope of use, and the price you pay for this is that sometimes you need a lot of low-level code to achieve a goal, which may seem not usable at first, but actually is. This is not to say things are always consistent or that the docs couldn't be better - but to say that judging them on "intuitiveness" is too vague to be actionable. If you @nav9 would like to help us improve the docs in a more specific way, please get in touch.
Here are some annoyances I've experienced. I've tried listing it and explaining why exactly they tend to get annoying. I'm not expecting you to defend the reason why it was created that way. I know that building such a complex library is an iterative process, and such design decisions just crept in over time. I also know it's difficult to refactor/fix them.
Here goes: 1. How to "show" the plot: The most common gripe. Plots should be shown by default (and should be non-blocking by default). 2. Ticks: Why are there so many ways to do this? plt.tick_params, plt.axis('off'), ax.set_xticks([]), ax.xaxis.set_ticks_position('none') . This needs to be simplified. I'd design it to be something like this. We create a figure object. fig1 = figure(). Instead of having separate plt and ax, everything can be controlled from the fig1 instance. Even if there are sub-plots. Couldn't ticks be programmed to automatically rotate if they are too close to each other, instead of overlapping? 3. Legends: Replace confusing terminology like bbox_to_anchor etc, and make the legend positioning simpler and wherever possible, make it automatic. So many ways to do one thing here too. And I had to google for "what is gca". Couldn't it be named better? 4. Naming and functions: Why can't background color just be called "background color"? There's plenty of other such naming that needs to be improved, as shown in point 3. Also, ax.set_facecolor is not intuitive. I'm not trying to set the color of the axis. I'm setting the background color of the plot. Why is it ax.set_xlim([xmin, xmax])? Couldn't it be ax.set_tick_limits(x_min, x_max, y_min, y_max)? Why do I have to use plt.ylabel("y label");plt.xlabel("x label"), when plt.label("xlabel", "ylabel") would do just fine? Why do I need to figure out what 221 means here ax1 = fig.add_subplot(221)? There should be a better, shorter way of specifying how many rows and columns of subplots I want, and not have to manually do an add_subplot. Look at this: plt.axis('off');fig.axes.get_xaxis().set_visible(False). Users are supposed to switch off the axis and then get xaxis and then set_visible to False? Couldn't this be made a lot simpler? There's also confusion and complexity in changing figure size. 5. Single widget for all subplots: Whether it's a legend or axes or a colorbar, a re-design is necessary to make it simpler to provide a single widget for all subplots. This solution is just unbelievably confusing. 6. Speed of dynamic updates: Updating the graph with and without blit is slow. Needs to be made faster (perhaps with a backend C++ implementation?). PyQtGraph is an alternative, but it's a nightmare to figure out how to use it. 7. Colors: I empathize with you since as an engineer, even I wasn't taught about the proper use of colors until I worked with Interaction Designers. You simply aren't supposed to take the liberty of using bright blue, green, red, pink and various other random color combinations to display plots. I had to go through a very cumbersome process to change the colors in this plot, since the default colors were horrendous. Colors are used with a purpose. To draw attention to specific things in a figure, to harmonize certain elements in a figure, and more. Some tips. If you have friends who are designers, they can tell you more.
The main point is to think about how the interfaces can be simplified, how function parameters and names can be made more intuitive, and how nitty-gritties can automatically be handled, instead of requiring users to manually program everything. It's like how Tom Cruise constantly thinks of how the audience would perceive a movie, while he's making the movie.
I'm currently working on a research paper, but I don't mind helping with the documentation or coding or design decisions whenever I can spare some time (which would be less). Do let me know how. I'll need some hand-holding.
1. How to "show" the plot: The most common gripe. Plots should be shown by default (and should be non-blocking by default).
When should the plot be shown? At the end of execution? After each artist is placed on the figure? Bear in mind that people use Matplotlib for lots of different things and lots of different coding environment - being explicit is a good thing here.
2. Ticks: Why are there so many ways to do this? plt.tick_params, plt.axis('off'), ax.set_xticks([]), ax.xaxis.set_ticks_position('none') .
Agreed that the tick interface is too confusing. However, we would have to be very careful to add yet another new way to do things.
This needs to be simplified. I'd design it to be something like this. We create a figure object. fig1 = figure(). Instead of having separate plt and ax, everything can be controlled from the fig1 instance.
Not clear what you are saying here. Our recommended API is to create the figure using plt and then everything else from the figure object. Often we will use a plt short hand to get the figure and the axes at the same time (plt.subplots and plt.subplot_mosaic).
Even if there are sub-plots. Couldn't ticks be programmed to automatically rotate if they are too close to each other, instead of overlapping?
Yes. But for every person who wanted that, we would have just as many who don't want the vertical space stolen, or complain that it overlap s the axes below, or spills off the page. You also are concerned with speed below - checking the size of fonts means we have to render them, and then check them all for an overlap. Not very fast.
3. Legends: Replace confusing terminology like bbox_to_anchor etc, and make the legend positioning simpler and wherever possible, make it automatic.
The default legend location is automatic, so not clear what you are referring to here. You don't need to use bbox_to_anchor at all - I've never used it once, and I've used Matplotlib for over 10 years now.
And I had to google for "what is gca". Couldn't it be named better?
Given that you never need to call gca, not clear how naming it better would help.
4. Naming and functions: Why can't background color just be called "background color"? There's plenty of other such naming that needs to be improved, as shown in point 3. Also, ax.set_facecolor is not intuitive. I'm not trying to set the color of the axis. I'm setting the background color of the plot.
"trunk"/"boot", "lift"/"elevator". For whatever reason we use facecolor and edgecolor. These are very consistent names throughout the library.
Why is it ax.set_xlim([xmin, xmax])? Couldn't it be ax.set_tick_limits(x_min, x_max, y_min, y_max)? Why do I have to use plt.ylabel("y label");plt.xlabel("x label"), when plt.label("xlabel", "ylabel") would do just fine?
Why do I need to figure out what 221 means here ax1 = fig.add_subplot(221)? There should be a better, shorter way of specifying how many rows and columns of subplots I want, and not have to manually do an add_subplot.
The trend I'm getting from these comments is that you have largely learned Matplotlib via Stackoverflow posts from 15 years ago. We strongly discourage add_subplot(221), and rather encourage axs = fig.subplots(2, 2); ax = axs[0, 0]. If you read our docs, we are pretty consistent about this.
Look at this: plt.axis('off');fig.axes.get_xaxis().set_visible(False). Users are supposed to switch off the axis and then get xaxis and then set_visible to False? Couldn't this be made a lot simpler?
Not clear what you are trying to accomplish here, or what you are suggesting could be improved.
Again, not clear what could be improved. fig.set_size_inches is pretty straight forward.
5. Single widget for all subplots: Whether it's a legend or axes or a colorbar, a re-design is necessary to make it simpler to provide a single widget for all subplots. This solution is just unbelievably confusing.
6. Speed of dynamic updates: Updating the graph with and without blit is slow. Needs to be made faster (perhaps with a backend C++ implementation?). PyQtGraph is an alternative, but it's a nightmare to figure out how to use it.
There are some slow things. There are also very slow ways to use Matplotlib, and fast ways.
7. Colors: I empathize with you since as an engineer, even I wasn't taught about the proper use of colors until I worked with Interaction Designers. You simply aren't supposed to take the liberty of using bright blue, green, red, pink and various other random color combinations to display plots. I had to go through a very cumbersome process to change the colors in this plot, since the default colors were horrendous. Colors are used with a purpose. To draw attention to specific things in a figure, to harmonize certain elements in a figure, and more. Some tips. If you have friends who are designers, they can tell you more.
The main point is to think about how the interfaces can be simplified, how function parameters and names can be made more intuitive, and how nitty-gritties can automatically be handled, instead of requiring users to manually program everything. It's like how Tom Cruise constantly thinks of how the audience would perceive a movie, while he's making the movie.
Our defaults are not that fiddly. Most of what you are concerned about above is customizing the look of plots, which by definition is fiddly. I think most plotting packages are similar - they provide a default, and then if you need to customize, you need to poke around to figure out how to do that.
I'm currently working on a research paper, but I don't mind helping with the documentation or coding or design decisions whenever I can spare some time (which would be less). Do let me know how. I'll need some hand-holding.
Where you could be the most help is bringing up individual issues that are not clear from the docs. Please note that the docs have seen a relatively major overhaul in 3.8, which can be previewed at https://matplotlib.org/devdocs/index.html
1. How to "show" the plot: The most common gripe. Plots should be shown by default (and should be non-blocking by default).
When should the plot be shown? At the end of execution? After each artist is placed on the figure? Bear in mind that people use Matplotlib for lots of different things and lots of different coding environment - being explicit is a good thing here.
"Show immediately by default" is a valid (though debatable) point-of-view. However it's only only relevant for direct use in REPLs (which gets less common). Library code will want to configure the whole plot before showing, Jupyter notebooks have an implicit show at the end of the cell.
In practice I don't see that we will get there a) because such a change is quite disruptive. b) because we'd need much better performance to be able to affort a redraw with every configuration command.
-> no action possible
2. Ticks: Why are there so many ways to do this? plt.tick_params, plt.axis('off'), ax.set_xticks([]), ax.xaxis.set_ticks_position('none') .
Agreed that the tick interface is too confusing. However, we would have to be very careful to add yet another new way to do things.
Fully agree. Part of the problem is that you have multiple related elements (tick markers, tick labels, the axis line (spine), grid lines and axis labels), and we (and users) sometimes do not clarify what we mean by "ticks" exactly.
In short:
use ax.set_xticks([]) to remove all ticks (resulting in no tick markers, tick labels and grid lines)
use .tick_params() for more detailed control of the individual aspects
plt.axis('off') deactivates all of (tick markers, tick labels, the axis line (spine), grid lines and axis labels)
This is often more than you want. Also axis() historically has a very peculiar API.
ax.xaxis.set_ticks_position('none') only removes the tick markers. You'll want this if you have grid lines but don't want the tick markers in addition (the grid line already takes the role of exactly displaying where the tick location is.
-> action unclear: Maybe improve docs (what exactly, e.g. #26619)
This needs to be simplified. I'd design it to be something like this. We create a figure object. fig1 = figure(). Instead of having separate plt and ax, everything can be controlled from the fig1 instance.
3. Legends: Replace confusing terminology like bbox_to_anchor etc, and make the legend positioning simpler and wherever possible, make it automatic.
The default legend location is automatic, so not clear what you are referring to here. You don't need to use bbox_to_anchor at all - I've never used it once, and I've used Matplotlib for over 10 years now.
bbox_to_anchor is only needed for advanced positioning. The name is somewhat cryptic, but I can't think of something that much better that would justify a renaming. Do you have ideas how to improve the parameter description or the legend guide?
And I had to google for "what is gca". Couldn't it be named better?
Given that you never need to call gca, not clear how naming it better would help.
gca() is from the pyplot interface, which is directly modelled after MATLAB. The direct correspondence is mandatory here. If you don't want to work in MATLAB style, don't use pyplot functions (with the exception of plt.subplots and plt.show)
Look at this: plt.axis('off');fig.axes.get_xaxis().set_visible(False). Users are supposed to switch off the axis and then get xaxis and then set_visible to False? Couldn't this be made a lot simpler?
Not clear what you are trying to accomplish here, or what you are suggesting could be improved.
5. Single widget for all subplots: Whether it's a legend or axes or a colorbar, a re-design is necessary to make it simpler to provide a single widget for all subplots. This solution is just unbelievably confusing.
6. Speed of dynamic updates: Updating the graph with and without blit is slow. Needs to be made faster (perhaps with a backend C++ implementation?). PyQtGraph is an alternative, but it's a nightmare to figure out how to use it.
There are some slow things. There are also very slow ways to use Matplotlib, and fast ways.
The rendering backend itself is in C. What is rather slow is the management of the complex figure logic on the python level. There are some inefficiencies that have been improved and some further ideas under discussion, but overall speed will not become super-fast without substantial API breakage and rewrite (which amounts to almost writing a completely new visualization libraray).
I know that building such a complex library is an iterative process, and such design decisions just crept in over time
The core Matplotlib object is the Artist - basically everything you see in any chart is a type of artist and that drives a lot of our design decisions. @paniterka has lightning talk How to stop crying when using Matplotlib that discusses this and like @jklymak said we'd much appreciate feedback on how we can better explain this in our user guide.
For whatever reason we use facecolor and edgecolor. These are very consistent names throughout the library.
The artist being our core object is for example the reason for facecolor/edgecolor. The Axes object is a type of artist, and that background you're thinking of is managed by what's called a patch artist object. Generally Patch artists are the abstraction we use for drawing polygons, therefore they all have facecolor and edgecolor properties. That's why those names are consistently used throughout the library.
plt.axis('off');fig.axes.get_xaxis().set_visible(False). Users are supposed to switch off the axis and then get xaxis and then set_visible to False? Couldn't this be made a lot simpler?
Like @timhoffm wrote, plt.axis('off') and bbox_inches="tight" should do the trick, as shown in the accepted answer on that page-. I think maybe the OO version may be clearer for why:
data=random.random((5,5))
fig, ax=plt.subplots() -# create plotting regionimg=ax.imshow(data, interpolation='nearest')# put a heatmap on the ax plotting regionimg.set_cmap('hot')
ax.axis('off') # turns off the xaxis and yaxis attached to ax axesfig.savefig("test.png", bbox_inches='tight') #save the figure with a tight bounding box, meaning no padding
Which what can we do to make that clearer? Should we add an example in our docs "save borderless image?" 'cause that comes up a lot?
Outdated recommendations on stackoverflow: not much we can do
Overall, we strike a delicate balance between API improvments and compatibility. We have already changed a lot (see the API change notes for the past releases), but we can't afford to break every user's plot. Concrete suggestions for improvements are welcome though and will be carefully considered.
I apologise this took up so much of your time. I had mentioned "I'm not expecting you to defend the reason...". I do appreciate your sincerity. The title of paniterka's video made me smile. Circa 1999, I used to go through documentation to figure out functions of libraries. Today, it's common to Google it, and it leads to StackOverflow and various blogs. Yes, you are right about that being the problem. At least I used to trust StackOverflow since the community tends to comment and update answers to keep information relevant. Seems like it's not done as often.
Although it is reasonable to expect people to RTFM or to spend time understanding the fundamentals of a library, it's also important to realize that many people are expected to use a multitude of libraries and languages, which ends up making them forget what they learnt and the pressures of quick delivery often does not allow them to spend time learning. So they end up googling for solutions. Long ago, when googling didn't help, I had to figure out the basics of d3.js and write a tutorial.
The latest trend is to use GitHub Co-pilot or ChatGPT or GPT4All. I asked ChatGPT "show me some sample code to add subplots to matplotlib", and it gave me sample code containing fig, axs = plt.subplots(2, 2, figsize=(10, 6)) instead of ax1 = fig.add_subplot(221). Even if the models get outdated, hopefully the use of LangChain could help models refer the latest version of the manual. I've given feedback to Google, referencing this discussion, asking if they could do anything to improve search results when programmers search for something, and the manual contains info that's more recent than what's available on StackOverflow or blogs.
I guess the solution is to simply use the wrappers (and to search the docs). Thanks everyone.
Not necessary! Like Tim said, user feedback is very helpful 'cause we've all got various levels of expert blindness.
it gave me sample code containing fig, axs = plt.subplots(2, 2, figsize=(10, 6))
I'm kinda excited about this. A while back I talked to a couple of the LLM folks to try and get them to train on our more recent docs & this is definitely an improvement! (ETA: also thanks for reaching out to Google about this)
So they end up googling for solutions
I hear and one of the underlying motivations of a lot of our larger documentation projects is to improve organization so that we can improve discoverablity so that it's easier to find things in the docs.
I would also connect this to #26388 and #26366. A lot of people are confused what is the difference between the MATLAB and object-oriented APIs and are not aware that one is being discouraged, which results with people coming up with unnecessarily convoluted code, e.g. where plt.gca() comes up. It's not only that StackOverflow posts are outdated, but many new tutorials teach the PLT-interface instead of the OO-interface (e.g. when you search medium.com for plt.plot you get 8.5k results, compared with 1.5k results for ax.plot). What people write out there is not something we could control, but a lot of Matplotlib user base does have this misconception and so it'd be good if the new docs are as clear as they can in explaining the mental model and super consistent in showing a recommended way to do things. Since it looks like @nav9 also has learnt the PLT-interface, maybe a good first issue for them would be to read through the relevant pages from the new docs that explain the OO-interface and internal logic of Matplotlib and give feedback?
3. Legends: Replace confusing terminology like bbox_to_anchor etc, and make the legend positioning simpler and wherever possible, make it automatic.
The default legend location is automatic, so not clear what you are referring to here. You don't need to use bbox_to_anchor at all - I've never used it once, and I've used Matplotlib for over 10 years now.
bbox_to_anchor is only needed for advanced positioning. The name is somewhat cryptic, but I can't think of something that much better that would justify a renaming. Do you have ideas how to improve the parameter description or the legend guide?
The name bbox_to_anchor becomes much clearer once a user understands what a bbox and anchor is. In the new docs I haven't found any tutorial that would explain what a bbox actually is and why it is an important concept for positioning (and searching for bbox mostly points to function names which all assume that you already know what it is). Maybe it would make sense to add a tutorial or a user guide chapter on positioning things, which would explain these concepts?
Maybe it would make sense to add a tutorial or a user guide chapter on positioning things, which would explain these concepts?
That's sort of supposed to be the transforms tutorial - Bbox's are part of that module, and yeah totally agree that a section on Bbox and maybe adding some clarity to the name would be useful.
@paniterka : I'd like to humbly draw your attention to the perspective of the end user. The end user does not want to spend time learning the nuances of a complex library, because:
They are under pressure to deliver results quickly.
They know that even if they learn the nuances, they'll have to re-learn another bunch of nuances of another library very soon, and then they'll forget the first library's nuances and have to re-learn it when they return to it.
That's the main point of this article. It's why our grandparents struggled with programming a VCR, and our parents struggled with learning how to use a feature phone and then a smartphone. I find myself with the same technical exhaustion, after having to learn the nuances of so many languages and frameworks. Even this guy faced the same thing. Even after writing a d3.js tutorial, I would never use it. I'd prefer a simpler graphing library.
Removal of unnecessary complexities is why Python became so much more popular than C, and GPT is so much more popular than older NLP algorithms. The solution the end user would want, is not a tutorial, but a simpler and faster way to get stuff done via matplotlib. If the API's help achieve that goal, then as an end user, I'm happy. If more powerful functionality can also be provided with a simpler API, then that's what people will be happy with, rather than having to pore over tutorials or do Google searches or consult ChatGPT.
Of course, this is the opinion of only one end user (me).
Going back to the reason I raised this issue: I encountered an error Matplotlib: TypeError: 'AxesSubplot' object is not subscriptable, when using:
for r in range(rows):
for c in range(cols):
ax[r][c].imshow(myimage, cmap="gray")
So I changed it to:
for r in range(rows):
for c in range(cols):
if rows == 1: ax[c].imshow(myimage, cmap="gray")
else: ax[r][c].imshow(myimage, cmap="gray")
...and went on with life. And today, the same error showed up for an edge case, and I had to add:
for r in range(rows):
for c in range(cols):
if rows == 1 and cols == 1:
ax.imshow(myimage, cmap="gray")
else:
if rows == 1: ax[c].imshow(myimage, cmap="gray")
else: ax[r][c].imshow(myimage, cmap="gray")
Rather than mark such design issues as non actionable, it'd help to have higher-level API's which make the subplot display simpler. A cursory search shows grid-strategy and patchworklib. Never knew these existed. These API's need better SEO.
Even after writing a d3.js tutorial, I would never use it.
You're hitting on something here, which is that Matplotib and d3 have a similar goal of being comprehensive general purpose building block visualization libraries - the goal is that folks can make visualizations sure, but also that they can build simpler tools using Matplotib or D3 - to do the latter, Matplotib and D3 provide a lot of primitives for folks to build custom visualizations. That comprehensiveness and flexibility is the strength of MPL and D3 and also the trade off is writing more code that feels like boilerplate to you b/c it's a standard in your domain.
but a simpler and faster way to get stuff done via matplotlib
Which is to say, we encourage the workflow pattern you're suggesting of use specialized/simplified APIs and drop down to us when you hit the limits of that API. This is mostly b/c like we've said up top, domain specific APIs are better suited to define what simpler and faster means for that domain.
ETA: but suggestions for more efficient/usable functions for writing that boilerplate are much appreciated - we provide functions like Axes.set and Axes.subplot_mosaic to try and balance flexible functionality with not having to learn all our internals.
@nav9 I can only agree with @story645 - matplotlib is a low-level visualization library with lots of customizability, so it looks like for your use case - when you use it only ocassionally and need results quickly - it might be a good solution if you invest some effort upfront to write your own stylesheets and helper functions that produce the level of abstraction you need, and interact mostly with that.
Regarding your example: have you tried using plt.subplots(..., squeeze=False) instead of writing the edge cases? See the docs:
If False, no squeezing at all is done: the returned Axes object is always a 2D array containing Axes instances, even if it ends up being 1x1.
squeeze is a good solution. Could've either been False by default, or could've been designed differently. Created an example to see it working:
import numpy
from PIL import Image
import matplotlib.pyplot as plt
imarray = numpy.random.rand(10,10,3) * 255
im = Image.fromarray(imarray.astype('uint8')).convert('RGBA')
#rows = 1; cols = 1;
#rows = 5; cols = 3;
rows = 1; cols = 5;
fig, ax = plt.subplots(rows, cols, squeeze=False)
fig.suptitle('random plots')
i = 0
for r in range(rows):
for c in range(cols):
ax[r][c].imshow(im)
i = i + 1
plt.show()
plt.close()
My earlier code was from a google search of the error which led here. From what I understand, y'all have already created everything that's required. People (like me) just need to RTFM instead of googling for solutions.
I presented the error message and the above code to ChatGPT without including squeeze=False and it said this is the solution ax[i].imshow(im) # Access subplot using linear index. It didn't suggest squeeze=False. I asked ChatGPT Instead, can I use fig, ax = plt.subplots(rows, cols, squeeze=False) and ax[r][c].imshow(im). It recognized the functionality and said yes.
Come to think of it, even if I looked at the docs, I'd have ignored squeeze, thinking it isn't relevant to what I'm searching for. The sentence extra dimensions are squeezed out from the returned array of just doesn't connect (in my mind). In the same way I noticed only now, that this answer already mentioned squeeze even before I edited it to include the above example.
Kind of similar to how Harvey announced the wrong Miss Universe. UX Stackexchange discussed how it could be avoided.
@timhoffm I've explained a lot and provided links in my previous comments. Fundamentally, it's about thinking how the end user would understand it. If we pick out specifics like this, the discussion will go on endlessly. @story645: It'd help if we first found out how programmers generally search for matplotlib functionality. Just because I google for it, it doesn't mean everyone would. This discussion has only our opinions. We could ask the programming community on Reddit.
I just searched for which subreddit would be appropriate, and went "Yikes!" when I saw this, this, this, this, this, this. Some constructive ones here, here.
Once the Matplotlib developers figure out what y'all want to know from the community, y'all could formulate questions/polls to gather opinions. It'd have to be done in a way that prevents a flame-war from erupting. It's about collecting opinions. Not about defending or explaining why Matplotlib was designed in the current way. The following subreddits can help: Python, dataisbeautiful, physics, datascience, learnmachinelearning, programming, data, bioinformatics, and more.
Perhaps it'd help to create a new subreddit "Improving Matplotlib", where developers can discuss SEO, design decisions, naming conventions, a more intuitive wrapper which retains the power of matplotlib but makes it easier to use, etc. Having larger community support can also help obtain volunteers who'd gladly fix popular StackOverflow answers.
It's about collecting opinions. Not about defending or explaining why Matplotlib was designed in the current way. .
Totally agree w/ you if we're running a broad usability survey - we did one a few years back and haven't again mostly cause we don't have the bandwidth. Which, the reddit suggestion sounds good (we have discourse.matplotlib.org and the scientific-python spaces) but would also require staffing.
I think what's happening here is that a lot of us are going into problem solver mode while you're trying to have this broader conversation. We're trying to sort through all the individual pain points you're bringing up - mostly cause some of that is immediately actionable - while you're aiming for this big picture discussion on usability. But I think a lot of this is conversation is circling around this comment from those reddit threads that's backed by previous usability surveys:
I like it now that I know how to use it
Which is why so many of the maintainer responses are in one way or another trying to teach usage. And that's probably what right now a user survey would focus on - how do we set expectations around what Matplotlib is for and how do we better communicate how to use it? For sure there are API changes that could be made too and features added, but I agree that the first step is probably figuring out a cohesive strategy/vision rather than fixing in piecemeal.
Activity
anntzer commented on Aug 28, 2023
The specific linked issue is already discussed at #19463.
Closing as the rest is non-actionable; feel free to propose more specific suggestions instead.
nav9 commented on Aug 28, 2023
Thank you anntzer, but as I mentioned initially, the issue is not just with subplots. There are various other aspects of matplotlib that need to be improved to make it usable. You can see the various "how to" questions asked on StackOverflow.
jklymak commented on Aug 28, 2023
Hundreds of thousands of people use Matplotlib across thousands of publications. Clearly Matplotlib is "usable". Better usability can always be discussed. If you have specific usability issues, please open a specific feature request.
story645 commented on Aug 28, 2023
Hi @nav9 if you'd like to discuss this further in the general, discourse.matplotlib.org is our discussion forum.
The major reason that 'usable' is not actionable for us is because Matplotlib aims to be a general purpose visualization library that any domain can build on top of for their own needs. These fields differ on what should be handled automatically and what default parameters should be and when possible we prefer to keep the functions as domain neutral as possible.
If you have the time, it would be a lot of help if you could go through some of those questions and surface some of the most common frustrations and asks. They definitely raise some good points, but our mostly volunteer team is currently spread a bit too thin to do that work. @paniterka has been doing some work in this space too.
paniterka commented on Aug 28, 2023
Thanks @story645 for the CC and I agree with your points. From my experience, which visualization library is perceived by the users as "intuitive" and for which reasons, really boils down to the user's mental model of how coding for visualization should work, and this bases on their previous experience (mostly which dataviz library they have used in the past). This is why MATLAB people find the PLT-API so usable :) This goes both ways though: once a user acquires a mental model for how MPL works, they also start seeing things as "intuitive", and this is also what we have seen for MPL in particular in the developer survey. Another point of tension is that MPL is a library with a very wide scope of use, and the price you pay for this is that sometimes you need a lot of low-level code to achieve a goal, which may seem not usable at first, but actually is. This is not to say things are always consistent or that the docs couldn't be better - but to say that judging them on "intuitiveness" is too vague to be actionable. If you @nav9 would like to help us improve the docs in a more specific way, please get in touch.
paniterka commented on Aug 28, 2023
Also, in terms of wrappers, @nav9 you can check out alternative APIs, maybe one of them will work for you:
https://matplotlib.org/mpl-third-party/#alternative-apis
nav9 commented on Aug 28, 2023
@paniterka Thank you. The alternative API's seem useful.
Here are some annoyances I've experienced. I've tried listing it and explaining why exactly they tend to get annoying. I'm not expecting you to defend the reason why it was created that way. I know that building such a complex library is an iterative process, and such design decisions just crept in over time. I also know it's difficult to refactor/fix them.
Here goes:
1. How to "show" the plot: The most common gripe. Plots should be shown by default (and should be non-blocking by default).
2. Ticks: Why are there so many ways to do this? plt.tick_params, plt.axis('off'), ax.set_xticks([]), ax.xaxis.set_ticks_position('none') . This needs to be simplified. I'd design it to be something like this. We create a figure object. fig1 = figure(). Instead of having separate plt and ax, everything can be controlled from the fig1 instance. Even if there are sub-plots. Couldn't ticks be programmed to automatically rotate if they are too close to each other, instead of overlapping?
3. Legends: Replace confusing terminology like bbox_to_anchor etc, and make the legend positioning simpler and wherever possible, make it automatic. So many ways to do one thing here too. And I had to google for "what is gca". Couldn't it be named better?
4. Naming and functions: Why can't background color just be called "background color"? There's plenty of other such naming that needs to be improved, as shown in point 3. Also, ax.set_facecolor is not intuitive. I'm not trying to set the color of the axis. I'm setting the background color of the plot. Why is it ax.set_xlim([xmin, xmax])? Couldn't it be ax.set_tick_limits(x_min, x_max, y_min, y_max)? Why do I have to use plt.ylabel("y label");plt.xlabel("x label"), when plt.label("xlabel", "ylabel") would do just fine? Why do I need to figure out what 221 means here ax1 = fig.add_subplot(221)? There should be a better, shorter way of specifying how many rows and columns of subplots I want, and not have to manually do an add_subplot. Look at this: plt.axis('off');fig.axes.get_xaxis().set_visible(False). Users are supposed to switch off the axis and then get xaxis and then set_visible to False? Couldn't this be made a lot simpler? There's also confusion and complexity in changing figure size.
5. Single widget for all subplots: Whether it's a legend or axes or a colorbar, a re-design is necessary to make it simpler to provide a single widget for all subplots. This solution is just unbelievably confusing.
6. Speed of dynamic updates: Updating the graph with and without blit is slow. Needs to be made faster (perhaps with a backend C++ implementation?). PyQtGraph is an alternative, but it's a nightmare to figure out how to use it.
7. Colors: I empathize with you since as an engineer, even I wasn't taught about the proper use of colors until I worked with Interaction Designers. You simply aren't supposed to take the liberty of using bright blue, green, red, pink and various other random color combinations to display plots. I had to go through a very cumbersome process to change the colors in this plot, since the default colors were horrendous. Colors are used with a purpose. To draw attention to specific things in a figure, to harmonize certain elements in a figure, and more. Some tips. If you have friends who are designers, they can tell you more.
The main point is to think about how the interfaces can be simplified, how function parameters and names can be made more intuitive, and how nitty-gritties can automatically be handled, instead of requiring users to manually program everything. It's like how Tom Cruise constantly thinks of how the audience would perceive a movie, while he's making the movie.
I'm currently working on a research paper, but I don't mind helping with the documentation or coding or design decisions whenever I can spare some time (which would be less). Do let me know how. I'll need some hand-holding.
jklymak commented on Aug 28, 2023
When should the plot be shown? At the end of execution? After each artist is placed on the figure? Bear in mind that people use Matplotlib for lots of different things and lots of different coding environment - being explicit is a good thing here.
Agreed that the tick interface is too confusing. However, we would have to be very careful to add yet another new way to do things.
Not clear what you are saying here. Our recommended API is to create the figure using plt and then everything else from the figure object. Often we will use a plt short hand to get the figure and the axes at the same time (plt.subplots and plt.subplot_mosaic).
Yes. But for every person who wanted that, we would have just as many who don't want the vertical space stolen, or complain that it overlap s the axes below, or spills off the page. You also are concerned with speed below - checking the size of fonts means we have to render them, and then check them all for an overlap. Not very fast.
The default legend location is automatic, so not clear what you are referring to here. You don't need to use bbox_to_anchor at all - I've never used it once, and I've used Matplotlib for over 10 years now.
Given that you never need to call gca, not clear how naming it better would help.
"trunk"/"boot", "lift"/"elevator". For whatever reason we use facecolor and edgecolor. These are very consistent names throughout the library.
Please see: https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.set.html. ax.set(xlim=[x_min, x_max], ylim=[y_min, y_max], ...) works just fine....
The trend I'm getting from these comments is that you have largely learned Matplotlib via Stackoverflow posts from 15 years ago. We strongly discourage add_subplot(221), and rather encourage axs = fig.subplots(2, 2); ax = axs[0, 0]. If you read our docs, we are pretty consistent about this.
Not clear what you are trying to accomplish here, or what you are suggesting could be improved.
Again, not clear what could be improved. fig.set_size_inches is pretty straight forward.
This is all massively easier than the first stack overflow post makes it seem. Please see the answer in that post: https://stackoverflow.com/a/68553479/3394386, and also https://matplotlib.org/stable/gallery/subplots_axes_and_figures/colorbar_placement.html# and follow up with any questions. If there is a complicated answer listed as the Answer for Stackoverflow, and it's from 2014, its possible things have been fixed.
There are some slow things. There are also very slow ways to use Matplotlib, and fast ways.
The color palette chosen for 2.0 went through a lot of thought and discussion before it was adopted. If you feel it is "horrendous" we provide a mechanism to provide your ow style sheets: https://matplotlib.org/stable/gallery/style_sheets/style_sheets_reference.html
Our defaults are not that fiddly. Most of what you are concerned about above is customizing the look of plots, which by definition is fiddly. I think most plotting packages are similar - they provide a default, and then if you need to customize, you need to poke around to figure out how to do that.
Where you could be the most help is bringing up individual issues that are not clear from the docs. Please note that the docs have seen a relatively major overhaul in 3.8, which can be previewed at https://matplotlib.org/devdocs/index.html
timhoffm commented on Aug 28, 2023
"Show immediately by default" is a valid (though debatable) point-of-view. However it's only only relevant for direct use in REPLs (which gets less common). Library code will want to configure the whole plot before showing, Jupyter notebooks have an implicit show at the end of the cell.
In practice I don't see that we will get there a) because such a change is quite disruptive. b) because we'd need much better performance to be able to affort a redraw with every configuration command.
-> no action possible
Fully agree. Part of the problem is that you have multiple related elements (tick markers, tick labels, the axis line (spine), grid lines and axis labels), and we (and users) sometimes do not clarify what we mean by "ticks" exactly.
In short:
This is often more than you want. Also axis() historically has a very peculiar API.
-> action unclear: Maybe improve docs (what exactly, e.g. #26619)
bbox_to_anchor is only needed for advanced positioning. The name is somewhat cryptic, but I can't think of something that much better that would justify a renaming. Do you have ideas how to improve the parameter description or the legend guide?
gca() is from the pyplot interface, which is directly modelled after MATLAB. The direct correspondence is mandatory here. If you don't want to work in MATLAB style, don't use pyplot functions (with the exception of plt.subplots and plt.show)
https://github.com/timhoffm/using-matplotlib/blob/main/01_Interfaces.ipynb
The linked Stackoverflow answer is not good. The correct solution is actually the answer below https://stackoverflow.com/a/45214623.
The rendering backend itself is in C. What is rather slow is the management of the complex figure logic on the python level. There are some inefficiencies that have been improved and some further ideas under discussion, but overall speed will not become super-fast without substantial API breakage and rewrite (which amounts to almost writing a completely new visualization libraray).
story645 commented on Aug 28, 2023
The core Matplotlib object is the Artist - basically everything you see in any chart is a type of artist and that drives a lot of our design decisions. @paniterka has lightning talk How to stop crying when using Matplotlib that discusses this and like @jklymak said we'd much appreciate feedback on how we can better explain this in our user guide.
The artist being our core object is for example the reason for facecolor/edgecolor. The Axes object is a type of artist, and that background you're thinking of is managed by what's called a patch artist object. Generally Patch artists are the abstraction we use for drawing polygons, therefore they all have facecolor and edgecolor properties. That's why those names are consistently used throughout the library.
Like @timhoffm wrote, plt.axis('off') and bbox_inches="tight" should do the trick, as shown in the accepted answer on that page-. I think maybe the OO version may be clearer for why:
Which what can we do to make that clearer? Should we add an example in our docs "save borderless image?" 'cause that comes up a lot?
timhoffm commented on Aug 29, 2023
@nav9 thanks for the detailed explanations. It is super-helpful to see where users struggle with the interface.
I think this falls into three categories:
Overall, we strike a delicate balance between API improvments and compatibility. We have already changed a lot (see the API change notes for the past releases), but we can't afford to break every user's plot. Concrete suggestions for improvements are welcome though and will be carefully considered.
nav9 commented on Aug 29, 2023
I apologise this took up so much of your time. I had mentioned "I'm not expecting you to defend the reason...". I do appreciate your sincerity. The title of paniterka's video made me smile. Circa 1999, I used to go through documentation to figure out functions of libraries. Today, it's common to Google it, and it leads to StackOverflow and various blogs. Yes, you are right about that being the problem. At least I used to trust StackOverflow since the community tends to comment and update answers to keep information relevant. Seems like it's not done as often.
Although it is reasonable to expect people to RTFM or to spend time understanding the fundamentals of a library, it's also important to realize that many people are expected to use a multitude of libraries and languages, which ends up making them forget what they learnt and the pressures of quick delivery often does not allow them to spend time learning. So they end up googling for solutions. Long ago, when googling didn't help, I had to figure out the basics of d3.js and write a tutorial.
The latest trend is to use GitHub Co-pilot or ChatGPT or GPT4All. I asked ChatGPT "show me some sample code to add subplots to matplotlib", and it gave me sample code containing fig, axs = plt.subplots(2, 2, figsize=(10, 6)) instead of ax1 = fig.add_subplot(221). Even if the models get outdated, hopefully the use of LangChain could help models refer the latest version of the manual. I've given feedback to Google, referencing this discussion, asking if they could do anything to improve search results when programmers search for something, and the manual contains info that's more recent than what's available on StackOverflow or blogs.
I guess the solution is to simply use the wrappers (and to search the docs). Thanks everyone.
story645 commented on Aug 29, 2023
Not necessary! Like Tim said, user feedback is very helpful 'cause we've all got various levels of expert blindness.
I'm kinda excited about this. A while back I talked to a couple of the LLM folks to try and get them to train on our more recent docs & this is definitely an improvement! (ETA: also thanks for reaching out to Google about this)
I hear and one of the underlying motivations of a lot of our larger documentation projects is to improve organization so that we can improve discoverablity so that it's easier to find things in the docs.
paniterka commented on Aug 29, 2023
I would also connect this to #26388 and #26366. A lot of people are confused what is the difference between the MATLAB and object-oriented APIs and are not aware that one is being discouraged, which results with people coming up with unnecessarily convoluted code, e.g. where plt.gca() comes up. It's not only that StackOverflow posts are outdated, but many new tutorials teach the PLT-interface instead of the OO-interface (e.g. when you search medium.com for plt.plot you get 8.5k results, compared with 1.5k results for ax.plot). What people write out there is not something we could control, but a lot of Matplotlib user base does have this misconception and so it'd be good if the new docs are as clear as they can in explaining the mental model and super consistent in showing a recommended way to do things. Since it looks like @nav9 also has learnt the PLT-interface, maybe a good first issue for them would be to read through the relevant pages from the new docs that explain the OO-interface and internal logic of Matplotlib and give feedback?
paniterka commented on Aug 29, 2023
The name bbox_to_anchor becomes much clearer once a user understands what a bbox and anchor is. In the new docs I haven't found any tutorial that would explain what a bbox actually is and why it is an important concept for positioning (and searching for bbox mostly points to function names which all assume that you already know what it is). Maybe it would make sense to add a tutorial or a user guide chapter on positioning things, which would explain these concepts?
story645 commented on Aug 29, 2023
That's sort of supposed to be the transforms tutorial - Bbox's are part of that module, and yeah totally agree that a section on Bbox and maybe adding some clarity to the name would be useful.
nav9 commented on Aug 29, 2023
@paniterka : I'd like to humbly draw your attention to the perspective of the end user. The end user does not want to spend time learning the nuances of a complex library, because:
That's the main point of this article. It's why our grandparents struggled with programming a VCR, and our parents struggled with learning how to use a feature phone and then a smartphone. I find myself with the same technical exhaustion, after having to learn the nuances of so many languages and frameworks. Even this guy faced the same thing. Even after writing a d3.js tutorial, I would never use it. I'd prefer a simpler graphing library.
Removal of unnecessary complexities is why Python became so much more popular than C, and GPT is so much more popular than older NLP algorithms. The solution the end user would want, is not a tutorial, but a simpler and faster way to get stuff done via matplotlib. If the API's help achieve that goal, then as an end user, I'm happy. If more powerful functionality can also be provided with a simpler API, then that's what people will be happy with, rather than having to pore over tutorials or do Google searches or consult ChatGPT.
Of course, this is the opinion of only one end user (me).
Going back to the reason I raised this issue: I encountered an error Matplotlib: TypeError: 'AxesSubplot' object is not subscriptable, when using:
for r in range(rows): for c in range(cols): ax[r][c].imshow(myimage, cmap="gray")So I changed it to:
for r in range(rows): for c in range(cols): if rows == 1: ax[c].imshow(myimage, cmap="gray") else: ax[r][c].imshow(myimage, cmap="gray")...and went on with life. And today, the same error showed up for an edge case, and I had to add:
for r in range(rows): for c in range(cols): if rows == 1 and cols == 1: ax.imshow(myimage, cmap="gray") else: if rows == 1: ax[c].imshow(myimage, cmap="gray") else: ax[r][c].imshow(myimage, cmap="gray")Rather than mark such design issues as non actionable, it'd help to have higher-level API's which make the subplot display simpler. A cursory search shows grid-strategy and patchworklib. Never knew these existed. These API's need better SEO.
story645 commented on Aug 29, 2023
You're hitting on something here, which is that Matplotib and d3 have a similar goal of being comprehensive general purpose building block visualization libraries - the goal is that folks can make visualizations sure, but also that they can build simpler tools using Matplotib or D3 - to do the latter, Matplotib and D3 provide a lot of primitives for folks to build custom visualizations. That comprehensiveness and flexibility is the strength of MPL and D3 and also the trade off is writing more code that feels like boilerplate to you b/c it's a standard in your domain.
Which is to say, we encourage the workflow pattern you're suggesting of use specialized/simplified APIs and drop down to us when you hit the limits of that API. This is mostly b/c like we've said up top, domain specific APIs are better suited to define what simpler and faster means for that domain.
ETA: but suggestions for more efficient/usable functions for writing that boilerplate are much appreciated - we provide functions like Axes.set and Axes.subplot_mosaic to try and balance flexible functionality with not having to learn all our internals.
paniterka commented on Aug 30, 2023
@nav9 I can only agree with @story645 - matplotlib is a low-level visualization library with lots of customizability, so it looks like for your use case - when you use it only ocassionally and need results quickly - it might be a good solution if you invest some effort upfront to write your own stylesheets and helper functions that produce the level of abstraction you need, and interact mostly with that.
Regarding your example: have you tried using plt.subplots(..., squeeze=False) instead of writing the edge cases? See the docs:
nav9 commented on Aug 30, 2023
squeeze is a good solution. Could've either been False by default, or could've been designed differently. Created an example to see it working:
import numpy from PIL import Image import matplotlib.pyplot as plt imarray = numpy.random.rand(10,10,3) * 255 im = Image.fromarray(imarray.astype('uint8')).convert('RGBA') #rows = 1; cols = 1; #rows = 5; cols = 3; rows = 1; cols = 5; fig, ax = plt.subplots(rows, cols, squeeze=False) fig.suptitle('random plots') i = 0 for r in range(rows): for c in range(cols): ax[r][c].imshow(im) i = i + 1 plt.show() plt.close()My earlier code was from a google search of the error which led here. From what I understand, y'all have already created everything that's required. People (like me) just need to RTFM instead of googling for solutions.
I presented the error message and the above code to ChatGPT without including squeeze=False and it said this is the solution ax[i].imshow(im) # Access subplot using linear index. It didn't suggest squeeze=False. I asked ChatGPT Instead, can I use fig, ax = plt.subplots(rows, cols, squeeze=False) and ax[r][c].imshow(im). It recognized the functionality and said yes.
Come to think of it, even if I looked at the docs, I'd have ignored squeeze, thinking it isn't relevant to what I'm searching for. The sentence extra dimensions are squeezed out from the returned array of just doesn't connect (in my mind). In the same way I noticed only now, that this answer already mentioned squeeze even before I edited it to include the above example.
Kind of similar to how Harvey announced the wrong Miss Universe. UX Stackexchange discussed how it could be avoided.
timhoffm commented on Aug 30, 2023
Can you think of a better wording?
story645 commented on Aug 30, 2023
Any suggestions on how we can make things more discoverable?
nav9 commented on Aug 30, 2023
@timhoffm I've explained a lot and provided links in my previous comments. Fundamentally, it's about thinking how the end user would understand it. If we pick out specifics like this, the discussion will go on endlessly.
@story645: It'd help if we first found out how programmers generally search for matplotlib functionality. Just because I google for it, it doesn't mean everyone would. This discussion has only our opinions. We could ask the programming community on Reddit.
I just searched for which subreddit would be appropriate, and went "Yikes!" when I saw this, this, this, this, this, this. Some constructive ones here, here.
Once the Matplotlib developers figure out what y'all want to know from the community, y'all could formulate questions/polls to gather opinions. It'd have to be done in a way that prevents a flame-war from erupting. It's about collecting opinions. Not about defending or explaining why Matplotlib was designed in the current way. The following subreddits can help: Python, dataisbeautiful, physics, datascience, learnmachinelearning, programming, data, bioinformatics, and more.
Perhaps it'd help to create a new subreddit "Improving Matplotlib", where developers can discuss SEO, design decisions, naming conventions, a more intuitive wrapper which retains the power of matplotlib but makes it easier to use, etc. Having larger community support can also help obtain volunteers who'd gladly fix popular StackOverflow answers.
story645 commented on Aug 30, 2023
Totally agree w/ you if we're running a broad usability survey - we did one a few years back and haven't again mostly cause we don't have the bandwidth. Which, the reddit suggestion sounds good (we have discourse.matplotlib.org and the scientific-python spaces) but would also require staffing.
I think what's happening here is that a lot of us are going into problem solver mode while you're trying to have this broader conversation. We're trying to sort through all the individual pain points you're bringing up - mostly cause some of that is immediately actionable - while you're aiming for this big picture discussion on usability. But I think a lot of this is conversation is circling around this comment from those reddit threads that's backed by previous usability surveys:
Which is why so many of the maintainer responses are in one way or another trying to teach usage. And that's probably what right now a user survey would focus on - how do we set expectations around what Matplotlib is for and how do we better communicate how to use it? For sure there are API changes that could be made too and features added, but I agree that the first step is probably figuring out a cohesive strategy/vision rather than fixing in piecemeal.