Fwiw the feature looks pretty natural to me, and it's certainly something I've been wanting to have before.

Now that #20054 is in I think this is now working properly on horizontal/vertical colorbars.

I didn't find any tests of the Zoom/Pan tools, am I missing those somewhere, or do we not test those ones and only test the widgets separately?

I agree with @jklymak concerns. We have two spaces, the data space (a, b) and the normalized "color" range (0, 1), which is associated with the color values. A the colored area in a colormap represents this color range and naively, zooming-in into the color range would result in a subrange and this a fraction of the color-range to be used, while the norm (i.e. the data-to-color relation) is unchanged. This however is less useful because you don't get additional detail. Actually, I'm wondering if it is useful at all.

What people usually want is to manipulate the norm. Technically, you would like to move the ticks on the colorbar, but that's not something we can directly support as a mouse interaction.

So two questions:

1. Is the naive color-range manipulation something that we consider supporting at all? If so, we cannot use zooming and panning tools for norm manipulation.
2. If not, is the changed semantics of the tools for colorbars as proposed here bearable in terms of user expectation or is that too confusing?

Mouse scrolling

There are standard conventions for mouse scrolling

scroll-up/down: vertical scroll
shift+scroll-up/down: horizontal scroll
ctrl+scroll-up/down: zoom in/out

For a regular axes the first to should translate to pan and the third should translate to zoom. (We don't have this implemented right now, but it would be a nice addition if anybody is interested in picking this up.)

We'd have to think if these could be resaonbly be adapted for colorbars in the above context.

Maybe a use-case for why I wanted this in the first place would be helpful. When making figures, coauthors will often say, well what if vmin/vmax were just a little higher/lower, how would that look? That could be done with adding widgets/sliders to control the vmin/vmax, but I don't want to be adding components, so I figured this was a good way to fit it in. Zoom/pan seemed natural to me, but if it isn't to other people we can consider some other interactions as well.

• set_xlim() - interesting, I hadn't considered that. It seems like set_xlim/set_ylim on a Colorbar axis should be dispatched to norm.vmin/norm.vmax and not zoomed in to a smaller data range. For example, that could be misleading if you have extensions on your colorbar, but the limits at the top aren't actually the norm limits...

• Zoom out / extend limits - This should be right click and drag I believe, the same as a normal axis because I just took most of the code from there. This also works with the home, back, and forward buttons too.

• Moving the ticks on the colorbar - I'm not sure I follow here? The ticks are moving in the above example. Do you mean you want to interact with just the ticks outside the axis and not inside the axis where the colors are?

As for the questions about whether this seems bearable for user expectations or not... I came up with this idea, so I'd say yes, but YMMV :) Another thing here is that you have to explicitly enter zoom/pan mode to get this available, so it isn't something that will just blindly be available for someone mousing over the image, so there is that safety net in a sense.

So maybe we start with set_xlim/set_ylim? Currently we do not have a colorbar.set_lim because we usually expect the user to change the colorbar by changing the vmin/vmax on the Norm. Hence the colorbar is a subordinate to the Norm. This PR inverts that and makes the colorbar actively control the norm. I will say this is also a confusion I see on stackoverflow that folks think the colorbar should somehow control the colors in the image. Currently, it is really easy currently to say "no it doesn't", the colorbar passively reflects the norm. If we no longer make that the case, then we need to think this through very carefully what the back and forth is and have thorough tests.

Again, maybe this is OK, but can we develop a programatic API first that doesn't rely on pan and zoom? i.e. colorbar.set_lim or colorbar.set_vlims, or?? That would make me a lot more comfortable rather than jumping to a GUI-only API, and would make the discussion a lot easier...

Some questions. If I set the vlim on the colorbar, and then I set it on the mappable, which is used at draw time? What if someone flips the limits? Norms often have other parameters than limits, in particulate everyone's favourite is symlognorm, or centered norm. How will this interact with those? Will the center move as well?

I'll put on the dev call for today. Feel free to stop by and pitch it. I don't know if we should discuss all the nitty gritty, but putting the PR on folks' radar and asking for comment here (or if you make a simpler colorbar.set_vlims PR) would be great.

I didn't find any tests of the Zoom/Pan tools, am I missing those somewhere, or do we not test those ones and only test the widgets separately?

Zoom is tested in lib/matplotlib/tests/test_backend_bases.py::test_interactive_zoom.

I guess the other UI for this is instead of mucking with the colorbar directly, a norm editor pops up in a separate helper window. This could also change the colormap.

@jklymak, you missed a good lengthy discussion today :) I think what you're proposing is essentially @anntzer's project here: https://github.com/anntzer/mplinorm

I'll be giving this some more thought over the next week and try to improve the UI.

OK, so if I zoom, I change the color limits of the ScalarMappable, but if I set the limits of the axis, i.e. with set_xlim it zooms in on part of the colorbar (at least after #20054) but leaves the mappable unchanged.

I think this is a mistake, will open a different issue about that next, but resolving that is probably a pre-req for merging this.

Further, I don't see how to expand the limits with this GUI API.

I would assume that right click zoom and right-click pan "zoom out" like they do with normal axes?

I think that the pan/zoom verbiage is semantically correct (if a bit jarring on first pass), but the UI needs a bit of tuning to make it clearer what is happening.

Although we implement the colorbar on an (increasingly standard) Axes, it is much more like the tick labels than it is like an normal "image". Under the hood we have transforms that go from data space -> x/y. The absolute position within the figure is meaningless, but by putting ticks on the Axes we can locally give meaning to the (relative) position. Even if we remove the ticks, we the relative positions with in an Axes still have meaning. When we adjust these transforms (by adjusting the x and y limits) we call this "panning" and "zooming" (depending on if we change the dynamic range or not).

Similarly, when we color map we have a transform (implemented in 2 parts: the norm and the colormap) that take data from dataspace -> RGB. The absolute color has no meaning (change the color map does not change the inherent meaning of the data any more than translating the whole axes around the figure changes the meaning!), but the relative colors tell us something about the relations between the data. If we then add a colorbar then we can get back access to the absolute values the same way ticks do.

In the images below, I think it is un-controversial to say "the y-axis was zoomed" or "the y-axis was panned". Doing exactly the same thing to the clim I think makes sense to use the same wording (there are some slight differences like when the curve goes out of the bounds it gets clipped but the color mapping saturates, could set over/under colors to transparent to get the same effect). Given the limitations of screens (we can not get a color axis sticking out of the screen!), I think implementing pan/zoom on the colorbar to "pan" and "zoom" in norm-space is the next best thing.

Please forgive the copy-pasta nature of this code!

import matplotlib.pyplot as plt
import numpy as np

th = np.linspace(start := 0, stop := 2 * np.pi, N := 1024)
data1d = np.sin(th)

data2d = np.sin(th[:, np.newaxis]) * np.cos(th[np.newaxis, :])

curve_axes = ["1d", "y zoom in", "y zoom out"]
image_axes = ["2d", "color zoom in", "color zoom out"]

fig, ax_dict = plt.subplot_mosaic(
    [
        curve_axes,
        image_axes,
    ],
    constrained_layout=True,
)

for an in curve_axes:
    ax = ax_dict[an]
    ax.plot(th, data1d)
    ax.set_title(an)
    if an == "y zoom in":
        ax.set_ylim(-0.01, 0.01)
    elif an == "y zoom out":
        ax.set_ylim(-100, 100)
    else:
        ax.set_ylim(-1, 1)

for an in image_axes:
    ax = ax_dict[an]
    im = ax.imshow(
        data2d,
        extent=[start - (stop - start) / (2 * N), stop + (stop - start) / (2 * N)] * 2,
    )
    ax.set_title(an)
    fig.colorbar(im, ax=ax, aspect=7)
    if an == "color zoom in":
        im.set_clim(-0.01, 0.01)
    elif an == "color zoom out":
        im.set_clim(-100, 100)
    else:
        im.set_clim(-1, 1)

fig.savefig("/tmp/zoom.png")


curve_axes = ["1d", "y pan up", "y pan down"]
image_axes = ["2d", "color pan up", "color pan down"]

fig, ax_dict = plt.subplot_mosaic(
    [
        curve_axes,
        image_axes,
    ],
    constrained_layout=True,
)

for an in curve_axes:
    ax = ax_dict[an]
    ax.plot(th, data1d)
    ax.set_title(an)
    if an == "y pan up":
        ax.set_ylim(0, 2)
    elif an == "y pan down":
        ax.set_ylim(-2, 0)
    else:
        ax.set_ylim(-1, 1)

for an in image_axes:
    ax = ax_dict[an]
    im = ax.imshow(
        data2d,
        extent=[start - (stop - start) / (2 * N), stop + (stop - start) / (2 * N)] * 2,
    )
    ax.set_title(an)
    fig.colorbar(im, ax=ax, aspect=7)
    if an == "color pan up":
        im.set_clim(0, 2)
    elif an == "color pan down":
        im.set_clim(-2, 0)
    else:
        im.set_clim(-1, 1)
        
fig.savefig("/tmp/pan.png")

plt.show()

As mentioned on the call, this only makes sense for continuous (i.e. not categorical) norms.

I don't have any problem with the semantics of zooming. I have practical concerns about the complexity of two-way linking of the colorbar to the mappable/norm it represents.

I do have another issue with the GUI aspect of zooming on a colorbar. @greglucas has a nice hefty colorbar up there, and you can still see him struggle to get the drag box to stay in the colorbar. I almost never have such large colorbars as I consider them a waste of space so I would be even more screwed. Below is a published example:

struggle to get the drag box to stay in the colorbar.

I don't think you need to keep the cursor in the colorbar; as long as the initial click is in it, the drag box will just be clipped to the axes.

Thats good - but it's still a little fiddly...

@jklymak, the colorbar and mappable are currently directly linked. cb.norm is cb.mappable.norm So, if you update the vmin/vmax of either the colorbar or the image mappable, that state should get propagated. I think this is the way it should be as well, I want my colorbar to represent the data in my image when I've updated the state and not lag behind or be out of sync somehow.

I suppose if you make your colorbar too small and can't interact with it, then that just means you can't take advantage of this feature as easily, it doesn't adversely affect anything. There was a discussion about trying to interact with the ticks instead of the axis/colored area to make it more explicit as well what we are changing.

@jklymak, the colorbar and mappable are currently directly linked. cb.norm is cb.mappable.norm So, if you update the vmin/vmax of either the colorbar or the image mappable, that state should get propagated. I think this is the way it should be as well, I want my colorbar to represent the data in my image when I've updated the state and not lag behind or be out of sync somehow.

Sure. But that does not necessarily have anything to do with the axes min/max. You are proposing that it does, and @tacaswell is proposing that zooming in on the colorbar is a "mistake" for some reason I've not heard. I'm not 100% against either proposal, but it needs some discussion. And just to reiterate, folks have asked to be able to zoom in on the colors on the colorbar, but to preserve the norm/colormap mapping. The ability to do so is new in #20054, so now is definitely the time to discuss if we don't want that behaviour, but except to implement these semantics, I'm not sure what the argument is against it.

I still feel the way forward here is if we are going to make this an action on the colorbar, we should first define the colorbar method that will do this, and leave the GUI discussion out of it.

Just a note about the limits corresponding to vmin/vmax. This is in the comments, which seems to indicate that a larger colorbar axes than just vmin and vmax is explicitly allowed in the old API (imagine contours on a plot, saturated at high values, but you still want them labeled on the colorbar.

        # copy the norm and change the vmin and vmax to the vmin and
        # vmax of the colorbar, not the norm.  This allows the situation
        # where the colormap has a narrower range than the colorbar, to
        # accommodate extra contours:

I guess I still just don't quite understand this use case and wonder if it can't be solved in a different manner... Is the real request to be able to use less of the colormap than is available by default because you want it to saturate at a different value? I've used this crude hack in the past to change the saturation point, so maybe a public function on a colormap to do something similar would be the better place for it than messing with limits on an axes?

cmap = mpl.cm.get_cmap('inferno')
# Remove the bottom 10% of the cmap
cmap_small = mpl.colors.LinearSegmentedColormap.from_list('inferno_small', cmap(np.linspace(0.1, 1.0, 256)))

pc = ax.contour(np.arange(1, 11), np.arange(1, 11), np.arange(100).reshape(10, 10), 
                levels=np.arange(0, 100, 10), vmin=0, vmax=50)
cb = fig.colorbar(pc)

is a use case where the norm goes from 0 to 50 but the colorbar goes to 100.

BTW, just to be clear, I'm not saying the above functionality should trump the proposed functionality, just that it is something we will break...

I was mostly thinking about the case where you make the range of the color bar narrow than the vmin/vmax which seems like a miss-feature to me (because then we have put colors in the figure that do not have a key to indicate what that color means), but making the top/bottom of the color bar bigger than the norm (and basically implementing the extend arrows by having a constant area in the color bar).

I think I am going to back away from "the color bar limits should always exactly match the norm limits" to "the norm limits should be contained in the color bar limits".

The first thing that occurred to me under that framing is we have a norm_tracks_limits setting on the color bar. If True we get the behavior with pan/zoom that @greglucas is proposing here, if it is False, then when the user sets the limits on the color bar we expand them to include vmin/vmax (there is precedent for this with the nonsingular step) and if they are bigger just let it happen. In all of these cases the "extend" arrows will still be consistent.

I think the constraints are:

• every possible color in the cmap should be shown with a correct value in the color bar
• ever tick / value along the axis of the color bar should be matched with the color that value would be in the data

every possible color in the cmap should be shown with a correct value in the color bar

That seems overly prescriptive. Folks have specifically asked to have the limits tighter than vmin and vmax. Why would we forbid that?

Are we decided this should go in 3.5? Seems we have already made the beta, and this is kind of a major change to include after the beta?

My (biased) vote is for 3.5, but I do understand the hesitation. My thinking is: the betas and rc's probably don't get a lot of interactive testing use, so I'm not sure how much this feature would even be tested there. It only affects interactive users, not static figures/plots.

See https://stackoverflow.com/questions/69257255/change-colorbar-limits-without-changing-the-values-of-the-data-it-represents-in for another instance of someone wanting the colorbar limits to be different than the vmin/vmax of the norm. I'm not sure I completely sympathize, and such users always have the ability to create a custom colormap, but it is often requested that the colorbar be "zoomed" without affecting the norm.

It's fine for 3.5, but not after rc.

OK, well lets try it....

Sorry for the late comment but #20471 should probably be reverted before 3.5 lands if this feature is merged.

This feature was merged, so we should definitely think about what to do with that example.

When I run the example without any zoom/pan clicked and interact with the colorbar it seems to be very finnicky, so I'm not sure what the expected outcome is supposed to be? I'm wondering if some of the other norm updates messed with the example too...

Instead of removing the example, we could keep it and add two colorbars to the image and label one "standard zoom/pan" and the other "manual interactivity" to demonstrate multiple ways to do this as well. It may even be beneficial to put an example in demonstrating how to turn off zoom/pan on that specific axes colorbar.ax.can_zoom = colorbar.ax.can_pan = lambda: False so that you can override it with your own methods.

(Side point, but I think(?) the "supported" API for disabling zoom/pan is ax.set_navigate(False)?)

Yes, I agree, but that example is actually turning ax.set_navigate(True) on that axes and then using pickers instead of zoom/pan. So, I was looking for a way to disable just the zoom/pan that was added on the colorbars. Not very elegant at all.

The easiest might be to remove that example, but I think we should make sure @richardsheridan's use-case is covered before just removing that.