Repository navigation
PGF Backend: Support interpolation='none'? #6740
Description
Activity
I found that in the case of a simple scaling, draw_image() receives as additional arguments dx and dy, the target width and height of the image, and transform is set to matplotlib.transforms.IdentityTransform. This is enough for my use case, and simple to implement. I've created a custom backend that does the necessary overrides: https://gist.github.com/f0k/f8c87ffd82488c2ed303989592bf4ebd
This implementation does not support any other transformation than IdentityTransform now, so it's probably not eligible to be merged. I'll leave this issue open as a feature request, but I don't have the time to work on a feature-complete implementation myself -- I'm fine with my workaround. Feel free to close the issue if you think there's not enough demand.
Another solution might be to extend the backend API so a backend can declare "I can handle scaling myself, but anything else is above my limits".Side note: The original backend received a pre-scaled image of 577 x 481 pixels, my modified backend (with option_scale_image() returning True) receives (dy, dx) of (570.0, 480.0). This may indicate a rounding issue somewhere.
Can you put in a PR making that change to the backend?
I could, but then the backend cannot cope with arbitrary affine image transforms any longer. I guess this is not what we want. Let's spell out the options (sorry for the wall of text above):
- A: Leave everything as it is
- B: Include what I have now: This will make interpolation='none' work for simple scaling, but break for rotation, shearing etc. (that'd be a regression)
- C: Extend on what I have now to support arbitrary affine transforms (probably nobody's there to implement that)
- D: Include what I have now, but use matplotlib's image transformation capabilities from within the backend if requested to do rotation, shearing etc. (ugly detour)
- E: Include what I have now, but extend the backend API so matplotlib will transform the image for the PGF backend if it needs to do rotation, shearing etc., but pass it on to the backend if it only needs to scale it (cleanest option in my opinion)
Option E could be realized as an extension of option_scale_image() so instead of returning True or False, the backend could specifically tell matplotlib whether it supports scaling (simplest case), scaling + rotation (harder) or every affine transform (hardest).
Option C would probably be solvable with \pgfsys@transformcm#1#2#3#4#5#6. It would still require a lot of tinkering (and writing/finding some good test cases), but not be impossible.
/edit: Okay, at least one test case is there: http://matplotlib.org/examples/api/demo_affine_image.html.Oh, and another thing: I based my modified backend on matplotlib 1.5.2, which has draw_image(self, gc, x, y, im, dx=None, dy=None, transform=None).
The current master (specifically, #5718) reduced this to draw_image(self, gc, x, y, im, dx=None, dy=None, transform=None), so basically this removes options B, D, E because the scaling shortcut is not available any more. I just got lucky it was still there in the version I have installed.Maybe Option C would be the best solution then. Option E would require peeling out the scaling factors from the transform, which wouldn't be too hard either if we know it's a scale transform.
I don't remember the exact details when I implemented the function back then, but I vaguely remember some thoughts concerning image transformation within PGF.. unfortunately I didn't document them.
First of all, the lack of documentation on how draw_image() should handle the transform parameter and how capabilities are announced / obeyed is a problem of course. But I think I also remember that handling the scaling externally lead to results that didn't match the original figure. Maybe some half-pixel mismatch that causes distortions for very small images? @f0k also mentioned some 'rounding' issues between pre-scaled and non-scaled images, maybe I observed something like that and decided not to chase it down and to go with the pre-scaled image for pixel-perfect matching with other backends.
I agree that the best solution is to move all transformations to the PGF layer. Thanks for the \pgfsys@transformcm#1#2#3#4#5#6 hint! But as @f0k already mentioned this might need some more work and we need a lot of test coverage to ensure perfect alignment.
But I'd also like to ask the question "is it really worth it?". We do have perfect matching with other backends, and I don't believe that 10x or 100x upscaled images grow considerably in size after being PNG compressed. Can't be more than a few kB, right?
Also remember that there is pcolor(), which emits vector graphics instead of pixel images. If your graph consists of only few points, I think this is technically the better solution.
But I'd also like to ask the question "is it really worth it?". We do have perfect matching with other backends, and I don't believe that 10x or 100x upscaled images grow considerably in size after being PNG compressed. Can't be more than a few kB, right?
It's difficult to hit an exact integer upscaling factor, though -- I can influence it via the dpi setting, but if it was easy to hit a certain factor, I could also just set the dpi so it hits a factor of 1. So most of the time, interpolation='nearest' will lead to some pixel blocks being larger and some being smaller. To avoid the picture looking funny, I'd have to crank up the dpi to a very large value.
Now I agree that for a 5x6 pixels image, the gain is negligible. I can blow it up to about 500x600 and rest assured that nobody will notice whether some of the 100x100 pixels are actually 99x99 or 101x101, without wasting too much space.
However, this was just a showcase, I'm actually plotting spectrogram excerpts. When they're blown up by a factor of 10 or 100, they get unmanageably large. When I don't blow them up, they get awkwardly distorted in the process of plotting. When I set interpolation to another interpolation algorithm, they get blurry.
Also remember that there is pcolor(), which emits vector graphics instead of pixel images.
Good to know! (Not for this case, but in general.)
go with the pre-scaled image for pixel-perfect matching with other backends
Is this tested anywhere?
- added this to the milestone on Aug 27, 2016
I'm currently trying to figure out the best way to unify the figures for my thesis, and intend to use the PGF backend to directly \include figures in my LaTeX document.
When creating PGF figures that include raster graphics (e.g., via imshow), the graphic gets stored as a PNG file and imported via \pgfimage, which is great. Unfortunately, it doesn't support interpolation='none' as the PDF backend does:
Checking the created PDF file in Evince, it indeed includes a PNG graphic of 5x6 pixels. The graphic for the PGF file is blown up, though:
I can manually fix this by replacing it with a 5x6 PNG file and tweaking the PGF output as follows:
Can we modify the PGF backend such that it emits the unscaled raster graphic and uses interpolate=false?
I'm willing to work on this myself, but would appreciate some guidance. Looking through the code, the image is written in https://github.com/matplotlib/matplotlib/blob/f1bab50/lib/matplotlib/backends/backend_pgf.py#L612.
The corresponding code in the PDF backend starts at https://github.com/matplotlib/matplotlib/blob/f1bab50/lib/matplotlib/backends/backend_pdf.py#L1600.
It overrides the option_scale_image() function to return True, and it takes an additional transform parameter in draw_image(). I guess this is where the magic happens -- if there's a transformation to apply, it will get the unmodified image and construct the PDF so the image is scaled appropriately.
The documentation on that parameter is scarce (https://github.com/matplotlib/matplotlib/blob/f1bab50/lib/matplotlib/backend_bases.py#L511), but it seems you can convert it into six values giving an affine transformation.
If I want the PGF backend to handle scaling, will the backend have to handle the full range of possible affine transformations (scaling would suffice for me)? Is the transform guaranteed to only be used for interpolation='none', so I can set interpolate=false in this case? Is there any better documentation on the transform?