Repository navigation
PDFPages Errors on savefig (missing encode) #9112
Description
With just a part of http://matplotlib.org/examples/pylab_examples/multipage_pdf.html
from matplotlib.backends.backend_pdf import PdfPages
import matplotlib.pyplot as plt
with PdfPages('multipage_pdf.pdf') as pdf:
plt.figure(figsize=(3, 3))
plt.plot(range(7), [3, 1, 4, 1, 5, 9, 2], 'r-o')
plt.title('Page One')
pdf.savefig() # saves the current figure into a pdf page
plt.close()Traceback
pdf.savefig()
File "/usr/lib/python3/dist-packages/matplotlib/backends/backend_pdf.py", line 2462, in savefig
figure.savefig(self, format='pdf', **kwargs)
File "/usr/lib/python3/dist-packages/matplotlib/figure.py", line 1572, in savefig
self.canvas.print_figure(*args, **kwargs)
File "/usr/lib/python3/dist-packages/matplotlib/backend_bases.py", line 2245, in print_figure
**kwargs)
File "/usr/lib/python3/dist-packages/matplotlib/backends/backend_cairo.py", line 483, in print_pdf
return self._save(fobj, 'pdf', *args, **kwargs)
File "/usr/lib/python3/dist-packages/matplotlib/backends/backend_cairo.py", line 516, in _save
surface = cairo.PDFSurface (fo, width_in_points, height_in_points)
File "/usr/lib/python3/dist-packages/cairocffi/surfaces.py", line 792, in __init__
_encode_filename(target), width_in_points, height_in_points)
File "/usr/lib/python3/dist-packages/cairocffi/surfaces.py", line 52, in _encode_filename
filename = filename.encode(sys.getfilesystemencoding())
AttributeError: 'PdfPages' object has no attribute 'encode'
The issue I see lies in Figure.savefig(self, *args, **kwargs) where Args get listed as:
Arguments: *fname*: A string containing a path to a filename, or a Python file-like object, or possibly some backend-dependent object such as :class:`~matplotlib.backends.backend_pdf.PdfPages`.
However the stack show above shows this "filename" passed down to cairosurfaces where it wants to encode the filename, which it can't do as it has a PDFPages object and not a filename.
I have been looking through the tests and we don't seem to try to save any tests for this. Maybe however this bug comes up only with the cairocffi.
Activity
anntzer commented on Aug 28, 2017
pycairo doesn't handle either this because PdfPages doesn't even have a write method:
Traceback (most recent call last): File "foo.py", line 8, in <module> pdf.savefig() # saves the current figure into a pdf page File "/usr/lib/python2.7/site-packages/matplotlib/backends/backend_pdf.py", line 2474, in savefig **kwargs) File "/usr/lib/python2.7/site-packages/matplotlib/figure.py", line 1573, in savefig self.canvas.print_figure(*args, **kwargs) File "/usr/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 2252, in print_figure **kwargs) File "/usr/lib/python2.7/site-packages/matplotlib/backends/backend_cairo.py", line 483, in print_pdf return self._save(fobj, 'pdf', *args, **kwargs) File "/usr/lib/python2.7/site-packages/matplotlib/backends/backend_cairo.py", line 516, in _save surface = cairo.PDFSurface (fo, width_in_points, height_in_points) TypeError: PDFSurface argument 1 must be None, or a filename (str), or a file object, or an object that has a "write" method (like StringIO).Even FigureCanvasPdf itself needs to check for whether the argument passed in is of class PdfPages. I think the proper fix is to make PdfPages more file-like, so that we don't have to carry these special casings around (in order e.g. for mpl_cairo to also support this).
OceanWolf commented on Aug 28, 2017
Hmm, I assume this worked at somepoint because I took the code out of our examples. I wonder when it broke, also I just tried running the test test_backend_pdf.test_multipage_pagecount, copy pasting that and same thing.
Why don't our tests flag this up? Do they use an older/newer version of cairo? I run debian testing.
anntzer commented on Aug 28, 2017
They probably don't use cairo at all but matplotlib's own pdf backend (did you set the backend to cairo either via the MPLBACKEND envvar or ~/.config/matplotlib/matplotlibrc?).
I don't think PdfPages+cairo can be made to work without major surgery because PdfPages seem to be tightly coupled to the matplotlib's own pdf backend (for example, as soon as the object is initialized the pdf header is already written to the stream, which cairo obviously cannot know about). If we really care about fixing this I think it'll involve moving PdfPages out of the backend_pdf into a generic MultiPage object that is fully decoupled from the backends, and simply add a flush_page (or similarly named) method to the renderer, that gets called at the end of MultiPage.savefig.
OceanWolf commented on Aug 28, 2017
Hmm, I set backend : GTK3Cairo in matplotlibrc, but I didn't think that had an effect as we use the pdf backend. From my understanding of savefig, it ignores the backend in use and switches to the non-gui backend. With the gui backends, this all seems hard-coded, that you can't run Qt5 with Cairo, only Agg, so it seems a bit weird that this happens.
That said, I know next to nothing about the differences between cairo and agg. only that cairo works with vector and agg works with raster AFAIK, and that they shouldn't give any differences in the end result.
I feel torn about what we should do here, as I don't know about the differences, so on the one hand I say as we promise in the docs (that I quoted above) that PDFPages works as a valid file, then it should; and on the other hand, why not auto detect that you use cairo when trying to save a pdf which won't work, and allow pdf to work independently of whatever backend you have specified in matplotlibrc?
anntzer commented on Aug 28, 2017
Wrong. savefig asks the backend "can you save in the format requested by the user"? (specifically, do you have a print_pdf method in this case), and only if the backend cannot, it will look up the default backend for that purpose (backend_pdf for pdf, backend_agg for png, etc.)
The cairo backend can actually output pdf natively (https://www.cairographics.org/manual/cairo-PDF-Surfaces.html), and I think this is a feature to be kept (well the current cairo backend is terrible in so many ways that I really wonder why you actually use gtk3cairo by default! but in the long run --or possibly even now with mpl_cairo-- native pdf output by a native cairo backend is likely going to be faster than the current pure python pdf output), it just cannot output PdfPages because that is a feature deeply tied to backend_pdf.
So I think the proper fix would be something like: in PdfPages.savefig, make sure that backend_pdf is actually going to be used. This does not involve setting the backend (because that is not how things are decided), but (ugh) patching the current figure canvas, if it is not a FigureCanvasPdf, to remove its print_pdf method, if any... but that appears not to be possible on the FigureCanvasGTK3Cairo, probably due to how the GTK3 extension classes are written! So perhaps a better way would be to allow one to set a print_foo method to None (similarly to https://docs.python.org/3/reference/datamodel.html#object.__hash__) to allow one to temporarily disable it.
The following patch appears to work
index 98aca9251..ce792a34f 100644 --- a/lib/matplotlib/backend_bases.py +++ b/lib/matplotlib/backend_bases.py @@ -2101,7 +2101,7 @@ class FigureCanvasBase(object): method_name = 'print_%s' % format # check if this canvas supports the requested format - if hasattr(self, method_name): + if getattr(self, method_name, None) is not None: return self # check if there is a default canvas for the requested format diff --git a/lib/matplotlib/backends/backend_pdf.py b/lib/matplotlib/backends/backend_pdf.py index 87a5c1e64..c284f6b40 100644 --- a/lib/matplotlib/backends/backend_pdf.py +++ b/lib/matplotlib/backends/backend_pdf.py @@ -2523,18 +2523,23 @@ class PdfPages(object): instance is provided, this figure is saved. If an int is specified, the figure instance to save is looked up by number. """ - if isinstance(figure, Figure): - figure.savefig(self, format='pdf', **kwargs) - else: + if not isinstance(figure, Figure): if figure is None: - figureManager = Gcf.get_active() + manager = Gcf.get_active() else: - figureManager = Gcf.get_fig_manager(figure) - if figureManager is None: - raise ValueError("No such figure: " + repr(figure)) - else: - figureManager.canvas.figure.savefig(self, format='pdf', - **kwargs) + manager = Gcf.get_fig_manager(figure) + if manager is None: + raise ValueError("No figure {}".format(figure)) + figure = manager.canvas.figure + patched = False + try: + if not isinstance(figure.canvas, FigureCanvasPdf): + patched = True + figure.canvas.print_pdf = None + figure.savefig(self, format='pdf', **kwargs) + finally: + if patched: + del figure.canvas.print_pdf def get_pagecount(self): """OceanWolf commented on Aug 28, 2017
I guess that also makes sense. See if you can print the canvas in its current state, if not, re-render using a different canvas.
Why I use GTK3Cairo? Because I prefer(ed) GTK3 as a Gui, and a choice between Agg and Cairo, I choice Cairo because I prefer to work in vector graphics, they remain editable, flexible, and I like that sense of freedom, and I also have a thing about using native support where possible because it feels more intuitive, cleaner, not reinventing the wheel. When you say in the long run, was that support for GTK3Cairo?
But as I say I have very little understanding of this, perhaps one of these days I will write a guide for MPL explaining it, and then people can correct me where my understanding goes astray.
I see code here that shows how to save a multiple-page PDFs using cairo for python, so we can probably adapt it for MPL, but a project for another time.
For now, I will just use the simple workaround, just changing the backend to the pdf_backend, i.e. with
anntzer commented on Aug 28, 2017
The current (gtk3)cairo backend is really slow (#8787), plots images in the wrong direction (#8798) and mishandles alpha (#8798). Mathtext rendering also leaves to be desired (e.g., \sqrt looks quite ugly).
At the end of the day, what gets rendered to your screen is always rasterized; meanwhile, if you save to a vector format, then backend_svg/backend_pdf can also do it and are automatically called when using e.g. backend_gtk3agg (as gtk3agg itself does not have the relevant print_* methods).
Long run (IMO) is to switch to a natively written cairo backend (https://github.com/anntzer/mpl_cairo). While images and alpha can be solved fairly easily in the current implementation, only a native implementation can fix the performance issue (and, in fact, do better than Agg).
OceanWolf commented on Aug 28, 2017
As an alternative fix, how about creating a new print_multi_page_pdf method which the pdf backend can just alias to print_pdf, none of the other backends will have it, so then in PdfPages you just call instead:
Of course, I would prefer to just have mpl work with cairo multipage pdfs, and I can't see why it shouldn't from a technical standpoint (just looping over figures like we do here with PdfPages) and thus avoid such a workaround.
And ooh, nice, mpl_cairo, I didn't know it existed :). It looks like quite a small library, I don't understand why python-cairo and cairocffi couldn't do what you do...
anntzer commented on Aug 28, 2017
Yup, I'd rather think of the proper solution for crossbackend multipage pdf support rather than introducing more workarounds.