Pgf plots with pdflatex broken · Issue #18826 · matplotlib/matplotlib · GitHub

Bug report

Bug summary

When I use a very minimal example with the PGF backend I get an LaTeX error.

Code for reproduction

import matplotlib.pyplot as plt
import numpy as np
plt.rcParams.update({"pgf.texsystem": "pdflatex"})
x = np.linspace(-0.5, 0.5, 200)
fig, ax = plt.subplots()
ax.plot(x, x)
plt.savefig('out.pgf')

Actual outcome

ValueError: Error processing '−0.4'
LaTeX Output:

! Package inputenc Error: Unicode character − (U+2212)
(inputenc)                not set up for use with LaTeX.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...

<*> ...ze{10.000000}{12.000000}\selectfont −0.4}

!  ==> Fatal error occurred, no output PDF file produced!

Expected outcome

Matplotlib version

Operating system: Linux 5.4.0-52-generic Wx image #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Matplotlib version: 3.3.2
Matplotlib backend (print(matplotlib.get_backend())): GTK3Agg
Python version: Python 3.8.5
Other libraries: Numpy ('1.18.4')
pdflatex --version:

pdfTeX 3.14159265-2.6-1.40.20 (TeX Live 2019/Debian)
kpathsea version 6.3.1
Copyright 2019 Han The Thanh (pdfTeX) et al.
There is NO warranty.  Redistribution of this software is
covered by the terms of both the pdfTeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the pdfTeX source.
Primary author of pdfTeX: Han The Thanh (pdfTeX) et al.
Compiled with libpng 1.6.37; using libpng 1.6.37
Compiled with zlib 1.2.11; using zlib 1.2.11
Compiled with xpdf version 4.01

Looks like one needs to add the following extra preamble in the three places in backend_pgf that start a tex subprocess:

\ifx\pdftexversion\relax  % implementation of \ifpdftex from the iftex package.
\else\usepackage[utf8]{inputenc}\DeclareUnicodeCharacter{2212}{-}
\fi

PR and tests left as an exercise to the reader.

Thank you for the quick answer and for confirming the bug. I do not feel comfortable, making the changes, because I do not have a proper understanding of pgf and the various latex versions. But I can try anyways.

Do you have an idea how to write a test for this? Would it be worthwhile changing the reference images https://github.com/matplotlib/matplotlib/tree/master/lib/matplotlib/tests/baseline_images/test_backend_pgf? If we include negative values for e.g. the x axis the bug is triggered.

As a general rule, we try hard not to add new baseline images because they cause the repo size to balloon very quickly. In this specific case my (unchecked) guess is that you can use the check_figures_equal decorator and check whether plt.figtext(.5, .5, "$-1$") and plt.figtext(.5, .5, $\N{MINUS SIGN}1$) generate the same plot.

Add test which shall detect issue matplotlib#18826

Add test which actually triggers issue matplotlib#18826

I have implemented the test (#18830). However since _update_converter (https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/testing/compare.py#L236) does not support pgf, the bug is not triggered by the test. I suspect that there is a deeper reason that pgf outputs cannot be compared in this fashion.

Apart from this I think using \DeclareUnicodeCharacter{2212}{-} does not really solve the problem. When I include the resulting pgf file into a LaTeX document and try to compile it with pdflatex I get the same error again. I think the actual solution would be to replace the MINUS SIGN with a - because pdflatex takes care of choosing the correct symbol in math environments on its own. Does that make sense to you?

I think we should aim for a solution in which it is possible to specify pgf.texsystem=pdflatex in matplotlib. This should result give pgf files which can be included into the following minimal LaTeX file:

\documentclass{article}
\usepackage{pgf}
\begin{document}
\input{foo.pgf}
\end{document}

Do you agree that this is the intended scenario?

Idea for fixing matplotlib#18826

By the way, commit 8c4df38 introduced the bug. Before, everything works like a charm (at least for pdflatex, not sure yet for the xelatex and luatlatex). What was the intend of this change?

I just realized that it says in the header of matplotlib-generated pgf files that one needs to include \usepackage[utf8]{inputenc}\DeclareUnicodeCharacter{2212}{-}. I was not aware of this. But is this really needed? Before the above mentioned commit, it was apparently not needed.

Regarding the test: something like the following triggers the issue:

@check_figures_equal(extensions=["pdf"])
@pytest.mark.backend("pgf")
def test_pgf_problem(fig_test, fig_ref):
    mpl.rcParams["pgf.texsystem"] = "pdflatex"
    plt.figure(fig_test.number)
    plt.figtext(.5, .5, "$-1$")
    plt.figure(fig_ref.number)
    plt.figtext(.5, .5, "$\N{MINUS SIGN}1$")

i.e. you need to request the use of the pgf backend (and then you can restrict the comparison to pdf, as other formats don't matter). The pgf backend will compile to a pdf and that's what will get compared.

Regarding \DeclareUnicodeCharacter: we can basically filter all inputs on Matplotlib's side to replace unicode characters by ascii ones, or let tex handle it. I think it's nicer to let tex handle them (especially given than the default engine, xetex, already handles them by default) so I wrote #14567, but I admittedly don't have a very strong opinion on that either.

By the way it looks like the instruction in backend_pgf is missing an \ensuremath{} around the minus (see the corresponding declaration in texmanager.py).

Looking at it again the approach of filtering on Matplotlib's side is basically just

diff --git i/lib/matplotlib/backends/backend_pgf.py w/lib/matplotlib/backends/backend_pgf.py
index a36dda160..9061d4e79 100644
--- i/lib/matplotlib/backends/backend_pgf.py
+++ w/lib/matplotlib/backends/backend_pgf.py
@@ -101,6 +101,7 @@ def common_texification(text):
     # Sometimes, matplotlib adds the unknown command \mathdefault.
     # Not using \mathnormal instead since this looks odd for the latex cm font.
     text = _replace_mathdefault(text)
+    text = text.replace("\N{UNICODE MINUS}", r"\ensuremath{-}")
     # split text into normaltext and inline math parts
     parts = re_mathsep.split(text)
     for i, s in enumerate(parts):
@@ -792,9 +793,6 @@ class FigureCanvasPgf(FigureCanvasBase):
 %% Make sure the required packages are loaded in your preamble
 %%   \\usepackage{pgf}
 %%
-%% and, on pdftex
-%%   \\usepackage[utf8]{inputenc}\\DeclareUnicodeCharacter{2212}{-}
-%%
 %% or, on luatex and xetex
 %%   \\usepackage{unicode-math}
 %%

which I guess does make things simpler for the end user at very little cost for us, so perhaps that's better indeed...

Add test and fix for issue matplotlib#18826

I like this fix very much since it simplifies the users life. Thanks for proposing it! I have updated the merge request (#18830).

And at least locally the tests run through.

Support minus signs for pgf backend in combination with pdflatex

Pgf plots with pdflatex broken · Issue #18826 · matplotlib/matplotlib · GitHub

Repository navigation

Pgf plots with pdflatex broken #18826

Bug report

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Pgf plots with pdflatex broken · Issue #18826 · matplotlib/matplotlib · GitHub

Repository navigation

Pgf plots with pdflatex broken #18826

Description

Bug report

Activity

anntzer commented on Oct 27, 2020

phiadaarr commented on Oct 28, 2020

anntzer commented on Oct 28, 2020

phiadaarr commented on Oct 29, 2020

phiadaarr commented on Oct 29, 2020

phiadaarr commented on Oct 29, 2020

anntzer commented on Oct 31, 2020

phiadaarr commented on Oct 31, 2020

phiadaarr commented on Oct 31, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions