Repository navigation
[Bug]: 3.5.0rc1: MPLCONFIGDIR no longer supported to load custom configurations? #21375
Description
Bug summary
Hello,
while packaging 3.5.0rc1, i notice that it's no longer possible to set a custom matplotlibrc, if the default one is "invalid".
# MPLCONFIGDIR=. PYTHONPATH=/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9 python3
Python 3.9.7 (default, Sep 24 2021, 09:43:00)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
rcfile /usr/share/matplotlib/mpl-data/matplotlibrc
<stdin>:1: MatplotlibDeprecationWarning:
The mpl_toolkits.legacy_colorbar rcparam was deprecated in Matplotlib 3.4 and will be removed two minor releases later. Please update your matplotlibrc.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 896, in <module>
defaultParams = rcsetup.defaultParams = {
File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 899, in <dictcomp>
rcParamsDefault[key]),
File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 674, in __getitem__
return dict.__getitem__(self, key)
KeyError: 'pcolormesh.snap'
(that's because /usr/share/matplotlib/mpl-data/matplotlibrc is a template file in Debian's setup); the same exact error happens even setting MATPLOTLIBRC=./matplotlibrc, as specified in the doc the doc).
This used to work, and it's the way how we set specific settings for our build environment (like backend, etc).
looking at the code in __init__.py it appears _rc_params_in_file() is always executed against what cbook._get_data_path("matplotlibrc") returns, which with a couple of hoops results in a static file (mind that we customize this location in debian as seen here, which is the path you see above ).
Without the possibility to either use MPLCONFIGDIR or MATPLOTLIBRC, building matplotlib in Debian becomes quite hard. Is this just an oversight and support should be re-introduced, or is this a conscious decision?
thanks,
Sandro
Code for reproduction
see aboveActual outcome
traceback
Expected outcome
no traceback
Operating system
Debian
Matplotlib Version
3.5.0rc1
Matplotlib Backend
No response
Python version
3.9.7
Jupyter version
No response
Other libraries
No response
Installation
source
Conda channel
No response
Activity
anntzer commented on Oct 18, 2021
Can you please provide a simple repro for devs not running debian? (e.g. using docker, or generic unix commands, etc.)
sandrotosi commented on Oct 18, 2021
i can try:
and finally, at the end you should get the error:
# build the doc MPLCONFIGDIR=. MATPLOTLIBDATA=/tmp/matplotlib/lib/matplotlib/mpl-data/ PYTHONPATH=/tmp/matplotlib/build/lib.linux-x86_64-3.9 xvfb-run -a /usr/bin/make -C doc html make[1]: Entering directory '/tmp/matplotlib/doc' Running Sphinx v4.2.0 Configuration error: There is a programmable error in your configuration file: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sphinx/config.py", line 328, in eval_config_file exec(code, namespace) File "/tmp/matplotlib/doc/conf.py", line 21, in <module> import matplotlib File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 895, in <module> defaultParams = rcsetup.defaultParams = { File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 898, in <dictcomp> rcParamsDefault[key]), File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 674, in __getitem__ return dict.__getitem__(self, key) KeyError: 'pcolormesh.snap'the debian build system is pretty crazy for mpl, let me know if you want me to explain some of its parts in details but this should get you going.
Just to cover all our bases, we do have some patches we apply https://salsa.debian.org/python-team/packages/matplotlib/-/tree/master/debian/patches
and while preparing for this reply, i noticed that not applying those patches (so skipping the quilt push step above), make the doc build move forward :( :( so it's some of Debian's customizations that's messing things up, suspecting https://salsa.debian.org/python-team/packages/matplotlib/-/blob/master/debian/patches/20_matplotlibrc_path_search_fix.patch even tho i'm not sure why, as that's the same patch we've had for a long time
anntzer commented on Oct 18, 2021
Unfortunately, the above results in
<elided> ========================================================================================================= short test summary info ========================================================================================================= ERROR build/lib.linux-x86_64-3.9/matplotlib/tests/test_backend_pdf.py - _pytest.nodes.Collector.CollectError: ImportError while importing test module '/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/tests/test_backend_pdf.py'. ERROR build/lib.linux-x86_64-3.9/matplotlib/tests/test_backend_pgf.py - _pytest.nodes.Collector.CollectError: ImportError while importing test module '/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/tests/test_backend_pgf.py'. ERROR build/lib.linux-x86_64-3.9/matplotlib/tests/test_pickle.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' ERROR test_axes_grid.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' ERROR test_axes_grid1.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' ERROR test_axisartist_angle_helper.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' ERROR test_axisartist_axis_artist.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' ERROR test_axisartist_axislines.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' ERROR test_axisartist_clip_path.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' ERROR test_axisartist_floating_axes.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' ERROR test_axisartist_grid_finder.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' ERROR test_axisartist_grid_helper_curvelinear.py - AttributeError: module 'matplotlib.cbook' has no attribute '_rename_parameter' !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 12 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! =============================================================================================== 1 skipped, 4 warnings, 12 errors in 10.06s ================================================================================================ Traceback (most recent call last): File "/tmp/matplotlib/tests.py", line 33, in <module> retcode = test(argv=extra_args) File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/_api/deprecation.py", line 205, in wrapper return func(*args, **kwargs) File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 1261, in test use(old_backend) File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 1144, in use plt.switch_backend(name) File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/pyplot.py", line 267, in switch_backend class backend_mod(matplotlib.backend_bases._Backend): File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/pyplot.py", line 268, in backend_mod locals().update(vars(importlib.import_module(backend_name))) File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_tkagg.py", line 1, in <module> from . import _backend_tk File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/backends/_backend_tk.py", line 14, in <module> from PIL import Image, ImageTk ImportError: cannot import name 'ImageTk' from 'PIL' (/usr/lib/python3/dist-packages/PIL/__init__.py) make: [debian/rules:35: build-arch-stamp] Error 1 (ignored) rm -f matplotlibrc touch build-arch-stamp dh_testdir # build the doc MPLCONFIGDIR=. MATPLOTLIBDATA=/tmp/matplotlib/lib/matplotlib/mpl-data/ PYTHONPATH=/tmp/matplotlib/build/lib.linux-x86_64-3.9 xvfb-run -a /usr/bin/make -C doc html make[1]: Entering directory '/tmp/matplotlib/doc' /bin/sh: 1: python: not found make[1]: *** [Makefile:34: html] Error 127 make[1]: Leaving directory '/tmp/matplotlib/doc' make: *** [debian/rules:24: build-indep-stamp] Error 2 root@localhost:/tmp/matplotlib#While we're at it, can you modify the above so that the docs can be built with (same options as previously) make O=-P (so that it can be debugged more easily)?
sandrotosi commented on Oct 19, 2021
I was able to replicate the error, but only when partially applying https://salsa.debian.org/python-team/packages/matplotlib/-/blob/master/debian/patches/20_matplotlibrc_path_search_fix.patch (ie either one of the 2 hunks without the other), but i was also able to get rid of the error reported initially by NOT applying that patch, which puzzles me: as i said before, we've been using variations of this patch for a long time
sandrotosi commented on Oct 19, 2021
oh apologies, i did not properly read the error: i think you need to run apt install python-is-python3 before ./debian/rules binary
now the sequence of commands should be
with no patches applied, this fails with
which is expected, since sphinx_panels is not available in debian (yet, guess i'm gonna have to package it)
sandrotosi commented on Oct 19, 2021
I've also tried to apply @QuLogic 's patch from fedora-python@4039d6d, but it fails the same way:
echo "backend : TkAgg" > matplotlibrc for v in 3.9; do \ LIB=$(python$v -c "from distutils.command.build import build ; from distutils.core import Distribution ; b = build(Distribution()) ; b.finalize_options() ; print(b.build_platlib)") ; \ PYTHONPATH=$LIB MATPLOTLIBDATA=/tmp/matplotlib/lib/matplotlib/mpl-data/ MPLCONFIGDIR=. xvfb-run -a python$v tests.py --verbose --full-trace ; \ done /tmp/matplotlib/tests.py:17: MatplotlibDeprecationWarning: The mpl_toolkits.legacy_colorbar rcparam was deprecated in Matplotlib 3.4 and will be removed two minor releases later. Please update your matplotlibrc. from matplotlib import test Traceback (most recent call last): File "/tmp/matplotlib/tests.py", line 17, in <module> from matplotlib import test File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 896, in <module> defaultParams = rcsetup.defaultParams = { File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 899, in <dictcomp> rcParamsDefault[key]), File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 675, in __getitem__ return dict.__getitem__(self, key) KeyError: 'pcolormesh.snap' make: [debian/rules:35: build-arch-stamp] Error 1 (ignored) rm -f matplotlibrc touch build-arch-stamp dh_testdir # build the doc MPLCONFIGDIR=. MATPLOTLIBDATA=/tmp/matplotlib/lib/matplotlib/mpl-data/ PYTHONPATH=/tmp/matplotlib/build/lib.linux-x86_64-3.9 xvfb-run -a /usr/bin/make -C doc html make[1]: Entering directory '/tmp/matplotlib/doc' Running Sphinx v4.2.0 Configuration error: There is a programmable error in your configuration file: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sphinx/config.py", line 328, in eval_config_file exec(code, namespace) File "/tmp/matplotlib/doc/conf.py", line 21, in <module> import matplotlib File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 896, in <module> defaultParams = rcsetup.defaultParams = { File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 899, in <dictcomp> rcParamsDefault[key]), File "/tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/__init__.py", line 675, in __getitem__ return dict.__getitem__(self, key) KeyError: 'pcolormesh.snap'so, the only reason it works without patch it's because matplotlib is reading the file available in the source dir:
note the rcfile: /tmp/matplotlib/build/lib.linux-x86_64-3.9/matplotlib/mpl-data/matplotlibrc which is printed via:
diff --git a/lib/matplotlib/__init__.py b/lib/matplotlib/__init__.py index ba9cd6c9..79f1eb0d 100644 --- a/lib/matplotlib/__init__.py +++ b/lib/matplotlib/__init__.py @@ -873,6 +873,7 @@ Please do not ask for support with these customizations active. # When constructing the global instances, we need to perform certain updates # by explicitly calling the superclass (dict.update, dict.items) to avoid # triggering resolution of _auto_backend_sentinel. +print(f' rcfile: {cbook._get_data_path("matplotlibrc")}') rcParamsDefault = _rc_params_in_file( cbook._get_data_path("matplotlibrc"), # Strip leading comment.I think this should show that MPLCONFIGDIR (or MATPLOTLIBRC) is no longer functional, because the command line clearly has MPLCONFIGDIR=. in it
QuLogic commented on Oct 19, 2021
What does this mean? Note, as of #19603, that file is the source of rcParam defaults. So you should be wary of changing it, and it should not, e.g., be moved to /etc (though you can still set the search path to look there.)
sandrotosi commented on Oct 19, 2021
Thanks, i think that clarified what's happening: everytime import matplotlib runs, it reads lib/matplotlib/mpl-data/matplotlibrc (regardless if the lines are commented, which initially tripped me off) to read the default for each and every config option.
In Debian, we dont ship mpl-data along with the module, but we split it in a separate package, and change the path to look for that file with this patch, but the patch is applied before the module is built (setup.py build), and so it's included in the installed module and used when running tests or building doc.
What complicates things even more, is that some of the packages we need to build matplotlib depends on matplotlib itself so there the old version of mpl installed on the same system where we're building the new version. So our patch lets mpl find the old matplotlibrc file, which does not contain new entries (like pcolormesh.snap) and that's where the process fails.
Im open to suggestions, the first thought i have is to change this to do something like:
if str(Path(__file__).with_name("mpl-data")) exists: return it else: return '/usr/share/matplotlib/mpl-data'the first branch will only work during build (which is unfortunate, because it means it's a test that fail for all installations, but oh well) and the second branch will be trigger when you install the package from the apt repo
what you think?
tacaswell commented on Oct 19, 2021
I assume the circular dependency is from the docs?
I wonder if the easiest path is for the debian packaging to move the matplotlibrc file out of the mpl-data folder and then patch
and leave the rest of the mpl-data related patches as-is?
sandrotosi commented on Oct 19, 2021
_get_data_path() is returning (in this case) exactly what get_data_path() would, via
matplotlib/lib/matplotlib/cbook/__init__.py
Line 512 in 9568e14
i think i need a way to differentiate between build-time and run-time, previously i'd just stick a matplotlibrc file crafted specifically for build in the right place and use MPLCONFIGDIR to load it up, but with the removal if the default rc params in 178515c and #19603 that no longer works
QuLogic commented on Oct 19, 2021
That is why the Fedora patch uses a relative path for get_data_path.
sandrotosi commented on Oct 19, 2021
yeah i tried to use that, but sadly it wont work, because parent.parent.parent... for a short path (like /tmp/matplotlib in this case, which the same path segments used in the building machines) doesnt work
tacaswell commented on Oct 19, 2021
but you are packaging mpl-data separately so it may no be there? By dropping the _get_data_path call on the _rcparams_in_file call for the default you can be sure it is to a file you packaging with core Matplotlib.
QuLogic commented on Oct 19, 2021
Why does that matter; it's the source path. Does Debian not install to a build root that mirrors the final location?
sandrotosi commented on Oct 21, 2021
the problem here is that the same code needs to support 2 different situations:
i think this will work:
diff --git a/lib/matplotlib/__init__.py b/lib/matplotlib/__init__.py index ba9cd6c9..d91654a5 100644 --- a/lib/matplotlib/__init__.py +++ b/lib/matplotlib/__init__.py @@ -524,7 +524,11 @@ def get_cachedir(): @_logged_cached('matplotlib data path: %s') def get_data_path(): """Return the path to Matplotlib data.""" - return str(Path(__file__).with_name("mpl-data")) + _data_path = Path(__file__).with_name("mpl-data") + if _data_path.exists(): + return str(_data_path) + else: + return '/usr/share/matplotlib/mpl-data'at least the build passes the previous error (i cannot test the second point until i manage to generate the final packages).
Now the problem seems to be with latex? (if you prefer me to open a new bug or retitle this one in a generic "support debian packaging of 3.5.0rc1" lemme know).
Using Sphinx-Gallery to convert rst text blocks to markdown for .ipynb files. generating gallery for gallery/lines_bars_and_markers... [100%] xcorr_acorr_demo.py generating gallery for gallery/images_contours_and_fields... [100%] watermark_image.py generating gallery for gallery/subplots_axes_and_figures... [100%] zoom_inset_axes.py generating gallery for gallery/statistics... [100%] violinplot.py generating gallery for gallery/pie_and_polar_charts... [100%] polar_scatter.py WARNING: /build/matplotlib-3.5.0~rc1/examples/text_labels_and_annotations/tex_demo.py failed to execute correctly: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sphinx_gallery/scrapers.py", line 377, in save_figures rst = scraper(block, block_vars, gallery_conf) File "/usr/lib/python3/dist-packages/sphinx_gallery/scrapers.py", line 170, in matplotlib_scraper fig.savefig(image_path, **these_kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/figure.py", line 3012, in savefig self.canvas.print_figure(fname, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backend_bases.py", line 2309, in print_figure result = print_method( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backend_bases.py", line 1643, in wrapper return func(*args, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/_api/deprecation.py", line 412, in wrapper return func(*inner_args, **inner_kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_agg.py", line 538, in print_png FigureCanvasAgg.draw(self) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_agg.py", line 434, in draw self.figure.draw(self.renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 73, in draw_wrapper result = draw(artist, renderer, *args, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 50, in draw_wrapper return draw(artist, renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/figure.py", line 2797, in draw self.tight_layout(**self._tight_parameters) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/figure.py", line 3190, in tight_layout kwargs = get_tight_layout_figure( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/tight_layout.py", line 319, in get_tight_layout_figure kwargs = _auto_adjust_subplotpars(fig, renderer, File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/tight_layout.py", line 82, in _auto_adjust_subplotpars bb += [ax.get_tightbbox(renderer, for_layout_only=True)] File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/axes/_base.py", line 4611, in get_tightbbox bb_xaxis = self.xaxis.get_tightbbox( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/axis.py", line 1105, in get_tightbbox self._update_label_position(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/axis.py", line 2083, in _update_label_position bboxes, bboxes2 = self._get_tick_boxes_siblings(renderer=renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/axis.py", line 1881, in _get_tick_boxes_siblings tlb, tlb2 = axis._get_tick_bboxes(ticks_to_draw, renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/axis.py", line 1085, in _get_tick_bboxes return ([tick.label1.get_window_extent(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/axis.py", line 1085, in <listcomp> return ([tick.label1.get_window_extent(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/text.py", line 910, in get_window_extent bbox, info, descent = self._get_layout(self._renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/text.py", line 309, in _get_layout _, lp_h, lp_d = renderer.get_text_width_height_descent( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_agg.py", line 257, in get_text_width_height_descent w, h, d = texmanager.get_text_width_height_descent( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 332, in get_text_width_height_descent dvifile = self.make_dvi(tex, fontsize) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 268, in make_dvi self._run_checked_subprocess( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 241, in _run_checked_subprocess raise RuntimeError( RuntimeError: latex was not able to process the following string: b'lp' Here is the full report generated by latex: This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2022/dev/Debian) (preloaded format=latex) restricted \write18 enabled. entering extended mode ! I can't find file `/build/matplotlib-3.5.0'. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/1acea6f6c115d0ec7a634ed0529287... (Press Enter to retry, or Control-D to exit) Please type another input file name ! Emergency stop. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/1acea6f6c115d0ec7a634ed0529287... No pages of output. Transcript written on texput.log. WARNING: /build/matplotlib-3.5.0~rc1/examples/text_labels_and_annotations/usetex_baseline_test.py failed to execute correctly: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sphinx_gallery/scrapers.py", line 377, in save_figures rst = scraper(block, block_vars, gallery_conf) File "/usr/lib/python3/dist-packages/sphinx_gallery/scrapers.py", line 170, in matplotlib_scraper fig.savefig(image_path, **these_kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/figure.py", line 3012, in savefig self.canvas.print_figure(fname, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backend_bases.py", line 2309, in print_figure result = print_method( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backend_bases.py", line 1643, in wrapper return func(*args, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/_api/deprecation.py", line 412, in wrapper return func(*inner_args, **inner_kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_agg.py", line 538, in print_png FigureCanvasAgg.draw(self) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_agg.py", line 434, in draw self.figure.draw(self.renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 73, in draw_wrapper result = draw(artist, renderer, *args, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 50, in draw_wrapper return draw(artist, renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/figure.py", line 2803, in draw mimage._draw_list_compositing_images( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/image.py", line 132, in _draw_list_compositing_images a.draw(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 50, in draw_wrapper return draw(artist, renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/axes/_base.py", line 3082, in draw mimage._draw_list_compositing_images( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/image.py", line 132, in _draw_list_compositing_images a.draw(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 50, in draw_wrapper return draw(artist, renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/text.py", line 729, in draw textrenderer.draw_tex(gc, x, y, clean_line, File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_agg.py", line 283, in draw_tex Z = texmanager.get_grey(s, size, self.dpi) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 307, in get_grey pngfile = self.make_png(tex, fontsize, dpi) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 284, in make_png dvifile = self.make_dvi(tex, fontsize) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 268, in make_dvi self._run_checked_subprocess( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 241, in _run_checked_subprocess raise RuntimeError( RuntimeError: latex was not able to process the following string: b'lg' Here is the full report generated by latex: This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2022/dev/Debian) (preloaded format=latex) restricted \write18 enabled. entering extended mode ! I can't find file `/build/matplotlib-3.5.0'. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/c12cf4c6a38f7f36051cd867a63734... (Press Enter to retry, or Control-D to exit) Please type another input file name ! Emergency stop. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/c12cf4c6a38f7f36051cd867a63734... No pages of output. Transcript written on texput.log. WARNING: /build/matplotlib-3.5.0~rc1/examples/text_labels_and_annotations/usetex_fonteffects.py failed to execute correctly: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sphinx_gallery/scrapers.py", line 377, in save_figures rst = scraper(block, block_vars, gallery_conf) File "/usr/lib/python3/dist-packages/sphinx_gallery/scrapers.py", line 170, in matplotlib_scraper fig.savefig(image_path, **these_kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/figure.py", line 3012, in savefig self.canvas.print_figure(fname, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backend_bases.py", line 2309, in print_figure result = print_method( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backend_bases.py", line 1643, in wrapper return func(*args, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/_api/deprecation.py", line 412, in wrapper return func(*inner_args, **inner_kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_agg.py", line 538, in print_png FigureCanvasAgg.draw(self) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_agg.py", line 434, in draw self.figure.draw(self.renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 73, in draw_wrapper result = draw(artist, renderer, *args, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 50, in draw_wrapper return draw(artist, renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/figure.py", line 2803, in draw mimage._draw_list_compositing_images( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/image.py", line 132, in _draw_list_compositing_images a.draw(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 50, in draw_wrapper return draw(artist, renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/text.py", line 685, in draw bbox, info, descent = self._get_layout(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/text.py", line 309, in _get_layout _, lp_h, lp_d = renderer.get_text_width_height_descent( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_agg.py", line 257, in get_text_width_height_descent w, h, d = texmanager.get_text_width_height_descent( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 332, in get_text_width_height_descent dvifile = self.make_dvi(tex, fontsize) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 268, in make_dvi self._run_checked_subprocess( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 241, in _run_checked_subprocess raise RuntimeError( RuntimeError: latex was not able to process the following string: b'lp' Here is the full report generated by latex: This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2022/dev/Debian) (preloaded format=latex) restricted \write18 enabled. entering extended mode ! I can't find file `/build/matplotlib-3.5.0'. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/1acea6f6c115d0ec7a634ed0529287... (Press Enter to retry, or Control-D to exit) Please type another input file name ! Emergency stop. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/1acea6f6c115d0ec7a634ed0529287... No pages of output. Transcript written on texput.log. generating gallery for gallery/text_labels_and_annotations... [100%] watermark_text.py generating gallery for gallery/pyplots... [100%] text_layout.py generating gallery for gallery/color... [100%] named_colors.py generating gallery for gallery/shapes_and_collections... [100%] scatter.py generating gallery for gallery/style_sheets... [100%] style_sheets_reference.py generating gallery for gallery/axes_grid1... [100%] simple_colorbar.py generating gallery for gallery/axisartist... [100%] simple_axisline3.py generating gallery for gallery/showcase... [100%] xkcd.py Animation size has reached 21251016 bytes, exceeding the limit of 20971520.0. If you're sure you want a larger animation embedded, set the animation.embed_limit rc parameter to a larger value (in MB). This and further frames will be dropped. generating gallery for gallery/event_handling... [100%] zoom_window.py generating gallery for gallery/frontpage... [100%] membrane.py WARNING: /build/matplotlib-3.5.0~rc1/examples/misc/multipage_pdf.py failed to execute correctly: Traceback (most recent call last): File "/build/matplotlib-3.5.0~rc1/examples/misc/multipage_pdf.py", line 36, in <module> pdf.savefig() File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_pdf.py", line 2738, in savefig figure.savefig(self, format="pdf", **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/figure.py", line 3012, in savefig self.canvas.print_figure(fname, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backend_bases.py", line 2309, in print_figure result = print_method( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backend_bases.py", line 1643, in wrapper return func(*args, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/_api/deprecation.py", line 386, in wrapper return func(*inner_args, **inner_kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/backend_pdf.py", line 2785, in print_pdf self.figure.draw(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 73, in draw_wrapper result = draw(artist, renderer, *args, **kwargs) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 50, in draw_wrapper return draw(artist, renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/figure.py", line 2803, in draw mimage._draw_list_compositing_images( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/image.py", line 132, in _draw_list_compositing_images a.draw(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/artist.py", line 50, in draw_wrapper return draw(artist, renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/axes/_base.py", line 3046, in draw self._update_title_position(renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/axes/_base.py", line 2996, in _update_title_position if title.get_window_extent(renderer).ymin < top: File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/text.py", line 910, in get_window_extent bbox, info, descent = self._get_layout(self._renderer) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/text.py", line 309, in _get_layout _, lp_h, lp_d = renderer.get_text_width_height_descent( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/backends/_backend_pdf_ps.py", line 103, in get_text_width_height_descent w, h, d = texmanager.get_text_width_height_descent( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 332, in get_text_width_height_descent dvifile = self.make_dvi(tex, fontsize) File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 268, in make_dvi self._run_checked_subprocess( File "/build/matplotlib-3.5.0~rc1/build/lib.linux-x86_64-3.9/matplotlib/texmanager.py", line 241, in _run_checked_subprocess raise RuntimeError( RuntimeError: latex was not able to process the following string: b'lp' Here is the full report generated by latex: This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2022/dev/Debian) (preloaded format=latex) restricted \write18 enabled. entering extended mode ! I can't find file `/build/matplotlib-3.5.0'. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/9eb8ed50c4f119894612567ddda7eb... (Press Enter to retry, or Control-D to exit) Please type another input file name ! Emergency stop. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/9eb8ed50c4f119894612567ddda7eb... No pages of output. Transcript written on texput.log. Extension error (sphinx_gallery.gen_gallery): Handler <function generate_gallery_rst at 0x7f4667938d30> for event 'builder-inited' threw an exception (exception: latex was not able to process the following string: b'lp' Here is the full report generated by latex: This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2022/dev/Debian) (preloaded format=latex) restricted \write18 enabled. entering extended mode ! I can't find file `/build/matplotlib-3.5.0'. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/9eb8ed50c4f119894612567ddda7eb... (Press Enter to retry, or Control-D to exit) Please type another input file name ! Emergency stop. <to be read again> \protect <*> /build/matplotlib-3.5.0~ rc1/doc/tex.cache/9eb8ed50c4f119894612567ddda7eb... No pages of output. Transcript written on texput.log. ) make[1]: *** [Makefile:34: html] Error 2 make[1]: Leaving directory '/build/matplotlib-3.5.0~rc1/doc' make: *** [debian/rules:24: build-indep-stamp] Error 2any idea what could possibly go wrong now? i found old reports suggesting to install packages that are already available in the build environment or setting rcParams that are no longer valid.
thanks!
jklymak commented on Oct 21, 2021
As a meta comment, how optional is mpl_data? Currently it has some data and images in it, but these are small (<1 Mb) It also has the canonical matplotlibrc. Finally it has default fonts, which take up the bulk of the data (~8 Mb).
If we don't consider mpl_data at all optional to the package, perhaps we just ask that downstream packages to not distribute it as optional.
If we do consider it optional, I dont' think we can continue to have the canonical matplotlibrc stored there. It needs to go somewhere we can be relatively certain folks will always have.
I'm not sure how optional we consider the fallback fonts. They seem pretty non-optional, though I can imagine a good packager removing them if they know they will not be needed.
QuLogic commented on Oct 21, 2021
I can't find it right now, but that's a known bug of tildes in the path (/tmp/matplotlib-3.5.0~rc1).
tacaswell commented on Oct 21, 2021
this would be my preference.
anntzer commented on Oct 21, 2021
I don't think any of mpl-data should be considered as optional. If debian wants to dedupe the files, I believe the onus is on their side to replace the files e.g. by symlinks, so that the contents effectively stay the same.
sandrotosi commented on Oct 22, 2021
thanks! the fix in #21414 worked (now there are other underlying issues outside of mpl that prevents to complete the packaging which i'm going to work on)
jklymak commented on Oct 22, 2021
@sandrotosi We discussed this on the weekly call, just to make sure we all agreed on the messaging here. For quite a while mpl_data has been considered a core part of matplotlib - as noted above we keep the canonical rcParam defaults there, and we keep the fallback fonts there. We appreciate that this directory used to be largely ancillary data in the past, but that usage has drifted substantially for the last few years. Hence, core Matplotlib expects lib/matplotlib/mpl_data to exist and be populated with the files we put there. Of course package managers can patch as they see fit, and move things around, or add softlinks to make the packages smaller.
It sounds like you already have this largely sorted to your satisfaction, but we just wanted to clarify how we currently view and use this directory. @tacaswell, @QuLogic and @anntzer can clarify any of the above if I have it wrong or have missed a subtlety (I just use conda on a Mac 😉 ).
Document that the `mpl-data` directory is required
Document that the `mpl-data` directory is required
Document that the `mpl-data` directory is required
Document that the `mpl-data` directory is required
Document that the `mpl-data` directory is required