Repository navigation
[Bug]: Coredump when importing matplotlib.pyplot #25823
Description
Bug summary
When I execute python -c 'import matplotlib.pyplot', python will coredump. This will occur when it is generating the ~/.cache/matplotlib/fontlist-v330.json file, assuming that it was not already successfully generated.
The root cause of the problem is that the /usr/share/fonts/google-noto-emoji/NotoColorEmoji.ttf font will cause an invalid pixel size exception to be thrown from the FreeType library, the exception is not subsequently handled in the C++ layer, and python will abort.
Code for reproduction
[ -f /usr/share/fonts/google-noto-emoji/NotoColorEmoji.ttf ] \
&& /bin/rm -f ~/.cache/matplotlib/fontlist-v330.json \
&& python -c "import matplotlib.pyplot"Actual outcome
[1] 229645 abort (core dumped) python -c
Expected outcome
No output
Additional information
It appears that this is only caused by the NotoColorEmoji font. If I update font_manager.py to skip this font, there is no issue.
This was at least working in 3.1.3. I have tried 3.4.3 and 3.6.1 and they both have the same issue. I'm using system_freetype=True in the mplsetup.py file. I have also tried compiling FreeType and this does not seem to resolve anything.
The root cause of the problem is that the /usr/share/fonts/google-noto-emoji/NotoColorEmoji.ttf font will cause an invalid pixel size exception to be thrown from the FreeType library, the exception is not subsequently handled in the C++ layer, and python will abort.
I am using RedHat 7.6, 7.3 works fine because it does not have the problematic font.
Here is a link to a similar issue using Pillow, issue. I'm not familiar with the code, but my understanding is that the correct solution would be the handle the FreeType exceptions in the ft2font_wrapper.cpp code and not propogate it into the Python interpreter.
Here is part of the stack trace.
#0 0x00007f7862cef207 in raise () from /lib64/libc.so.6
#1 0x00007f7862cf08f8 in abort () from /lib64/libc.so.6
#2 0x00007f784cfb2420 in _Unwind_SetGR (context=, index=, val=)
at /tmp/gcc-v9.3.0p3/gcc.source/libgcc/unwind-dw2.c:1442
#3 0x00007f784cfbe797 in __gcc_personality_v0 (version=, actions=, exception_class=, ue_header=0x15d9bf0,
context=0x7fff105961e0) at /tmp/gcc-v9.3.0p3/gcc.source/libgcc/unwind-c.c:231
#4 0x00007f785a69a7b3 in _Unwind_RaiseException_Phase2 () at /tmp/gcc-v9.3.0p4lnx86/gcc.source/libgcc/unwind.inc:64
#5 0x00007f785a69acf1 in _Unwind_RaiseException (exc=0x15d9bf0) at /tmp/gcc-v9.3.0p4lnx86/gcc.source/libgcc/unwind.inc:136
#6 0x00007f785a74a358 in __cxa_throw () from .../lib/64bit/libstdc++.so.6
#7 0x00007f784cfb12a1 in throw_ft_error(std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int) [clone .cold] ()
from .../lib/python3.9/site-packages/matplotlib-3.6.1-py3.9-linux-x86_64.egg/matplotlib/ft2font.cpython-39-x86_64-linux-gnu.so
#8 0x00007f784cfb9868 in PyFT2Font_init(PyFT2Font*, _object*, _object*) () at /tmp/gcc-v9.3.0p3/gcc.source/libgcc/unwind-dw2.c:1442
Operating system
RedHat 7.6
Matplotlib Version
3.6.1
Matplotlib Backend
Call to getBackend also crashes (same reason as above)
Python version
3.9.16
Jupyter version
No response
Installation
None
Activity
ksunden commented on May 6, 2023
This is somewhat surprising. I can state that my main dev machine has NotoColorEmoji installed and I do not see this.
It is true that non-scalable fonts are not supported by Matplotlib, but there is handling python-side for rejecting such fonts (though that is after calling into the ft2font cpp library).
QuLogic commented on May 6, 2023
But with what version of FreeType?
QuLogic commented on May 6, 2023
The only thing in PyFT2Font_init that I can see that calls throw_ft_error is the FT2Font constructor, but that is wrapped in the CALL_CPP_FULL macro that handles all exceptions, so this seems a system-specific problem.
john-odonovan commented on May 6, 2023
The version of FreeType is 2.10.4
Here is the size of the NotoColorEmoji.ttf file for reference
$ ls -l /usr/share/fonts/google-noto-emoji/NotoColorEmoji.ttf
-rw-r--r--. 1 root root 7297112 May 3 2018 /usr/share/fonts/google-noto-emoji/NotoColorEmoji.ttf
I have checked this in 5+ different machines, and the only one which works is the 7.3 machine without the NotoColorEmoji font. All others are using RH7.6. I'll check on some other RH8 machines.
john-odonovan commented on May 6, 2023
Some more debugging, but I'm not clear on how this is an issue;
In FT2Font::FT2Font, the call to FT_Set_Char_Size fails with an error code of 23. As a result throw_ft_error will be called.
error = FT_Set_Char_Size(face, 12 * 64, 0, 72 * (unsigned int)hinting_factor, 72); if (error) { FT_Done_Face(face); throw_ft_error("Could not set the fontsize", error); }throw_ft_error will throw a runtime exception with the following message;
"Could not set the fontsize (invalid pixel size; error code 0x17)"
FT2Font::FT2Font was called from within the CALL_CPP_FULL macro as @QuLogic pointed out, which does have a try/catch block and explicitly catches std::runtime_error (+ has a catch all as well).
CALL_CPP_FULL( "FT2Font", (self->x = new FT2Font(open_args, hinting_factor, fallback_fonts)), Py_CLEAR(self->py_file), -1);The only strange thing I see is that when these files are compiled, gcc is being used as opposed to g++. It looks like gcc does recognize .cpp as a C++ extension and has no issue compiling the code, but I wonder if there is some impact on exception handling from doing this? I tried forcing g++ by setting CC but this led to other issues with C code that could not be compiled by g++.
john-odonovan commented on May 7, 2023
I manually compiled the ft2font related files with g++ and it made no difference, also used GCC 12.2.0 and still a crash.
john-odonovan commented on May 8, 2023
I went back to try and figure out the difference between 3.1.3 which was working and 3.4.3/3.6.1, ... It looks like the issue is related to some compiler options.
If I remove the -fvisibility=hidden -flto from the compile line for the objects in the src directory (ft2font.cpp. ft2font_wrapper.cpp), then everything works fine. The old 3.1.3 build did not have these options. I'm not sure what the interaction between these options and exceptions is though.
john-odonovan commented on May 8, 2023
The issue seems to be the addition of -flto to the compile line for src/ft2font.cpp. If I remove -flto from this compile line, leaving it on the others, then no crash.
john-odonovan commented on May 8, 2023
Hi,
I'm not really going to go much further to see why LTO and exceptions are not playing well together. In the *setup.cfg.template file, there is an entry to disable LTO. Once I do this, the crashes in matplotlib disappear.
if [ -f setup.cfg.template ]; then sed -e 's/#tests = .*/tests = False/' \ -e 's/#system_freetype = .*/system_freetype = True/' \ -e 's/#enable_lto = .*/enable_lto = False/' \ setup.cfg.template > setup.cfg fi if [ -f mplsetup.cfg.template ]; then sed -e 's/#tests = .*/tests = False/' \ -e 's/#system_freetype = .*/system_freetype = True/' \ -e 's/#enable_lto = .*/enable_lto = False/' \ mplsetup.cfg.template > mplsetup.cfg fiIf someone wants to investigate further, then just setting error = 23 in the FT2Font constructor after the call to FT_Set_Char_Size should be sufficient to replicate the problem, but remove the ~/.cache/matplotlib first to make sure that the code is called.
error = FT_Set_Char_Size(face, 12 * 64, 0, 72 * (unsigned int)hinting_factor, 72); // error = 23; if (error) { FT_Done_Face(face); throw_ft_error("Could not set the fontsize", error); }ksunden commented on May 8, 2023
What version of GCC do you have?
ksunden commented on May 8, 2023
Even if I put the error = 23; in, I do not get a core dump.
On import no error happens, though when trying to render text I get a ValueError since no fonts are found since all of them are rejected because of the error.
This is without changing compiler options/etc.
john-odonovan commented on May 8, 2023
I'm using GCC 9.3 and I've also tried 12.2. I could try recompiling everything including Python with 12.2 and see what happens.
ksunden commented on May 8, 2023
I'm on gcc 11.3.
When I use freetype 2.10.1 (the closest version we had SHA hash for already in setupext.py) I do not see any issue.
When I use freetype 2.10.4/2.11.1 I initially got undefined symbol: BrotliDecoderDecompress but after passing --with-brotli=no to the freetype configure, I got the same as 2.10.1. No errors.
Full Diff:john-odonovan commented on May 9, 2023
I recompiled everything with GCC 12.2.0, also checked the top-level Python build to make sure it was linked with g++ (--with-cxx-main on configure commands). I updated to use the system FreeType (2.8.0). I'm not really sure what steps I would take next. For now I will just go with enable_lto=False, to resolve the issue.
tacaswell commented on May 11, 2023
Can you reproduce this with 3.7.x or main?
jklymak commented on Apr 3, 2024
I'm going to close as an obscure compiler issue, but feel free to ask for a re-open.