Repository navigation
Appveyor build failing #9176
Description
The appveyor build is failing fairly consistently on the python 3.5 build (eg. see https://ci.appveyor.com/project/matplotlib/matplotlib/build/1.0.2957/job/dqhano9lykge7ffb#L1375). The actual failure is somewhere in test_artist:
lib\matplotlib\tests\test_artist.py ...ssCommand exited with code -1073740791
Activity
tacaswell commented on Oct 4, 2017
@kbrose did a bunch of work to speed up the path simplifications in #8776
dopplershift commented on Oct 4, 2017
It should be noted, though, that PR passed AppVeyor at the time.
dopplershift commented on Oct 5, 2017
There's #9282 that I just opened--at least as far as test_artist.py...-1073740791 is concerned. I went looking back through the AppVeyor logs and noticed that when my jsanimation PR went in is when these started happening. I swear when I saw the first one I looked and checked that the failure wasn't related to my changes (i.e. previous builds on master had the same problem). Apparently I was wrong (my bad).
At any rate while I think I have the failing test identified...I have no idea why it's causing the problem.
Kojoley commented on Oct 5, 2017
@cgohlke these exceptions are properly handled (other way python would crash), I do not why, but faulthandler on python 3.6 prints every C++ exception. I have even checked this under debugger + after rewriting convert_rect(py_converters.cpp) to not use exceptions these messages do not show.
tacaswell commented on Oct 5, 2017
Adding '-v' seems to also change the behavior #9229
kbrose commented on Oct 6, 2017
So to clarify, does it look like this is related to the changes I made, or unrelated? If people suspect my modifications may have caused some of the problems I can try to find some time to dig into it.
Kojoley commented on Oct 6, 2017
Crash 1
I have some success in reproducing the crash in test_artist.py. It always happens in lib/matplotlib/tests/test_artist.py::test_cull_markers. The smallest set of tests required (for me) to trigger the crash is following:
And it happens only with specific PYTHONHASHSEED. With 1 2 4 7 8 11 13 15 ... it crashes, while with 3 5 6 9 10 12 14 16 ... does not. Reordering the tests seems to affect something so it does not crash too.
matplotlib/src/ft2font_wrapper.cpp
Line 349 in 062beb6
Crash 2
This crash happened to me several times upon stopping pytest with Ctrl+C
matplotlib/src/file_compat.h
Line 136 in 062beb6
matplotlib/src/ft2font_wrapper.cpp
Line 364 in 062beb6
cgohlke commented on Oct 6, 2017
@Kojoley are your builds using Appveyor or freetype built with setup_external_compile.py?
In that case, crashes are not really unexpected. See comments at #6969 (diff)
Kojoley commented on Oct 6, 2017
@cgohlke I cannot catch what are you saying. Freetype builds successfully by setup_external_compile all the time for me with every python version I had tried (2.7, 3.4, 3.5, 3.6).
cgohlke commented on Oct 6, 2017
When using setup_external_compile on Python 3.5 and 3.6, freetype is built with the Visual Studio 2010 compiler, which uses a different C runtime than Python and the rest of the matplotlib C extensions, which are built with Visual Studio 2015. This can cause strange crashes.
Kojoley commented on Oct 6, 2017
You are right about different runtimes and it might be a root cause if ft2font would not be a mpl python extension that builds with setuptools. It should not be hard to build freetype with VS2017 (using cmake or convert/upgrade project) so I will check this.
cgohlke commented on Oct 6, 2017
Yes, the ft2font Python extension is built with VS2015, but the freetype library is built with VS2010 and linked into the ft2font extension.
13 remaining items
Kojoley commented on Oct 11, 2017
Maybe I was fooled by GitHub
but my latest conda's Python 3.6.2 does not have the fix.
cgohlke commented on Oct 11, 2017
Python 3.6.3 contains this bug fix. Unfortunately it also contains a faulthandler version that prints all the caught C++ exception messages. Probably it should be disabled on Appveyor for that version?
Kojoley commented on Oct 11, 2017
Indeed Python 3.6.3 does have it.
Kojoley commented on Oct 11, 2017
As we see, on windows faulthandler is completely useless, so I would just remove it from appveyor, for a while at least.
anntzer commented on Oct 11, 2017
Can we replace 3.5 by 3.4 on Appveyor?
cgohlke commented on Oct 11, 2017
3.4 contains that bug too.
Kojoley commented on Oct 11, 2017
I am for removing 3.5 from appveyor, but patch -u -p 1 -i ...\1224.patch -d path\to\python could be used if we leave it.
What about faulthandler and C++ exceptions there is a thread https://bugs.python.org/issue31701 and a fix python/cpython#3929
anntzer commented on Oct 11, 2017
If I understand correctly we did not see the bug before because it's due to a change in the CRT? or something else?
Kojoley commented on Oct 11, 2017
@anntzer I had seen problems with descriptors a year ago and that's why pytest-xdist is not activated on appveyor. Nothing changed to CRT, it just sometimes crashes on a bad value, while other time not (we do not know what happens here, because it is distributed without sources). When I installed debug python (python_d) and run simple pip install . on mpl sources it crashed python (thanks to versioneer). After some time I figured that subprocess.Popen with PIPE's would crash the python_d.
tacaswell commented on Oct 12, 2017
👍 to dropping 3.5 from the test matrix and documenting this issue in the windows install docs.
dstansby commented on Oct 14, 2017
Thanks a lot to everyone who chased this one!
tacaswell commented on Oct 15, 2017
Thanks @Kojoley and @cgohlke !