Repository navigation
Tox.ini does not work out of the box #11361
Description
Bug report
Running tox on both my local environment and apparently on Travis (see #11360) causes an issue because "Matplotlib test data is not installed".
As part of #11360, I also tried to create a custom pytz testenv that invokes pytest -m pytz and it failed during test discovery because it could not find mpl_toolkits. Ideally, running tox out of the box should work, and it should also be possible to use it to run specific subsets of the tests.
Code for reproduction
$ tox -e py36Actual outcome
Output is spammy, hidden under a <details> tag:
$ tox -e py36
GLOB sdist-make: LOCATION/matplotlib/setup.py
py36 inst-nodeps: LOCATION/matplotlib/.tox/dist/matplotlib-2.0.2+8083.g391c0cb30.zip
py36 installed: atomicwrites==1.1.5,attrs==18.1.0,cycler==0.10.0,kiwisolver==1.0.1,matplotlib==2.0.2+8083.g391c0cb30,more-itertools==4.2.0,numpy==1.14.3,pluggy==0.6.0,py==1.5.3,pyparsing==2.2.0,pytest==3.6.0,python-dateutil==2.7.3,six==1.11.0
py36 runtests: PYTHONHASHSEED='735566734'
py36 runtests: commands[0] | sh -c rm -f $HOME/.matplotlib/fontList*
WARNING:test command found but not installed in testenv
cmd: /usr/bin/sh
env: LOCATION/matplotlib/.tox/py36
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
py36 runtests: commands[1] | LOCATION/matplotlib/.tox/py36/bin/python LOCATION/matplotlib/tests.py --processes=-1 --process-timeout=300
Python byte-compilation optimization level: 0
LOCATION/matplotlib/.tox/py36/lib/python3.6/site-packages/matplotlib/__init__.py:1440: UserWarning: Matplotlib is not built with the correct FreeType version to run tests. Set local_freetype=True in setup.cfg and rebuild. Expect many image comparison failures below. Expected freetype version 2.6.1. Found freetype version 2.9.1. Freetype build type is not local
"" if ft2font.__freetype_build_type__ == 'local' else "not "
Traceback (most recent call last):
File "LOCATION/matplotlib/tests.py", line 56, in <module>
recursionlimit=args.recursionlimit)
File "LOCATION/matplotlib/.tox/py36/lib/python3.6/site-packages/matplotlib/__init__.py", line 1456, in test
raise ImportError("Matplotlib test data is not installed")
ImportError: Matplotlib test data is not installed
ERROR: InvocationError for command 'LOCATION/matplotlib/.tox/py36/bin/python LOCATION/matplotlib/tests.py --processes=-1 --process-timeout=300' (exited with code 1)Expected outcome
Should run the tests properly.
Matplotlib version
- Operating system: Linux
- Matplotlib version: master (git checkout 391c0cb3089ac9336b7ca80)
- Python version: 3.6.5
- Other libraries:
$ pip freeze pluggy==0.6.0 py==1.5.3 six==1.11.0 tox==3.0.0 virtualenv==16.0.0
Activity
jklymak commented on Jun 1, 2018
I guess I don't know what tox is, but it looks like you didn't setup the setup.cfg in your environment for tests? https://matplotlib.org/devel/contributing.html#building-matplotlib-for-image-comparison-tests
pganssle commented on Jun 1, 2018
@jklymak That may indeed be the case, though since tox is only used for tests and creates its own environment, it should probably be setting those values correctly itself.
anntzer commented on Jun 1, 2018
diff --git a/tox.ini b/tox.ini index 9407b70d6..e3bb93c7d 100644 --- a/tox.ini +++ b/tox.ini @@ -7,10 +7,13 @@ envlist = py35, py36 [testenv] -changedir = /tmp +setenv = + PIP_ISOLATED = 1 + PIP_CONFIG_FILE = /dev/null + MPLLOCALFREETYPE = 1 +usedevelop = True +changedir = {envtmpdir} commands = - sh -c 'rm -f $HOME/.matplotlib/fontList*' - {envpython} {toxinidir}/tests.py --processes=-1 --process-timeout=300 + pytest --pyargs matplotlib deps = - numpy pytestappears to make tox work. Perhaps also set MPLCONFIGDIR to e.g. {envtmpdir}/mplconfigdir or similar, and bikeshed the pytest default args; and also define the [test] extras_require (sphinx, pillow, etc.)