<divid="unreleased-message"> You are reading an old version of the documentation (v1.3.1). For the latest version see <ahref="https://matplotlib.org/stable/devel/testing.html">https://matplotlib.org/stable/devel/testing.html</a></div>
<spanid="id1"></span><h1>Testing<aclass="headerlink" href="#testing" title="Permalink to this headline">¶</a></h1>
<p>Matplotlib has a testing infrastructure based on <aclass="reference external" href="http://somethingaboutorange.com/mrl/projects/nose/">nose</a>, making it easy
to write new tests. The tests are in <ttclass="xref py py-mod docutils literal"><spanclass="pre">matplotlib.tests</span></tt>, and
customizations to the nose testing infrastructure are in
<ttclass="xref py py-mod docutils literal"><spanclass="pre">matplotlib.testing</span></tt>. (There is other old testing cruft around,
please ignore it while we consolidate our testing to these locations.)</p>
<divclass="section" id="requirements">
<h2>Requirements<aclass="headerlink" href="#requirements" title="Permalink to this headline">¶</a></h2>
<p>The following software is required to run the tests:</p>
<blockquote>
<div><ulclass="simple">
<li><aclass="reference external" href="http://somethingaboutorange.com/mrl/projects/nose/">nose</a>, version 1.0 or later</li>
<li><aclass="reference external" href="http://pages.cs.wisc.edu/~ghost/">Ghostscript</a> (to render PDF
<h2>Running the tests<aclass="headerlink" href="#running-the-tests" title="Permalink to this headline">¶</a></h2>
<p>Running the tests is simple. Make sure you have nose installed and run
the script <ttclass="file docutils literal"><spanclass="pre">tests.py</span></tt> in the root directory of the distribution.
The script can take any of the usual <aclass="reference external" href="http://somethingaboutorange.com/mrl/projects/nose/1.0.0/usage.html">nosetest arguments</a>, such as</p>
the <ttclass="xref py py-obj docutils literal"><spanclass="pre">*</span></tt> at the end: this will copy only the images we need to include
in the <ttclass="xref py py-obj docutils literal"><spanclass="pre">git</span></tt> repository. The files ending in <ttclass="xref py py-obj docutils literal"><spanclass="pre">_pdf.png</span></tt> and
<ttclass="xref py py-obj docutils literal"><spanclass="pre">_svg.png</span></tt> are converted from the <ttclass="xref py py-obj docutils literal"><spanclass="pre">pdf</span></tt> and <ttclass="xref py py-obj docutils literal"><spanclass="pre">svg</span></tt> originals on the fly
and do not need to be in the respository. Put these new files under
source code revision control (with <ttclass="xref py py-obj docutils literal"><spanclass="pre">git</span><spanclass="pre">add</span></tt>). When rerunning the
tests, they should now pass.</p>
<p>There are two optional keyword arguments to the <ttclass="xref py py-obj docutils literal"><spanclass="pre">image_comparison</span></tt>
decorator:</p>
<blockquote>
<div><ulclass="simple">
<li><ttclass="xref py py-obj docutils literal"><spanclass="pre">extensions</span></tt>: If you only wish to test some of the image formats
(rather than the default <ttclass="xref py py-obj docutils literal"><spanclass="pre">png</span></tt>, <ttclass="xref py py-obj docutils literal"><spanclass="pre">svg</span></tt> and <ttclass="xref py py-obj docutils literal"><spanclass="pre">pdf</span></tt> formats), pass a
list of the extensions to test.</li>
<li><ttclass="xref py py-obj docutils literal"><spanclass="pre">tol</span></tt>: This is the image matching tolerance, the default <ttclass="xref py py-obj docutils literal"><spanclass="pre">1e-3</span></tt>.
If some variation is expected in the image between runs, this
value may be adjusted.</li>
</ul>
</div></blockquote>
</div>
<divclass="section" id="known-failing-tests">
<h2>Known failing tests<aclass="headerlink" href="#known-failing-tests" title="Permalink to this headline">¶</a></h2>
<p>If you’re writing a test, you may mark it as a known failing test with
the <ttclass="xref py py-func docutils literal"><spanclass="pre">knownfailureif()</span></tt>
decorator. This allows the test to be added to the test suite and run
on the buildbots without causing undue alarm. For example, although
the following test will fail, it is an expected failure:</p>
<h2>Creating a new module in matplotlib.tests<aclass="headerlink" href="#creating-a-new-module-in-matplotlib-tests" title="Permalink to this headline">¶</a></h2>
<p>We try to keep the tests categorized by the primary module they are
testing. For example, the tests related to the <ttclass="docutils literal"><spanclass="pre">mathtext.py</span></tt> module
are in <ttclass="docutils literal"><spanclass="pre">test_mathtext.py</span></tt>.</p>
<p>Let’s say you’ve added a new module named <ttclass="docutils literal"><spanclass="pre">whizbang.py</span></tt> and you want
to add tests for it in <ttclass="docutils literal"><spanclass="pre">matplotlib.tests.test_whizbang</span></tt>. To add
this module to the list of default tests, append its name to
<ttclass="docutils literal"><spanclass="pre">default_test_modules</span></tt> in <ttclass="file docutils literal"><spanclass="pre">lib/matplotlib/__init__.py</span></tt>.</p>
</div>
<divclass="section" id="using-tox">
<h2>Using tox<aclass="headerlink" href="#using-tox" title="Permalink to this headline">¶</a></h2>
<p><aclass="reference external" href="http://tox.testrun.org/">Tox</a> is a tool for running tests against
multiple Python environments, including multiple versions of Python
(e.g., 2.6, 2.7, 3.2, etc.) and even different Python implementations
<p>Tox is configured using a file called <ttclass="docutils literal"><spanclass="pre">tox.ini</span></tt>. You may need to
edit this file if you want to add new environments to test (e.g.,
<ttclass="docutils literal"><spanclass="pre">py33</span></tt>) or if you want to tweak the dependencies or the way the
tests are run. For more info on the <ttclass="docutils literal"><spanclass="pre">tox.ini</span></tt> file, see the <aclass="reference external" href="http://tox.testrun.org/latest/config.html">Tox
Configuration Specification</a>.</p>
</div>
<divclass="section" id="using-travis-ci">
<h2>Using Travis CI<aclass="headerlink" href="#using-travis-ci" title="Permalink to this headline">¶</a></h2>
<p><aclass="reference external" href="http://travis-ci.org/">Travis CI</a> is a hosted CI system “in the
cloud”.</p>
<p>Travis is configured to receive notifications of new commits to GitHub
repos (via GitHub “service hooks”) and to run builds or tests when it
sees these new commits. It looks for a YAML file called
<ttclass="docutils literal"><spanclass="pre">.travis.yml</span></tt> in the root of the repository to see how to test the
project.</p>
<p>Travis CI is already enabled for the <aclass="reference external" href="https://github.com/matplotlib/matplotlib/">main matplotlib GitHub
repository</a> – for
example, see <aclass="reference external" href="http://travis-ci.org/#!/matplotlib/matplotlib">its Travis page</a>.</p>
<p>If you want to enable Travis CI for your personal matplotlib GitHub
repo, simply enable the repo to use Travis CI in either the Travis CI
UI or the GitHub UI (Admin | Service Hooks). For details, see <aclass="reference external" href="http://about.travis-ci.org/docs/user/getting-started/">the
Travis CI Getting Started page</a>. This
generally isn’t necessary, since any pull request submitted against
the main matplotlib repository will be tested.</p>
<p>Once this is configured, you can see the Travis CI results at