<liclass="toctree-l2"><aclass="reference internal" href="MEP/MEP12.html">MEP12: Improve Gallery and Examples</a></li>
<liclass="toctree-l2"><aclass="reference internal" href="MEP/MEP13.html">MEP13: Use properties for Artists</a></li>
<liclass="toctree-l2"><aclass="reference internal" href="MEP/MEP14.html">MEP14: Text handling</a></li>
<liclass="toctree-l2"><aclass="reference internal" href="MEP/MEP15.html">MEP15: Fix axis autoscaling when limits are specified for one axis only</a></li>
<spanid="id1"></span><h1>Testing<aclass="headerlink" href="#testing" title="Link to this heading">#</a></h1>
<p>Matplotlib uses the <aclass="reference external" href="http://doc.pytest.org/en/latest/">pytest</a> framework.</p>
<p>The tests are in <codeclass="file docutils literal notranslate"><spanclass="pre">lib/matplotlib/tests</span></code>, and customizations to the pytest
testing infrastructure are in <aclass="reference internal" href="../api/testing_api.html#module-matplotlib.testing" title="matplotlib.testing"><codeclass="xref py py-mod docutils literal notranslate"><spanclass="pre">matplotlib.testing</span></code></a>.</p>
<sectionid="requirements">
<spanid="testing-requirements"></span><h2>Requirements<aclass="headerlink" href="#requirements" title="Link to this heading">#</a></h2>
<p>To run the tests you will need to
<aclass="reference internal" href="development_setup.html#installing-for-devs"><spanclass="std std-ref">set up Matplotlib for development</span></a>. Note in
particular the <aclass="reference internal" href="../install/dependencies.html#test-dependencies"><spanclass="std std-ref">additional dependencies</span></a> for testing.</p>
<divclass="admonition note">
<pclass="admonition-title">Note</p>
<p>We will assume that you want to run the tests in a development setup.</p>
<p>While you can run the tests against a regular installed version of
Matplotlib, this is a far less common use case. You still need the
<aclass="reference internal" href="../install/dependencies.html#test-dependencies"><spanclass="std std-ref">additional dependencies</span></a> for testing.
You have to additionally get the reference images from the repository,
because they are not distributed with pre-built Matplotlib packages.</p>
</div>
</section>
<sectionid="running-the-tests">
<spanid="run-tests"></span><h2>Running the tests<aclass="headerlink" href="#running-the-tests" title="Link to this heading">#</a></h2>
<p>In the root directory of your development repository run:</p>
<p><codeclass="docutils literal notranslate"><spanclass="pre">pytest</span></code> can be configured via many <aclass="reference external" href="https://pytest.org/en/stable/how-to/usage.html" title="(in pytest v9.0.3)"><spanclass="xref std std-doc">command-line parameters</span></a>. Some particularly useful ones are:</p>
<p>If you want to use <codeclass="docutils literal notranslate"><spanclass="pre">pytest</span></code> as a module (via <codeclass="docutils literal notranslate"><spanclass="pre">python</span><spanclass="pre">-m</span><spanclass="pre">pytest</span></code>), then you will need
to avoid clashes between <codeclass="docutils literal notranslate"><spanclass="pre">pytest</span></code>'s import mode and Python's search path:</p>
<ul>
<li><p>On more recent Python, you may <aclass="reference external" href="https://docs.python.org/3/using/cmdline.html#cmdoption-P" title="(in Python v3.14)"><codeclass="xref std std-option docutils literal notranslate"><spanclass="pre">disable</span><spanclass="pre">"unsafe</span><spanclass="pre">import</span>
<spanclass="pre">paths"</span></code></a> (i.e., stop adding the current directory to the import path) with the
<li><p>On any Python, set <codeclass="docutils literal notranslate"><spanclass="pre">pytest</span></code>'s <aclass="reference external" href="https://pytest.org/en/stable/explanation/pythonpath.html" title="(in pytest v9.0.3)"><spanclass="xref std std-doc">import mode</span></a> to the older <codeclass="docutils literal notranslate"><spanclass="pre">prepend</span></code> mode (but note that this will break
<h3>Viewing image test output<aclass="headerlink" href="#viewing-image-test-output" title="Link to this heading">#</a></h3>
<p>The output of <aclass="reference internal" href="#image-comparison"><spanclass="std std-ref">image-based</span></a> tests is stored in a
<codeclass="docutils literal notranslate"><spanclass="pre">result_images</span></code> directory. These images can be compiled into one HTML page, containing
hundreds of images, using the <codeclass="docutils literal notranslate"><spanclass="pre">visualize_tests</span></code> tool:</p>
<p>Pytest determines which functions are tests by searching for files whose names
begin with <codeclass="docutils literal notranslate"><spanclass="pre">"test_"</span></code> and then within those files for functions beginning with
<codeclass="docutils literal notranslate"><spanclass="pre">"test"</span></code> or classes beginning with <codeclass="docutils literal notranslate"><spanclass="pre">"Test"</span></code>.</p>
<p>Some tests have internal side effects that need to be cleaned up after their
execution (such as created figures or modified <aclass="reference internal" href="../api/matplotlib_configuration_api.html#matplotlib.rcParams" title="matplotlib.rcParams"><codeclass="xref py py-obj docutils literal notranslate"><spanclass="pre">rcParams</span></code></a>). The pytest fixture
<codeclass="docutils literal notranslate"><spanclass="pre">matplotlib.testing.conftest.mpl_test_settings</span></code> will automatically clean
these up; there is no need to do anything further.</p>
</section>
<sectionid="random-data-in-tests">
<h2>Random data in tests<aclass="headerlink" href="#random-data-in-tests" title="Link to this heading">#</a></h2>
<p>Random data is a very convenient way to generate data for examples,
however the randomness is problematic for testing (as the tests
must be deterministic!). To work around this set the seed in each test.
For numpy's default random number generator use:</p>
<p>and then use <codeclass="docutils literal notranslate"><spanclass="pre">rng</span></code> when generating the random numbers.</p>
<p>The seed is <aclass="reference internal" href="../project/history.html#project-history"><spanclass="std std-ref">John Hunter's</span></a> birthday.</p>
</section>
<sectionid="writing-an-image-comparison-test">
<spanid="image-comparison"></span><h2>Writing an image comparison test<aclass="headerlink" href="#writing-an-image-comparison-test" title="Link to this heading">#</a></h2>
<p>Writing an image-based test is only slightly more difficult than a simple
test. The main consideration is that you must specify the "baseline", or
expected, images in the <aclass="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.image_comparison" title="matplotlib.testing.decorators.image_comparison"><codeclass="xref py py-obj docutils literal notranslate"><spanclass="pre">image_comparison</span></code></a>
decorator. For example, this test generates a single image and automatically
<p>The first time this test is run, there will be no baseline image to compare
against, so the test will fail. Copy the output images (in this case
<codeclass="file docutils literal notranslate"><spanclass="pre">result_images/test_lines/test_line_dashes.png</span></code>) to the correct
subdirectory of <codeclass="file docutils literal notranslate"><spanclass="pre">baseline_images</span></code> tree in the source directory (in this
case <codeclass="file docutils literal notranslate"><spanclass="pre">lib/matplotlib/tests/baseline_images/test_lines</span></code>). Put this new
file under source code revision control (with <codeclass="docutils literal notranslate"><spanclass="pre">git</span><spanclass="pre">add</span></code>). When rerunning
the tests, they should now pass.</p>
<p>If you wish to compare multiple file formats, then omit the extension from the
baseline image name and optionally pass the <em>extensions</em> argument:</p>
<p>It is preferred that new tests use <codeclass="docutils literal notranslate"><spanclass="pre">style='mpl20'</span></code> as this leads to smaller
figures and reflects the newer look of default Matplotlib plots. Also, if the
texts (labels, tick labels, etc) are not really part of what is tested, use the
<codeclass="docutils literal notranslate"><spanclass="pre">remove_text=True</span></code> argument or add the <codeclass="docutils literal notranslate"><spanclass="pre">text_placeholders</span></code> fixture as this
will lead to smaller figures and reduce possible issues with font mismatch on
<h3>Compare two methods of creating an image<aclass="headerlink" href="#compare-two-methods-of-creating-an-image" title="Link to this heading">#</a></h3>
<p>Baseline images take a lot of space in the Matplotlib repository.
An alternative approach for image comparison tests is to use the
<aclass="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.check_figures_equal" title="matplotlib.testing.decorators.check_figures_equal"><codeclass="xref py py-obj docutils literal notranslate"><spanclass="pre">check_figures_equal</span></code></a> decorator, which should be
used to decorate a function taking two <aclass="reference internal" href="../api/_as_gen/matplotlib.figure.Figure.html#matplotlib.figure.Figure" title="matplotlib.figure.Figure"><codeclass="xref py py-obj docutils literal notranslate"><spanclass="pre">Figure</span></code></a> parameters and draws the same
images on the figures using two different methods (the tested method and the
baseline method). The decorator will arrange for setting up the figures and
then collect the drawn results and compare them.</p>
<p>For example, this test compares two different methods to draw the same
circle: plotting a circle using a <aclass="reference internal" href="../api/_as_gen/matplotlib.patches.Circle.html#matplotlib.patches.Circle" title="matplotlib.patches.Circle"><codeclass="xref py py-obj docutils literal notranslate"><spanclass="pre">matplotlib.patches.Circle</span></code></a> patch
vs plotting the circle using the parametric equation of a circle</p>
<p>Both comparison decorators have a tolerance argument <codeclass="docutils literal notranslate"><spanclass="pre">tol</span></code> that is used to specify the
tolerance for difference in color value between the two images, where 255 is the maximal
difference. The test fails if the average pixel difference is greater than this value.</p>
<p>See the documentation of <aclass="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.image_comparison" title="matplotlib.testing.decorators.image_comparison"><codeclass="xref py py-obj docutils literal notranslate"><spanclass="pre">image_comparison</span></code></a> and
<aclass="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.check_figures_equal" title="matplotlib.testing.decorators.check_figures_equal"><codeclass="xref py py-obj docutils literal notranslate"><spanclass="pre">check_figures_equal</span></code></a> for additional information
<h2>Creating a new module in matplotlib.tests<aclass="headerlink" href="#creating-a-new-module-in-matplotlib-tests" title="Link to this heading">#</a></h2>
<p>We try to keep the tests categorized by the primary module they are
testing. For example, the tests related to the <codeclass="docutils literal notranslate"><spanclass="pre">mathtext.py</span></code> module
are in <codeclass="docutils literal notranslate"><spanclass="pre">test_mathtext.py</span></code>.</p>
</section>
<sectionid="using-github-actions-for-ci">
<h2>Using GitHub Actions for CI<aclass="headerlink" href="#using-github-actions-for-ci" title="Link to this heading">#</a></h2>
<p><aclass="reference external" href="https://docs.github.com/en/actions">GitHub Actions</a> is a hosted CI system
"in the cloud".</p>
<p>GitHub Actions is configured to receive notifications of new commits to GitHub
repos and to run builds or tests when it sees these new commits. It looks for a
YAML files in <codeclass="docutils literal notranslate"><spanclass="pre">.github/workflows</span></code> to see how to test the project.</p>
<p>GitHub Actions is already enabled for the <aclass="reference external" href="https://github.com/matplotlib/matplotlib/">main Matplotlib GitHub repository</a> -- for example, see <aclass="reference external" href="https://github.com/matplotlib/matplotlib/actions?query=workflow%3ATests">the Tests
workflows</a>.</p>
<p>GitHub Actions should be automatically enabled for your personal Matplotlib
fork once the YAML workflow files are in it. It generally isn't necessary to
look at these workflows, since any pull request submitted against the main
Matplotlib repository will be tested. The Tests workflow is skipped in forked
repositories but you can trigger a run manually from the <aclass="reference external" href="https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow">GitHub web interface</a>.</p>
<p>Tox is configured using a file called <codeclass="docutils literal notranslate"><spanclass="pre">tox.ini</span></code>. You may need to
edit this file if you want to add new environments to test (e.g.,
<codeclass="docutils literal notranslate"><spanclass="pre">py33</span></code>) or if you want to tweak the dependencies or the way the
tests are run. For more info on the <codeclass="docutils literal notranslate"><spanclass="pre">tox.ini</span></code> file, see the <aclass="reference external" href="https://tox.readthedocs.io/en/latest/config.html">Tox
Configuration Specification</a>.</p>
</section>
<sectionid="building-old-versions-of-matplotlib">
<h2>Building old versions of Matplotlib<aclass="headerlink" href="#building-old-versions-of-matplotlib" title="Link to this heading">#</a></h2>
<p>When running a <codeclass="docutils literal notranslate"><spanclass="pre">git</span><spanclass="pre">bisect</span></code> to see which commit introduced a certain bug,
you may (rarely) need to build very old versions of Matplotlib. The following
<h2>Testing released versions of Matplotlib<aclass="headerlink" href="#testing-released-versions-of-matplotlib" title="Link to this heading">#</a></h2>
<p>Running the tests on an installation of a released version (e.g. PyPI package
or conda package) also requires additional setup.</p>
<divclass="admonition note">
<pclass="admonition-title">Note</p>
<p>For an end-user, there is usually no need to run the tests on released
versions of Matplotlib. Official releases are tested before publishing.</p>
</div>
<sectionid="install-additional-dependencies">
<h3>Install additional dependencies<aclass="headerlink" href="#install-additional-dependencies" title="Link to this heading">#</a></h3>
<p>Install the <aclass="reference internal" href="../install/dependencies.html#test-dependencies"><spanclass="std std-ref">additional dependencies for testing</span></a>.</p>
</section>
<sectionid="obtain-the-reference-images">
<h3>Obtain the reference images<aclass="headerlink" href="#obtain-the-reference-images" title="Link to this heading">#</a></h3>
<p>Many tests compare the plot result against reference images. The reference
images are not part of the regular packaged versions (pip wheels or conda
packages). If you want to run tests with reference images, you need to obtain
the reference images matching the version of Matplotlib you want to test.</p>
<p>To do so, either download the matching source distribution
<codeclass="docutils literal notranslate"><spanclass="pre">matplotlib-X.Y.Z.tar.gz</span></code> from <aclass="reference external" href="https://pypi.org/project/matplotlib/">PyPI</a>
or alternatively, clone the git repository and <codeclass="docutils literal notranslate"><spanclass="pre">git</span><spanclass="pre">checkout</span><spanclass="pre">vX.Y.Z</span></code>. Copy
the folder <codeclass="file docutils literal notranslate"><spanclass="pre">lib/matplotlib/tests/baseline_images</span></code> to the folder
<codeclass="file docutils literal notranslate"><spanclass="pre">matplotlib/tests</span></code> of your the matplotlib installation to test.
<liclass="toc-h3 nav-item toc-entry"><aclass="reference internal nav-link" href="#compare-two-methods-of-creating-an-image">Compare two methods of creating an image</a></li>
</ul>
</li>
<liclass="toc-h2 nav-item toc-entry"><aclass="reference internal nav-link" href="#creating-a-new-module-in-matplotlib-tests">Creating a new module in matplotlib.tests</a></li>
<liclass="toc-h2 nav-item toc-entry"><aclass="reference internal nav-link" href="#using-github-actions-for-ci">Using GitHub Actions for CI</a></li>
<liclass="toc-h2 nav-item toc-entry"><aclass="reference internal nav-link" href="#building-old-versions-of-matplotlib">Building old versions of Matplotlib</a></li>
<liclass="toc-h2 nav-item toc-entry"><aclass="reference internal nav-link" href="#testing-released-versions-of-matplotlib">Testing released versions of Matplotlib</a><ulclass="nav section-nav flex-column">