Faster image comparison decorator by mdboom · Pull Request #98 · matplotlib/matplotlib · GitHub

This is a riff on the ideas in the comments on the pull requests by Paul @ivanov to reduce memory consumption when running the regression tests. See:

#96

and

#56

This is different from the other two in that it:

Calls the test code to generate the figure object only once, and then calls savefig three times on the same figure -- this results in an ~ 12% speedup.
Automatically closes all figures and restores rcParams and the unit registry after every test. The latter prevents some nasty side effect problems presented by some tests. This also revealed a memory leak in pyplot.close('all'). This saves massive amounts of memory vs. the current master, but it roughly equivalent to Paul's approach in pull request 96.
Runs Inkscape in "shell" mode so that a single inkscape process is fired up and re-used to convert each SVG to PNG. This results in an additional ~ 32% speedup.

1) Run the test itself that generates figure(s) only once, then savefig it to each output format. This appears to have an approx 2.75x speedup on run time. 2) Cleanup better after each test: Close all figures Reset rcParams to their defaults Reset the unit registry to defaults This requires the following changes to the tests themselves: 1) The tests no longer need to call 'savefig' -- just create figures and the decorator will find them. 2) Tests that don't use the image_comparison decorator should use the new cleanup decorator to make sure that cleanup happens.

…cape up multiple times.

…cape server and starting a new one to prevent memory leaks in Inkscape from becoming problematic.

…n test setup function to reset rcParams state.

This is moved over from here:

#97

@WeatherGod, @ivanov and @mdboom were participating over there.

hmm, i'm getting some errors in python2.5:

Failure: AttributeError ('function' object has no attribute 'im_class') ... ERROR matplotlib.tests.test_backend_svg.test_visibility.test ... ok matplotlib.tests.test_basic.test_simple ... ok matplotlib.tests.test_basic.test_simple_knownfail ... KNOWNFAIL: Test known to fail matplotlib.tests.test_basic.test_override_builtins ... ok matplotlib.tests.test_cbook.Test_delete_masked_points.test_bad_first_arg ... ok matplotlib.tests.test_cbook.Test_delete_masked_points.test_datetime ... ok matplotlib.tests.test_cbook.Test_delete_masked_points.test_rgba ... ok matplotlib.tests.test_cbook.Test_delete_masked_points.test_string_seq ... ok matplotlib.tests.test_cbook.test_is_string_like ... ok matplotlib.tests.test_cbook.test_restrict_dict ... ok matplotlib.tests.test_mlab.test_colinear_pca ... ok matplotlib.tests.test_mlab.test_recarray_csv_roundtrip ... FAIL matplotlib.tests.test_mlab.test_rec2csv_bad_shape ... ok matplotlib.tests.test_mlab.test_prctile ... ok matplotlib.tests.test_transforms.test_Affine2D_from_values ... ok Failure: AttributeError ('function' object has no attribute 'im_class') ... ERROR matplotlib.tests.test_axes.test_arc_ellipse.test ... ok matplotlib.tests.test_axes.test_arc_ellipse.test ... ok matplotlib.tests.test_axes.test_arc_ellipse.test ...

and then stalls and never finishes. Is this a nose issue? (I'm on nose 1.0.0)

Ah. I'm having issues with nose 1.0 as well. Investigating...

…11.3

This latest commit seems to fix the nose-1.0.0 issue on Python 2.7. @ivanov -- do you mind kicking the tires with Python 2.5? Python 3.x will have its own set of issues, I suspect, but I'll defer that to when I merge this into the py3 branch.

see http://bugs.python.org/issue5982 for the source

@mdboom - I'm getting a bunch of ERROR: Failure: AttributeError ('staticmethod' object has no attribute '__func__') messages, but once I got around that with ivanov@be6af2f - it still stalls on the third matplotlib.tests.test_axes.test_arc_ellipse.test ...

It's probably hitting some sort of deadlock with the Inkscape server. I've got 0.48.1. But it also could be a difference in the behavior of the "subprocess" module in Python 2.5 vs. 2.7, or some sort of platform difference. If you revert lib/matplotlib/testing/compare.py to what is in master, does the stalling go away? If so, maybe we should defer the Inkscape server part of this pull request until we better understand how to do it in a robust way.

mdboom, on 2011-04-28 05:37, wrote:

If you revert lib/matplotlib/testing/compare.py to what is in master, does the stalling go away?

That did the trick, Mike! I still have a couple of failures on
that box, but I also get them on master, so I'll have to look
into that later.

I still needed to use the get(1) business, instead of
func - see https://github.com/ivanov/matplotlib/commits/faster-image-comp

…mage_comparison_decorator Fix for Python 2.5 compatibility.

…er than running it as a server, which mysteriously causes a deadlock on some systems.

looks good now, Mike, thanks!

Faster image comparison decorator

mdboom added 5 commits April 26, 2011 16:29

Fix memory leak in pyplot.close('all')

354dee8

Run inkscape for SVG conversion as a server -- prevents starting inks…

a074823

…cape up multiple times.

Do N SVG->PNG conversions with inkscape before shutting down the inks…

9086ef1

…cape server and starting a new one to prevent memory leaks in Inkscape from becoming problematic.

Reduce the number of dictionary copies around each test. Call the mai…

84bd649

…n test setup function to reset rcParams state.

Fix new test decorator to work with nose-1.0.0 in addition to nose-0.…

ea16278

…11.3

obscure way of getting to the function, but works in 2.5

be6af2f

see http://bugs.python.org/issue5982 for the source

reverting testing/compare.py back to master

5e50b90

mdboom added 3 commits April 28, 2011 12:46

Fix indentation.

cc644a8

Merge remote-tracking branch 'ivanov/faster-image-comp' into faster_i…

43ffb29

…mage_comparison_decorator Fix for Python 2.5 compatibility.

Revert back to using inkscape once for each SVG->PNG conversion, rath…

bffb15d

…er than running it as a server, which mysteriously causes a deadlock on some systems.

mdboom added a commit that referenced this pull request May 2, 2011

Merge pull request #98 from mdboom/faster_image_comparison_decorator.

fdf5fae

Faster image comparison decorator

mdboom merged commit fdf5fae into matplotlib:master May 2, 2011

richbwood mentioned this pull request Dec 19, 2012

test_pcolormesh hangs #1609

Closed

anntzer mentioned this pull request Mar 8, 2017

Investigate alternative svg renderers for the test suite #8242

Closed

Faster image comparison decorator by mdboom · Pull Request #98 · matplotlib/matplotlib · GitHub

Repository navigation

Faster image comparison decorator - #98

Faster image comparison decorator#98
mdboom merged 11 commits into
matplotlib:masterfrom
mdboom:faster_image_comparison_decorator

mdboom commented Apr 27, 2011

Uh oh!

mdboom commented Apr 27, 2011

Uh oh!

ivanov commented Apr 28, 2011

Uh oh!

mdboom commented Apr 28, 2011

Uh oh!

mdboom commented Apr 28, 2011

Uh oh!

ivanov commented Apr 28, 2011

Uh oh!

mdboom commented Apr 28, 2011

Uh oh!

ivanov commented Apr 28, 2011

Uh oh!

ivanov commented Apr 28, 2011

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Faster image comparison decorator by mdboom · Pull Request #98 · matplotlib/matplotlib · GitHub

Repository navigation

Faster image comparison decorator - #98

Faster image comparison decorator#98mdboom merged 11 commits intomatplotlib:mastermatplotlib/matplotlib:masterfrom mdboom:faster_image_comparison_decoratormdboom/matplotlib:faster_image_comparison_decoratorCopy head branch name to clipboard

Conversation

mdboom commented Apr 27, 2011

Uh oh!

mdboom commented Apr 27, 2011

Uh oh!

ivanov commented Apr 28, 2011

Uh oh!

mdboom commented Apr 28, 2011

Uh oh!

mdboom commented Apr 28, 2011

Uh oh!

ivanov commented Apr 28, 2011

Uh oh!

mdboom commented Apr 28, 2011

Uh oh!

ivanov commented Apr 28, 2011

Uh oh!

ivanov commented Apr 28, 2011

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Faster image comparison decorator#98
mdboom merged 11 commits into
matplotlib:masterfrom
mdboom:faster_image_comparison_decorator