I could not find a similar issue so forgive me if its a duplicate. We are trying to get a couple of our tests suites to execute on mpl 1.x and 2.0 without having to duplicate the baseline images. The classic style helps a lot but there are still many small issues mainly related to text positioning. Is there a way I can force it to behave the same on mpl 1.x and 2.0?
The difference is subtle but enough to cause issues when testing - the text baseline (or center line I guess) is different for both. This was run for mpl 1.5.3 and mpl 2.0.0.
I understand how the label placements are different,
What I'm wondering is if that is really a problem. I would argue that while the mpl 2.0 version might not pass the letter of your image comparison test, it does pass the spirit of it. And while this is just speculation, the difference might come from different versions of freetype.
I'm offering that removing the text and comparing the arrays of ticks objects is a way to work around that and still have a meaningful test for the time being.
If its really a problem is up for you to decide. From my perspective any change that cannot be worked around to core upstream libraries (which matplotlib to some extent definitely is) causes a ton of problems downstream. But from what I gather that seems to be mpl's stance as well.
I apparently have a long comment on another computer that I never hit send on....
This is a change due to #6200 in which we re-generated the test images so this slipped through.
We should add a rcParam to reset the baseline on y-ticks, but we can not add it to the classic style as it would require use to either re-re-generate all of the test images.
There has to be some judgment to what needs to be work-around-able as every bug could really be a feature (obligator xkcd https://xkcd.com/1172/) 😈 .
We should add a rcParam to reset the baseline on y-ticks, but we can not add it to the classic style as it would require use to either re-re-generate all of the test images.
Sounds good. This likely only affects people that do pixel level tests for matplotlib plots and, as long as its documented somewhere on how to do it, it would solve that particular issue.
With pixel-perfect comparisons I just mean some non-image aware RMS misfit over all pixel value similar to what matplotlib and also pytest-mpl is doing. The actual comparison fct is here if someone cares: https://github.com/obspy/obspy
We are indeed thinking about moving to pytest but adding additional packages brings its own share of problems.
Any way I can force the baseline of the y-ticks right now?
Activity
phobson commented on Jan 23, 2017
When you test, would it be possible to use the remove_text=True option, and write a separate assertion that the text of the labels are similar?
krischer commented on Jan 23, 2017
This would pass as the text positioning would then no longer show. I guess I don't really understand what you are hinting at.
Here are both pictures - open in different tabs and rapidly switch back and forth to easily see the issue:
mpl 1.5

mpl 2.0

phobson commented on Jan 23, 2017
I understand how the label placements are different,
What I'm wondering is if that is really a problem. I would argue that while the mpl 2.0 version might not pass the letter of your image comparison test, it does pass the spirit of it. And while this is just speculation, the difference might come from different versions of freetype.
I'm offering that removing the text and comparing the arrays of ticks objects is a way to work around that and still have a meaningful test for the time being.
krischer commented on Jan 23, 2017
Ah I see - I was misreading your comment.
Good idea. I'll give it a shot! Thanks.
If its really a problem is up for you to decide. From my perspective any change that cannot be worked around to core upstream libraries (which matplotlib to some extent definitely is) causes a ton of problems downstream. But from what I gather that seems to be mpl's stance as well.
tacaswell commented on Jan 24, 2017
I apparently have a long comment on another computer that I never hit send on....
This is a change due to #6200 in which we re-generated the test images so this slipped through.
We should add a rcParam to reset the baseline on y-ticks, but we can not add it to the classic style as it would require use to either re-re-generate all of the test images.
There has to be some judgment to what needs to be work-around-able as every bug could really be a feature (obligator xkcd https://xkcd.com/1172/) 😈 .
krischer commented on Jan 24, 2017
Sounds good. This likely only affects people that do pixel level tests for matplotlib plots and, as long as its documented somewhere on how to do it, it would solve that particular issue.
tacaswell commented on Jan 24, 2017
@krischer What project are you working on that is also doing pixel-perfect comparisons?
Have you looked at using pytest-mpl?
krischer commented on Jan 24, 2017
A couple of projects but the biggest one with the most images tests is this here: https://github.com/obspy/obspy
With pixel-perfect comparisons I just mean some non-image aware RMS misfit over all pixel value similar to what matplotlib and also pytest-mpl is doing. The actual comparison fct is here if someone cares: https://github.com/obspy/obspy
We are indeed thinking about moving to pytest but adding additional packages brings its own share of problems.
Any way I can force the baseline of the y-ticks right now?
tacaswell commented on Jan 24, 2017
not without patching mpl.
dstansby commented on Feb 24, 2017
This is fixed on 2.0.x
update (#7)