| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Since parametrizing the test allows it to run in parallel, this makes it flaky, as one process can overwrite the test result image of another. Our standard way for dealing with tests that use the same baseline image is to pass duplicate filenames to `image_comparison`, because that is serialized.
|
No, that's only for check_figures_equal, not image_comparison. And in that case, the filename doesn't actually matter since it doesn't correspond with checked-in files, so it can be changed at will. |
Sorry, something went wrong.
|
Sorry about the confusion with check_figures_equal. This approach here unrolls the parametrization into a helper loop. Exchanging the declarative parametrization for to code logic makes it less readable. But what bothers me more is that pytest will now not give the information which of the parameters failed. I Would prefer to solve this in the administrative logic if possible. I quickly searched for disabling parallelization for some tests, but didn’t find anything. Could we set a lock based on the filename in image_comparison? That would prevent simultaneous access to the test file. AFAICS a good test could still overwrite a bad test image result so that you see the failure but may not have the offending test image anymore. But that‘s no difference in the manual solution proposed here. Alternatively/additionally, could image_comparison carry a counter and number the test image stem.png, stem1.png, ... ? |
Sorry, something went wrong.
|
We could use a lock, Cartopy uses filelock for this purpose. But I think that might be a bit large for 3.2.x, so how about we merge this (I can retarget directly if needed) and try that out for 3.3.0? |
Sorry, something went wrong.
|
I don‘t see a problem backporting a lock to 3.2.1. It‘s an internal bug fix. But if you want to bring this explicit fix back I’m also fine with that. |
Sorry, something went wrong.
|
It's not just a lock, but also a new dependency. And also 3.2.1 is scheduled for Friday, and that will take some time to test, I think. |
Sorry, something went wrong.
…735-on-v3.2.x Backport PR #16735 on branch v3.2.x (Make test_stem less flaky.)
| Back | FazBrowse Home | New Git URL |
PR Summary
Since parametrizing the test allows it to run in parallel, this makes it flaky, as one process can overwrite the test result image of another.
Our standard way for dealing with tests that use the same baseline image is to pass duplicate filenames to image_comparison, because that is serialized.
This is basically the same as #16656 for test_stem.
PR Checklist