Have you actually tried this before and after the regression you mentioned as example?

@gvanrossum: The regression I linked above 1 is not caused by a single change but rather accumulation of multiple slowdown caused by multiple changes of which the most significant ones being python/cpython#91666 and python/cpython#27077. Since there is no benchmark in pyperformance to exercise coverage performance, the slowdown was unnoticed. Once this benchmark is added, it will help us pinpoint which commits caused the most slowdown with a simple git bisect between 3.10 and main branches.

Footnotes

1. https://github.com/python/pyperformance/pull/213#issue-1261503016 ↩

Nevertheless it would be interesting to see the numbers of running this BM on the state before and after said commits to demonstrate its usefulness.

I am running the benchmarks now, will post soon.

@gvanrossum Benchmarking before and after python/cpython@944fffe commit

Results:

Question for @ericsnowcurrently -- Do you agree this is useful?

Yeah, it's a helpful indicator of impact.