Impossible to add a benchmark that exercises newer Python features #281
Description
Activity
Yeah, pyperformance should probably check this somehow and refuse to run certain benchmarks based on the advertised Python version support.
Another option could be to disable them by default, and enable them when we drop older version support. But that seems less ideal.
Reacted by Alex Waygood
It appears to be currently impossible to add a benchmark that uses a feature added in Python >3.7. Even if you add the requires-python field in the pyproject.toml file for a specific benchmark, the benchmark is run on all Python versions in CI. This means that the CI will fail if the benchmark includes a feature that is new in 3.8, for example.
This issue currently blocks both of these open PRs: