Output python-versions, containing all Python versions, for cache-busting - #611
Output python-versions, containing all Python versions, for cache-busting#611kurtmckee wants to merge 1 commit into
Conversation
Sorry, something went wrong.
| pythonVersions.push( | ||
| `${installed.resolvedPythonVersion}-pypy${installed.resolvedPyPyVersion}` | ||
| ); |
There was a problem hiding this comment.
Do we need to change pypy and python version parts order ?
Sorry, something went wrong.
There was a problem hiding this comment.
I'm fine with either order.
I did it this way for potential parsing purposes later in the workflow. It seemed like if someone wanted to determine all of the Python versions (regardless of CPython vs PyPy implementation), it would be easier if the Python version appeared at the beginning of the string. If -pypy appeared later in the string it could be stripped off or extracted out as needed.
However, I don't view the ordering as a critical need, and can change the order if desired.
Sorry, something went wrong.
| core.info(`Successfully set up ${installed.impl} (${pythonVersion})`); | ||
| } | ||
| } | ||
| core.setOutput('python-versions', pythonVersions.sort().join(',')); |
There was a problem hiding this comment.
Do we need to sort versions ?
Sorry, something went wrong.
There was a problem hiding this comment.
I sorted it to help stabilize the cache key. If one developer writes their workflow YAML file as:
- uses: actions/setup-python@v4
with:
python-version: |
3.11
3.10
3.9
3.8
3.7and the versions are later rearranged to:
- uses: actions/setup-python@v4
with:
python-version: |
3.7
3.8
3.9
3.10
3.11sorting the versions will provide stability to the cache key.
I currently think this is beneficial behavior, but can remove the sorting if desired.
Sorry, something went wrong.
|
@dmitry-shibanov I just saw the new support for "allow-prereleases", which is fantastic! I would love to get additional feedback about this PR, or to have it merged, so that I can use this new output as a cache key. I don't know if I communicated the use case well, so to summarize: Setting up a GitHub runner for each Python version is comparatively expensive for some of my projects -- it can take 30 seconds to spin up a runner, 5 seconds to set up the tox environment, but the test suite only takes 2 seconds to run for each Python version. Therefore I install all Python versions on a single runner:
The cache needs to be invalidated if any Python version changes, and this PR makes that possible. Python 3.12 releases are coming out with frequency, and I would like to take advantage of the new "allow-prereleases" feature and have this python-versions output available for cache busting. Thanks for your time, and your work on setup-python! |
Sorry, something went wrong.
|
@dmitry-shibanov, please let me know if you have additional feedback. I'm interested in having this PR merge for effective cache-busting when multiple Python versions are installed. |
Sorry, something went wrong.
|
Rebased on main and resolved CI failures. |
Sorry, something went wrong.
|
I'm closing this PR because I've written a GitHub action to address my need for strong cache-busting based on installed Python versions. |
Sorry, something went wrong.
Description:
This introduces a python-versions output, which is a comma-separated string of all installed Python versions and implementations. This is useful for cache-busting when multiple Python versions are installed, as described in #606.
For example, it is now possible to cache a .tox/ test directory and invalidate the cache when any of the Python versions change. Caching the .tox/ directory offers a significant speedup to test setup times, but if a non-default Python's version changes (say, if 3.11.1 is the default but 3.10.x increments to 3.10.y), it is currently very difficult to invalidate the cache. This new output variable allows for trivial cache invalidation in this situation.
To demonstrate the new output variable, I created a workflow with a single setup-python step that installs multiple CPython and PyPy versions, then used echo to show the value of python-versions.
The value displayed is:
which can be used directly in a cache key, or written to a file and included in a call to hashFiles(). The versions are sorted to avoid sensitivity to the order of Python versions specific in the workflow, but are sorted merely as strings for this purpose, not as semantic versions. Therefore a user can create a workflow that installs Python versions in any order they want, but the python-versions output string will be stable.
NOTE: This work depends on #610 to remove trailing newlines in PyPy versions.
Related issue:
Closes #606
Check list: