CI: Add GHA workflow to upload nightly wheels by matthewfeickert · Pull Request #22733 · matplotlib/matplotlib · GitHub

PR Summary

Resolves [ENH]: Add a nightly wheel build #21635
Resolves document where nightly wheels are published #9994
Closes Also upload a subset of nightly wheels #21637 (supersedes)

This PR adds a nightlies.yml GitHub Action workflow that uses the gh CLI API to download the 'wheels' artifact from the latest completed build of the cibuildwheel workflow from the 'main' branch (idea taken from actions/download-artifact#3 (comment)).

gh run example:

$ gh run --repo matplotlib/matplotlib list --branch main --workflow cibuildwheel.yml
STATUS  NAME                                                                  WORKFLOW         BRANCH  EVENT         ID          ELAPSED   AGE
✓       Merge pull request #22719 from oscargus/incorrectdeprecationwarning   Build CI wheels  main    push          2067808346  1h11m7s   5h
✓       Merge pull request #22138 from stanleyjs/subfigure-clear              Build CI wheels  main    push          2067068506  1h11m8s   7h
✓       Merge pull request #22698 from oscargus/fixdoclinks                   Build CI wheels  main    push          2063032633  1h24m7s   21h
✓       Merge pull request #22711 from andrew-fennell/RangeSlider-bugfix      Build CI wheels  main    push          2062983684  1h25m25s  21h
✓       Merge pull request #22263 from jklymak/doc-version-switcher-condense  Build CI wheels  main    push          2061330182  1h14m20s  1d
✓       Merge pull request #22361 from anntzer/datetex                        Build CI wheels  main    push          2061179011  1h18m39s  1d
✓       Merge pull request #22721 from anntzer/style                          Build CI wheels  main    push          2059315555  1h3m24s   1d
X       PendingDeprecationWarning:                                            Build CI wheels  main    pull_request  2059201532  0s        1d
X       change test_aspect_equal_error() into test_aspect_equal()             Build CI wheels  main    pull_request  2057642015  0s        1d
✓       Merge pull request #22356 from timhoffm/triplot                       Build CI wheels  main    push          2057490803  1h2m25s   1d
✓       Merge pull request #22360 from anntzer/multilinetex                   Build CI wheels  main    push          2057057079  1h3m4s    1d
✓       Merge pull request #22418 from anntzer/ov                             Build CI wheels  main    push          2057027071  1h0m11s   1d
✓       Merge pull request #22722 from anntzer/cmf                            Build CI wheels  main    push          2056818352  1h8m22s   1d
✓       Merge pull request #22697 from oscargus/removecleanuptestcase         Build CI wheels  main    push          2056286785  1h3m30s   1d
✓       Merge pull request #22716 from jklymak/doc-set-canonical              Build CI wheels  main    push          2056271472  1h1m49s   1d
✓       Merge pull request #22556 from oscargus/parse_math_rcparams           Build CI wheels  main    push          2056265621  1h15m47s  2d
-       Locale font                                                           Build CI wheels  main    pull_request  2051994021  4s        2d
-       Locale font                                                           Build CI wheels  main    pull_request  2051993409  3s        2d
✓       Merge pull request #22163 from daniilS/tk_toolbar_colours             Build CI wheels  main    push          2044974231  1h13m49s  4d
-       Provide axis('equal') for Axes3D                                      Build CI wheels  main    pull_request  2040650565  3s        5d

For details on a run, try: gh run view <run-id>

This is done by getting the fields event, status, and databaseId from the output of gh run list as JSON and then using jq to filter for 'push' events (which correspond to merged PRs to the 'main' branch) that have completed runs. This is then sorted by the databaseIds in descending order to get the latest completed build run number.

filter and download example:

$ gh run --repo matplotlib/matplotlib list --branch main --workflow cibuildwheel.yml --json event,status,databaseId
[
  {
    "databaseId": 2067808346,
    "event": "push",
    "status": "completed"
  },
  {
    "databaseId": 2067068506,
    "event": "push",
    "status": "completed"
  },
  ...,
  {
    "databaseId": 2044974231,
    "event": "push",
    "status": "completed"
  },
  {
    "databaseId": 2040650565,
    "event": "pull_request",
    "status": "completed"
  }
]
$ cat runs.json | jq -c '[ .[] | select(.event == "push") | select(.status == "completed") ] | sort_by(.databaseId) | reverse'
[{"databaseId":2067808346,"event":"push","status":"completed"},{"databaseId":2067068506,"event":"push","status":"completed"},{"databaseId":2063032633,"event":"push","status":"completed"},{"databaseId":2062983684,"event":"push","status":"completed"},{"databaseId":2061330182,"event":"push","status":"completed"},{"databaseId":2061179011,"event":"push","status":"completed"},{"databaseId":2059315555,"event":"push","status":"completed"},{"databaseId":2057490803,"event":"push","status":"completed"},{"databaseId":2057057079,"event":"push","status":"completed"},{"databaseId":2057027071,"event":"push","status":"completed"},{"databaseId":2056818352,"event":"push","status":"completed"},{"databaseId":2056286785,"event":"push","status":"completed"},{"databaseId":2056271472,"event":"push","status":"completed"},{"databaseId":2056265621,"event":"push","status":"completed"},{"databaseId":2044974231,"event":"push","status":"completed"}]
$ cat runs.json | jq -c '[ .[] | select(.event == "push") | select(.status == "completed") ] | sort_by(.databaseId) | reverse | .[0].databaseId'
2067808346
$ gh run --repo matplotlib/matplotlib download 2067808346 --name wheels
$ ls *.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp310-cp310-macosx_10_12_universal2.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp310-cp310-macosx_10_12_x86_64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp310-cp310-macosx_11_0_arm64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp310-cp310-win32.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp310-cp310-win_amd64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp38-cp38-macosx_10_12_universal2.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp38-cp38-macosx_10_12_x86_64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp38-cp38-macosx_11_0_arm64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp38-cp38-win32.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp38-cp38-win_amd64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp39-cp39-macosx_10_12_universal2.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp39-cp39-macosx_10_12_x86_64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp39-cp39-macosx_11_0_arm64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp39-cp39-win32.whl
matplotlib-3.6.0.dev1935+gda9533d507-cp39-cp39-win_amd64.whl

The subset of all the downloaded wheels that are x86_64.manylinux* are then uploaded to the scipy-wheels-nightly organization using the anaconda-client CLI API.

N.B.: anaconda-client must currently be installed from GitHub as there have been no uploads to PyPI since 2016. c.f. anaconda/anaconda-client#540

The workflow runs nightly as a CRON job ('schedule' in GHA parlance) at 01:23 UTC (the time has no meaning (it is just 0123) beyond making sure it misses the wall of CRON jobs that slams CI providers at 00:00 and 00:01) and on demand through workflow dispatch.

The status of the build on my fork is currently summarized in #21635 (comment).

PR Checklist

Tests and Styling

Has pytest style unit tests (and pytest passes).
Is Flake 8 compliant (install flake8-docstrings and run flake8 --docstring-convention=all).

(No Python code was changed, so checking these off as main is passing)

Documentation

New features are documented, with examples if plot related. (N/A as no new features)
New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there). (N/A as no new features)
API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there). (N/A as no API changes)
Documentation is sphinx and numpydoc compliant (the docs should build without error).

c.f. #21637 (comment)

ANACONDA_ORG_UPLOAD_TOKEN is still the right secret but I do not think we will know if this works until we merge it.

Not sure if it is worth it, but you could manually download a wheel and then upload it with the token to test in advance that the ANACONDA_ORG_UPLOAD_TOKEN is valid. Though given that this PR enables workflow dispatch if there was something wrong with the token you could just change the value of the repo secret and rerun the job (which runs in under 1 minute) so probably not worth the time to test it out before.

Thank you for opening your first PR into Matplotlib!

If you have not heard from us in a while, please feel free to ping @matplotlib/developers or anyone who has commented on the PR. Most of our reviewers are volunteers and sometimes things fall through the cracks.

You can also join us on gitter for real-time discussion.

For details on testing, writing docs, and our review process, please see the developer guide

We strive to be a welcoming and open project. Please follow our Code of Conduct.

Okay, this passed CI before I rebased it to allow for uploading all the wheels (c.f. #22733 (comment)) so this should be ready for review. I'm going to tag multiple people as there as lots of discussion in Issue #9994 (apologies if this annoys anyone — not the intention).

Tagging:

@matplotlib/developers as was suggested by the very nice welcome bot. 😍
@ianhi as they already have been thinking about this work a lot and lead it with PR Also upload a subset of nightly wheels #21637.
@matthew-brett, @jklymak, @QuLogic given discussion and input in Issue document where nightly wheels are published #9994 and PR Also upload a subset of nightly wheels #21637.
@greglucas as they pointed out documentation is needed in Also upload a subset of nightly wheels #21637 (comment).
@tacaswell as he'll need to verify that ANACONDA_ORG_UPLOAD_TOKEN is still correct in the repo secrets (c.f. Also upload a subset of nightly wheels #21637 (comment)). Maybe he could do a manual run first to verify this?
@dstansby to make the sunpy team aware of this PR given document where nightly wheels are published #9994 (comment). 👋

Oh, this is clever to re-use the existing built wheels!

@QuLogic Question before I push next: Are you cool with rebasing during code review? Or would you prefer that the commit history not get revised until after final approval and then a final rebase be done to keep things more atomic and clean? The dev docs seem neutral on this point but I want to make your review as easy and not annoying as possible. 👍

We are very pro re-basing.

Add nightlies.yml GitHub Action workflow that uses the gh CLI API to download the 'wheels' artifact from the latest completed build of the cibuildwheel workflow from the 'main' branch. This is done by getting the fields 'event', 'status', and 'databaseId' from the output of `gh run list` as JSON and then using jq to filter for 'push' events (which correspond to merged PRs to the 'main' branch) that have completed runs. This is then sorted by the databaseIds in descending order to get the latest completed build run number. Then the 'wheels' artifact can be downloaded. The downloaded wheels are then uploaded to the scipy-wheels-nightly Anaconda Cloud organization (https://anaconda.org/scipy-wheels-nightly) using the anaconda-client CLI API.

Describe that the wheels are available on the scipy-wheels-nightly Anaconda Cloud organization (https://anaconda.org/scipy-wheels-nightly) and that a user should use --index-url to install from this package index.

Thanks @matthewfeickert! Congratulations on your first PR to Matplotlib 🎉 We hope to hear from you again.

@tacaswell: The token is incorrect:

Error:  ('Authentication token does not have the sufficient scope to perform this action expected: api:write', 401)

Congratulations on your first PR to Matplotlib 🎉 We hope to hear from you again.

Thanks for a ⚡ fast and helpful review! 😄 Truly appreciate all the help here — makes for a good first time contributor experience. 👍

I guess the "good news" RE: the token scope problem is that this shouldn't require any code changes at least.

https://github.com/matplotlib/matplotlib/runs/5780119640?check_suite_focus=true <- looks like it works now.

[edit to add extra word that made the comment way less obnoxious 🤦🏻 ]

Looks like in CI the given command doesn't necessarily work. because it tries to pull in the matlpotlib dependencies from the same source. See https://github.com/matplotlib/ipympl/runs/5780345225?check_suite_focus=true

Looking in indexes: https://pypi.anaconda.org/scipy-wheels-nightly/simple
Collecting matplotlib
  Downloading https://pypi.anaconda.org/scipy-wheels-nightly/simple/matplotlib/3.6.0.dev1948%2Bgd8ede1a710/matplotlib-3.6.0.dev1948%2Bgd8ede1a710-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.9/11.9 MB 59.0 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement cycler>=0.10 (from matplotlib) (from versions: none)
ERROR: No matching distribution found for cycler>=0.10
Error: Process completed with exit code 1.

Perhaps it should be --extra-index-url?

i think extra needs to point to pypi?

follow along at matplotlib/ipympl#449 for experimentation

https://github.com/matplotlib/matplotlib/runs/5780119640?check_suite_focus=true <- like it works now.

YAY! :D

@ianhi @QuLogic This is what I was assuming people would be doing (given how we test scipy for pyhf)

$ docker run --rm -ti python:3.10 /bin/bash
root@1d34811c6d56:/# python -m venv venv && . venv/bin/activate
(venv) root@1d34811c6d56:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@1d34811c6d56:/# python -m pip --quiet install matplotlib  # Get dependencies from PyPI
(venv) root@d092a0562fab:/# python -m pip install --upgrade --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple matplotlib
Looking in indexes: https://pypi.anaconda.org/scipy-wheels-nightly/simple
Requirement already satisfied: matplotlib in /venv/lib/python3.10/site-packages (3.5.1)
Collecting matplotlib
  Downloading https://pypi.anaconda.org/scipy-wheels-nightly/simple/matplotlib/3.6.0.dev1948%2Bgd8ede1a710/matplotlib-3.6.0.dev1948%2Bgd8ede1a710-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.9/11.9 MB 5.8 MB/s eta 0:00:00
Requirement already satisfied: packaging>=20.0 in /venv/lib/python3.10/site-packages (from matplotlib) (21.3)
Requirement already satisfied: numpy>=1.19 in /venv/lib/python3.10/site-packages (from matplotlib) (1.22.3)
Requirement already satisfied: fonttools>=4.22.0 in /venv/lib/python3.10/site-packages (from matplotlib) (4.31.2)
Requirement already satisfied: cycler>=0.10 in /venv/lib/python3.10/site-packages (from matplotlib) (0.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /venv/lib/python3.10/site-packages (from matplotlib) (1.4.2)
Requirement already satisfied: pillow>=6.2.0 in /venv/lib/python3.10/site-packages (from matplotlib) (9.0.1)
Requirement already satisfied: python-dateutil>=2.7 in /venv/lib/python3.10/site-packages (from matplotlib) (2.8.2)
Requirement already satisfied: pyparsing>=2.2.1 in /venv/lib/python3.10/site-packages (from matplotlib) (3.0.7)
Requirement already satisfied: six>=1.5 in /venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)

Though it seems that this isn't how you think people will approach it, so I guess this should get updated.

edit: Oops I see that this doesn't even work properly, so yeah needs to get fixed

(venv) root@d092a0562fab:/# python -m pip uninstall -y matplotlib
Found existing installation: matplotlib 3.5.1
Uninstalling matplotlib-3.5.1:
  Successfully uninstalled matplotlib-3.5.1
(venv) root@d092a0562fab:/# python -m pip install --upgrade --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple matplotlib
Looking in indexes: https://pypi.anaconda.org/scipy-wheels-nightly/simple
Collecting matplotlib
  Downloading https://pypi.anaconda.org/scipy-wheels-nightly/simple/matplotlib/3.6.0.dev1948%2Bgd8ede1a710/matplotlib-3.6.0.dev1948%2Bgd8ede1a710-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.9/11.9 MB 18.3 MB/s eta 0:00:00
Requirement already satisfied: python-dateutil>=2.7 in /venv/lib/python3.10/site-packages (from matplotlib) (2.8.2)
Requirement already satisfied: packaging>=20.0 in /venv/lib/python3.10/site-packages (from matplotlib) (21.3)
Requirement already satisfied: fonttools>=4.22.0 in /venv/lib/python3.10/site-packages (from matplotlib) (4.31.2)
Requirement already satisfied: cycler>=0.10 in /venv/lib/python3.10/site-packages (from matplotlib) (0.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /venv/lib/python3.10/site-packages (from matplotlib) (1.4.2)
Requirement already satisfied: numpy>=1.19 in /venv/lib/python3.10/site-packages (from matplotlib) (1.22.3)
Requirement already satisfied: pillow>=6.2.0 in /venv/lib/python3.10/site-packages (from matplotlib) (9.0.1)
ERROR: Could not find a version that satisfies the requirement pyparsing<3.0.0,>=2.2.1 (from matplotlib) (from versions: none)
ERROR: No matching distribution found for pyparsing<3.0.0,>=2.2.1

I think the pyparsing issue is one we should look into un-pinning on main.

Looks like the magic working line is this one:
(edit no it's not - see below for the actual working one)

pip install --upgrade --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple --extra-index-url https://pypi.org/simple matplotlib

actually no it doesn't :/

still got 3.5.1

https://github.com/matplotlib/ipympl/runs/5780465750?check_suite_focus=true

Looking in indexes: https://pypi.anaconda.org/scipy-wheels-nightly/simple, https://pypi.org/simple
Collecting matplotlib
  Downloading matplotlib-3.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.9/11.9 MB 40.6 MB/s eta 0:00:00

I think the pyparsing issue is one we should look into un-pinning on main.

Yeah this is necessary. You currently have to do the following:

$ docker run --rm -ti python:3.10 /bin/bash
root@ad729996566a:/# python -m venv venv && . venv/bin/activate
(venv) root@ad729996566a:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@ad729996566a:/# python -m pip --quiet install matplotlib  # Get dependencies 
(venv) root@ad729996566a:/# python -m pip --quiet install --upgrade 'pyparsing<3.0'  # Fix pyparsing upper bound problem
(venv) root@ad729996566a:/# python -m pip install --upgrade --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple --pre matplotlib
Looking in indexes: https://pypi.anaconda.org/scipy-wheels-nightly/simple
Collecting matplotlib
  Downloading https://pypi.anaconda.org/scipy-wheels-nightly/simple/matplotlib/3.6.0.dev1948%2Bgd8ede1a710/matplotlib-3.6.0.dev1948%2Bgd8ede1a710-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.9/11.9 MB 18.5 MB/s eta 0:00:00
Requirement already satisfied: pyparsing<3.0.0,>=2.2.1 in /venv/lib/python3.10/site-packages (from matplotlib) (2.4.7)
Requirement already satisfied: cycler>=0.10 in /venv/lib/python3.10/site-packages (from matplotlib) (0.11.0)
Requirement already satisfied: packaging>=20.0 in /venv/lib/python3.10/site-packages (from matplotlib) (21.3)
Requirement already satisfied: fonttools>=4.22.0 in /venv/lib/python3.10/site-packages (from matplotlib) (4.31.2)
Requirement already satisfied: python-dateutil>=2.7 in /venv/lib/python3.10/site-packages (from matplotlib) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /venv/lib/python3.10/site-packages (from matplotlib) (1.4.2)
Requirement already satisfied: numpy>=1.19 in /venv/lib/python3.10/site-packages (from matplotlib) (1.22.3)
Requirement already satisfied: pillow>=6.2.0 in /venv/lib/python3.10/site-packages (from matplotlib) (9.0.1)
Requirement already satisfied: six>=1.5 in /venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)
Installing collected packages: matplotlib
Successfully installed matplotlib-3.6.0.dev1948+gd8ede1a710
(venv) root@ad729996566a:/# python -m pip show matplotlib
Name: matplotlib
Version: 3.6.0.dev1948+gd8ede1a710
Summary: Python plotting package
Home-page: https://matplotlib.org
Author: John D. Hunter, Michael Droettboom
Author-email: matplotlib-users@python.org
License: PSF
Location: /venv/lib/python3.10/site-packages
Requires: cycler, fonttools, kiwisolver, numpy, packaging, pillow, pyparsing, python-dateutil
Required-by:

edit: I originally was uninstalling and then installing from the nightlies,

(venv) root@ad729996566a:/# python -m pip --quiet uninstall -y matplotlib  # Remove so that the dev version can be installed, as stable gets preference...though could maybe just use --pre

but you can just use --pre as I show above.

ok I think the best possible option is:

pip install --upgrade --pre --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple --extra-index-url https://pypi.org/simple matplotlib

this way you get the most up to date mpl dependencies (but download the deps from pypi)

ok I think the best possible option is:
pip install --upgrade --pre --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple --extra-index-url https://pypi.org/simple matplotlib
this way you get the most up to date mpl dependencies (but download the deps from pypi)

Yeah that works!

$ docker run --rm -ti python:3.10 /bin/bash
root@17a63da795c7:/# python -m venv venv && . venv/bin/activate
(venv) root@17a63da795c7:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@17a63da795c7:/# python -m pip install --upgrade --pre --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple --extra-index-url https://pypi.org/simple matplotlib
Looking in indexes: https://pypi.anaconda.org/scipy-wheels-nightly/simple, https://pypi.org/simple
Collecting matplotlib
  Downloading https://pypi.anaconda.org/scipy-wheels-nightly/simple/matplotlib/3.6.0.dev1948%2Bgd8ede1a710/matplotlib-3.6.0.dev1948%2Bgd8ede1a710-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.9/11.9 MB 25.5 MB/s eta 0:00:00
Collecting cycler>=0.10
  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.4.2-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 24.1 MB/s eta 0:00:00
Collecting pyparsing<3.0.0,>=2.2.1
  Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.8/67.8 KB 5.0 MB/s eta 0:00:00
Collecting packaging>=20.0
  Downloading packaging-21.3-py3-none-any.whl (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 KB 3.5 MB/s eta 0:00:00
Collecting fonttools>=4.22.0
  Downloading fonttools-4.31.2-py3-none-any.whl (899 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 899.5/899.5 KB 23.8 MB/s eta 0:00:00
Collecting python-dateutil>=2.7
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 KB 23.1 MB/s eta 0:00:00
Collecting pillow>=6.2.0
  Downloading Pillow-9.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 31.0 MB/s eta 0:00:00
Collecting numpy>=1.19
  Downloading https://pypi.anaconda.org/scipy-wheels-nightly/simple/numpy/1.23.0.dev0%2B967.g41cf10dcb/numpy-1.23.0.dev0%2B967.g41cf10dcb-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.0/17.0 MB 10.4 MB/s eta 0:00:00
Collecting six>=1.5
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, pyparsing, pillow, numpy, kiwisolver, fonttools, cycler, python-dateutil, packaging, matplotlib
Successfully installed cycler-0.11.0 fonttools-4.31.2 kiwisolver-1.4.2 matplotlib-3.6.0.dev1948+gd8ede1a710 numpy-1.23.0.dev0+967.g41cf10dcb packaging-21.3 pillow-9.0.1 pyparsing-2.4.7 python-dateutil-2.8.2 six-1.16.0
(venv) root@17a63da795c7:/# python -m pip show matplotlib
Name: matplotlib
Version: 3.6.0.dev1948+gd8ede1a710
Summary: Python plotting package
Home-page: https://matplotlib.org
Author: John D. Hunter, Michael Droettboom
Author-email: matplotlib-users@python.org
License: PSF
Location: /venv/lib/python3.10/site-packages
Requires: cycler, fonttools, kiwisolver, numpy, packaging, pillow, pyparsing, python-dateutil
Required-by:

I'll open a fix PR for the docs after dinner.

Whoops, it looks like we fixed the compatibility in with pyparsing 3 in https://github.com/matplotlib/matplotlib/pull/21501/files and removed the pinning in the CI requirement files but forgot to remove it in setup.py 🤦🏻

#22751 to fix the pyparsing issue.

PR #22752 to make the docs more robust.

Duplicate uploads also fail.

I think that is OK though?

We have at least one build on main per day most days and I am not sure it is worth the logic to not get rejected on the days we do not (or on days when someone has pushed the button to run it early).

Duplicate uploads also fail.

I think that is OK though?

We have at least one build on main per day most days and I am not sure it is worth the logic to not get rejected on the days we do not (or on days when someone has pushed the button to run it early).

If no one minds I don't mind throwing in some additional logic to check just before the upload stage. This is pretty easy with pip index versions and some sed

$ python -m pip index \
    --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple \
    --pre \
    versions matplotlib | \
  grep matplotlib | \
  sed 's/.*(\(.*\))/\1/'
WARNING: pip index is currently an experimental command. It may be removed/changed in a future release without prior warning.
3.6.0.dev1948+gd8ede1a710

So all you'd need to do would be something like

$ LAST_NIGHTLY_VERSION="$(python -m pip index \
    --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple \
    --pre \
    versions matplotlib | \
  grep matplotlib | \
  sed 's/.*(\(.*\))/\1/')"
WARNING: pip index is currently an experimental command. It may be removed/changed in a future release without prior warning.
$ echo "${LAST_NIGHTLY_VERSION}"
3.6.0.dev1948+gd8ede1a710

and then just check if that shows up as version of the wheels that just got downloaded from the GitHub Actions workflow artifact

$ [ "$(find dist -type f -iname "matplotlib-${LAST_NIGHTLY_VERSION}*.whl" | wc --lines)" -gt "0" ]
$ echo $?
0

(maybe a more elegant way to do that but that's not the worst)

I'll make an Issue from this and I'm happy to PR it, unless you would prefer to avoid touching it as much as possible.

edit: Answer is way easier: Use the --skip-existing option

          anaconda --token ${{ secrets.ANACONDA_ORG_UPLOAD_TOKEN }} upload \
            --user scipy-wheels-nightly \
            --skip-existing \
            dist/matplotlib-*.whl

matthewfeickert commented Mar 30, 2022

View reviewed changes

Comment thread .github/workflows/nightlies.yml Outdated

matthewfeickert commented Mar 30, 2022

View reviewed changes

Comment thread .github/workflows/nightlies.yml

matthewfeickert commented Mar 30, 2022

View reviewed changes

Comment thread .github/workflows/nightlies.yml Outdated

github-actions Bot reviewed Mar 30, 2022

View reviewed changes

matthewfeickert force-pushed the ci/upload-nightly-wheels branch from 0f78d7c to 5527d13 Compare March 30, 2022 20:29

matthewfeickert requested a review from ianhi March 30, 2022 20:33

tacaswell added this to the v3.6.0 milestone Mar 31, 2022

tacaswell approved these changes Mar 31, 2022

View reviewed changes

ianhi mentioned this pull request Mar 31, 2022

Also upload a subset of nightly wheels #21637

Closed

QuLogic reviewed Mar 31, 2022

View reviewed changes

Comment thread .github/workflows/nightlies.yml Outdated

Comment thread doc/users/installing/index.rst Outdated

Comment thread .github/workflows/nightlies.yml

Comment thread .github/workflows/nightlies.yml Outdated

Comment thread .github/workflows/nightlies.yml Outdated

matthewfeickert force-pushed the ci/upload-nightly-wheels branch from d66a1b2 to 1cb0a7a Compare March 31, 2022 03:34

matthewfeickert commented Mar 31, 2022

View reviewed changes

Comment thread .github/workflows/nightlies.yml Outdated

matthewfeickert requested a review from QuLogic March 31, 2022 03:38

QuLogic reviewed Mar 31, 2022

View reviewed changes

Comment thread .github/workflows/nightlies.yml Outdated

matthewfeickert added 2 commits March 30, 2022 23:17

DOC: Add instructions on installing nightly build wheels

7dda3c6

Describe that the wheels are available on the scipy-wheels-nightly Anaconda Cloud organization (https://anaconda.org/scipy-wheels-nightly) and that a user should use --index-url to install from this package index.

matthewfeickert force-pushed the ci/upload-nightly-wheels branch from 1cb0a7a to 7dda3c6 Compare March 31, 2022 04:25

matthewfeickert requested a review from QuLogic March 31, 2022 04:27

matthewfeickert mentioned this pull request Mar 31, 2022

[MNT]: Add codespell to pre-commit hooks #22740

Closed

QuLogic approved these changes Mar 31, 2022

View reviewed changes

QuLogic merged commit 91f3465 into matplotlib:main Mar 31, 2022

matthewfeickert deleted the ci/upload-nightly-wheels branch March 31, 2022 06:38

tacaswell mentioned this pull request Apr 1, 2022

[ENH]: Does pyplot need to call switch_backend #22739

Closed

ianhi mentioned this pull request Apr 1, 2022

use new matplotlib nightly wheel in tests matplotlib/ipympl#449

Merged

matthewfeickert mentioned this pull request Apr 1, 2022

DOC: Correct nightly wheels pip install command #22752

Merged

6 tasks done

This was referenced Apr 1, 2022

Duplicate uploads of nightly wheels to scipy-wheels-nightly Anaconda cloud package index fails #22757

Closed

Add HEAD of dependencies GHA workflow scikit-hep/mplhep#373

Closed

MNT: Skip existing wheels during nightly wheel upload #22759

Merged

dstansby mentioned this pull request Apr 1, 2022

Use Matplotlib nightly wheels in devdeps testing sunpy/sunpy#6019

Closed

This was referenced Apr 1, 2022

ci: Add matplotlib nightly wheels to HEAD of dependencies testing scikit-hep/pyhf#1835

Merged

DEV: Add codespell to pre-commit hooks #22777

Merged

tacaswell mentioned this pull request May 15, 2022

Fix variable initialization due to jump bypassing it #23051

Merged

6 tasks

bmcfee mentioned this pull request Jul 14, 2022

Nightly builds for bleeding edge dependencies librosa/librosa#1528

Open

matthewfeickert mentioned this pull request Feb 7, 2024

Add nightly wheel upload to Anaconda Cloud scikit-hep/awkward#3011

Closed

CI: Add GHA workflow to upload nightly wheels by matthewfeickert · Pull Request #22733 · matplotlib/matplotlib · GitHub

Repository navigation

CI: Add GHA workflow to upload nightly wheels - #22733

CI: Add GHA workflow to upload nightly wheels#22733QuLogic merged 2 commits intomatplotlib:mainmatplotlib/matplotlib:mainfrom matthewfeickert:ci/upload-nightly-wheelsmatthewfeickert/matplotlib:ci/upload-nightly-wheelsCopy head branch name to clipboard

Conversation

matthewfeickert commented Mar 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklist

Uh oh!

Uh oh!

Uh oh!

matthewfeickert Mar 30, 2022

Choose a reason for hiding this comment

Uh oh!

tacaswell Mar 31, 2022

Choose a reason for hiding this comment

Uh oh!

matthewfeickert Mar 31, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

matthewfeickert commented Mar 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tacaswell commented Mar 31, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matthewfeickert commented Mar 31, 2022

Uh oh!

tacaswell commented Mar 31, 2022

Uh oh!

Uh oh!

Uh oh!

QuLogic commented Mar 31, 2022

Uh oh!

QuLogic commented Mar 31, 2022

Uh oh!

matthewfeickert commented Mar 31, 2022

Uh oh!

tacaswell commented Apr 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianhi commented Apr 1, 2022

Uh oh!

QuLogic commented Apr 1, 2022

Uh oh!

ianhi commented Apr 1, 2022

Uh oh!

ianhi commented Apr 1, 2022

Uh oh!

matthewfeickert commented Apr 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tacaswell commented Apr 1, 2022

Uh oh!

ianhi commented Apr 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianhi commented Apr 1, 2022

Uh oh!

matthewfeickert commented Apr 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianhi commented Apr 1, 2022

Uh oh!

matthewfeickert commented Apr 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tacaswell commented Apr 1, 2022

Uh oh!

tacaswell commented Apr 1, 2022

Uh oh!

CI: Add GHA workflow to upload nightly wheels#22733
QuLogic merged 2 commits into
matplotlib:mainfrom
matthewfeickert:ci/upload-nightly-wheels

matthewfeickert commented Mar 30, 2022 •
edited
Loading

matthewfeickert commented Mar 30, 2022 •
edited
Loading

tacaswell commented Apr 1, 2022 •
edited
Loading

matthewfeickert commented Apr 1, 2022 •
edited
Loading

ianhi commented Apr 1, 2022 •
edited
Loading

matthewfeickert commented Apr 1, 2022 •
edited
Loading

matthewfeickert commented Apr 1, 2022 •
edited
Loading

matthewfeickert commented Apr 1, 2022 •
edited
Loading