Maintainer information#

Making a release#

For branch organization we use a variation of the GitHub Flow with the latest release branch named stable (due to ReadTheDocs constraints).

Making a major release#

Assume for concreteness that we are releasing version 0.20.0.

Preparation#

1. Make a tracking issue with a title like 0.20.0 release planning. Add the checklist:

   • [ ] Update packages

   • [ ] Look for open PRs to add to the release milestone

   • [ ] Make sure all PRs in the release milestone are merged

   • [ ] Write release notes

   • [ ] Tidy changelog

2. Make a release notes blog post at pyodide-blog: https://github.com/pyodide/pyodide-blog

3. Generate the list of contributors for the release at the end of the release notes blog post with:

   git shortlog -s 0.19.0.. | cut -f2- | grep -v '\[bot\]' | sort --ignore-case | tr '\n' ';' | sed 's/;/, /g;s/, $//' | fold -s
   

   where 0.19.0 is the tag for the last major release.

4. Read the changelog and tidy it up by adding subsections, organizing, and proof reading it. Make a pull request with these changes titled Rearrange changelog for 0.20.0 release and merge it.

5. Make sure all the PRs that we want to release are merged and that the release notes are ready.

Releasing#

1. Switch to the main branch

2. From the root of the repository run:

   ./tools/bump_version.py 0.20.0 --tag
   

   This makes a release commit and tags it.

3. Push the release commit and tag to upstream. This triggers the release CI.

   git push upstream main 0.20.0
   

4. Wait for CI to pass and release to be created.

5. Rename the stable branch to a release branch for the previous major version. For instance if last release was, 0.20.0, the corresponding release branch would be 0.20.X:

   git fetch upstream stable:stable
   git branch 0.20.X stable
   git push -u upstream 0.20.X
   

6. Create a new stable branch:

   git switch main
   git switch -C stable
   git push upstream stable --force
   

7. Set the version back to next development version with:

   git switch main
   ./tools/bump_version.py 0.21.0 --dev
   git push upstream main
   

Making a minor release#

Assume for concreteness that we are releasing version 0.27.2.

Preparation#

1. Go through the commits on the main branch since the last release, find ones you want to backport and add the needs backport label to the pull requests. You can do this manually in the web interface on the GitHub PR or you can use

   ./tools/backport.py add-pr <pr-number>
   

2. List out the needs backport PRs that are missing changelog entries with

   ./tools/backport.py missing-changelogs
   

   and double check that every PR that should have a changelog does have one.

3. Read the changelog and tidy it up by adding subsections, organizing, and proof reading it. Make a pull request with these changes titled e.g., Rearrange changelog for 0.27.2 release and merge it.

4. Make the backport branch (on top of stable):

   ./tools/backport.py backport-branch
   

5. Make the update-changelog branch (on top of main) with:

   ./tools/backport.py changelog-branch
   

6. Open PRs for these two branches with:

   ./tools/backport.py open-release-prs
   

7. Use the backport branch PR as the release tracker.

8. Make sure that the CI passes on the backports branch and it is approved. When it does pass, set the date for the release in the changelog with:

   ./tools/backport.py set-date
   git switch backports-for-0.27.2
   git push -f
   git switch changelog-for-0.27.2
   git push -f
   

   Then merge the two PRs.

9. Run

   ./tools/backport.py clear-prs
   

   to clear all the needs backport labels.

Releasing#

1. Switch to the stable branch and git pull.

2. From the root of the repository run:

   ./tools/backport.py bump-version --tag
   

   This makes a release commit and tags it.

3. Push the release commit and tag to upstream/stable. This triggers the release CI.

   git push upstream stable 0.27.2
   

4. Wait for CI to pass and the release to be created.

Making an alpha release#

Assume for concreteness that we are releasing 0.28.0a1.

Preparation#

Any single maintainer can decide on their own to make an alpha release, it is not required to discuss it with other maintainers.

Name the first alpha release x.x.xa1 and in subsequent alphas increment the final number. No preparation is necessary. Do not make any changes to the changelog.

Release instructions#

1. Switch to the main branch and git pull.

2. From the root of the repository run:

   ./tools/bump_version.py 0.28.0a1 --tag
   

   This makes a release commit and tags it.

3. Push the release commit and tag to upstream/main. This triggers the release CI.

   git push upstream main 0.28.0a1
   

4. Put the version back with:

   git revert 0.28.0a1 -n && git commit -m "Back to development version"
   git push upstream main
   

5. Wait for CI to pass and the release to be created.

Fixing documentation for a released version#

Cherry pick the corresponding documentation commits to the stable branch. Use git commit --amend to add [skip ci] to the commit message.

Updating the Docker image#

Anyone with an account on hub.docker.com can follow the following steps:

1. Make whatever changes are needed to the Dockerfile.

2. Build the docker image with docker build . in the Pyodide root directory. If the build succeeds, docker will give you a hash for the built image.

3. Use python ./tools/docker_image_tag.py to find out what the new image tag should be. Tag the image with:

   docker image tag <image-hash> <your-docker-username>/pyodide-env:<image-tag>
   

4. Push the image with:

   docker image push <your-docker-username>/pyodide-env:<image-tag>
   

5. Replace the image in .circleci/config.yml with your newly created image. Open a pull request with your changes to Dockerfile and .circleci/config.yml.

6. When the tests pass and the pull request is approved, a maintainer must copy the new image into the pyodide dockerhub account.

7. Then replace the image tag in .circleci/config.yml, .devcontainer/devcontainer.json, and run_docker with the new image under the pyodide dockerhub account.

Its also possible to update the docker image by pushing your changes to the Dockerfile to a branch in the pyodide/pyodide repo (not on a fork) and clicking Run workflow on https://github.com/pyodide/pyodide/actions/workflows/docker_image.yml.

Updating packages#

Before updating the Python version and before making a major Pyodide release, we try to update all packages that are not too much trouble. Run

make -C packages update-all

to update all packages and make a pull request with these changes. There will be build/test failures, revert the packages that fail the build or tests and make a note to update them independently.

Updating pyodide-build#

To change the version of pyodide-build, change the commit of the pyodide-build submodule.

cd pyodide-build
git checkout "<COMMIT HASH>"

To test with a fork of pyodide-build, change the .gitmodules file to point to your fork and update the commit hash

# .gitmodules
[submodule "pyodide-build"]
	path = pyodide-build
	url = https://github.com/<yourfork>/pyodide-build

git submodule sync
cd pyodide-build
git checkout "<COMMIT HASH>"

Upgrading the ABI#

Each year, we aim to release a new Pyodide ABI version with a new CPython and Emscripten version.

Similar to the CPython alpha release, we stabilize the ABI early before releasing the new Pyodide version so that package maintainers can update their packages to use the new ABI (#5580).

0. Stabilize and freeze the ABI#

Discuss with the maintainers to decide what ABI changes to make before the release.

Once the ABI changes are decided, create a 20XX_0 branch (targeting year) from the main branch. All PRs that change the ABI should be merged to the 20XX_0 branch.

1. Disable USE_PREBUILT_PACKAGES#

Pyodide CI uses prebuilt packages built with the ABI of the last release. To upgrade the ABI, you need to disable them, otherwise the CI will fail.

• Find the USE_PREBUILT_PACKAGES environment variable in .circleci/config.yml and set it to false.

2. Upgrade Emscripten and CPython version#

Update the Emscripten and CPython versions respectively.

Updating the Emscripten version is not mandatory, but it is recommended to benefit from the latest features and bug fixes.

Updating Emscripten and CPython separately is often easier than updating them together.

2.1. Upgrade the Emscripten version#

Updating Emscripten requires the following three steps:

1. Rebase the patches in emsdk/patches onto the new Emscripten version.

2. Update the Emscripten version in Makefile.envs.

3. Update the struct_info JSON file in src/js/ to match the version of the file in Emscripten.

All three steps are automated by tools/update_emscripten.py. To update, run:

./tools/update_emscripten.py new_version

If there are rebase conflicts, you will have to manually finish the rebase. Once the rebase is completed, rerun update_emscripten.py. It will start over the rebase from scratch but reuse your conflict resolutions using the git rerere feature.

After this is done, commit all the changes and open a PR. There are frequently complicated CI failures.

2.2. Upgrade CPython version#

Prerequisites#

The desired version of CPython must be available at:

1. The specific release section of https://www.python.org/downloads

2. https://hub.docker.com/_/python

3. https://github.com/actions/python-versions/releases

Steps#

1. Follow the steps in Updating the Docker image to create a Docker image for the new Python version.

2. Make sure you are in a Python virtual environment with the new version of Python and with requirements.txt installed. (It is also possible to work in the Docker image as an alternative.)

3. Update the Python version in Makefile.envs.

4. Update the Python version in the following locations:

   • .github/workflows/main.yml

   • docs/conf.py

   • docs/development/contributing.md

   • docs/development/building-and-testing-packages.md

   • environment.yml

   • .pre-commit-config.yaml

   • pyproject.toml

   (TODO: make this list shorter.)

5. Check python/cpython-source-deps for updated versions of bundled C libraries. Pyodide bundles zstd from this repository (see ZSTDTARBALL in cpython/Makefile). Update the tag and URL if a newer version is available, and verify the build still works.

6. Rebase the patches:

   • Clone CPython and cd into it. Checkout the Python version you are upgrading from. For instance, if the old version is 3.13.2, use git checkout v3.13.2 (Python tags have a leading v). Run:

     git am ~/path/to/pyodide/cpython/patches/*
     

   • Rebase the patches onto the new version of Python. For instance, if updating from Python v3.13.2 to Python 3.14.1:

     git rebase v3.13.2 --onto v3.14.1
     

   • Resolve conflicts and drop patches that have been upstreamed. If you have conflicts, make sure you are using diff3:

     git config --global merge.conflictstyle diff3
     

   • Generate the new patches:

     rm ~/path/to/pyodide/cpython/patches/*
     git format-patch v3.14.1 -o ~/path/to/pyodide/cpython/patches/
     

7. Try to build Python with make -C cpython. Fix any build errors. If you modify the Python source in-tree after a failed build, it may be useful to run make rebuild.

8. Try to finish the build with a top-level make. Fix compile errors in src/core and any link errors. It may be useful to apply upgrade_pythoncapi.py --no-compat to the C extension in src/core.

   The file most tightly coupled to the CPython version is src/core/stack_switching/pystate.c. Consult the greenlet TPythonState.cpp to figure out how to fix it.

9. Run:

   python tools/make_test_list.py
   

   Then run the core tests with pytest src/tests/test_core_python.py and either fix the failures or update src/tests/python_tests.yaml to skip or xfail them.

10. Try to build packages with:

    pyodide build-recipes '*'
    

11. Fix failing package tests.

12. Update standard library stubs in src/templates. We currently have webbrowser.py and ssl.py that we implement ourselves. If you are updating the Python version, you may need to update these stubs. Review the Python docs for the standard library modules and check if the APIs have changed. Search for in version 3.XX to find the relevant changes.

Old major Python upgrades#

3. Release a new Pyodide alpha version#

After all the changes are merged, release a new Pyodide alpha version so that people can build packages with the new ABI.

4. Update packages and release new package set#

TODO: Update the process after PEP 783 is accepted.

Go to pyodide/pyodide-recipes and follow the guidelines in MAINTAINERS.md. Youll need to create a tag for the existing package sets and update default_cross_build_env_url in the main branch.

If you updated the Python version, also update the python version used in the CI. When opening the PR, include the full build label to trigger a full build of all packages; otherwise, only a small set of packages will be built.

Most likely, some packages will fail to build. You can disable them by adding _disabled: true to the recipes of the failing packages. You dont need to handle all the build failures in the same PRyou can open multiple PRs to fix the build failures or let the package maintainers handle them.

Create a new tag for the new package set when the build is successful. This will create a new release with the new package set.

5. Re-enable USE_PREBUILT_PACKAGES and upgrade the ABI in the main branch#

Go back to pyodide/pyodide and update the PYODIDE_PREBUILT_PACKAGES_BASE variable in Makefile.envs to the new package set you created in the previous step.

Then re-enable USE_PREBUILT_PACKAGES in .circleci/config.yml and open a PR to merge it into the main branch.

6. Release a new Pyodide version#

After enabling most of the packages that were available in the previous ABI version, we are ready to release a new Pyodide version with the new ABI.

Update What is the PyEmscripten Platform? to match the new ABI.

Merge the 20XX_0 branch into main and open a PR, then release a new Pyodide version with the new ABI.