FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Add docs for updating external dependencies by sethmlarson · Pull Request #1280 · python/devguide · GitHub

/ devguide Public

Add docs for updating external dependencies - #1280

Open
sethmlarson wants to merge 7 commits into
python:mainfrom
sethmlarson:sbom-externals
ConversationCommits (7)ChecksFiles changed
Open

Add docs for updating external dependencies#1280
sethmlarson wants to merge 7 commits into
python:mainfrom
sethmlarson:sbom-externals

Conversation

sethmlarson commented Feb 27, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

sethmlarson requested a review from zooba February 27, 2024 21:57
hugovk reviewed Feb 28, 2024
View reviewed changes
Comment thread developer-workflow/sbom.rst Outdated
Comment thread developer-workflow/sbom.rst Outdated
Comment thread developer-workflow/sbom.rst Outdated
Comment thread developer-workflow/sbom.rst Outdated
zooba reviewed Feb 28, 2024
View reviewed changes

zooba left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Thanks for starting this! It's important workflow that we've never properly documented (not just for SBOMs)

Comment thread developer-workflow/sbom.rst Outdated
Updating external dependencies (cpython-source-deps)
----------------------------------------------------

Dependencies for Windows CPython builds are `stored in a separate repository <https://github.com/python/cpython-source-deps>`_

zooba Feb 28, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Some binaries are also stored in https://github.com/python/cpython-bin-deps, though generally they should also have sources in the source-deps repo. Is this distinction important here?

sethmlarson Feb 29, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Do any of the cpython-bin-deps get shipped along with the CPython artifacts? If they're derived from the cpython-source-deps repository I think we should be okay.

zooba Mar 4, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think the only one that isn't derived from cpython-source-deps is vcruntime140.dll, which comes from our repo to make sure we always get the latest one and not whichever GHA build machine we're on.

Comment thread developer-workflow/sbom.rst Outdated
SBOM tooling in the CPython repository matches these git refs in order to build the :cpy-file:`Misc/externals.spdx.json`
SBOM file. When updating external dependencies for a CPython branch:

1. Push the update to the ``cpython-source-deps`` repository and create a new git tag.

zooba Feb 28, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Maybe worth noting that this can only be done by a core committer, and we don't accept PRs for it (because we need to verify the sources have come from the right source and are unmodified, and our trust boundary for this is "has the commit bit").

Also might be worth noting that sometimes there's a build step involved and the core committer will then push a tag to cpython-bin-deps that will actually be used in the build. Tcl/Tk, libffi and OpenSSL are all in this group.

In practice for contributors, what this usually means is that they should post an issue requesting the updated version, wait for a core dev to say the tags are ready, and then the contributor can continue with the following steps.

sethmlarson Mar 4, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I've addressed this comment in b32b691. Do you think we should cover the cpython-bin-deps part here as well?

zooba Mar 6, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Not in the same note, but it ought to be documented somewhere. At the very least, we should mention the cpython-bin-deps repo at least once so that someone reading this knows to look there.

sethmlarson and others added 2 commits February 29, 2024 08:26
Apply suggestions from code review
6646cf8 
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Add PCbuild/build.bat --regen option to docs
cefc01d
ezio-melotti reviewed Mar 4, 2024
View reviewed changes
Comment thread developer-workflow/sbom.rst Outdated
Comment thread developer-workflow/sbom.rst Outdated
Comment thread developer-workflow/sbom.rst Outdated
Comment thread developer-workflow/sbom.rst Outdated
sethmlarson and others added 3 commits March 4, 2024 14:19
Apply suggestions from code review
a154e01 
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Update developer-workflow/sbom.rst
cc0709a 
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Add warning for non-core developers and external dependencies
b32b691
willingc reviewed Mar 6, 2024
View reviewed changes

willingc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Overall, this is a nice improvement. Perhaps adding subsections would add context and clarity (not suggested wording but I see 3 distinct parts):

  • Process for updating dependencies: who and how (make a subsection and not a note)
  • Background on how the SBOM is built
  • Steps for a core dev updating the external dependencies

Comment thread developer-workflow/sbom.rst
Comment thread developer-workflow/sbom.rst Outdated
Comment thread developer-workflow/sbom.rst Outdated
Comment thread developer-workflow/sbom.rst
Comment thread developer-workflow/sbom.rst Outdated
Comment thread developer-workflow/sbom.rst
builds of CPython for Windows in the script :cpy-file:`PCbuild/get_externals.bat`.

In this script the libraries to fetch are designated by ``{name}-{version}``
Git refs being added to the ``libraries`` variable.

willingc Mar 6, 2024

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Would be helpful to clarify where the libraries variable is.

willingc Mar 11, 2024

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This is still unclear.

Apply suggestions from code review
3588722 
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>

sethmlarson commented Mar 11, 2024

Copy link
Copy Markdown
Contributor Author

@willingc Apologies, didn't mean to mark the PR as ready for more review. I won't be able to get this one complete until later in March after I'm back from a trip.

willingc commented Oct 12, 2024

Copy link
Copy Markdown
Collaborator

Ping @sethmlarson. What do we need to do to reboot this PR or move it to draft status? Thanks!

StanFromIreland commented Sep 13, 2025

Copy link
Copy Markdown
Member

Related issue: #298

StanFromIreland added the needs: PR update An update or rebase to an existing PR is needed. label Jun 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

hugovk hugovk left review comments
zooba zooba left review comments
willingc willingc left review comments
ezio-melotti ezio-melotti left review comments

Assignees

No one assigned

Labels

needs: PR update An update or rebase to an existing PR is needed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

Back | FazBrowse Home | New Git URL