gh-112088: Run autoreconf in GHA check_generated_files by vstinner · Pull Request #112090 · python/cpython · GitHub

The "Check if generated files are up to date" job of GitHub Actions now runs the "autoreconf -ivf -Werror" command instead of the "make regen-configure" command to avoid depending on the unstable quay.io server.

Issue: Consider avoiding the dependency on quay.io server in the GitHub Action "Check if generated files are up to date" job #112088

I think this is a good idea. It doesn't appear there is any reason for the quay.io image anymore given ubuntu-22.04 has a sufficiently good autoconf.

cc @erlend-aasland

Let's update https://github.com/python/cpython/blob/d4f83e1e3a19e2f881115f20d58ae6a019ddb48f/Doc/using/configure.rst#generated-files as well.

And backport?

@hugovk: I completed the PR to address your review. Would you mind to review the updated PR?

Let's update https://github.com/python/cpython/blob/d4f83e1e3a19e2f881115f20d58ae6a019ddb48f/Doc/using/configure.rst#generated-files as well.

Done.

And backport?

I added the backport to 3.11 and 3.12 labels.

The main point of this PR is that Tools/build/regen-configure.sh now uses the same container image than the GitHub Action job, so the job doesn't have to run autoreconf in a container, but can run it directly.

The "Check if generated files are up to date" job of GitHub Actions now runs the "autoreconf -ivf -Werror" command instead of the "make regen-configure" command to avoid depending on the external quay.io server. Add Tools/build/regen-configure.sh script to regenerate the configure with an Ubuntu container image. The "quay.io/tiran/cpython_autoconf:271" container image (https://github.com/tiran/cpython_autoconf) is no longer used.

Oh, there is a fix in the Changelog entry causing the Docs job to fail: fixed.

@erlend-aasland: It would be nice if you could review this change, since you made many configure changes last months ;-)

Also, the devguide must be updated.

This looks good to me. (I'm a little bit puzzled by the aclocal.m4 and configure changes, but they are also fine.)

Thanks for doing this.

IIRC, we did the autoconf version bump before the 3.12 freeze, so IMO we should backport this to 3.12.

This looks good to me. (I'm a little bit puzzled by the aclocal.m4 and configure changes, but they are also fine.)

I'm not sure if I picked the wrong pkg-config package. But well, for now, my main worry is more to make the whole Python workflow more reliable by avoiding depending on the external quay.io server which had multiple outages in several days.

With my change, the Python workflow no longer pulls an external container image, but simply reuse what's available on Ubuntu 22.04 in the GitHub Action.

IIRC, we did the autoconf version bump before the 3.12 freeze, so IMO we should backport this to 3.12.

Oh. So we cannot backport this change to 3.11?

…H-112090) The "Check if generated files are up to date" job of GitHub Actions now runs the "autoreconf -ivf -Werror" command instead of the "make regen-configure" command to avoid depending on the external quay.io server. Add Tools/build/regen-configure.sh script to regenerate the configure with an Ubuntu container image. The "quay.io/tiran/cpython_autoconf:271" container image (https://github.com/tiran/cpython_autoconf) is no longer used. (cherry picked from commit d9fd33a) Co-authored-by: Victor Stinner <vstinner@python.org>

Merged. Thanks for reviews. Let's start with a backport to 3.12.

Oh. So we cannot backport this change to 3.11?

No, 3.11 uses a patched Autoconf 2.69 from Alpine Linux, not a vanilla 2.69 installation.

@vstinner I was doing some work on release-tools adjacent to this, @zware pointed me to this PR. What are your thoughts on backporting the regen-configure makefile target to all supported versions so that release-tools can depend on it being there?

For 3.11 and earlier autoconf version configure generation udpates, ask the release managers (@pablogsal and @ambv) for those branches. (it makes sense for 3.12)

…112090) The "Check if generated files are up to date" job of GitHub Actions now runs the "autoreconf -ivf -Werror" command instead of the "make regen-configure" command to avoid depending on the external quay.io server. Add Tools/build/regen-configure.sh script to regenerate the configure with an Ubuntu container image. The "quay.io/tiran/cpython_autoconf:271" container image (https://github.com/tiran/cpython_autoconf) is no longer used. (cherry picked from commit d9fd33a)

) (#112159) gh-112088: Run autoreconf in GHA check_generated_files (#112090) The "Check if generated files are up to date" job of GitHub Actions now runs the "autoreconf -ivf -Werror" command instead of the "make regen-configure" command to avoid depending on the external quay.io server. Add Tools/build/regen-configure.sh script to regenerate the configure with an Ubuntu container image. The "quay.io/tiran/cpython_autoconf:271" container image (https://github.com/tiran/cpython_autoconf) is no longer used. (cherry picked from commit d9fd33a)

…112090) The "Check if generated files are up to date" job of GitHub Actions now runs the "autoreconf -ivf -Werror" command instead of the "make regen-configure" command to avoid depending on the external quay.io server. Add Tools/build/regen-configure.sh script to regenerate the configure with an Ubuntu container image. The "quay.io/tiran/cpython_autoconf:271" container image (https://github.com/tiran/cpython_autoconf) is no longer used.

I can see using of aclocal 1.16.5 in this PR.
But in docs we write "Autoconf 2.71 and aclocal 1.16.4 are required to regenerate the configure script". (https://docs.python.org/3.14/using/configure.html#build-requirements).

Should we provide some changes to docs for update them?
@vstinner

Should we provide some changes to docs for update them?

Yes. Do you want to propose a PR to update the doc?

Yes. Do you want to propose a PR to update the doc?

Yes, I do.

bedevere-app Bot mentioned this pull request Nov 14, 2023

Consider avoiding the dependency on quay.io server in the GitHub Action "Check if generated files are up to date" job #112088

Closed

vstinner added the skip news label Nov 14, 2023

vstinner force-pushed the avoid_quay_io branch from 026ed0e to 7f3e933 Compare November 14, 2023 22:41

vstinner marked this pull request as ready for review November 14, 2023 23:36

vstinner requested review from corona10, erlend-aasland, ezio-melotti and hugovk as code owners November 14, 2023 23:36

bedevere-app Bot added the awaiting core review label Nov 14, 2023

gpshead approved these changes Nov 15, 2023

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels Nov 15, 2023

hugovk reviewed Nov 15, 2023

View reviewed changes

Comment thread Makefile.pre.in Outdated

vstinner force-pushed the avoid_quay_io branch from cc67cbd to e713876 Compare November 15, 2023 12:47

vstinner added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Nov 15, 2023

vstinner force-pushed the avoid_quay_io branch from 96437cb to 4f8987f Compare November 15, 2023 13:05

hugovk approved these changes Nov 15, 2023

View reviewed changes

erlend-aasland approved these changes Nov 15, 2023

View reviewed changes

vstinner removed the needs backport to 3.11 only security fixes label Nov 15, 2023

vstinner merged commit d9fd33a into python:main Nov 15, 2023

vstinner deleted the avoid_quay_io branch November 15, 2023 20:47

bedevere-app Bot removed the awaiting merge label Nov 15, 2023

This comment was marked as outdated.

Sign in to view

bedevere-app Bot removed the needs backport to 3.12 only security fixes label Nov 15, 2023

zware mentioned this pull request Nov 15, 2023

Use manifest for quay.io/tiran/cpython_autoconf instead of tag python/release-tools#75

Closed

This comment was marked as outdated.

Sign in to view

This comment was marked as duplicate.

Sign in to view

sethmlarson mentioned this pull request Nov 16, 2023

Backport regen-configure target to supported branches #112160

Closed

gh-112088: Run autoreconf in GHA check_generated_files by vstinner · Pull Request #112090 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-112088: Run autoreconf in GHA check_generated_files - #112090

gh-112088: Run autoreconf in GHA check_generated_files#112090vstinner merged 1 commit intopython:mainpython/cpython:mainfrom vstinner:avoid_quay_iovstinner/cpython:avoid_quay_ioCopy head branch name to clipboard

Conversation

vstinner commented Nov 14, 2023 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gpshead left a comment

Choose a reason for hiding this comment

Uh oh!

vstinner commented Nov 15, 2023

Uh oh!

hugovk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vstinner commented Nov 15, 2023

Uh oh!

vstinner commented Nov 15, 2023

Uh oh!

vstinner commented Nov 15, 2023

Uh oh!

vstinner commented Nov 15, 2023

Uh oh!

erlend-aasland commented Nov 15, 2023

Uh oh!

erlend-aasland left a comment

Choose a reason for hiding this comment

Uh oh!

erlend-aasland commented Nov 15, 2023

Uh oh!

vstinner commented Nov 15, 2023

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

vstinner commented Nov 15, 2023

Uh oh!

erlend-aasland commented Nov 15, 2023

Uh oh!

sethmlarson commented Nov 15, 2023

Uh oh!

gpshead commented Nov 15, 2023

Uh oh!

This comment was marked as outdated.

This comment was marked as duplicate.

efimov-mikhail commented Oct 14, 2024

Uh oh!

vstinner commented Oct 14, 2024

Uh oh!

efimov-mikhail commented Oct 14, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

gh-112088: Run autoreconf in GHA check_generated_files#112090
vstinner merged 1 commit into
python:mainfrom
vstinner:avoid_quay_io

vstinner commented Nov 14, 2023 •
edited by bedevere-app Bot
Loading