Repository navigation
bpo-45020: Freeze some of the modules imported during startup. - #28335
ericsnowcurrently merged 8 commits into
Conversation
Sorry, something went wrong.
|
Eric, I am going to do the test_query change separately, right now, so I can backport it and keep it in sync across versions. |
Sorry, something went wrong.
gvanrossum
left a comment
There was a problem hiding this comment.
However, wait for Terry and his separate fix for idle_test/test_query.py (which currently has a failing test, I'm not sure why).
Sorry, something went wrong.
gvanrossum
left a comment
There was a problem hiding this comment.
I wonder if this would ease the pain of generating all those modules a bit?
It reduces the number of scrollback lines by 2/3rd.
diff --git a/Tools/scripts/freeze_modules.py b/Tools/scripts/freeze_modules.py
index 0c2b826280..f3d821529a 100644
--- a/Tools/scripts/freeze_modules.py
+++ b/Tools/scripts/freeze_modules.py
@@ -511,9 +511,8 @@ def regen_makefile(modules):
# Note that we freeze the module to the target .h file
# instead of going through an intermediate file like we used to.
rules.append(f'{header}: Programs/_freeze_module {pyfile}')
- rules.append(f'\t$(srcdir)/Programs/_freeze_module {src.frozenid} \\')
- rules.append(f'\t\t$(srcdir)/{pyfile} \\')
- rules.append(f'\t\t$(srcdir)/{header}')
+ rules.append(f'\t$(srcdir)/Programs/_freeze_module {src.frozenid} '
+ f'$(srcdir)/{pyfile} $(srcdir)/{header}')
rules.append('')
frozenfiles[-1] = frozenfiles[-1].rstrip(" \\")
Sorry, something went wrong.
|
The reason there are failing test is that a) certain test files, such as asyncio and multiprocessing, are allowed to contain required randomly failing tests and b) when one of the github actions fail, all are rerun, giving bogus tests that passed on one run a chance to fail on the rerun. This is what happened to this time. I have now spent more time on these bogus failures than I did making the PR and I think this policy is rather rude. If there is a failure on a required test on the third run, either of you are welcome to hit the rerun button on the details page. Or merge if you can. I am off to sleep. |
Sorry, something went wrong.
|
That’s a problem. Could you file a bpo bug about the random failures? I
think we should fix the GitHub action’s approach to retrying *and* attempt
to fix those failing tests (I know from other large CI jobs that it is hard
to reach perfection in the latter).
On Tue, Sep 14, 2021 at 23:47 Terry Jan Reedy ***@***.***> wrote:
The reason there are failing test is that a) certain test files, such as
asyncio and multiprocessing, are allowed to contain required randomly
failing tests and b) when one of the github actions fail, *all* are
rerun, giving bogus tests that passed on one run a chance to fail on the
rerun. This is what happened to this time. I have now spent more time on
these bogus failures than I did making the PR and I think this policy is
rather rude. If there is a failure on a required test on the third run,
either of you are welcome to hit the rerun button on the details page. Or
merge if you can. I am off to sleep.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
<#28335 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAWCWMVSIFS5MVWECSLMILDUCA6P7ANCNFSM5EAQFRXQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
--Guido (mobile)
|
Sorry, something went wrong.
|
@terryjreedy, thanks for doing that. (FTR, PR gh-28344) |
Sorry, something went wrong.
Doing this provides significant performance gains for runtime startup (~15% with all the imported modules frozen). We don't yet freeze all the imported modules because there are a few hiccups in the build systems we need to sort out first. (See bpo-45186 and bpo-45188.)
Note that in PR GH-28320 we added a command-line flag (-X frozen_modules=[on|off]) that allows users to opt out of (or into) using frozen modules. The default is still "off" but we will change it to "on" as soon as we can do it in a way that does not cause contributors pain.
FYI, most of this PR was already reviewed in #28107. Also, almost all the changes in this PR are generated code (not generated: Tools/scripts/freeze_modules.py and Lib/test/*.py).
Here are things we'll be doing in follow-up PRs:
https://bugs.python.org/issue45020