FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

GH-119169: Simplify `os.walk()` exception handling by barneygale · Pull Request #121435 · python/cpython · GitHub

/ cpython Public

GH-119169: Simplify os.walk() exception handling - #121435

Merged
barneygale merged 1 commit into
python:mainfrom
barneygale:gh-119186-use-for-loop
Jul 8, 2024
ConversationCommits (1)ChecksFiles changed
Merged

GH-119169: Simplify os.walk() exception handling#121435
barneygale merged 1 commit into
python:mainfrom
barneygale:gh-119186-use-for-loop

Conversation

barneygale commented Jul 6, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Handle errors from os.scandir() and ScandirIterator similarly, which lets us loop over directory entries with for.

$ ./python -m timeit -s 'import os' 'list(os.walk("."))'
5 loops, best of 5: 41.5 msec per loop
5 loops, best of 5: 40.1 msec per loop
# --> 3.5% faster

pythonGH-119186: Slightly speed up os.walk()
abce81c 
Handle errors from `os.scandir()` and `ScandirIterator` similarly, which
lets us loop over directory entries with `for`.
barneygale added the performance Performance or resource usage label Jul 6, 2024
barneygale changed the title GH-119186: Simplify os.walk() exception handling GH-119169: Simplify os.walk() exception handling Jul 6, 2024
bedevere-app Bot mentioned this pull request Jul 6, 2024
Open
serhiy-storchaka approved these changes Jul 8, 2024
View reviewed changes

serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM.

This is not so much optimization as simplification.

barneygale commented Jul 8, 2024

Copy link
Copy Markdown
Contributor Author

Thanks Serhiy

barneygale merged commit db00fee into python:main Jul 8, 2024

bedevere-bot commented Jul 8, 2024

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 LTO + PGO 3.x has failed when building commit db00fee.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/568/builds/6911) and take a look at the build logs.
  4. Check if the failure is related to this commit (db00fee) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/568/builds/6911

Failed tests:

  • test_smtplib

Failed subtests:

  • test_with_statement_QUIT_failure - test.test_smtplib.SMTPSimTests.test_with_statement_QUIT_failure
  • test_with_statement - test.test_smtplib.SMTPSimTests.test_with_statement

Summary of the results of the build (if available):

==

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/test/test_smtplib.py", line 1243, in test_with_statement_QUIT_failure
    smtp.noop()
    ~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 523, in noop
    return self.docmd("noop")
           ~~~~~~~~~~^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 430, in docmd
    return self.getreply()
           ~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 403, in getreply
    raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/test/test_smtplib.py", line 1233, in test_with_statement
    code, message = smtp.noop()
                    ~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 523, in noop
    return self.docmd("noop")
           ~~~~~~~~~~^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 430, in docmd
    return self.getreply()
           ~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.lto-pgo/build/Lib/smtplib.py", line 403, in getreply
    raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed

bedevere-bot commented Jul 8, 2024

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 FIPS Only Blake2 Builtin Hash 3.x has failed when building commit db00fee.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/469/builds/8522) and take a look at the build logs.
  4. Check if the failure is related to this commit (db00fee) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/469/builds/8522

Failed tests:

  • test_smtplib

Failed subtests:

  • test_with_statement_QUIT_failure - test.test_smtplib.SMTPSimTests.test_with_statement_QUIT_failure
  • test_with_statement - test.test_smtplib.SMTPSimTests.test_with_statement

Summary of the results of the build (if available):

==

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_smtplib.py", line 1243, in test_with_statement_QUIT_failure
    smtp.noop()
    ~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 523, in noop
    return self.docmd("noop")
           ~~~~~~~~~~^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 430, in docmd
    return self.getreply()
           ~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 403, in getreply
    raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_smtplib.py", line 1233, in test_with_statement
    code, message = smtp.noop()
                    ~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 523, in noop
    return self.docmd("noop")
           ~~~~~~~~~~^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 430, in docmd
    return self.getreply()
           ~~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/smtplib.py", line 403, in getreply
    raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed

noahbkim pushed a commit to hudson-trading/cpython that referenced this pull request Jul 11, 2024
pythonGH-119169: Simplify os.walk() exception handling (python#121435)
946baeb 
Handle errors from `os.scandir()` and `ScandirIterator` similarly, which
lets us loop over directory entries with `for`.
estyxx pushed a commit to estyxx/cpython that referenced this pull request Jul 17, 2024
pythonGH-119169: Simplify os.walk() exception handling (python#121435)
7a70032 
Handle errors from `os.scandir()` and `ScandirIterator` similarly, which
lets us loop over directory entries with `for`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

serhiy-storchaka serhiy-storchaka approved these changes

Assignees

No one assigned

Labels

performance Performance or resource usage

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

Back | FazBrowse Home | New Git URL