FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-117636: Remove redundant type check in `os.path.join()` by nineteendo · Pull Request #117638 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

gh-117636: Remove redundant type check in os.path.join() - #117638

Merged
hauntsaninja merged 9 commits into
python:mainfrom
nineteendo:speedup-os.path.join
Apr 14, 2024
ConversationCommits (9)ChecksFiles changed
Merged

gh-117636: Remove redundant type check in os.path.join()#117638
hauntsaninja merged 9 commits into
python:mainfrom
nineteendo:speedup-os.path.join

Conversation

nineteendo commented Apr 8, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmark:

ntpath.py

script 
::test.bat
@echo off
echo 1 item && python -m timeit -s "import before.ntpath" "before.ntpath.join('foo')" && python -m timeit -s "import after.ntpath" "after.ntpath.join('foo')"
echo 10 items && python -m timeit -s "import before.ntpath; paths = ['foo'] * 10" "before.ntpath.join(*paths)" && python -m timeit -s "import after.ntpath; paths = ['foo'] * 10" "after.ntpath.join(*paths)"
echo 100 items && python -m timeit -s "import before.ntpath; paths = ['foo'] * 100" "before.ntpath.join(*paths)" && python -m timeit -s "import after.ntpath; paths = ['foo'] * 100" "after.ntpath.join(*paths)"
1 item
500000 loops, best of 5: 699 nsec per loop # before
500000 loops, best of 5: 648 nsec per loop # after
# -> 1.08x faster
10 items
50000 loops, best of 5: 5.48 usec per loop # before
50000 loops, best of 5: 5.52 usec per loop # after
# -> no difference
100 items
5000 loops, best of 5: 54.1 usec per loop # before
5000 loops, best of 5: 54 usec per loop # after
# -> no difference

posixpath.py

script 
# test.sh
echo 1 item && python -m timeit -s "import before.posixpath" "before.posixpath.join('foo')" && python -m timeit -s "import after.posixpath" "after.posixpath.join('foo')"
echo 10 items && python -m timeit -s "import before.posixpath; paths = ['foo'] * 10" "before.posixpath.join(*paths)" && python -m timeit -s "import after.posixpath; paths = ['foo'] * 10" "after.posixpath.join(*paths)"
echo 100 items && python -m timeit -s "import before.posixpath; paths = ['foo'] * 100" "before.posixpath.join(*paths)" && python -m timeit -s "import after.posixpath; paths = ['foo'] * 100" "after.posixpath.join(*paths)"
1 item
1000000 loops, best of 5: 335 nsec per loop # before
1000000 loops, best of 5: 271 nsec per loop # after
# -> 1.24x faster
10 items
50000 loops, best of 5: 3.43 usec per loop # before
100000 loops, best of 5: 3.35 usec per loop # after
# -> 1.02x faster
100 items
10000 loops, best of 5: 34.4 usec per loop # before
10000 loops, best of 5: 34.1 usec per loop # after
# -> no difference

bedevere-app Bot mentioned this pull request Apr 8, 2024
Closed
nineteendo marked this pull request as ready for review April 9, 2024 05:52

nineteendo commented Apr 9, 2024

Copy link
Copy Markdown
Contributor Author

@serhiy-storchaka, you're the one who added this check, so you can probably best evaluate if it's still needed.

serhiy-storchaka commented Apr 9, 2024

Copy link
Copy Markdown
Member

The first check was for cases like join(None) or join(['a', 'b']). The second checks was for cases like join('', b'a'), join(b'', 'a'), join(None, 'a'), or something like this.

It seems that after adding fspath() they are no longer needed.

erlend-aasland linked an issue Apr 9, 2024 that may be closed by this pull request
Remove redundant type check in os.path.join() #117636
Closed

nineteendo commented Apr 10, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Apparently this unnecessary concatenation has been here since the first iteration of posixpath! So it wasn't even intended as a type check:

cpython/Lib/posixpath.py

Lines 11 to 14 in c636014

def cat(a, b):
if b[:1] = '/': return b
if a = '' or a[-1:] = '/': return a + b
return a + '/' + b

hauntsaninja approved these changes Apr 13, 2024
View reviewed changes

hauntsaninja left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Thank you!

nineteendo commented Apr 13, 2024

Copy link
Copy Markdown
Contributor Author

@serhiy-storchaka, can this be merged?

hauntsaninja merged commit 9ee94d1 into python:main Apr 14, 2024
nineteendo deleted the speedup-os.path.join branch April 14, 2024 21:11
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
pythongh-117636: Remove redundant type check in os.path.join() (pyt…
c946fa0 
…hon#117638)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

hauntsaninja hauntsaninja approved these changes
serhiy-storchaka Awaiting requested review from serhiy-storchaka

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Remove redundant type check in os.path.join()

3 participants

Back | FazBrowse Home | New Git URL