FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-124397: Add free-threading support for iterators. by rhettinger · Pull Request #148894 · python/cpython · GitHub

Repository navigation

gh-124397: Add free-threading support for iterators. - #148894

Merged
rhettinger merged 25 commits into
python:mainfrom
rhettinger:iterator_synchronization
May 1, 2026
Merged

rhettinger merged 25 commits into
python:mainfrom
rhettinger:iterator_synchronization

Conversation

rhettinger commented Apr 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

rhettinger added the type-feature A feature request or enhancement label Apr 22, 2026
rhettinger changed the title Issue-124397: Add free-threading support for iterators. gh-124397: Add free-threading support for iterators. Apr 22, 2026
rhettinger marked this pull request as draft April 22, 2026 22:39
rhettinger requested a review from colesbury April 23, 2026 02:00
rhettinger force-pushed the iterator_synchronization branch from 21fca13 to 4c2bad0 Compare April 23, 2026 02:33
rhettinger marked this pull request as ready for review April 23, 2026 02:52

colesbury left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM

Comment thread Doc/library/threading.rst Outdated
Comment thread Doc/library/threading.rst
Comment thread Lib/threading.py Outdated
Comment thread Lib/threading.py Outdated
Comment thread Lib/threading.py Outdated
Comment thread Lib/threading.py
Comment thread Lib/threading.py Outdated
Comment thread Doc/library/threading.rst Outdated

read-the-docs-community Bot commented Apr 28, 2026 •
edited
Loading

Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #32503627 | 📁 Comparing fbae726 against main (76b3923)

  🔍 Preview build  

69 files changed · + 1 added · ± 68 modified

+ Added

± Modified

Copy link
Copy Markdown
Member

I concur with @ZeroIntensity who wrote that threading is a bad place for this. I'm not sure that threading is a good home for the 3 added functions (serialize, synchronized, concurrent_tee). IMO itertools would be a better home for them.

For example, I would expect @threading.synchronized (which sounds like synchronized in Java) to work on any function, such as:

import threading

class Demo:
    _next_id = 0

    @threading.synchronized
    def get_id(self):
        self._next_id += 1
        return self._next_id

But calling get_id() fails with:

Traceback (most recent call last):
  File "/home/vstinner/python/main/x.py", line 12, in <module>
    print(demo.get_id())
          ~~~~~~~~~~~^^
  File "/home/vstinner/python/main/Lib/threading.py", line 930, in inner
    return serialize(iterator)
  File "/home/vstinner/python/main/Lib/threading.py", line 869, in __init__
    self._iterator = iter(iterable)
                     ~~~~^^^^^^^^^^
TypeError: 'int' object is not iterable

concurrent_tee() sounds very specific, whereas currently threading only contains very generic tooling to write multithreaded code. itertools contains many functions, so should we expect to pop up more itertools functions in threading? As an user, it's uneasy for me to choose between threading.concurrent_tee() and itertools.tee() who are in two different modules. I would expect them to be in the same module, and I expect itertools.tee() to mention concurrent_tee() and explain when it should be used instead.

Copy link
Copy Markdown
Contributor Author

Here are some GRASP-like thoughts on which module these should go into.

These are only useful in the context of multithreaded programs. The threading module is where people look for tools to manage and synchronize concurrency conflicts.

The implementation of the tools is almost entirely about lock management and synchronized result buffering. There is almost no logic that isn't threading related.

In contrast, the itertools module is scoped to only include general purpose tools for creating iterators. Nothing in it has any thread management. Most of the tools are roughly equivalent to just a plain generator.

ZeroIntensity commented Apr 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

I'd be able to live with these existing under threading, but as Victor said, serialize and synchronized are extremely misleading names when in the threading module. itertools.serialize and itertools.synchronized make sense because itertools implies that they're about iteration; on the other hand, threading does not. There isn't anything in their names that distinguishes these from general concurrency ideas.

If we're going to keep these under threading, let's call them something like serialize_iterator and synchronized_iterator.

rhettinger requested a review from AA-Turner as a code owner April 30, 2026 21:11

ZeroIntensity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Thanks for humoring me.

@vstinner, are you comfortable with this living under threading, as long as "iterator" is in the name?

Comment thread Doc/library/threading.rst
Comment thread Lib/threading.py Outdated

ZeroIntensity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM, thank you!

The failing CI is due to a Ubuntu outage.

Comment thread Lib/threading.py Outdated
Comment thread Lib/threading.py Outdated
Comment thread Doc/library/threading.rst Outdated
Comment thread Lib/threading.py Outdated
rhettinger and others added 6 commits May 1, 2026 13:57
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
rhettinger merged commit 91e871a into python:main May 1, 2026
51 checks passed
rhettinger deleted the iterator_synchronization branch May 1, 2026 21:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic-free-threading type-feature A feature request or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants


Back | FazBrowse Home | New Git URL