| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| import time | ||
| import unittest | ||
| from threading import Thread, Barrier, iter_locked | ||
| from test.support import threading_helper | ||
|
|
||
|
|
||
| threading_helper.requires_working_threading(module=True) | ||
|
|
||
| class non_atomic_iterator: | ||
|
|
||
| def __init__(self, it): | ||
| self.it = iter(it) | ||
|
|
||
| def __iter__(self): | ||
| return self | ||
|
|
||
| def __next__(self): | ||
| a = next(self.it) | ||
| t = time.perf_counter() + 1e-6 | ||
| while time.perf_counter() < t: | ||
| pass | ||
| b = next(self.it) | ||
|
Comment thread
Copy link
Copy Markdown
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityAdd time.sleep(0) (or with small non-zero argument) or Event.wait() with a timeout to force or significantly increase the chance of concurrent access.
Sorry, something went wrong.
eendebakpt reacted with thumbs up emoji
All reactions
|
||
| return a, b | ||
|
|
||
| def count(): | ||
| i = 0 | ||
| while True: | ||
| i += 1 | ||
| yield i | ||
|
|
||
| class iter_lockedThreading(unittest.TestCase): | ||
|
|
||
| @threading_helper.reap_threads | ||
| def test_iter_locked(self): | ||
| number_of_threads = 10 | ||
| number_of_iterations = 8 | ||
| barrier = Barrier(number_of_threads) | ||
| def work(it): | ||
| while True: | ||
| try: | ||
| a, b = next(it) | ||
| assert a + 1 == b | ||
| except StopIteration: | ||
| break | ||
|
|
||
| data = tuple(range(400)) | ||
| for it in range(number_of_iterations): | ||
| iter_locked_iterator = iter_locked(non_atomic_iterator(data,)) | ||
| worker_threads = [] | ||
| for ii in range(number_of_threads): | ||
| worker_threads.append( | ||
| Thread(target=work, args=[iter_locked_iterator])) | ||
|
|
||
| with threading_helper.start_threads(worker_threads): | ||
| pass | ||
|
|
||
| barrier.reset() | ||
|
|
||
| @threading_helper.reap_threads | ||
| def test_iter_locked_generator(self): | ||
| number_of_threads = 5 | ||
| number_of_iterations = 4 | ||
| barrier = Barrier(number_of_threads) | ||
| def work(it): | ||
| barrier.wait() | ||
| for _ in range(1_000): | ||
| try: | ||
| next(it) | ||
| except StopIteration: | ||
| break | ||
|
|
||
| for it in range(number_of_iterations): | ||
| generator = iter_locked(count()) | ||
| worker_threads = [] | ||
| for ii in range(number_of_threads): | ||
| worker_threads.append( | ||
| Thread(target=work, args=[generator])) | ||
|
|
||
| with threading_helper.start_threads(worker_threads): | ||
| pass | ||
|
|
||
| barrier.reset() | ||
|
|
||
| if __name__ == "__main__": | ||
| unittest.main() | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Add :meth:`threading.iter_locked` to make concurrent iteration over an iterable execute using a lock. |
| Back | FazBrowse Home | New Git URL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityI'm not sure non_atomic_iterator is a good example of something that exists in the wild, because it's not thread-safe on GIL-ful builds either. The thread can arbitrarily switch during execution of Python code, so this would need a lock anyway if concurrent iteration were a use case. For free-threading, __next__ itself should have its own locking to prevent races. How was this design reached?
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityThe iter_locked allows the non_atomic_iterator to be used in a threaded-setting, whether in the normal or FT build. Do you have suggestions for other tests?
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.