FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

fix: Make `download_ranges` compatible with `asyncio.create_task(..)` by chandra-siri · Pull Request #1591 · googleapis/python-storage · GitHub

This repository was archived by the owner on Mar 31, 2026. It is now read-only.
/ python-storage Public archive

fix: Make download_ranges compatible with asyncio.create_task(..) - #1591

Merged
chandra-siri merged 7 commits into
mainfrom
concurrency_for_download_ranges
Oct 29, 2025
Merged

fix: Make download_ranges compatible with asyncio.create_task(..)#1591
chandra-siri merged 7 commits into
mainfrom
concurrency_for_download_ranges

Conversation

chandra-siri commented Oct 27, 2025
edited
Loading

Copy link
Copy Markdown
Collaborator

Multiple download_ranges should be able to run via multiple async tasks in a single event loop.

ie when a user runs a code like below , it works

... # some code 
await mrd.download_ranges(...)
await mrd.download_ranges()

but when a user runs code like below it doesn't work. This PR is fixing this issue. More details in internal b/449916743

... # some code 
task1 = asyncio.create_task(mrd.download_ranges(..))
task2 = asyncio.create_task(mrd.download_ranges(..))
_ = await asyncio.gather(task1, task2)

This PR achieves the above goal by

  • converts read_id_to_writable_buffer_dict into a class attribute instead of local variable of download_ranges
  • makes read_id random int, instead of ordered i + j
  • Provide an id (random int) for each method.
  • Add two more maps for keeping track of read_id to function_id and buffer
  • Because reads on client-to-server stream can be made one at a time, hence asyncio.lock is used.

product-auto-label Bot added size: m Pull request size is medium. api: storage Issues related to the googleapis/python-storage API. labels Oct 27, 2025
chandra-siri force-pushed the concurrency_for_download_ranges branch from 58fdf9d to 76b2327 Compare October 28, 2025 10:39
chandra-siri changed the title Concurrency support for download_ranges fix: "download_ranges" should be able to run from multiple coroutines Oct 28, 2025
chandra-siri marked this pull request as ready for review October 28, 2025 11:34
chandra-siri requested review from a team October 28, 2025 11:34
chandra-siri changed the title fix: "download_ranges" should be able to run from multiple coroutines fix: Make download_ranges compatible with asyncio.create_task Oct 28, 2025
chandra-siri changed the title fix: Make download_ranges compatible with asyncio.create_task fix: Make download_ranges compatible with asyncio.create_task(..) Oct 28, 2025
Pulkit0110 previously approved these changes Oct 28, 2025
chandra-siri merged commit faf8b83 into main Oct 29, 2025
15 checks passed
chandra-siri deleted the concurrency_for_download_ranges branch October 29, 2025 06:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: storage Issues related to the googleapis/python-storage API. size: m Pull request size is medium.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL