This repository was archived by the owner on Mar 31, 2026. It is now read-only.
Commit d90f0ee
authored
feat(experimental): add bidi stream retry manager. (#1632)
- Add bidi stream retry manager. It will the main class that would be
responsible for retries.
- Integrate MRD with bidi stream retry manager and reads resumption
strategy
- Added conformance tests for all error scenarios and smart resumption
for bidi reads
- Refactored MRD's `download_ranges()` method, which will now use retry
manager's `execute()` method to send the requests
- Modified the logic to parse and handle the
`BidiReadObjectRedirectedError`.
- Modified cloud build configuration to also pull the latest changes of
the current PR.
- Formatted few more files using black formatter1 parent a994dbf commit d90f0ee
16 files changed
Lines changed: 1274 additions & 307 deletions
File tree
- cloudbuild
- google/cloud/storage
- _experimental/asyncio
- retry
- _media/requests
- tests
- conformance
- unit
- asyncio
- retry
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -6,6 +6,7 @@ sudo apt-get update && sudo apt-get install -y git python3-pip python3-venv | |||
| 6 | 6 | # Clone the repository and checkout the specific commit from the build trigger. | |
| 7 | 7 | git clone https://github.com/googleapis/python-storage.git | |
| 8 | 8 | cd python-storage | |
| 9 | + git fetch origin "refs/pull/${_PR_NUMBER}/head" | ||
| 9 | 10 | git checkout ${COMMIT_SHA} | |
| 10 | 11 | ||
| 11 | 12 | ||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -68,7 +68,7 @@ steps: | |||
| 68 | 68 | # Execute the script on the VM via SSH. | |
| 69 | 69 | # Capture the exit code to ensure cleanup happens before the build fails. | |
| 70 | 70 | set +e | |
| 71 | - gcloud compute ssh ${_VM_NAME} --zone=${_ZONE} --internal-ip --ssh-key-file=/workspace/.ssh/google_compute_engine --command="ulimit -n {_ULIMIT}; COMMIT_SHA=${COMMIT_SHA} _ZONAL_BUCKET=${_ZONAL_BUCKET} bash run_zonal_tests.sh" | ||
| 71 | + gcloud compute ssh ${_VM_NAME} --zone=${_ZONE} --internal-ip --ssh-key-file=/workspace/.ssh/google_compute_engine --command="ulimit -n {_ULIMIT}; COMMIT_SHA=${COMMIT_SHA} _ZONAL_BUCKET=${_ZONAL_BUCKET} _PR_NUMBER=${_PR_NUMBER} bash run_zonal_tests.sh" | ||
| 72 | 72 | EXIT_CODE=$? | |
| 73 | 73 | set -e | |
| 74 | 74 | ||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -16,6 +16,7 @@ | |||
| 16 | 16 | ||
| 17 | 17 | from google.api_core import exceptions | |
| 18 | 18 | ||
| 19 | + | ||
| 19 | 20 | def raise_if_no_fast_crc32c(): | |
| 20 | 21 | """Check if the C-accelerated version of google-crc32c is available. | |
| 21 | 22 | ||
@@ -24,7 +25,7 @@ def raise_if_no_fast_crc32c(): | |||
| 24 | 25 | raises google.api_core.exceptions.FailedPrecondition: If the C extension is not available. | |
| 25 | 26 | returns: True if the C extension is available. | |
| 26 | 27 | rtype: bool | |
| 27 | - | ||
| 28 | + | ||
| 28 | 29 | """ | |
| 29 | 30 | if google_crc32c.implementation != "c": | |
| 30 | 31 | raise exceptions.FailedPrecondition( | |
0 commit comments