FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat(storage): add option to disable checksums and improve robustness of full_object_checksum validation by chandra-siri · Pull Request #17665 · googleapis/google-cloud-python · GitHub

/ google-cloud-python Public

feat(storage): add option to disable checksums and improve robustness of full_object_checksum validation - #17665

Merged
chandra-siri merged 19 commits into
googleapis:mainfrom
chandra-siri:feat/appendable-checksum-robustness
Jul 10, 2026
ConversationCommits (19)ChecksFiles changed
Merged

feat(storage): add option to disable checksums and improve robustness of full_object_checksum validation#17665
chandra-siri merged 19 commits into
googleapis:mainfrom
chandra-siri:feat/appendable-checksum-robustness

Conversation

chandra-siri commented Jul 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

This PR introduces performance options and robustness improvements to the asynchronous append operations in AsyncAppendableObjectWriter.

Key Changes:

  1. Disable Checksums for Performance:

    • Added the enable_checksum optional parameter (default: True) to AsyncAppendableObjectWriter.append().
    • When set to False, the writer skips calculation of chunk-level CRC32C checksums in _WriteResumptionStrategy, improving append throughput.
    • Updated unit tests to verify that chunk-level checksums are omitted when disabled.

  2. Robustness of Finalization Checksum Validation:

    • Implemented type checking (isinstance(int)) and range checking ([0, 2**32-1]) for full_object_checksum in finalize().
    • Wrapped stream receiving and response processing inside a try...finally block. This guarantees that the underlying stream is closed and local writer state variables are reset on mismatch exceptions/errors, preventing stream leaks.
    • Simplified the finalization request building code into a clean if-elif-else format.
    • Added corresponding unit tests verifying validation error exceptions and proper stream cleanup.

chandra-siri added 11 commits July 8, 2026 06:58
feat(storage): support full_object_checksum in AsyncAppendableObjectW…
78d500d 
…riter
test(storage): add unit tests for AsyncAppendableObjectWriter checksu…
b231244 
…m validation
test(storage): add integration tests for AsyncAppendableObjectWriter …
5b518be 
…checksum validation
style: run ruff format on test_async_appendable_object_writer.py
27933da
refactor(storage): conditionally attach object_checksums to finalizat…
22ffa69 
…ion request
refactor(storage): replace warning with comment when checksum is None
7a58986
Revert "refactor(storage): replace warning with comment when checksum…
9e02476 
… is None"

This reverts commit 7a58986.
test(storage): assert exceptions.InvalidArgument on checksum mismatch…
063142b 
… in integration tests
feat(storage): validate type and range of full_object_checksum
7dd7cd8
fix(storage): close stream and reset local state in finalize on error
0a73153
feat(storage): allow option to disable checksums for faster async app…
f7b8e4c 
…end operations
chandra-siri requested a review from a team as a code owner July 8, 2026 14:02

gemini-code-assist Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

chandra-siri added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jul 8, 2026

chandra-siri commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

review / submit after #17658

snippet-bot Bot commented Jul 9, 2026 •
edited
Loading

Copy link
Copy Markdown

No region tags are edited in this PR.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

chandra-siri changed the title feat(storage): support full_object_checksum and add option to disable checksums in AsyncAppendableObjectWriter feat(storage): add option to disable checksums and improve robustness of full_object_checksum validation Jul 9, 2026
chandra-siri removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jul 9, 2026

chandra-siri commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

/gemini review

gemini-code-assist Bot reviewed Jul 9, 2026
View reviewed changes

gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Code Review

This pull request introduces an enable_checksum parameter to control CRC32C checksum calculations during async object writes, adds validation for full_object_checksum in finalize(), and ensures proper stream cleanup. Key feedback includes removing undocumented parameters from the download_ranges docstring, wrapping both send() and recv() in the try...finally block for robust cleanup, and explicitly rejecting boolean values in the full_object_checksum type check along with adding a corresponding unit test.

Comment thread packages/google-cloud-storage/google/cloud/storage/asyncio/async_multi_range_downloader.py
Comment thread packages/google-cloud-storage/google/cloud/storage/asyncio/async_appendable_object_writer.py Outdated
Comment thread packages/google-cloud-storage/google/cloud/storage/asyncio/async_appendable_object_writer.py Outdated
Comment thread packages/google-cloud-storage/tests/unit/asyncio/test_async_appendable_object_writer.py
chandra-siri enabled auto-merge (squash) July 9, 2026 15:26
chandra-siri merged commit a5a717d into googleapis:main Jul 10, 2026
30 checks passed
release-please Bot mentioned this pull request Jul 10, 2026
Merged
noahdietz pushed a commit that referenced this pull request Jul 13, 2026
chore: release main (#17646)
7c18c24 
🤖 I have created a release *beep* *boop*
---


<details><summary>google-apps-chat: 0.10.2</summary>

##
[0.10.2](google-apps-chat-v0.10.1...google-apps-chat-v0.10.2)
(2026-07-13)


### Features

* update googleapis and regenerate
([#17678](#17678))
([670917c](670917c))
</details>

<details><summary>google-auth: 2.56.0</summary>

##
[2.56.0](google-auth-v2.55.2...google-auth-v2.56.0)
(2026-07-13)


### Features

* **auth:** Implement python mtls helpers
([#17495](#17495))
([e7baed1](e7baed1))


### Bug Fixes

* **google-auth:** add aiohttp bound for Python 3.14
([#17654](#17654))
([b2ec761](b2ec761))
* **google-auth:** add bounds for urllib3 and packaging dependencies
([#17647](#17647))
([e9c6265](e9c6265))
* **google-auth:** add cryptography bound for Python 3.14
([#17649](#17649))
([e9ca4c4](e9ca4c4))
* **google-auth:** add gRPC extra and clean up obsolete TODOs
([#17644](#17644))
([6da41e8](6da41e8)),
closes
[#1735](#1735)
[#1736](#1736)
[#1739](#1739)
* **google-auth:** raise rsa extra lower bound to 4.0
([#17652](#17652))
([4087828](4087828))
</details>

<details><summary>google-cloud-apigee-registry: 0.10.1</summary>

##
[0.10.1](google-cloud-apigee-registry-v0.10.0...google-cloud-apigee-registry-v0.10.1)
(2026-07-13)


### Features

* update googleapis and regenerate
([#17678](#17678))
([670917c](670917c))
</details>

<details><summary>google-cloud-binary-authorization: 1.19.0</summary>

##
[1.19.0](google-cloud-binary-authorization-v1.18.0...google-cloud-binary-authorization-v1.19.0)
(2026-07-13)


### Features

* update googleapis and regenerate
([#17678](#17678))
([670917c](670917c))
</details>

<details><summary>google-cloud-documentai-toolbox: 0.17.1</summary>

##
[0.17.1](google-cloud-documentai-toolbox-v0.17.0...google-cloud-documentai-toolbox-v0.17.1)
(2026-07-13)


### Bug Fixes

* bump jinja2 from 3.1.0 to 3.1.6 in
/packages/google-cloud-documentai-toolbox
([#17630](#17630))
([4cfb931](4cfb931))
* bump pillow from 10.0.0 to 12.2.0 in
/packages/google-cloud-documentai-toolbox
([#17631](#17631))
([1838c35](1838c35))
</details>

<details><summary>google-cloud-memorystore: 0.5.2</summary>

##
[0.5.2](google-cloud-memorystore-v0.5.1...google-cloud-memorystore-v0.5.2)
(2026-07-13)


### Features

* update googleapis and regenerate
([#17678](#17678))
([670917c](670917c))
</details>

<details><summary>google-cloud-storage: 3.13.0</summary>

##
[3.13.0](google-cloud-storage-v3.12.1...google-cloud-storage-v3.13.0)
(2026-07-13)


### Features

* **storage:** add option to disable checksums and improve robustness of
full_object_checksum validation
([#17665](#17665))
([a5a717d](a5a717d))
* **storage:** support full_object_checksum in
AsyncAppendableObjectWriter
([#17658](#17658))
([e08d5ca](e08d5ca))
</details>

<details><summary>google-devicesandservices-health: 0.1.1</summary>

##
[0.1.1](google-devicesandservices-health-v0.1.0...google-devicesandservices-health-v0.1.1)
(2026-07-13)


### Features

* update googleapis and regenerate
([#17678](#17678))
([670917c](670917c))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

v-pratap v-pratap approved these changes
krishnamd-jkp Awaiting requested review from krishnamd-jkp
+1 more reviewer
gemini-code-assist[bot] gemini-code-assist[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

Back | FazBrowse Home | New Git URL