Push batch / transaction onto a thread-local stack inside 'with'. by tseaver · Pull Request #518 · googleapis/google-cloud-python · GitHub

Prep. for #514.

Coverage remained the same when pulling e5e4518 on tseaver:push_batch_xact_on_stack_in_context_mgr into 77cfe10 on GoogleCloudPlatform:master.

Add explicit test of using the stack.

Coverage remained the same when pulling 8190f4c on tseaver:push_batch_xact_on_stack_in_context_mgr into 77cfe10 on GoogleCloudPlatform:master.

Nesting batches "works": it just "suspends" the outer batch until the inner one exits. It does mean that the "inner" changes will be sent to the back-end before the "outer" ones, but folks using Batch aren't supposed to care about that, right?

The problem I was trying to solve was to make discovery of any current batch / transaction straightforward, even if the application does weird stuff like nesting (maybe accidentally).

If nesting is not a desirable feature, then we can close this PR and do something like your suggestion instead.

Motivating example:

from gcloud.datastore.batch import _BATCHES, Batch

def put(*entities):
    current = _BATCHES.top
    in_batch = current is not None
    if not in_batch:
        current = Batch()
    for entity in entities:
        current.put(entity)
    if not in_batch:
        current.commit()

def delete(*keys):
    current = _BATCHES.top
    in_batch = current is not None
    if not in_batch:
        current = Batch()
    for key in keys:
        current.delete(key)
    if not in_batch:
        current.commit()

which will all Just Work(TM) even in the face of nesting.

I keep confusing myself. The key focus here is that with must be called and when __exit__ occurs the stack is happy. Hence there is no such thing as interleaving.

That said, I still think the maintenance cost of a stack is not worth the minimal gain in functionality (which many users may never see). Users motivated enough to want nested / conditional transactions can accomplish the same thing with the API provided byBatch. The complexity becomes their burden, not ours.

I'm torn between "it already works as intended" and "let's not add code that we don't need to".

That's actually why I made _BATCHES private (only capitalized as a sop to pylint :): Nobody but us chickens should touch it.

It's not us touching it that worries me, it's bugs from exotic usage in the wild that we don't / won't have bandwidth or desire to deal with.

For example, what if someone calls a method that uses a context manager under the hood, and they expect

with Transaction() as txn:
    funky_save()

to save in txn when instead it spawns a new transaction and doesn't save there.

As I said, I don't think the burden of a larger surface area justifies the small set of use cases that this enables syntactically.

I'm much more comfortable

Clearly documenting nesting is not allowed
Throwing an error if nesting is attempted

I don't see how your example is problematic: if funky_save manages its own transaction, separate from the surrounding one, their changes will still be saved, but just not "atomically" with txn's changes: that would be what they want, presumably (or they wouldn't code funky_save that way). We aren't doing damage by supporting nesting.

@silvolu Do the other mappings support nesting?

I'm not saying the functionality is damaging, I'm saying the maintenance burden on us is not worth it.

Hmm, the maintenance / explainability of the consistent / nested one seems easier to me than maintaining / explaining the "Highlander" case.

I suppose you're right. I'm probably just being paranoid. Let's hope it doesn't bite us.

LGTM

…ext_mgr Push batch / transaction onto a thread-local stack inside 'with'.

Also: - Changing api.get() back to accept only `keys` and returns only a list (a lot of headache for not much gain). - Factored out behavior to extract shared dataset_id from a set of keys into _get_dataset_id_from_keys(). - Updated docstrings and other tests that rely on changed / removed methods. See googleapis#518 for some context.

* chore: Prepare for mono repository migration * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

Source-Link: googleapis/synthtool@453a5d9 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:81ed5ecdfc7cac5b699ba4537376f3563f6f04122c4ec9e735d3b3dc1d43dd32

…p/templates/python_library/.kokoro (#518) Source-Link: https://togithub.com/googleapis/synthtool/commit/bb171351c3946d3c3c32e60f5f18cee8c464ec51 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f62c53736eccb0c4934a3ea9316e0d57696bb49c1a7c86c726e9bb8a2f87dadf

Source-Link: https://togithub.com/googleapis/synthtool/commit/352b9d4c068ce7c05908172af128b294073bf53c Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:3e3800bb100af5d7f9e810d48212b37812c1856d20ffeafb99ebe66461b61fc7

…pprove] (#518) Source-Link: https://togithub.com/googleapis/synthtool/commit/e3a1277ac35fc88c09db1930533e24292b132ced Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:452901c74a22f9b9a3bd02bce780b8e8805c97270d424684bff809ce5be8c2a2

Source-Link: googleapis/synthtool@7197a00 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:c43f1d918bcf817d337aa29ff833439494a158a0831508fda4ec75dc4c0d0320 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>

Co-authored-by: Anthonios Partheniou <partheniou@google.com>

Source-Link: googleapis/synthtool@1b71c10 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:00c9d764fd1cd56265f12a5ef4b99a0c9e87cf261018099141e2ca5158890416 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* updated CHANGELOG.md [ci skip] * updated setup.py Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

…ination (#518) * chore: Update gapic-generator-python to v1.18.4 PiperOrigin-RevId: 657207628 Source-Link: googleapis/googleapis@33fe71e Source-Link: googleapis/googleapis-gen@e02739d Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZTAyNzM5ZDEyMmVkMTViZDVlZjU3NzFjNTdmMTJhODNkNDdhMWRkYSJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

Fixes #518 We'll need to identify the longer term fix here, but this will unblock current CI failures

Fixes #518 Unpin pytest-asyncio version. Adjust the event_loop fixture to make it compatible with pytest-asyncio>=0.17 Use the @pytest_asyncio.fixture decorator for apropriately.

* updated CHANGELOG.md [ci skip] * updated setup.cfg [ci skip] * updated setup.py Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Closes #517

Push batch / transaction onto a thread-local stack inside 'with'.

e5e4518

Prep. for #514.

dhermes mentioned this pull request Jan 8, 2015

Decouple Transaction from Connection #514

Closed

8 tasks

dhermes reviewed Jan 8, 2015
View reviewed changes

Comment thread gcloud/datastore/batch.py

This comment was marked as spam.

Sign in to view

Give '_Batches' methods more natural names.

8190f4c

Add explicit test of using the stack.

tseaver added a commit that referenced this pull request Jan 8, 2015

Merge pull request #518 from tseaver/push_batch_xact_on_stack_in_cont…

7c01fcc

…ext_mgr Push batch / transaction onto a thread-local stack inside 'with'.

tseaver merged commit 7c01fcc into googleapis:master Jan 8, 2015

dhermes mentioned this pull request Jan 8, 2015

Removing dataset_id from allocate_ids and get_entities #510

Merged

tseaver deleted the push_batch_xact_on_stack_in_context_mgr branch January 8, 2015 22:14

dhermes mentioned this pull request Jan 8, 2015

Factor out mutation-setting logic from 'Connection'. #524

Merged

dhermes mentioned this pull request Jan 9, 2015

Adding api.delete() and removing Key.delete(). #527

Merged

dhermes added the api: datastore Issues related to the Datastore API. label Dec 31, 2015

parthea added a commit that referenced this pull request Aug 21, 2025

chore(deps): update dependency google-auth to v2.12.0 (#518 )

777206c

Co-authored-by: Anthonios Partheniou <partheniou@google.com>

parthea added a commit that referenced this pull request Sep 16, 2025

chore(deps): update dependency google-auth to v2.12.0 (#518 )

e377e59

Co-authored-by: Anthonios Partheniou <partheniou@google.com>

parthea pushed a commit that referenced this pull request Nov 24, 2025

chore: release 0.26.6 (#518 )

78f4611

* updated CHANGELOG.md [ci skip] * updated setup.py Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

parthea pushed a commit that referenced this pull request Nov 24, 2025

test: temporarily pin pytest-asyncio to < 0.17.0 (#522 )

49e9fa7

Fixes #518 We'll need to identify the longer term fix here, but this will unblock current CI failures

parthea pushed a commit that referenced this pull request Nov 26, 2025

chore: release 1.16.0 (#518 )

6d8a257

* updated CHANGELOG.md [ci skip] * updated setup.cfg [ci skip] * updated setup.py Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

parthea pushed a commit that referenced this pull request Mar 2, 2026

feat: add support for Python 3.10 (#518 )

9d6792f

parthea pushed a commit that referenced this pull request Mar 6, 2026

test: update system test for mtls (#518 )

bcdc44a

parthea pushed a commit that referenced this pull request Mar 9, 2026

chore(deps): update dependency pyjwt to v2.10.0 (#518 )

9da0fde

parthea pushed a commit that referenced this pull request Mar 9, 2026

tests: use 120 sec timeout for 'blob.delete' during teardown (#518 )

f5fc1e1

Closes #517

Push batch / transaction onto a thread-local stack inside 'with'. by tseaver · Pull Request #518 · googleapis/google-cloud-python · GitHub

Uh oh!

Push batch / transaction onto a thread-local stack inside 'with'. - #518

Push batch / transaction onto a thread-local stack inside 'with'.#518
tseaver merged 2 commits into
googleapis:masterfrom
tseaver:push_batch_xact_on_stack_in_context_mgr

tseaver commented Jan 8, 2015

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

This comment was marked as spam.

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Push batch / transaction onto a thread-local stack inside 'with'. by tseaver · Pull Request #518 · googleapis/google-cloud-python · GitHub

Uh oh!

Push batch / transaction onto a thread-local stack inside 'with'. - #518

Push batch / transaction onto a thread-local stack inside 'with'.#518tseaver merged 2 commits intogoogleapis:mastergoogleapis/google-cloud-python:masterfrom tseaver:push_batch_xact_on_stack_in_context_mgrCopy head branch name to clipboard

Conversation

tseaver commented Jan 8, 2015

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

This comment was marked as spam.

Uh oh!

coveralls commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

tseaver commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

dhermes commented Jan 8, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Push batch / transaction onto a thread-local stack inside 'with'.#518
tseaver merged 2 commits into
googleapis:masterfrom
tseaver:push_batch_xact_on_stack_in_context_mgr