gh-146452: Improve locking granularity in pickle's batch_dict_exact by scopreon · Pull Request #150025 · python/cpython · GitHub

Improve critical section locking in batch_dict_exact. This originally held a lock for the entirey of batch_dict_exact_impl, which is not necessary. We can instead have localised locks for sections which require it. It wrapped possibly expensive calls to save() etc

Remove batch_dict_exact_impl which existed to allow the wrapping with critical sections. Called by batch_dict_exact.

Additionally handle the case where PyDict_Next returns 0 in the size 1 special case (modified during pickling).

Note: this spun out of looking into #149816 89, it was fixed in #146452.

Test was added in https://github.com/python/cpython/pull/146470/changes

Issue: Segfault in _pickle.c:batch_dict_exact when pickling dict with concurrent mutation (free-threading) #146452

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

All commit authors signed the Contributor License Agreement.

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

…ing) Replace the coarse critical section wrapping the entire function with fine-grained sections covering only PyDict_Next + Py_INCREF. Also handle PyDict_Next returning 0 in the single-item fast path.

We discussed this in person. This assertion could fail if the dictionary gets mutated by another thread between the size check in the caller and this check. The code should work correctly in this case (it will just emit a SETITEMS that sets 0 items), so removing the assertion is the right change.

Thanks @scopreon for the PR, and @JelleZijlstra for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

Sorry, @scopreon and @JelleZijlstra, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 57a0e570d36f41b953a91bbaf4262a5d05d0391b 3.14

GH-150039 is a backport of this pull request to the 3.15 branch.

…exact and fix race condition (GH-150025) (#150039) gh-146452: Improve locking granularity in pickle's batch_dict_exact and fix race condition (GH-150025) Remove assertion that could fail in rare race condition. Replace the coarse critical section wrapping the entire function with fine-grained sections covering only PyDict_Next + Py_INCREF. Also handle PyDict_Next returning 0 in the single-item fast path. (cherry picked from commit 57a0e57) Co-authored-by: Saul Cooperman <58375603+scopreon@users.noreply.github.com>

GH-150062 is a backport of this pull request to the 3.14 branch.

#150062) [3.14] gh-146452: Improve locking granularity in pickle's batch_dict_exact and fix race condition (GH-150025) Remove assertion that could fail in rare race condition. Replace the coarse critical section wrapping the entire function with fine-grained sections covering only PyDict_Next + Py_INCREF. Also handle PyDict_Next returning 0 in the single-item fast path. (cherry picked from commit 57a0e57)

I propose to revert these changes, since they cause Out of Memory on the main branch when running test_free_threading.test_pickle. Because of that, the GitHub Action CI jobs fail with exit code 143 (issue gh-150114).

I don't think that this change is correct: sometimes (race condition), test_pickle_dumps_with_concurrent_dict_mutation() of test_free_threading.test_pickle enters an unlimited loop and exhaust the system memory. See the issue gh-150257.

The following code was added to _pickle to fix the crash gh-146452:

/* gh-146452: Wrap the dict iteration in a critical section to prevent
   concurrent mutation from invalidating PyDict_Next() iteration state. */
static int
batch_dict_exact(PickleState *state, PicklerObject *self, PyObject *obj)
{
    int ret;
    Py_BEGIN_CRITICAL_SECTION(obj);
    ret = batch_dict_exact_impl(state, self, obj);
    Py_END_CRITICAL_SECTION();
    return ret;
}

This PR replaces this function with "fine-grained sections".

Problem: Py_BEGIN_CRITICAL_SECTION(obj); and Py_END_CRITICAL_SECTION(); is put around a single PyDict_Next() call, whereas the PyDict_Next() documentation recommends putting the whole loop in the critical section.

By the way, the 3.15 backport of this PR is different since there was not batch_dict_exact_impl() function. Maybe a 3.15 backport of the previous fix was missed?

I prepared 3 reverts:

main: Revert "gh-146452: Improve locking granularity in pickle's batch_dict_exact and fix race condition (#150025)" #150261
3.15: [3.15] Revert "[3.15] gh-146452: Improve locking granularity in pickle's batch_dict_exact and fix race condition (GH-150025) (#150039)" #150262
3.14: [3.14] Revert "[3.14] gh-146452: Improve locking granularity in pickle's batch_dict_… (#150062)" #150263

Once the 3.15 revert will be merged, the commit e62a611 should be backported to the 3.15 branch.

…_exact and fix race condition (#150025)" (#150261) This reverts commit 57a0e57.

…e's batch_dict_exact and fix race condition (GH-150025) (#150039)" (#150262) Revert "[3.15] gh-146452: Improve locking granularity in pickle's batch_dict_exact and fix race condition (GH-150025) (#150039)" This reverts commit 66ade28.

Thanks for spotting this regression @vstinner , and for the explanation. Sorry for overlooking that.

bedevere-app Bot added the awaiting review label May 18, 2026

scopreon changed the title Improve locking granularity in pickle's batch_dict_exact gh-146452: Improve locking granularity in pickle's batch_dict_exact May 18, 2026

bedevere-app Bot mentioned this pull request May 18, 2026

Segfault in _pickle.c:batch_dict_exact when pickling dict with concurrent mutation (free-threading) #146452

Closed

scopreon force-pushed the gh-149816-89 branch from 08500b2 to 8f6a3e3 Compare May 18, 2026 22:18

Improve locking granularity in pickle's batch_dict_exact (free-thread…

ac03980

…ing) Replace the coarse critical section wrapping the entire function with fine-grained sections covering only PyDict_Next + Py_INCREF. Also handle PyDict_Next returning 0 in the single-item fast path.

scopreon force-pushed the gh-149816-89 branch from 8f6a3e3 to ac03980 Compare May 18, 2026 22:31

JelleZijlstra approved these changes May 18, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting review labels May 18, 2026

JelleZijlstra reviewed May 18, 2026

View reviewed changes

JelleZijlstra approved these changes May 18, 2026

View reviewed changes

JelleZijlstra added needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels May 18, 2026

JelleZijlstra merged commit 57a0e57 into python:main May 18, 2026
63 checks passed

bedevere-app Bot removed the awaiting merge label May 18, 2026

miss-islington-app Bot assigned JelleZijlstra May 18, 2026

bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label May 18, 2026

scopreon mentioned this pull request May 19, 2026

[3.14] gh-146452: Improve locking granularity in pickle's batch_dict_… #150062

Merged

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label May 19, 2026

vstinner mentioned this pull request May 22, 2026

GitHub free threading CI failing with SIGTERM #150114

Closed

vstinner mentioned this pull request May 22, 2026

test_free_threading.test_pickle takes more than 4 minutes and consumes up to 22 GiB of memory #150257

Closed

vstinner added a commit that referenced this pull request May 22, 2026

Revert "gh-146452 : Improve locking granularity in pickle's batch_dict…

b770b23

…_exact and fix race condition (#150025)" (#150261) This reverts commit 57a0e57.

gh-146452: Improve locking granularity in pickle's batch_dict_exact by scopreon · Pull Request #150025 · python/cpython · GitHub

Repository navigation

gh-146452: Improve locking granularity in pickle's batch_dict_exact - #150025

gh-146452: Improve locking granularity in pickle's batch_dict_exact#150025
JelleZijlstra merged 1 commit into
python:mainfrom
scopreon:gh-149816-89

scopreon commented May 18, 2026 •
edited
Loading

Uh oh!

Uh oh!

bedevere-app Bot commented May 18, 2026

Uh oh!

python-cla-bot Bot commented May 18, 2026 •
edited
Loading

Uh oh!

Uh oh!

bedevere-app Bot commented May 18, 2026

Uh oh!

JelleZijlstra May 18, 2026

Uh oh!

Uh oh!

miss-islington-app Bot commented May 18, 2026

Uh oh!

miss-islington-app Bot commented May 18, 2026

Uh oh!

bedevere-app Bot commented May 18, 2026

Uh oh!

bedevere-app Bot commented May 19, 2026

Uh oh!

vstinner commented May 22, 2026 •
edited
Loading

Uh oh!

Uh oh!

vstinner commented May 22, 2026

Uh oh!

scopreon commented May 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

                   assert(self->proto > 0);
                   dict_size = PyDict_GET_SIZE(obj);
-                  assert(dict_size);

gh-146452: Improve locking granularity in pickle's batch_dict_exact by scopreon · Pull Request #150025 · python/cpython · GitHub

Repository navigation

gh-146452: Improve locking granularity in pickle's batch_dict_exact - #150025

gh-146452: Improve locking granularity in pickle's batch_dict_exact#150025JelleZijlstra merged 1 commit intopython:mainpython/cpython:mainfrom scopreon:gh-149816-89scopreon/cpython:gh-149816-89Copy head branch name to clipboard

Conversation

scopreon commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app Bot commented May 18, 2026

Uh oh!

python-cla-bot Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app Bot commented May 18, 2026

Uh oh!

JelleZijlstra May 18, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app Bot commented May 18, 2026

Uh oh!

miss-islington-app Bot commented May 18, 2026

Uh oh!

bedevere-app Bot commented May 18, 2026

Uh oh!

bedevere-app Bot commented May 19, 2026

Uh oh!

vstinner commented May 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented May 22, 2026

Uh oh!

scopreon commented May 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gh-146452: Improve locking granularity in pickle's batch_dict_exact#150025
JelleZijlstra merged 1 commit into
python:mainfrom
scopreon:gh-149816-89

scopreon commented May 18, 2026 •
edited
Loading

python-cla-bot Bot commented May 18, 2026 •
edited
Loading

vstinner commented May 22, 2026 •
edited
Loading