gh-142659: Optimize set_swap_bodies for intersection_update by Siyet · Pull Request #148155 · python/cpython · GitHub

Summary

Replace the general-purpose set_swap_bodies() with a specialized set_replace_body() that exploits the invariant that the source argument is always a uniquely-referenced temporary set about to be discarded.

Follow-up to the observation in #132290 (comment) and #142659.

Changes

set_swap_bodies() was designed for arbitrary two-way swaps between any two sets, but it is only called from set_intersection_update() and set_intersection_update_multi_impl(), where the second argument (tmp) is always a freshly created temporary with Py_REFCNT == 1.

The new set_replace_body() exploits this invariant:

Skip atomic stores on src: the temporary is not visible to other threads, so plain assignments suffice (saves atomic fence overhead in the free-threaded build).
Skip copy_small_table for src: use plain memcpy instead of per-entry atomic stores when writing back to src's smalltable (Py_GIL_DISABLED path).
Remove frozenset hash swap: neither argument is ever a frozenset in these call sites (enforced via assert).
Simplify shared-marking logic: src is never shared (enforced via assert), so only one direction of the shared-marking check is needed — propagate shared status from dst to src for proper deallocation of old entries.
Remove unused Py_hash_t h variable.

All assumptions are guarded by assert() to document and enforce the contract.

Issue: Minor optimization for set object, swap bodies #142659

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@Siyet Could you add a few benchmarks for the affected cases?

Benchmarks on a dedicated server (AMD EPYC 1 vCPU, 2 GB RAM, Ubuntu 24.04, no background load, taskset -c 0, 50 rounds):

Default build (with GIL):

Benchmark	Before (ns)	After (ns)	Change
small (5 & 5)	1010 ± 19	1022 ± 21	+1.2%
medium (100 & 100)	5010 ± 76	4930 ± 58	-1.6%
large (10k & 10k)	1141519 ± 12612	1137019 ± 11591	-0.4%
multi-arg (100 & 100 & 50)	6042 ± 50	6048 ± 88	+0.1%
empty result (100 & 0)	4323 ± 47	4301 ± 59	-0.5%

Free-threading build (--disable-gil):

Benchmark	Before (ns)	After (ns)	Change
small (5 & 5)	1910 ± 18	1931 ± 22	+1.1%
medium (100 & 100)	6716 ± 63	6650 ± 91	-1.0%
large (10k & 10k)	2407935 ± 22615	2404563 ± 25709	-0.1%
multi-arg (100 & 100 & 50)	8371 ± 91	8284 ± 82	-1.0%
empty result (100 & 0)	5896 ± 111	5828 ± 65	-1.1%

The end-to-end improvement is modest (~1%) because set_replace_body is a small fraction of the total intersection_update time — most time is spent in set_intersection (iteration, hashing, insertion). The main value of this change is code clarity and correctness: the new function documents its contract via assertions, removes dead branches (frozenset hash swap, bidirectional shared-marking), and skips unnecessary atomic operations on the uniquely-referenced temporary.

Benchmark script

"""Benchmark set.intersection_update for gh-142659.

Measures the performance of intersection_update across different set sizes
and usage patterns. Each benchmark recreates the set on every iteration
to measure the full intersection_update path including set_replace_body.
"""
import statistics
import sys
import timeit

ROUNDS = 50

BENCHMARKS = {
    "small (5 & 5)": {
        "setup": "",
        "stmt": "set(range(5)).intersection_update({3, 4, 5, 6, 7})",
        "number": 2_000_000,
    },
    "medium (100 & 100)": {
        "setup": "b = set(range(50, 150))",
        "stmt": "set(range(100)).intersection_update(b)",
        "number": 500_000,
    },
    "large (10k & 10k)": {
        "setup": "b = set(range(5000, 15000))",
        "stmt": "set(range(10000)).intersection_update(b)",
        "number": 2_000,
    },
    "multi-arg (100 & 100 & 50)": {
        "setup": "b = set(range(50, 150)); c = set(range(75, 125))",
        "stmt": "set(range(100)).intersection_update(b, c)",
        "number": 500_000,
    },
    "empty result (100 & 0 overlap)": {
        "setup": "b = set(range(200, 300))",
        "stmt": "set(range(100)).intersection_update(b)",
        "number": 500_000,
    },
}


def main():
    print(f"Python {sys.version}")
    print(f"Rounds: {ROUNDS}")
    print()
    print(f"{'Benchmark':<32} {'Mean (ns)':>10} {'Stdev':>10} {'Min':>10} {'Max':>10}")
    print("-" * 78)

    for name, bench in BENCHMARKS.items():
        times = timeit.repeat(
            bench["stmt"],
            setup=bench["setup"],
            number=bench["number"],
            repeat=ROUNDS,
        )
        per_iter = [t / bench["number"] * 1e9 for t in times]
        mean = statistics.mean(per_iter)
        stdev = statistics.stdev(per_iter)
        print(
            f"{name:<32} {mean:>10.1f} {stdev:>9.1f} {min(per_iter):>10.1f} {max(per_iter):>10.1f}"
        )


if __name__ == "__main__":
    main()

Replace the general-purpose set_swap_bodies() with a specialized set_replace_body() that exploits the invariant that src is always a uniquely-referenced temporary about to be discarded.

Suggested change

      
                assert(Py_REFCNT(src) == 1);
          
                assert(_PyObject_IsUniquelyReferenced(src));

This assertion is going to be prone to gc.get_objects()/gc.get_referrers() issues like in #148180.

I'm not sure if we should do anything about that.

Will gc.get_objects and similar ever see this set? They use a stop-the-world heap traversal, which can't happen until set_intersection_update / set_intersection_update_multi_impl returns, and by the time that happens, this src set is already gone.

I agree with @zhuyifei1999 - src is a local variable that gets freed before the function returns, so gc can't observe it.

Switched to _PyObject_IsUniquelyReferenced per @eendebakpt's suggestion regardless, since I believe it's the correct API for free-threaded builds.

If the called is required to cleanup src immediately afterwards, maybe it is cleaner to let the set_replace_body steal the reference and do the decref.

In set_intersection_update_multi_impl, I placed Py_DECREF(tmp) after Py_END_CRITICAL_SECTION().

My reasoning was that moving the decref inside set_replace_body would run the deallocation (and decref of all keys) while still holding the critical section lock, which I thought might be undesirable.

By keeping it outside, the caller has control over when deallocation happens.

This is my first PR though and I'm still getting familiar with the project, so if I'm wrong here - happy to rework it.

set_replace_body is a function internal to cpython so I don't think you need to mention it in the news entry as this won't affect user programs.

Done, removed the NEWS entry in 100c491.

Could you replace these two asserts with

assert(PySet_CheckExact((PyObject *) dst));
assert(PySet_CheckExact((PyObject *) src));

I'd prefer to keep the current assertions. dst can be a subclass of set (e.g. class S(set): pass; S([1,2]).intersection_update({2})), so PySet_CheckExact(dst) would fire on that legitimate case. make_new_set_basetype() normalizes the type for src, but dst is the original caller object.

The current form (!PyType_IsSubtype(..., &PyFrozenSet_Type)) checks exactly what we need: neither argument is a frozenset.

Could you use PySet_Check instead?

Good point, switched to PySet_Check in 0c0a00a.

This memcpy and the one below it on line 1528 can be pulled out of the macro.

Also, I find this entire section for copying the smalltable a bit awkward. Right now, this section is the only time copy_small_table is called so we could just move the for loop in copy_small_table into this function. Additionally, we could make copy_small_table exist for GIL builds and have it use memcpy as the default unless the GIL is disabled. Something roughly like this:

static void
copy_small_table(setentry *dest, setentry *src)
{
#ifdef Py_GIL_DISABLED
    for (Py_ssize_t i = 0; i < PySet_MINSIZE; i++) {
        _Py_atomic_store_ptr_release(&dest[i].key, src[i].key);
        _Py_atomic_store_ssize_relaxed(&dest[i].hash, src[i].hash);
    }
#else
    memcpy(dest, src, PySet_MINSIZE * sizeof(setentry));
#endif
}

Good suggestion, done in 100c491. Moved the #ifdef Py_GIL_DISABLED branch inside copy_small_table so it uses memcpy in GIL builds and atomic stores in free-threaded builds. The callsite is clean now - no preprocessor conditionals.

Sorry if I was not clear. There were two options that I suggested:

Never use copy_small_table and move the logic into set_replace_body
Never use memcpy in set_replace_body and instead call copy_small table

I'm not sure which option is best. However, if you want to implement the second option, I would make sure to no longer call memcpy in set_replace_body.

Wait actually on second thought I realized that you probably should not make that change. Never mind.

No worries. Thanks for the thorough review, really appreciate it!

Remove NEWS entry for internal-only change. Refactor copy_small_table to be available in all builds: memcpy on GIL-enabled, atomic stores on free-threaded. This removes the #ifdef from set_replace_body callsite.

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

This PR is stale because it has been open for 30 days with no activity.

dst is locked in all usages of set_replace_body afaics. Can we update the name to reflect this? And maybe add an assert on dst.

This PR is stale because it has been open for 90 days with no activity.

Siyet requested a review from rhettinger as a code owner April 6, 2026 08:34

bedevere-app Bot added the awaiting review label Apr 6, 2026

bedevere-app Bot mentioned this pull request Apr 6, 2026

Minor optimization for set object, swap bodies #142659

Open

aleksandr.tseluyko added 2 commits April 6, 2026 16:07

pythongh-142659 : Optimize set_swap_bodies for intersection_update

554d5c1

Replace the general-purpose set_swap_bodies() with a specialized set_replace_body() that exploits the invariant that src is always a uniquely-referenced temporary about to be discarded.

Add news entry for pythongh-142659

37a95e6

Siyet force-pushed the gh-142659-optimize-set-replace-body branch from d3f1a20 to 37a95e6 Compare April 6, 2026 13:08

eendebakpt reviewed Apr 6, 2026

View reviewed changes

Use _PyObject_IsUniquelyReferenced instead of Py_REFCNT check

1be8796

Siyet requested review from eendebakpt and zhuyifei1999 April 7, 2026 06:51

brijkapadia reviewed Apr 11, 2026

View reviewed changes

Siyet requested a review from colesbury April 11, 2026 22:32

pythongh-142659 : Move GIL/free-threading branch into copy_small_table

100c491

Remove NEWS entry for internal-only change. Refactor copy_small_table to be available in all builds: memcpy on GIL-enabled, atomic stores on free-threaded. This removes the #ifdef from set_replace_body callsite.

eendebakpt added the skip news label Apr 12, 2026

pythongh-142659 : Use PySet_Check in set_replace_body assertions

0c0a00a

Siyet requested a review from brijkapadia April 12, 2026 07:54

github-actions Bot added the stale Stale PR or inactive for long period of time. label May 18, 2026

eendebakpt reviewed May 18, 2026

View reviewed changes

github-actions Bot removed the stale Stale PR or inactive for long period of time. label May 26, 2026

github-actions Bot added the stale Stale PR or inactive for long period of time. label Aug 24, 2026

-                      memcpy(a->smalltable, b->smalltable, sizeof(tab));
-                      memcpy(b->smalltable, tab, sizeof(tab));
+                      memcpy(dst->smalltable, src->smalltable, sizeof(tab));
+                      memcpy(src->smalltable, tab, sizeof(tab));

+                  assert(!PyType_IsSubtype(Py_TYPE(dst), &PyFrozenSet_Type));
+                  assert(!PyType_IsSubtype(Py_TYPE(src), &PyFrozenSet_Type));
+                  assert(Py_REFCNT(src) == 1);

	assert(Py_REFCNT(src) == 1);
	assert(_PyObject_IsUniquelyReferenced(src));

+                 moving dst's old contents into src for proper cleanup on Py_DECREF.
-                   t=set(a); a.clear(); a.update(b); b.clear(); b.update(t); del t
+                 The caller guarantees that src is a uniquely-referenced temporary set

		@@ -0,0 +1,3 @@
		Optimize :meth:`set.intersection_update` by replacing the general-purpose
		``set_swap_bodies()`` with a specialized ``set_replace_body()`` that skips

-                  if (a_table == a->smalltable || b_table == b->smalltable) {
-                      memcpy(tab, a->smalltable, sizeof(tab));
+                  assert(!PyType_IsSubtype(Py_TYPE(dst), &PyFrozenSet_Type));

               static void
-              set_swap_bodies(PySetObject *a, PySetObject *b)
+              set_replace_body(PySetObject *dst, PySetObject *src)

gh-142659: Optimize set_swap_bodies for intersection_update by Siyet · Pull Request #148155 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-142659: Optimize set_swap_bodies for intersection_update - #148155

gh-142659: Optimize set_swap_bodies for intersection_update#148155Siyet wants to merge 5 commits intopython:mainpython/cpython:mainfrom Siyet:gh-142659-optimize-set-replace-bodySiyet/cpython:gh-142659-optimize-set-replace-bodyCopy head branch name to clipboard

Conversation

Siyet commented Apr 6, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Uh oh!

bedevere-app Bot commented Apr 6, 2026

Uh oh!

eendebakpt commented Apr 6, 2026

Uh oh!

Siyet commented Apr 6, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-app Bot commented Apr 12, 2026

Uh oh!

github-actions Bot commented May 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gh-142659: Optimize set_swap_bodies for intersection_update#148155
Siyet wants to merge 5 commits into
python:mainfrom
Siyet:gh-142659-optimize-set-replace-body

Siyet commented Apr 6, 2026 •
edited by bedevere-app Bot
Loading