Repository navigation
Improve performance by replacing PyTuple_Pack with PyTuple_FromArray #140009
Description
Feature or enhancement
Proposal:
The PyTuple_Pack method is relatively slow because it is a variadic function. For various cases (in particular tuples of size 1 and 2) we can replace PyTuple_Pack with PyTuple_FromArray (introduced in #111489)
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
Linked PRs
- gh-140009: Improve performance of list_extend_dictitems by using PyTuple_FromArray #140010
- gh-140009: Optimize dict object by replacing PyTuple_Pack with PyTuple_FromArray #144529
- gh-140009: Optimize dict object by replacing PyTuple_Pack with PyTuple_FromArray #144531
- gh-140009: Improve performance of datetime extension module by using PyTuple_FromArray #144532
- gh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _warnings #144760
- gh-140009: Optimize dict.items() symmetric difference via PyTuple_FromArray #144771
- gh-140009: Optimize JSON parsing with object_pairs_hook via PyTuple_FromArray #144772
- gh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _PyCode_ConstantKey #144773
- gh-140009: Replace PyTuple_Pack with PyTuple_FromArray in codeobject.c and _pickle.c #144829
Pinned by eendebakpt
Note: the remaining cases of PyTuple_Pack(1, ...) and PyTuple_Pack(2, ...) are for the most part not performance critical. If opening a PR, please create proper benchmarks (using pyperf, hyperfine or some other benchmarking package) showing the performance gain for the change.
Activity
StanFromIreland commented on Oct 12, 2025
eendebakpt commented on Oct 12, 2025
gh-140009: Improve performance of `list_extend_dictitems` by using `P…
kurtmetzger commented on Oct 15, 2025
If I were to work on this, is it basically changing instances that look like this:
result = PyTuple_Pack(1, basestate);
to this?
PyObject *items[1] = {basestate}; result = PyTuple_FromArray(items, 1);
for all instances where the array would only be 1 or two items?
Thanks
eendebakpt commented on Oct 15, 2025
@kurtmetzger Yes, that would be it! But I would wait with pr's for #140132 (comment). If that is accepted, the PyTuple_FromSingle would be even better.
kurtmetzger commented on Oct 15, 2025
@eendebakpt I see, so if the other gets accepted, it creates custom functions for arrays of lengths 1 and 2 that should be used like so:
PyTuple_MakePair(a, b)
or
PyTuple_MakeSingle(a)
to replace the current PyTuple_Pack calls?
eendebakpt commented on Oct 16, 2025
Yes. Note however that #140079 by @sergey-miryanov is there as well, which overlaps.
Add NEWS entry for pythongh-140009
pythongh-140009: Optimize dict object by replacing PyTuple_Pack with …
pythongh-140009: Optimize dict object by replacing PyTuple_Pack with …
eendebakpt commented on Feb 8, 2026
Note: the remaining cases of PyTuple_Pack(1, ...) and PyTuple_Pack(2, ...) are for the most part not performance critical. If opening a PR, please create proper benchmarks (using pyperf, hyperfine or some other benchmarking package) showing the performance gain for the change.
pythongh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _warn…
pythongh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _warn…
pythongh-140009: Use PyTuple_FromArray in dictitems_xor_lock_held
pythongh-140009: Use PyTuple_FromArray in _json object_pairs_hook path
pythongh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _PyCo…
pythongh-140009: Use PyTuple_FromArray in dictitems_xor_lock_held
pythongh-140009: Add news entry for json object_pairs_hook optimization
pythongh-140009: Add news entry for dict.items() symmetric difference…
pythongh-140009: Trigger CI rerun
eendebakpt commented on May 19, 2026
Closing this. The most relevant cases have been covered by #145247