This is just an expirement to improve Sergey's PR to solve this issue #139951

cc @sergey-miryanov @markshannon

Is it correct that pack(1, None) crashes for now in test_tuple_pack?

Is it correct that pack(1, None) crashes for now in test_tuple_pack?

That's because test C _testlimitedcapi function replaces Python None with C NULL, and PyTuple_Pack() arguments must not be NULL.

The approach here is to check for tuples of immutables at creation time. An alternative could be to do this inside the garbage collector (and untrack the tuple if needed). Was that considered?

The issue solved here is about a use case where many tuples are created in a tight loop. But there could potentially also be tuples of immutables in a normal workflow that visited many times by the GC.

One datapoint: after starting up my "normal" work and running this script

import gc
gc.collect()

def candidate(obj):
   return all( not gc.is_tracked(x) for x in obj )

for immutable_type in (tuple, frozenset):

    number_of_objects_tracked  =0 
    number_of_candidates=0
    number_of_immutable_candidates = 0
    
    for obj in gc.get_objects():
        number_of_objects_tracked += 1
        if type(obj) is immutable_type:
            number_of_candidates +=1        
            if  candidate(obj):
                number_of_immutable_candidates+=1
    
    print(f'type {immutable_type}')
    print(f'  {number_of_objects_tracked=}')
    print(f'  {number_of_candidates=}')
    print(f'  {number_of_immutable_candidates=}')

I get as output:

type <class 'tuple'>
  number_of_objects_tracked=703468
  number_of_candidates=82177
  number_of_immutable_candidates=582
type <class 'frozenset'>
  number_of_objects_tracked=703478
  number_of_candidates=4799
  number_of_immutable_candidates=4787

So for me the amount of tuples would not be reduced a lot, but it would be interesting to apply the same approach to the frozenset. Could someone run the above script on their "typical" workflow or application?

An alternative could be to do this inside the garbage collector (and untrack the tuple if needed). Was that considered?

This is already doing in the GC (see untrack_tuples in gc.c). Also, you may be interested in #139389

I've created another PR with tests:
#140575

We can also adapt changes to functions _PyTuple_FromArraySteal, tuple_concat, tuple_repeat, tuple_subscript if we want to.
I have no strong opinion about it.

Is this change still relevant after commit 0c01090 and commit 88ad41f?

Is this change still relevant after commit 0c01090 and commit 88ad41f?

Only if we want to implement it to _PyTuple_FromArraySteal, tuple_concat, tuple_repeat, tuple_subscript as @efimov-mikhail said above.

We both haven't strong opinion on this.

You may create a new PR for these functions if you want to update these functions.

I've decided to close this PR.
@sergey-miryanov, feel free to experiment with those functions if you want to.

@efimov-mikhail Thanks!