Do you know if there any more tests with the same issues (I'm not familiar with RustPython, so I don't know where to even try finding a list)? I'd rather get them all done in one go.

We are copying the tests from CPython directly into Lib/test, this structure might look familiar:)

Anyway, I've gathered a list using a small script:

• Lib/test/audit-tests.py
• Lib/test/test_fileutils.py
• Lib/test/test_optimizer.py
• Lib/test/test_monitoring.py
• Lib/test/test_type_cache.py
• Lib/test/test_thread_local_bytecode.py
• Lib/test/test_c_stack_unwind.py
• Lib/test/test_free_threading/test_dict_watcher.py
• Lib/test/test_gdb/test_jit.py

I've looked at each of those tests, all but test_monitoring.py requires either _testcapi or _testinternalcapi it seems. so this should be the last PR regarding this (hopefully).

import ast
import pathlib

ROOT = pathlib.Path(__file__).parent
TEST_DIR = ROOT / "Lib/test"


class Visitor(ast.NodeVisitor):
    def __init__(self):
        self.found = False

    def visit_Call(self, node):
        func = node.func

        if not isinstance(func, ast.Attribute):
            return

        if func.attr != "import_module":
            return

        args = node.args
        if len(args) != 1:
            return

        arg = args[0]
        if not isinstance(arg, ast.Constant):
            return

        value = arg.value

        self.found = value in ("_testcapi", "_testinternalcapi")

    def visit_ClassDef(self, node):
        for bnode in node.body:
            if isinstance(bnode, ast.Assign):
                return self.generic_visit(bnode)

    def visit_FuncionDef(self, node):
        return

    def visit_AsyncFunctionDef(self, node):
        return


bad = set()
for child in TEST_DIR.glob("**/*.py"):
    if "test_capi" in child.parts:
        continue

    rchild = child.relative_to(ROOT)
    try:
        source = child.read_text(encoding="utf-8")
        mod = ast.parse(source)
    except:
        bad.add(rchild)
        continue

    visitor = Visitor()
    visitor.visit(mod)

    if visitor.found:
        print(rchild)


# print("\ncould not parse:\n" + "\n".join(map(str, bad)))

Interestingly this uncovers UB in INSTRUMENTED_JUMP, I have a patch at #152376 to fix it and allow merging. I thought it would be accidental but after two re-runs it still hits it.

Interestingly this uncovers UB in INSTRUMENTED_JUMP, I have a patch at #152376 to fix it and allow merging. I thought it would be accidental but after two re-runs it still hits it.

I see that it just crashes without any information:/
Weird, I wonder if the test isn't isolated. maybe it somehow relies on importing _testcapi once at the top?

Anway, I've subscribed to #152376 so I'll update the PR once it's merged. puting on draft for now

I see that it just crashes without any information:/

See the "Display logs" step, which has the UBSan output.

Seems like #152376 fixed it. tysm for looking into it!

Thanks @ShaharNaveh for the PR, and @StanFromIreland for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

GH-152660 is a backport of this pull request to the 3.15 branch.

GH-152661 is a backport of this pull request to the 3.14 branch.

Thanks @ShaharNaveh for the PR, and @StanFromIreland for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

Sorry, @ShaharNaveh and @StanFromIreland, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 189ab8388762c8102b2d72ad3a1a921f7f2aef10 3.13

Skipping backport to 3.13 here.