gh-116146: Add C-API to create module from spec and initfunc by itamaro · Pull Request #139196 · python/cpython · GitHub

C-API WG decision: capi-workgroup/decisions#77

Issue: Expose a C-API function to allow custom importers to create a module using an init function #116146

This looks reasonable.

I'm proposing a replacement for Init functions in PEP-793, but that PEP adds PyModule_FromSlotsAndSpec and PyModule_Exec which should work for similar use cases.
Would it make sense to switch the spec & initfunc arguments to match that proposal?

I'm not sure if the word Builtin in the name is necessary -- is that to match the internal create_builtin? Would something like PyImport_LoadModuleFromInitfuncAndSpec work better?

Do you want to submit this to the C API WG?

Would it make sense to switch the spec & initfunc arguments to match that proposal?

I think it should be fine! Is the PEP already implemented on main?

I'm not sure if the word Builtin in the name is necessary -- is that to match the internal create_builtin? Would something like PyImport_LoadModuleFromInitfuncAndSpec work better?

Yes, I used it to reflect what it does internally. PyImport_LoadModuleFromInitfuncAndSpec sounds great to me!

Do you want to submit this to the C API WG?

Will do. Should we first decide whether to change this to match PEP 793, or go to the WP with options and get their input?

Is the PEP already implemented on main?

No, it's not even accepted :)
I have a slight preference to align the two proposals, in case both make it. I'm checking if you have strong reasons for the order here. No need to update the PR now.

Will do. Should we first decide whether to change this to match PEP 793, or go to the WP with options and get their input?

Just go to the WG :)

Just go to the WG :)

done :) capi-workgroup/decisions#77

- use `PyImport_CreateModuleFromInitfunc` for the new API - no need to introduce `create_builtin_ex` - update `create_builtin` and all callers - update exception message

- remove unused `found` variable - use `my_test_extension` instead of `embedded_ext` (the former is free-threading-ready, the latter prints a warning)

Co-authored-by: Kumar Aditya <kumaraditya@python.org>

I left my suggestions as a PR: itamaro#27

* Test single-phase init as well; don't use private APIs in test * Doc update --------- Co-authored-by: Itamar Oren <itamarost@gmail.com>

Co-authored-by: Victor Stinner <vstinner@python.org>

All commit authors signed the Contributor License Agreement.

…on under the free-threaded build

thank you @encukou @vstinner @kumaraditya303 for the review and suggestions!

@encukou are you planning to merge gh-141197 soon? until that is merged, this PR has a docs failure due to referring to the function that you document in gh-141197.

The new function PyImport_CreateModuleFromInitfunc() should be documented in Doc/whatsnew/3.15.rst.

Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Kumar Aditya <kumaraditya@python.org> Co-authored-by: Victor Stinner <vstinner@python.org>

LGTM. Thanks for the updates!

Congrats @itamaro!

…ythonGH-139196) Co-authored-by: Kumar Aditya <kumaraditya@python.org> Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>

bedevere-app Bot mentioned this pull request Sep 21, 2025

Expose a C-API function to allow custom importers to create a module using an init function #116146

Closed

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from f9cd705 to 4363e5a Compare September 21, 2025 19:25

encukou mentioned this pull request Sep 23, 2025

_PyImport_SwapPackageContext would be helpful if exposed #139227

Closed

encukou reviewed Sep 23, 2025

View reviewed changes

Comment thread Python/import.c Outdated

itamaro mentioned this pull request Sep 24, 2025

Expose a C-API to allow custom importers to create a module directly from spec & initfunc capi-workgroup/decisions#77

Closed

6 tasks

vstinner reviewed Sep 26, 2025

View reviewed changes

Comment thread Python/import.c Outdated

itamaro added 5 commits November 8, 2025 08:09

pythongh-116146 : Add new C-API to create builtin from spec and initfunc

403c7e2

Apply review feedback and CAPI WG decision

074a510

- use `PyImport_CreateModuleFromInitfunc` for the new API - no need to introduce `create_builtin_ex` - update `create_builtin` and all callers - update exception message

Add tests for the PyImport_CreateModuleFromInitfunc API

d38c0e4

Document the new PyImport_CreateModuleFromInitfunc C-API

d5be6e5

Add news entry

781a730

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from 4363e5a to 781a730 Compare November 8, 2025 18:52

itamaro added 2 commits November 8, 2025 10:54

Fix lint

7af34eb

fix CI

35e9e78

- remove unused `found` variable - use `my_test_extension` instead of `embedded_ext` (the former is free-threading-ready, the latter prints a warning)

itamaro changed the title gh-116146: Add new C-API to create builtin from spec and initfunc gh-116146: Add C-API to create module from spec and initfunc Nov 9, 2025

itamaro marked this pull request as ready for review November 9, 2025 01:01

itamaro requested review from brettcannon, ericsnowcurrently, kumaraditya303, ncoghlan and warsaw as code owners November 9, 2025 01:01

bedevere-app Bot added the awaiting review label Nov 9, 2025

kumaraditya303 reviewed Nov 9, 2025

View reviewed changes

Comment thread Python/import.c Outdated

Apply Kumar's review suggestion

6aa2fb2

Co-authored-by: Kumar Aditya <kumaraditya@python.org>

PyImport_CreateModuleFromInitfunc: Doc & test update (#27 )

f652399

* Test single-phase init as well; don't use private APIs in test * Doc update --------- Co-authored-by: Itamar Oren <itamarost@gmail.com>

vstinner reviewed Nov 12, 2025

View reviewed changes

Comment thread Doc/c-api/import.rst Outdated

Comment thread Include/cpython/import.h Outdated

Comment thread Programs/_testembed.c Outdated

Comment thread Programs/_testembed.c Outdated

Comment thread Programs/_testembed.c Outdated

Comment thread Python/import.c

Apply Victor's review suggestions

c4f1cc8

Co-authored-by: Victor Stinner <vstinner@python.org>

itamaro added 2 commits November 12, 2025 08:17

Set error if initfunc is NULL

29b1b68

Account for the RuntimeWarning when import a singlephase init extensi…

1165950

…on under the free-threaded build

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from 8f16533 to 1165950 Compare November 12, 2025 16:18

vstinner reviewed Nov 12, 2025

View reviewed changes

Comment thread Misc/NEWS.d/next/C_API/2025-11-08-10-51-50.gh-issue-116146.pCmx6L.rst Outdated

encukou reviewed Nov 13, 2025

View reviewed changes

Comment thread Doc/c-api/import.rst Outdated

kumaraditya303 reviewed Nov 13, 2025

View reviewed changes

Comment thread Include/cpython/import.h Outdated

kumaraditya303 reviewed Nov 13, 2025

View reviewed changes

Comment thread Programs/_testembed.c Outdated

kumaraditya303 reviewed Nov 13, 2025

View reviewed changes

Comment thread Python/import.c Outdated

Apply suggestions from code review

79e5d0e

Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Kumar Aditya <kumaraditya@python.org> Co-authored-by: Victor Stinner <vstinner@python.org>

itamaro requested a review from AA-Turner as a code owner November 13, 2025 15:31

Mention new C-API in whatsnew

fbfde0a

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from fabafb5 to fbfde0a Compare November 13, 2025 15:32

vstinner approved these changes Nov 13, 2025

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting review labels Nov 13, 2025

kumaraditya303 approved these changes Nov 13, 2025

View reviewed changes

Merge branch 'main' into pythongh-116146 -create-builtin-initfunc

f485fcb

encukou merged commit 1e4e59b into python:main Nov 14, 2025
46 checks passed

bedevere-app Bot removed the awaiting merge label Nov 14, 2025

encukou added a commit to encukou/cpython that referenced this pull request Nov 14, 2025

Link reference to PyModule_Exec added in python#139196

816023d

itamaro deleted the gh-116146-create-builtin-initfunc branch November 15, 2025 23:40

itamaro mentioned this pull request Dec 1, 2025

Assertion failures from calling create_builtin with invalid object #142029

Closed

itamaro mentioned this pull request Dec 11, 2025

gh-142029: Raise ModuleNotFoundError instead of crashing on nonexsistent module name given to create_builtin() #142054

Merged

gh-116146: Add C-API to create module from spec and initfunc by itamaro · Pull Request #139196 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-116146: Add C-API to create module from spec and initfunc - #139196

gh-116146: Add C-API to create module from spec and initfunc#139196encukou merged 15 commits intopython:mainpython/cpython:mainfrom itamaro:gh-116146-create-builtin-initfuncitamaro/cpython:gh-116146-create-builtin-initfuncCopy head branch name to clipboard

Conversation

itamaro commented Sep 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

encukou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

itamaro commented Sep 23, 2025

Uh oh!

encukou commented Sep 24, 2025

Uh oh!

itamaro commented Sep 24, 2025

Uh oh!

Uh oh!

Uh oh!

encukou commented Nov 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

python-cla-bot Bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

itamaro commented Nov 12, 2025

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vstinner commented Nov 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gh-116146: Add C-API to create module from spec and initfunc#139196
encukou merged 15 commits into
python:mainfrom
itamaro:gh-116146-create-builtin-initfunc

itamaro commented Sep 21, 2025 •
edited
Loading

python-cla-bot Bot commented Nov 12, 2025 •
edited
Loading