gh-134160: Start "Extending and embedding" with a Diataxis-style tutorial by encukou · Pull Request #142314 · python/cpython · GitHub

The "Extending and embedding" section of the docs starts with a "tutorial", which is now outdated (it uses soft-deprecated API), but it also doesn't quite work as a tutorial (in the Diátaxis sense).

This PR pulls out the bits needed for a simple extension module: it's as simple as it can get to expose a C function. Topics that need lengthy explanation are left out (this includes crucial ones like refcounting & error handling, put also modern things like ABI info or free-threading support).
The code is updated to modern, non-deprecated API -- specifically, PEP 793's PyModExport.

The remainder of the existing chapter is renamed to "Using the C API: Assorted topics", to mirror the later chapter “Defining Extension Types: Assorted Topics”. This title was somewhat fitting even without the tutorial part taken out.
Care is taken to not remove any information, unless it's duplicated or no longer relevant. An “assorted topics” section works nicely here.

It would be nice to pull more bits out into dedicated explanation or tutorial pages; that's out of scope for this PR.

I apologize for any typos; I found reviewers are much better at finding them than I am (especially after I've been rewriting drafts for days).

Issue: Docs Examples Focus on Outdated Single-Phase Init Modules and Static Types #134160

📚 Documentation preview 📚: https://cpython-previews--142314.org.readthedocs.build/

See: https://diataxis.fr/tutorials/#ruthlessly-minimise-explanation

Suggested change

      
               first-extension-module.rst
          
               extending.rst
          
               newtypes_tutorial.rst
          
               newtypes.rst
          
               building.rst
          
               windows.rst
          
               embedding.rst
          
               first-extension-module
          
               extending
          
               newtypes_tutorial
          
               newtypes
          
               building
          
               windows
          
               embedding

Why change this? All of CPython's toctrees have the .rst suffixes.

It’s not required, so is a little cleaner (like we don’t have to do import './module.py'), but maybe Sphinx didn’t support that when it was first created and now that’s the style we have.

Oh. Makes sense!
But I think the first such change should be its own PR, so it's easily revertable if anything goes wrong. This PR is big enough on its own.

Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Co-authored-by: Éric <merwok@netwok.org>

I've changed things so the first compilation is after adding #includes. Not finding Python.h headers is a reasonable failure mode, and it makes sense to debug that together with any other tool issues.

The “now run your build tool” section stood out as the most no-tutorial-i part of the tutorial, so I tried the tools from the PyPA recommendation list and chose the most straightforward one: meson-python.
(It'd be setuptools if it weren't for pypa/distutils#387)

I put notes on using other tools in an appendix.

As a big proponent of meson-python IMO it's a great choice for this.

Do people actually need to run ensurepip if they have run python -m venv .venv ?

This tutorial could show the venv commands directly (and avoid ensurepip step) instead of referring to them.

I added “You need to be able to install Python packages” to the intro, leaving exact steps for setting that up out of scope here.
Does that work?

With a link to ensurepip docs, or Python setup and usage (if that includes notes about venv/pip)?

This note seems to contradict the intent of not providing options in a tutorial 🙂

This is partly setup you're expected to have before starting, and part troubleshooting advice.

I added it to the introduction, and made this note smaller. Does that work?

Will check!

Yay, thanks for doing this! I'm excited to finally see a better tutorial.

I tried to avoid being overly nitpicky in my review.

Co-authored-by: Daniele Nicolodi <daniele@grinta.net> Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

Mostly just typos left. I left a few minor wording suggestions, so feel free to reject those if you want to.

These two sentences feel very related.

Suggested change

      
            extension.
          
            Unlike Python, C has an explicit compilation step.
          
            extension, because unlike Python, C has an explicit compilation step.

Juxtaposition is a valid construct! 🙂

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

Thank you for those! I must admit I'm blind to typos at this point...

I'll merge soon, but I'm happy to take more suggestions. Docs are never done :)

I don't have any more complaints, LGTM!

…e tutorial (pythonGH-142314) Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Co-authored-by: Éric <merwok@netwok.org> Co-authored-by: Daniele Nicolodi <daniele@grinta.net> Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

encukou added 7 commits December 5, 2025 17:29

New first chapter of a C API tutorial

4d31b9a

Reword more

4c2b982

Start from the beginning

1525795

TMP

dc844dd

Full draft

32ae620

Ruthlessly miminise explanation

13d5efa

See: https://diataxis.fr/tutorials/#ruthlessly-minimise-explanation

Re-add some reference tragets

7f1014e

encukou added the docs Documentation in the Doc dir label Dec 5, 2025

github-project-automation Bot added this to Docs PRs Dec 5, 2025

github-project-automation Bot moved this to Todo in Docs PRs Dec 5, 2025

bedevere-app Bot added the awaiting core review label Dec 5, 2025

bedevere-app Bot mentioned this pull request Dec 5, 2025

Docs Examples Focus on Outdated Single-Phase Init Modules and Static Types #134160

Open

Typo fix

4c61bf4

merwok reviewed Dec 5, 2025

View reviewed changes

Comment thread Doc/extending/first-extension-module.rst Outdated

merwok reviewed Dec 5, 2025

View reviewed changes

Comment thread Doc/extending/first-extension-module.rst Outdated

merwok reviewed Dec 5, 2025

View reviewed changes

Comment thread Doc/extending/first-extension-module.rst Outdated

merwok reviewed Dec 5, 2025

View reviewed changes

Comment thread Doc/extending/first-extension-module.rst

merwok reviewed Dec 5, 2025

View reviewed changes

Comment thread Doc/extending/first-extension-module.rst

merwok reviewed Dec 5, 2025

View reviewed changes

Comment thread Doc/extending/first-extension-module.rst Outdated

merwok reviewed Dec 5, 2025

View reviewed changes

Comment thread Doc/extending/first-extension-module.rst Outdated

StanFromIreland reviewed Dec 5, 2025

View reviewed changes

Comment thread Doc/extending/first-extension-module.rst Outdated

encukou and others added 4 commits December 10, 2025 11:32

Apply suggestions from code review

0d91185

Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com> Co-authored-by: Éric <merwok@netwok.org>

Address more review comments

2902a8b

tart with the headers

276d8cd

str->bytes is *encoding*

2306fcd

encukou added 3 commits December 12, 2025 14:03

Merge in the main branch

a312862

Link to the list of recommended tools in the Python Packaging User Guide

ae85969

Fix line highlight

27af529

Move signature check up

7069412

encukou added the skip news label Dec 12, 2025

encukou added 2 commits December 12, 2025 16:22

Use meson-python

54e2999

Use 'sampleproject' for the project name

f92562c

merwok reviewed Dec 13, 2025

View reviewed changes

ZeroIntensity reviewed Dec 15, 2025

View reviewed changes

dnicolodi reviewed Dec 16, 2025

View reviewed changes

Comment thread Doc/extending/first-extension-module.rst Outdated

encukou and others added 4 commits December 16, 2025 14:18

Apply suggestions from code review

a17458a

Co-authored-by: Daniele Nicolodi <daniele@grinta.net> Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

Address more review

638d2a0

Use :c:func:system, not system

2fc1511

Expand the explanation of Py_RETURN_NONE

24c6eaa

ngoldbaum reviewed Dec 16, 2025

View reviewed changes

Comment thread Doc/includes/capi-extension/spammodule-01.c Outdated

encukou added 2 commits December 17, 2025 11:37

Merge in the main branch

8e622ac

Add a comment for <stdlib.h>

772152c

ZeroIntensity reviewed Dec 17, 2025

View reviewed changes

Apply suggestions from code review

72fd39f

Co-authored-by: Peter Bierma <zintensitydev@gmail.com>

ZeroIntensity approved these changes Dec 19, 2025

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels Dec 19, 2025

encukou merged commit 049c252 into python:main Dec 19, 2025
32 checks passed

github-project-automation Bot moved this from Todo to Done in Docs PRs Dec 19, 2025

bedevere-app Bot removed the awaiting merge label Dec 19, 2025

encukou deleted the modexport-tutorial branch December 19, 2025 16:48

encukou mentioned this pull request Dec 19, 2025

Implement PEP 793 – PyModExport #140550

Open

25 tasks

+                 preferably in a :mod:`virtual environment <venv>`.
+                 You can also use another tool that can build and install
+                 ``pyproject.toml``-based projects, like
+                 `uv <https://docs.astral.sh/uv/>`_ (``uv pip install .``).

+              .. tip::
+                 If you don't have ``pip`` installed, run ``python -m ensurepip``,
+                 preferably in a :mod:`virtual environment <venv>`.

		extension.
		Unlike Python, C has an explicit compilation step.

	extension.
	Unlike Python, C has an explicit compilation step.
	extension, because unlike Python, C has an explicit compilation step.

gh-134160: Start "Extending and embedding" with a Diataxis-style tutorial by encukou · Pull Request #142314 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-134160: Start "Extending and embedding" with a Diataxis-style tutorial - #142314

gh-134160: Start "Extending and embedding" with a Diataxis-style tutorial#142314encukou merged 25 commits intopython:mainpython/cpython:mainfrom encukou:modexport-tutorialencukou/cpython:modexport-tutorialCopy head branch name to clipboard

Conversation

encukou commented Dec 5, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

encukou commented Dec 10, 2025

Uh oh!

encukou commented Dec 12, 2025

Uh oh!

ngoldbaum commented Dec 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

encukou commented Dec 19, 2025

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gh-134160: Start "Extending and embedding" with a Diataxis-style tutorial#142314
encukou merged 25 commits into
python:mainfrom
encukou:modexport-tutorial

encukou commented Dec 5, 2025 •
edited by github-actions Bot
Loading