FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-149083: use sentinel to fix _functools.reduce() signature by skirpichev · Pull Request #149591 · python/cpython · GitHub

/ cpython Public

gh-149083: use sentinel to fix _functools.reduce() signature - #149591

Merged
JelleZijlstra merged 3 commits into
python:mainfrom
skirpichev:use-sentinel-in-c-reduce/149083
May 10, 2026
Merged

gh-149083: use sentinel to fix _functools.reduce() signature#149591
JelleZijlstra merged 3 commits into
python:mainfrom
skirpichev:use-sentinel-in-c-reduce/149083

Conversation

skirpichev commented May 9, 2026
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

skirpichev changed the title gh-149536: use sentinel to fix _functools.reduce() signature gh-149083: use sentinel to fix _functools.reduce() signature May 9, 2026
skirpichev force-pushed the use-sentinel-in-c-reduce/149083 branch from 6c65b61 to bbae869 Compare May 9, 2026 04:29
skirpichev marked this pull request as ready for review May 9, 2026 09:23
skirpichev requested a review from rhettinger as a code owner May 9, 2026 09:23
skirpichev requested a review from JelleZijlstra May 9, 2026 11:03

Copy link
Copy Markdown
Member Author

CC @JelleZijlstra

AFAIK, it's the first case of using sentinel's in C extensions. I used it just to fix signature, NULL is fine as C default.

JelleZijlstra merged commit c6fd7de into python:main May 10, 2026
58 checks passed
JelleZijlstra added awaiting merge needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels May 10, 2026

Copy link
Copy Markdown

Thanks @skirpichev for the PR, and @JelleZijlstra for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

bedevere-app Bot commented May 10, 2026

Copy link
Copy Markdown

GH-149653 is a backport of this pull request to the 3.15 branch.

bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 10, 2026
JelleZijlstra pushed a commit that referenced this pull request May 10, 2026
…H-149591) (#149653)

gh-149083: use sentinel to fix _functools.reduce() signature (GH-149591)
(cherry picked from commit c6fd7de)

Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
skirpichev deleted the use-sentinel-in-c-reduce/149083 branch May 10, 2026 23:52
pytorchmergebot pushed a commit to pytorch/pytorch that referenced this pull request Jun 1, 2026
….15 (#185682)

## Summary

Python 3.15 will expose an introspectable signature for `functools.reduce` using a PEP 661 sentinel default: `(function, iterable, /, initial=functools._initial_missing)`. The CPython change is [#149591](python/cpython#149591), which merged into CPython main on 2026-05-10, after 3.15.0b1 was tagged. So on 3.15.0b1 the default is still `<unrepresentable>` and `inspect.signature()` raises `ValueError`, which `substitute_in_graph` swallows and skips the check. From 3.15.0b2 the check will run.

When it runs, it compares positional parameter names, keyword-only names, and default values. The polyfill declares its own local sentinel:

```python
_initial_missing = object()

@substitute_in_graph(functools.reduce)
def reduce(function, iterable, initial=_initial_missing, /):
    ...
```

That local `object()` is a different instance from `functools._initial_missing`, so the defaults dict compares unequal and `substitute_in_graph` raises:

```
File ".../torch/_dynamo/polyfills/functools.py", line N, in <module>
  @substitute_in_graph(functools.reduce)
TypeError: Signature mismatch between <built-in function reduce> and <function reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

The polyfills loader imports every polyfill module, so this will break `import torch._dynamo` entirely on Python 3.15.0b2+. Same failure mode as #185403's `struct.pack` fix, just one step ahead of the next beta.

The fix imports `functools._initial_missing` instead of declaring a local one. The polyfill's internal `if initial is _initial_missing` identity check still works because both sides now reference the same object. No behavior change on any Python version. `functools._initial_missing` has existed in `functools` since well before 3.10 (it's used by the pure-Python `functools.reduce` fallback), so the import is safe on every supported version.

Part of #184352 (Python 3.15 support). Similar to #185403.

## Test plan

Verified on a locally-built Python 3.15-dev (CPython main at `heads/3.15:863c7e0`, which includes [#149591](python/cpython#149591)). The upstream signature is exposed natively:

```
$ python3.15 -c "import functools, inspect; print(inspect.signature(functools.reduce))"
(function, iterable, /, initial=_initial_missing)
$ python3.15 -c "import functools, inspect; sig = inspect.signature(functools.reduce); print(sig.parameters['initial'].default is functools._initial_missing)"
True
```

Extracted the actual `substitute_in_graph` from `torch/_dynamo/decorators.py` and ran it against both polyfill versions with no monkey-patching.

Unmodified polyfill (`_initial_missing = object()` declared locally):

```
TypeError: Signature mismatch between <built-in function reduce> and <function unmodified_reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

Fixed polyfill (`from functools import _initial_missing`): no `TypeError` at the signature check.

Regression on Python 3.12.13 (oldest currently supported):

```
$ python3 -c "import inspect, functools; inspect.signature(functools.reduce)"
ValueError: no signature found for builtin <built-in function reduce>
```

The signature check is silently skipped on Python <= 3.14, so the fix is a no-op there. `functools._initial_missing` is confirmed present on 3.12.13. Eager-vs-polyfill parity verified on every input shape:

```
reduce(lambda a, b: a+b, [1,2,3,4])       == 10     (matches functools.reduce)
reduce(lambda a, b: a+b, [1,2,3], 100)    == 106    (matches)
reduce(lambda a, b: a+b, [])              -> TypeError("reduce() of empty iterable with no initial value")  (matches)
reduce(lambda a, b: a+b, [], 99)          == 99     (matches)
```

Pre-commit checks: `python3 -m py_compile torch/_dynamo/polyfills/functools.py` and `git diff --staged --check` both clean.

Pull Request resolved: #185682
Approved by: https://github.com/ezyang

Co-authored-by: Edward Z. Yang via mergedog <ezyang@meta.com>
khushi-411 pushed a commit to khushi-411/pytorch that referenced this pull request Jun 1, 2026
….15 (pytorch#185682)

## Summary

Python 3.15 will expose an introspectable signature for `functools.reduce` using a PEP 661 sentinel default: `(function, iterable, /, initial=functools._initial_missing)`. The CPython change is [pytorch#149591](python/cpython#149591), which merged into CPython main on 2026-05-10, after 3.15.0b1 was tagged. So on 3.15.0b1 the default is still `<unrepresentable>` and `inspect.signature()` raises `ValueError`, which `substitute_in_graph` swallows and skips the check. From 3.15.0b2 the check will run.

When it runs, it compares positional parameter names, keyword-only names, and default values. The polyfill declares its own local sentinel:

```python
_initial_missing = object()

@substitute_in_graph(functools.reduce)
def reduce(function, iterable, initial=_initial_missing, /):
    ...
```

That local `object()` is a different instance from `functools._initial_missing`, so the defaults dict compares unequal and `substitute_in_graph` raises:

```
File ".../torch/_dynamo/polyfills/functools.py", line N, in <module>
  @substitute_in_graph(functools.reduce)
TypeError: Signature mismatch between <built-in function reduce> and <function reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

The polyfills loader imports every polyfill module, so this will break `import torch._dynamo` entirely on Python 3.15.0b2+. Same failure mode as pytorch#185403's `struct.pack` fix, just one step ahead of the next beta.

The fix imports `functools._initial_missing` instead of declaring a local one. The polyfill's internal `if initial is _initial_missing` identity check still works because both sides now reference the same object. No behavior change on any Python version. `functools._initial_missing` has existed in `functools` since well before 3.10 (it's used by the pure-Python `functools.reduce` fallback), so the import is safe on every supported version.

Part of pytorch#184352 (Python 3.15 support). Similar to pytorch#185403.

## Test plan

Verified on a locally-built Python 3.15-dev (CPython main at `heads/3.15:863c7e0`, which includes [pytorch#149591](python/cpython#149591)). The upstream signature is exposed natively:

```
$ python3.15 -c "import functools, inspect; print(inspect.signature(functools.reduce))"
(function, iterable, /, initial=_initial_missing)
$ python3.15 -c "import functools, inspect; sig = inspect.signature(functools.reduce); print(sig.parameters['initial'].default is functools._initial_missing)"
True
```

Extracted the actual `substitute_in_graph` from `torch/_dynamo/decorators.py` and ran it against both polyfill versions with no monkey-patching.

Unmodified polyfill (`_initial_missing = object()` declared locally):

```
TypeError: Signature mismatch between <built-in function reduce> and <function unmodified_reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

Fixed polyfill (`from functools import _initial_missing`): no `TypeError` at the signature check.

Regression on Python 3.12.13 (oldest currently supported):

```
$ python3 -c "import inspect, functools; inspect.signature(functools.reduce)"
ValueError: no signature found for builtin <built-in function reduce>
```

The signature check is silently skipped on Python <= 3.14, so the fix is a no-op there. `functools._initial_missing` is confirmed present on 3.12.13. Eager-vs-polyfill parity verified on every input shape:

```
reduce(lambda a, b: a+b, [1,2,3,4])       == 10     (matches functools.reduce)
reduce(lambda a, b: a+b, [1,2,3], 100)    == 106    (matches)
reduce(lambda a, b: a+b, [])              -> TypeError("reduce() of empty iterable with no initial value")  (matches)
reduce(lambda a, b: a+b, [], 99)          == 99     (matches)
```

Pre-commit checks: `python3 -m py_compile torch/_dynamo/polyfills/functools.py` and `git diff --staged --check` both clean.

Pull Request resolved: pytorch#185682
Approved by: https://github.com/ezyang

Co-authored-by: Edward Z. Yang via mergedog <ezyang@meta.com>
gplutop7 pushed a commit to gplutop7/pytorch that referenced this pull request Jul 15, 2026
….15 (pytorch#185682)

## Summary

Python 3.15 will expose an introspectable signature for `functools.reduce` using a PEP 661 sentinel default: `(function, iterable, /, initial=functools._initial_missing)`. The CPython change is [pytorch#149591](python/cpython#149591), which merged into CPython main on 2026-05-10, after 3.15.0b1 was tagged. So on 3.15.0b1 the default is still `<unrepresentable>` and `inspect.signature()` raises `ValueError`, which `substitute_in_graph` swallows and skips the check. From 3.15.0b2 the check will run.

When it runs, it compares positional parameter names, keyword-only names, and default values. The polyfill declares its own local sentinel:

```python
_initial_missing = object()

@substitute_in_graph(functools.reduce)
def reduce(function, iterable, initial=_initial_missing, /):
    ...
```

That local `object()` is a different instance from `functools._initial_missing`, so the defaults dict compares unequal and `substitute_in_graph` raises:

```
File ".../torch/_dynamo/polyfills/functools.py", line N, in <module>
  @substitute_in_graph(functools.reduce)
TypeError: Signature mismatch between <built-in function reduce> and <function reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

The polyfills loader imports every polyfill module, so this will break `import torch._dynamo` entirely on Python 3.15.0b2+. Same failure mode as pytorch#185403's `struct.pack` fix, just one step ahead of the next beta.

The fix imports `functools._initial_missing` instead of declaring a local one. The polyfill's internal `if initial is _initial_missing` identity check still works because both sides now reference the same object. No behavior change on any Python version. `functools._initial_missing` has existed in `functools` since well before 3.10 (it's used by the pure-Python `functools.reduce` fallback), so the import is safe on every supported version.

Part of pytorch#184352 (Python 3.15 support). Similar to pytorch#185403.

## Test plan

Verified on a locally-built Python 3.15-dev (CPython main at `heads/3.15:863c7e0`, which includes [pytorch#149591](python/cpython#149591)). The upstream signature is exposed natively:

```
$ python3.15 -c "import functools, inspect; print(inspect.signature(functools.reduce))"
(function, iterable, /, initial=_initial_missing)
$ python3.15 -c "import functools, inspect; sig = inspect.signature(functools.reduce); print(sig.parameters['initial'].default is functools._initial_missing)"
True
```

Extracted the actual `substitute_in_graph` from `torch/_dynamo/decorators.py` and ran it against both polyfill versions with no monkey-patching.

Unmodified polyfill (`_initial_missing = object()` declared locally):

```
TypeError: Signature mismatch between <built-in function reduce> and <function unmodified_reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

Fixed polyfill (`from functools import _initial_missing`): no `TypeError` at the signature check.

Regression on Python 3.12.13 (oldest currently supported):

```
$ python3 -c "import inspect, functools; inspect.signature(functools.reduce)"
ValueError: no signature found for builtin <built-in function reduce>
```

The signature check is silently skipped on Python <= 3.14, so the fix is a no-op there. `functools._initial_missing` is confirmed present on 3.12.13. Eager-vs-polyfill parity verified on every input shape:

```
reduce(lambda a, b: a+b, [1,2,3,4])       == 10     (matches functools.reduce)
reduce(lambda a, b: a+b, [1,2,3], 100)    == 106    (matches)
reduce(lambda a, b: a+b, [])              -> TypeError("reduce() of empty iterable with no initial value")  (matches)
reduce(lambda a, b: a+b, [], 99)          == 99     (matches)
```

Pre-commit checks: `python3 -m py_compile torch/_dynamo/polyfills/functools.py` and `git diff --staged --check` both clean.

Pull Request resolved: pytorch#185682
Approved by: https://github.com/ezyang

Co-authored-by: Edward Z. Yang via mergedog <ezyang@meta.com>
aws-kingrj pushed a commit to amazon-contributing/upstream-to-pytorch that referenced this pull request Jul 29, 2026
….15 (pytorch#185682)

## Summary

Python 3.15 will expose an introspectable signature for `functools.reduce` using a PEP 661 sentinel default: `(function, iterable, /, initial=functools._initial_missing)`. The CPython change is [pytorch#149591](python/cpython#149591), which merged into CPython main on 2026-05-10, after 3.15.0b1 was tagged. So on 3.15.0b1 the default is still `<unrepresentable>` and `inspect.signature()` raises `ValueError`, which `substitute_in_graph` swallows and skips the check. From 3.15.0b2 the check will run.

When it runs, it compares positional parameter names, keyword-only names, and default values. The polyfill declares its own local sentinel:

```python
_initial_missing = object()

@substitute_in_graph(functools.reduce)
def reduce(function, iterable, initial=_initial_missing, /):
    ...
```

That local `object()` is a different instance from `functools._initial_missing`, so the defaults dict compares unequal and `substitute_in_graph` raises:

```
File ".../torch/_dynamo/polyfills/functools.py", line N, in <module>
  @substitute_in_graph(functools.reduce)
TypeError: Signature mismatch between <built-in function reduce> and <function reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

The polyfills loader imports every polyfill module, so this will break `import torch._dynamo` entirely on Python 3.15.0b2+. Same failure mode as pytorch#185403's `struct.pack` fix, just one step ahead of the next beta.

The fix imports `functools._initial_missing` instead of declaring a local one. The polyfill's internal `if initial is _initial_missing` identity check still works because both sides now reference the same object. No behavior change on any Python version. `functools._initial_missing` has existed in `functools` since well before 3.10 (it's used by the pure-Python `functools.reduce` fallback), so the import is safe on every supported version.

Part of pytorch#184352 (Python 3.15 support). Similar to pytorch#185403.

## Test plan

Verified on a locally-built Python 3.15-dev (CPython main at `heads/3.15:863c7e0`, which includes [pytorch#149591](python/cpython#149591)). The upstream signature is exposed natively:

```
$ python3.15 -c "import functools, inspect; print(inspect.signature(functools.reduce))"
(function, iterable, /, initial=_initial_missing)
$ python3.15 -c "import functools, inspect; sig = inspect.signature(functools.reduce); print(sig.parameters['initial'].default is functools._initial_missing)"
True
```

Extracted the actual `substitute_in_graph` from `torch/_dynamo/decorators.py` and ran it against both polyfill versions with no monkey-patching.

Unmodified polyfill (`_initial_missing = object()` declared locally):

```
TypeError: Signature mismatch between <built-in function reduce> and <function unmodified_reduce at 0x...>:
  (function, iterable, /, initial=_initial_missing)
  != (function, iterable, initial=<object object at 0x...>, /)
```

Fixed polyfill (`from functools import _initial_missing`): no `TypeError` at the signature check.

Regression on Python 3.12.13 (oldest currently supported):

```
$ python3 -c "import inspect, functools; inspect.signature(functools.reduce)"
ValueError: no signature found for builtin <built-in function reduce>
```

The signature check is silently skipped on Python <= 3.14, so the fix is a no-op there. `functools._initial_missing` is confirmed present on 3.12.13. Eager-vs-polyfill parity verified on every input shape:

```
reduce(lambda a, b: a+b, [1,2,3,4])       == 10     (matches functools.reduce)
reduce(lambda a, b: a+b, [1,2,3], 100)    == 106    (matches)
reduce(lambda a, b: a+b, [])              -> TypeError("reduce() of empty iterable with no initial value")  (matches)
reduce(lambda a, b: a+b, [], 99)          == 99     (matches)
```

Pre-commit checks: `python3 -m py_compile torch/_dynamo/polyfills/functools.py` and `git diff --staged --check` both clean.

Pull Request resolved: pytorch#185682
Approved by: https://github.com/ezyang

Co-authored-by: Edward Z. Yang via mergedog <ezyang@meta.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL