gh-102433: Use `inspect.getattr_static` in `typing._ProtocolMeta.instancecheck` by AlexWaygood · Pull Request #103034 · python/cpython · GitHub

Use getattr_static rather than hasattr in _ProtocolMeta.__instancecheck__, in order to avoid unexpectedly evaluating descriptors and __getattr__ methods in the process of calling isinstance() against runtime-checkable protocols.

This slows down isinstance() checks against runtime-checkable protocols by around 46% (on my machine), but the consensus in python/typing#1363 is that the performance penalty is worth it because:

Any isinstance() check causing an expensive (or otherwise badly behaved) property or __getattr__ method to be evaluated is pretty surprising behaviour.
Bringing the runtime's understanding of runtime-checkable protocols closer to the way static type checkers understand these classes is highly desirable.
Runtime-checkable protocols should not be used for performance-critical code anyway (this is now clearly documented following Typing: Document performance pitfalls of protocols decorated with @runtime_checkable #102936.)

In various subtle edge cases, this change will mean that some classes that used to be considered structural "subclasses" of a certain runtime-checkable protocol will no longer be considered subclasses of that protocol (and vice versa). As such, this can't be backported. (Perhaps we should consider documenting the existing behaviour for 3.11 and 3.10.)

Issue: isinstance on runtime_checkable Protocol has side-effects for @property methods #102433

Note that most discussion has actually taken place in a python/typing issue rather than a CPython issue:

How should isinstance() work at runtime when comparing classes with properties against runtime-checkable protocols? typing#1363

I'd be very interested to know if any maintainers of libraries that use type annotations at runtime have any concerns about this proposed change. Off the top of my head, I can think of:

pydantic (@samuelcolvin?)
cattrs (@Tinche?)
beartype (@leycec)
typeguard (@agronholm?)
fastAPI (@tiangolo?)

I’m on vacation, please find another reviewer.

Thanks for tagging me, I'll discuss with the pydantic team and respond in a few days.

Thanks so much for the apropos ping, @AlexWaygood. You do the good typing work! Sadly, this...

This slows down isinstance() checks against runtime-checkable protocols by around 46% (on my machine)...

...is a non-starter. That's measurably unacceptable degradation. The existing Protocol API works. There's no demonstrable justification for both breaking that implementation in backwards-incompatible edge cases and imposing additional runtime costs while doing so.

Python ≥ 3.11's renewed focus on runtime performance is germane here. Efficiency is no longer orthogonal to our goals in the Python ecosystem. Efficiency is increasingly a goal in and of itself. Do not sacrifice quantifiable speed at the golden altar of well-intended but ultimately futile attempts to force parity between static and runtime type-checkers.

Static and runtime type-checkers will inevitably disagree. These are profoundly different understandings of what type-checking even means. Reducing the scope of that disagreement is useful – but preserving (or ideally improving) runtime performance is substantially more useful.

Static type-checking serves Python's runtime needs by improving the robustness and reliability of Python at runtime. Static analysis is subordinate to runtime. Runtime always comes first. Not last. We're all here for what Python accomplishes at runtime. To paraphrase President Truman:

The buck stops at runtime.

Should be fine for cattrs, we have separate build and run stages for the de/serializers.

Thanks for the feedback @leycec! To be clear, I fully agree with your sentiment here, and have no wish to make the CPython runtime subservient to the needs of static typing:

Static type-checking serves Python's runtime needs by improving the robustness and reliability of Python at runtime. Static analysis is subordinate to runtime. Runtime always comes first. Not last. We're all here for what Python accomplishes at runtime.

I realise that in my summary of this PR at the top, I maybe should have been clearer about the bug that this PR is trying to fix. Ultimately this PR is first and foremost about fixing unexpected behaviour at runtime. Improving consistency between how static type checkers and the runtime see certain classes is a "nice to have" that comes along with the fix.

Here is a demonstration of the user-visible bug that this PR is trying to fix (which has been reported to the CPython repo multiple times, so is clearly surprising behaviour for users). Properties, other descriptors, and __getattr__ methods are all currently "called" if you compare an object against a runtime-checkable protocol using isinstance(). This is the issue that this PR is trying to fix.

REPL session demonstrating the bug as it exists on CPython `main`

>>> import time
>>> from typing import runtime_checkable, Protocol
>>> @runtime_checkable
... class HasX(Protocol):
...     x: int
...
>>> class A:
...     @property
...     def x(self):
...         raise RuntimeError('no')
...
>>> class B:
...     @property
...     def x(self):
...         time.sleep(3600)
...         return 42
...
>>> class C:
...     def __getattr__(self, name):
...         raise RuntimeError('no')
...
>>> isinstance(A(), HasX)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in __instancecheck__
    if all(hasattr(instance, attr) and
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in <genexpr>
    if all(hasattr(instance, attr) and
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "<stdin>", line 4, in x
RuntimeError: no
>>> isinstance(C(), HasX)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in __instancecheck__
    if all(hasattr(instance, attr) and
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in <genexpr>
    if all(hasattr(instance, attr) and
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "<stdin>", line 3, in __getattr__
RuntimeError: no
>>> isinstance(B(), HasX)  # this will take an hour to evaluate

I don't like (at all) that this results in a 46% performance degradation for simple isinstance() checks against runtime-checkable protocols, and I suggested a few workarounds in python/typing#1363 that would still fix the bug for most users of runtime-checkable protocols, while having a much smaller performance degradation (e.g. python/typing#1363 (comment)). These were criticized, however, on the grounds that:

they weren't particularly "principled" solutions, which might have bugs in subtle edge cases
they felt like premature optimizations, since isinstance() checks against runtime-checkable protocols are abysmally slow anyway, so users should probably not be using runtime-checkable protocols for performance-critical code.

There are three options here that I can see:

Go with this solution (take the performance hit for now, but see if we can optimise _ProtocolMeta.__instancecheck__ and/or inspect.getattr_static in followup PRs).
Go with a "less principled" solution that doesn't use inspect.getattr_static, and therefore has a smaller performance degradation, but might have bugs in subtle edge cases.
Declare the bug as "wontfix"; document clearly in the typing documentation that you should be careful about comparing instances of a class X against a runtime-checkable protocol if class X has properties with side effects, which would be triggered during the course of an isinstance() check against a runtime-checkable protocol.

IMHO, correct behavior should always come first, then maybe good performance.

As far as typeguard is concerned, I don't see a problem here.

I'm pretty sure this shouldn't affect pydantic since we tend to avoid isinstance() checks unless they're unavoidable, we also don't have a validator for Protocol, and if we do, I guess we would need our own logic.

I'm therefore neutral on it.

FWIW (and it's not worth much since it's probably too late), my personal opinion is that torturing isinstance to provide partial runtime type checking is a mistake - it would be better if isinstance just checked that the first arg's type inherits from the second arg - no magic, predicable behaviour, predictably fast. Then python could provide a second method for more powerful type checking with worse performance and more magic. I would argue that the decision to contort isinstance is the route cause headaches like this, and many more.

FWIW (and it's not worth much since it's probably too late), my personal opinion is that torturing isinstance to provide partial runtime type checking is a mistake - it would be better if isinstance just checked that the first arg's type inherits from the second arg - no magic, predicable behaviour, predictably fast. Then python could provide a second method for more powerful type checking with worse performance and more magic. I would argue that the decision to contort isinstance is the route cause headaches like this, and many more.

FWIW I agree fully with this, and I'm quite glad that protocols are by default not runtime-checkable. But, as you say, it's almost certainly too late to change this now without going through a painful deprecation period that (in my opinion) would cause more trouble than it's worth.

I agree with you both. The checking of run-time protocols would probably be best left to run-time type checkers, but getting rid of run-time checkable protocols would be a PITA at this point.

typeguard is lazy and uses instance() on run-time protocols, and doesn't currently provide support for static protocols. That could be changed with enough motivation and free time though.

Note the recent comment on the issue reinforcing that the current behavior of @runtime_checkable executing arbitrary properties causes (and hides) real bugs.

There are three options here that I can see:

Go with this solution (take the performance hit for now, but see if we can optimise _ProtocolMeta.__instancecheck__ and/or inspect.getattr_static in followup PRs).

Go with a "less principled" solution that doesn't use inspect.getattr_static, and therefore has a smaller performance degradation, but might have bugs in subtle edge cases.

Declare the bug as "wontfix"; document clearly in the typing documentation that you should be careful about comparing instances of a class X against a runtime-checkable protocol if class X has properties with side effects, which would be triggered during the course of an isinstance() check against a runtime-checkable protocol.

I agree that these are the options.

I think the advice we would have to document in the case of (3) is basically un-followable. If you are doing isinstance(x, SomeProtocol) in your code, it's because you don't already know the type of x, so there's no way you can "be careful" that x isn't of a type with dangerous or costly executable properties.

I agree with @agronholm that we should prioritize correct behavior first, and then seek to optimize it. IMO correct behavior (purely from a runtime perspective, ignoring static typing) is that we should not execute arbitrary descriptor code in an isinstance check, and inspect.getattr_static is how to reliably do that. So I favor (1).

I'm going to merge this once the CI passes.

We've already merged two optimisation PRs into main, which, taken together, basically make up for the performance hit that this PR gives:

There are two more PRs (which are slightly riskier) that have yet to be decided on:

gh-74690: typing: Simplify and optimise _ProtocolMeta.__instancecheck__ #103159
gh-74690: typing: Call _get_protocol_attrs and _callable_members_only at protocol class creation time, not during isinstance() checks #103160

If anybody has any opinions on either of those, feedback is very welcome! Once decisions have been taken on the two open PRs, I'll create another PR adding a separate NEWS entry summarising all of the performance work that's been done (and the internal changes to typing.py that were needed for the performance optimisations).

I'd also like to look into optimising inspect.getattr_static itself, as well, since the core reason for the performance hit here is obviously that inspect.getattr_static is very slow compared to hasattr.

Many thanks to everybody who took their time to provide feedback on this PR, I really appreciate it!

Sorry for my delay replying here. 🙈 Thanks a lot for pinging me. 🙏

As FastAPI and my other tools heavily depend and interact with Pydantic (or copy its ideas), if this doesn't affect Pydantic (as Samuel confirmed above), it doesn't affect FastAPI and friends. 🚀

…a.__instancecheck__` (python#103034)

Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). (From OE-Core rev: 2b1d07c7deb4f0247765bc737fb11a1747143edf) Signed-off-by: Wang Mingyu <wangmy@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>

Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). (From OE-Core rev: a37154b9166323d05cca970ebb37bee0d5250893) Signed-off-by: Wang Mingyu <wangmy@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

Changelog: ============ - Fix use of @deprecated on classes with __new__ but no __init__. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using isinstance() would cause AttributeError to be raised if using Python 3.7. - Change deprecated @runtime to formal API @runtime_checkable in the error message. - Fix regression in 4.6.0 where attempting to define a Protocol that was generic over a ParamSpec or a TypeVarTuple would cause TypeError to be raised. - typing_extensions is now documented at https://typing-extensions.readthedocs.io/en/latest/. - Add typing_extensions.Buffer, a marker class for buffer types, as proposed by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12. - Backport two CPython PRs fixing various issues with typing.Literal: python/cpython#23294 and python/cpython#23383. Both CPython PRs were originally, and both were backported to Python >=3.9.1, but no earlier. - A side effect of one of the changes is that equality comparisons of Literal objects will now raise a TypeError if one of the Literal objects being compared has a mutable parameter. (Using mutable parameters with Literal is not supported by PEP 586 or by any major static type checkers.) - Literal is now reimplemented on all Python versions <= 3.10.0. - Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise TypeError when the protocol is not decorated with @runtime_checkable. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 (see python/cpython#74690 for details). - A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered "frozen" at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on isinstance() checks comparing objects to the protocol. See "What's New in Python 3.12" for more details. - isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). - Backport the ability to define __init__ methods on Protocol classes, a change made in Python 3.11 (originally implemented in python/cpython#31628 - Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12. - Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex, SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as the versions from the typing module, but isinstance() checks against the typing_extensions versions are >10x faster at runtime on Python <3.12. - Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. - Add typing_extensions.get_original_bases, a backport of types.get_original_bases, introduced in Python 3.12 (CPython PR python/cpython#101827, originally - This function should always produce correct results when called on classes constructed using features from typing_extensions. - Constructing a call-based TypedDict using keyword arguments for the fields now causes a DeprecationWarning to be emitted. This matches the behaviour of typing.TypedDict on 3.11 and 3.12. - Backport the implementation of NewType from 3.10 (where it is implemented as a class rather than a function). This allows user-defined NewTypes to be pickled. - Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed. - Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695. - Backport changes to the repr of typing.Unpack that were made in order to implement PEP 692 (backport of python/cpython#104048). Signed-off-by: Wang Mingyu <wangmy@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [typing-extensions](https://togithub.com/python/typing_extensions) ([changelog](https://togithub.com/python/typing_extensions/blob/main/CHANGELOG.md)) | `==4.5.0` -> `==4.6.3` | [![age](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/compatibility-slim/4.5.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/confidence-slim/4.5.0)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>python/typing_extensions</summary> ### [`v4.6.3`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-463-June-1-2023) [Compare Source](https://togithub.com/python/typing_extensions/compare/4.6.2...4.6.3) - Fix a regression introduced in v4.6.0 in the implementation of runtime-checkable protocols. The regression meant that doing `class Foo(X, typing_extensions.Protocol)`, where `X` was a class that had `abc.ABCMeta` as its metaclass, would then cause subsequent `isinstance(1, X)` calls to erroneously raise `TypeError`. Patch by Alex Waygood (backporting the CPython[https://github.com/python/cpython/pull/105152](https://togithub.com/python/cpython/pull/105152)l/105152). - Sync the repository's LICENSE file with that of CPython. `typing_extensions` is distributed under the same license as CPython itself. - Skip a problematic test on Python 3.12.0b1. The test fails on 3.12.0b1 due to a bug in CPython, which will be fixed in 3.12.0b2. The `typing_extensions` test suite now passes on 3.12.0b1. ### [`v4.6.2`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-462-May-25-2023) [Compare Source](https://togithub.com/python/typing_extensions/compare/4.6.1...4.6.2) - Fix use of `@deprecated` on classes with `__new__` but no `__init__`. Patch by Jelle Zijlstra. - Fix regression in version 4.6.1 where comparing a generic class against a runtime-checkable protocol using `isinstance()` would cause `AttributeError` to be raised if using Python 3.7. ### [`v4.6.1`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-461-May-23-2023) [Compare Source](https://togithub.com/python/typing_extensions/compare/4.6.0...4.6.1) - Change deprecated `@runtime` to formal API `@runtime_checkable` in the error message. Patch by Xuehai Pan. - Fix regression in 4.6.0 where attempting to define a `Protocol` that was generic over a `ParamSpec` or a `TypeVarTuple` would cause `TypeError` to be raised. Patch by Alex Waygood. ### [`v4.6.0`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-460-May-22-2023) [Compare Source](https://togithub.com/python/typing_extensions/compare/4.5.0...4.6.0) - `typing_extensions` is now documented at https://typing-extensions.readthedocs.io/en/latest/. Patch by Jelle Zijlstra. - Add `typing_extensions.Buffer`, a marker class for buffer types, as proposed by PEP 688. Equivalent to `collections.abc.Buffer` in Python 3.12. Patch by Jelle Zijlstra. - Backport two CPython PRs fixing various issues with `typing.Literal`: [https://github.com/python/cpython/pull/23294](https://togithub.com/python/cpython/pull/23294)3294 [https://github.com/python/cpython/pull/23383](https://togithub.com/python/cpython/pull/23383)ll/23383. Both CPython PRs were originally by Yurii Karabas, and both were backported to Python >=3.9.1, but no earlier. Patch by Alex Waygood. A side effect of one of the changes is that equality comparisons of `Literal` objects will now raise a `TypeError` if one of the `Literal` objects being compared has a mutable parameter. (Using mutable parameters with `Literal` is not supported by PEP 586 or by any major static type checkers.) - `Literal` is now reimplemented on all Python versions <= 3.10.0. The `typing_extensions` version does not suffer from the bug that was fixed in [https://github.com/python/cpython/pull/29334](https://togithub.com/python/cpython/pull/29334)9334. (The CPython bugfix was backported to CPython 3.10.1 and 3.9.8, but no earlier.) - Backport [CPython PR 26067](https://togithub.com/python/cpython/pull/26067) (originally by Yurii Karabas), ensuring that `isinstance()` calls on protocols raise `TypeError` when the protocol is not decorated with `@runtime_checkable`. Patch by Alex Waygood. - Backport several significant performance improvements to runtime-checkable protocols that have been made in Python 3.12 ([https://github.com/python/cpython/issues/74690](https://togithub.com/python/cpython/issues/74690)es/74690 for details). Patch by Alex Waygood. A side effect of one of the performance improvements is that the members of a runtime-checkable protocol are now considered “frozen” at runtime as soon as the class has been created. Monkey-patching attributes onto a runtime-checkable protocol will still work, but will have no impact on `isinstance()` checks comparing objects to the protocol. See ["What's New in Python 3.12"](https://docs.python.org/3.12/whatsnew/3.12.html#typing) for more details. - `isinstance()` checks against runtime-checkable protocols now use `inspect.getattr_static()` rather than `hasattr()` to lookup whether attributes exist (backport[https://github.com/python/cpython/pull/103034](https://togithub.com/python/cpython/pull/103034)3034). This means that descriptors and `__getattr__` methods are no longer unexpectedly evaluated during `isinstance()` checks against runtime-checkable protocols. However, it may also mean that some objects which used to be considered instances of a runtime-checkable protocol on older versions of `typing_extensions` may no longer be considered instances of that protocol using the new release, and vice versa. Most users are unlikely to be affected by this change. Patch by Alex Waygood. - Backport the ability to define `__init__` methods on Protocol classes, a change made in Python 3.11 (originally implemented[https://github.com/python/cpython/pull/31628](https://togithub.com/python/cpython/pull/31628)ll/31628 by Adrian Garcia Badaracco). Patch by Alex Waygood. - Speedup `isinstance(3, typing_extensions.SupportsIndex)` by >10x on Python <3.12. Patch by Alex Waygood. - Add `typing_extensions` versions of `SupportsInt`, `SupportsFloat`, `SupportsComplex`, `SupportsBytes`, `SupportsAbs` and `SupportsRound`. These have the same semantics as the versions from the `typing` module, but `isinstance()` checks against the `typing_extensions` versions are >10x faster at runtime on Python <3.12. Patch by Alex Waygood. - Add `__orig_bases__` to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples. Other TypedDicts and NamedTuples already had the attribute. Patch by Adrian Garcia Badaracco. - Add `typing_extensions.get_original_bases`, a backport of [`types.get_original_bases`](https://docs.python.org/3.12/library/types.html#types.get_original_bases), introduced in Python 3.12 (CPython[https://github.com/python/cpython/pull/101827](https://togithub.com/python/cpython/pull/101827)l/101827, originally by James Hilton-Balfe). Patch by Alex Waygood. This function should always produce correct results when called on classes constructed using features from `typing_extensions`. However, it may produce incorrect results when called on some `NamedTuple` or `TypedDict` classes that use `typing.{NamedTuple,TypedDict}` on Python <=3.11. - Constructing a call-based `TypedDict` using keyword arguments for the fields now causes a `DeprecationWarning` to be emitted. This matches the behaviour of `typing.TypedDict` on 3.11 and 3.12. - Backport the implementation of `NewType` from 3.10 (where it is implemented as a class rather than a function). This allows user-defined `NewType`s to be pickled. Patch by Alex Waygood. - Fix tests and import on Python 3.12, where `typing.TypeVar` can no longer be subclassed. Patch by Jelle Zijlstra. - Add `typing_extensions.TypeAliasType`, a backport of `typing.TypeAliasType` from PEP 695. Patch by Jelle Zijlstra. - Backport changes to the repr of `typing.Unpack` that were made in order to implement [PEP 692](https://peps.python.org/pep-0692/) (backport of [https://github.com/python/cpython/pull/104048](https://togithub.com/python/cpython/pull/104048)4048). Patch by Alex Waygood. </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/allenporter/flux-local).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

It was caused by the new [4.6 version of typing-extensions](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md#release-460-may-22-2023) used by jsonschema-spec to validate protocol. > isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backporting python/cpython#103034). This means that descriptors and getattr methods are no longer unexpectedly evaluated during isinstance() checks against runtime-checkable protocols. However, it may also mean that some objects which used to be considered instances of a runtime-checkable protocol on older versions of typing_extensions may no longer be considered instances of that protocol using the new release, and vice versa. Most users are unlikely to be affected by this change. Patch by Alex Waygood. Temporary workaround will be downgrading typing-extensions to version below 4.6: pip install "typing-extensions<4.6" Bug #50048

…python>=3.12.0` Any objects using `__getattr__`, e.g. `Parameter` break on older versions [see PR](python/cpython#103034)

inspect.getattr_static uses inspect._check_instance which uses object.__getattribute__(obj, "__dict__"). This fails for bound instance methods (although obj.__dict__ does work), so they cannot fulfill runtime_checkable Protocols anymore.

from typing import Protocol, runtime_checkable


@runtime_checkable
class Labelled(Protocol):
    label: str


class A:
    def f(self):
        pass


A.f.label = "f"

a = A()
print(a.f.__dict__)  # {'label': 'f'}
print(object.__getattribute__(a.f, "__dict__"))  # AttributeError

assert isinstance(a.f, Labelled)  # AssertionError

Is this an intended/known change? Could it be noted in the changelog here as an example of case that will no longer work?

@GDYendell, I will admit that I did indeed fail to consider how this would change the behaviour of runtime-checkable protocols when used in conjunction with bound instance methods that have extra attributes monkeypatched onto them.

Can you possibly open a new issue about this? I haven't got the time to look at it right now, and comment threads on long-merged PRs are unfortunately likely to get forgotten about.

AlexWaygood added 3 commits March 24, 2023 19:50

Implementation

212fce2

Tests

ef71395

docs

4407d60

AlexWaygood requested a review from carljm March 25, 2023 17:08

AlexWaygood requested review from Fidget-Spinner, JelleZijlstra and gvanrossum as code owners March 25, 2023 17:08

bedevere-bot mentioned this pull request Mar 25, 2023

implement winreg QueryValue / SetValue using QueryValueEx / SetValueEx #102344

Closed

bedevere-bot added the awaiting core review label Mar 25, 2023

AlexWaygood changed the title gh-102344: Use inspect.getattr_static in typing._ProtocolMeta.__instancecheck__ gh-102433: Use inspect.getattr_static in typing._ProtocolMeta.__instancecheck__ Mar 25, 2023

bedevere-bot mentioned this pull request Mar 25, 2023

isinstance on runtime_checkable Protocol has side-effects for @property methods #102433

Closed

AlexWaygood added topic-typing 3.12 only security fixes stdlib Standard Library Python modules in the Lib/ directory labels Mar 25, 2023

Wrong issue number

44eba6b

AlexWaygood mentioned this pull request Mar 25, 2023

How should isinstance() work at runtime when comparing classes with properties against runtime-checkable protocols? python/typing#1363

Closed

gvanrossum removed their request for review March 25, 2023 21:09

Merge branch 'main' into runtime-protocols-getattr_static

9989eea

AlexWaygood merged commit 6d59c9e into python:main Apr 2, 2023

bedevere-bot removed the awaiting merge label Apr 2, 2023

AlexWaygood deleted the runtime-protocols-getattr_static branch April 2, 2023 13:22

AlexWaygood mentioned this pull request Apr 2, 2023

gh-74690: typing: Simplify and optimise _ProtocolMeta.__instancecheck__ #103159

Merged

AlexWaygood mentioned this pull request Apr 5, 2023

gh-74690: Further optimise typing._ProtocolMeta.__instancecheck__ #103280

Merged

gaogaotiantian pushed a commit to gaogaotiantian/cpython that referenced this pull request Apr 8, 2023

pythongh-102433 : Use inspect.getattr_static in `typing._ProtocolMet…

3a0530c

…a.__instancecheck__` (python#103034)

warsaw pushed a commit to warsaw/cpython that referenced this pull request Apr 11, 2023

pythongh-102433 : Use inspect.getattr_static in `typing._ProtocolMet…

c3c6139

…a.__instancecheck__` (python#103034)

AlexWaygood mentioned this pull request May 24, 2023

[Bug] typing_extensions.Protocol unsupported under typing_extensions ≥ 4.6.0 beartype/beartype#241

Closed

This was referenced May 31, 2023

typing_extensions >= 4.6.0 causes pip unit test failure tensorflow/tensorflow#60687

Closed

runtime checkable protocols potential raising error with custom getattribute #105134

Closed

This was referenced Jun 10, 2023

'_io.BufferedReader' object has no attribute 'decode' python-openapi/openapi-spec-validator#209

Closed

BaseFilePathHandler has different working logic in Python 3.7 and Python 3.8, causing error reading files. p1c2u/jsonschema-path#14

Closed

GDYendell mentioned this pull request Nov 20, 2024

Bound instance methods fail to satisfy runtime_checkable Protocol #127059

Closed

dangotbanned mentioned this pull request Dec 14, 2024

chore: Add some Compliant Protocols narwhals-dev/narwhals#1522

Merged

10 tasks

coderabbitai Bot mentioned this pull request Jul 30, 2026

Invalidate semantic model caches koxudaxi/datamodel-code-generator#3666

Merged

coderabbitai Bot mentioned this pull request Aug 7, 2026

fix: add core presentation renderer registration seam for plugin channels MervinPraison/PraisonAI#3783

Merged

5 tasks

gh-102433: Use `inspect.getattr_static` in `typing._ProtocolMeta.__instancecheck__` by AlexWaygood · Pull Request #103034 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-102433: Use inspect.getattr_static in typing._ProtocolMeta.__instancecheck__ - #103034

gh-102433: Use inspect.getattr_static in typing._ProtocolMeta.__instancecheck__#103034AlexWaygood merged 11 commits intopython:mainpython/cpython:mainfrom AlexWaygood:runtime-protocols-getattr_staticAlexWaygood/cpython:runtime-protocols-getattr_staticCopy head branch name to clipboard

Conversation

AlexWaygood commented Mar 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood commented Mar 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gvanrossum commented Mar 25, 2023

Uh oh!

samuelcolvin commented Mar 25, 2023

Uh oh!

leycec commented Mar 26, 2023

Uh oh!

Tinche commented Mar 26, 2023

Uh oh!

AlexWaygood commented Mar 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood commented Mar 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agronholm commented Mar 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samuelcolvin commented Mar 27, 2023

Uh oh!

AlexWaygood commented Mar 27, 2023

Uh oh!

agronholm commented Mar 27, 2023

Uh oh!

carljm commented Mar 27, 2023

Uh oh!

AlexWaygood commented Apr 2, 2023

Uh oh!

AlexWaygood commented Apr 2, 2023

Uh oh!

tiangolo commented Apr 2, 2023

Uh oh!

GDYendell commented Nov 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood commented Nov 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

gh-102433: Use `inspect.getattr_static` in `typing._ProtocolMeta.instancecheck` by AlexWaygood · Pull Request #103034 · python/cpython · GitHub

gh-102433: Use inspect.getattr_static in typing._ProtocolMeta.instancecheck - #103034

gh-102433: Use inspect.getattr_static in typing._ProtocolMeta.instancecheck#103034
AlexWaygood merged 11 commits into
python:mainfrom
AlexWaygood:runtime-protocols-getattr_static

AlexWaygood commented Mar 25, 2023 •
edited
Loading

AlexWaygood commented Mar 25, 2023 •
edited
Loading

AlexWaygood commented Mar 26, 2023 •
edited
Loading

AlexWaygood commented Mar 26, 2023 •
edited
Loading

agronholm commented Mar 26, 2023 •
edited
Loading

GDYendell commented Nov 20, 2024 •
edited
Loading