allow object type in dict.get and dict.pop - #15471
Open
randolf-scholz wants to merge 4 commits into
Open
randolf-scholz wants to merge 4 commits into
randolf-scholz wants to merge 4 commits into
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
This comment has been minimized.
This comment has been minimized.
This was referenced Feb 28, 2026
This comment has been minimized.
Copy link
Copy Markdown
Member
|
I count 28 errors going away in the ecosystem and 9 errors being added |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
Author
|
Well those 9 errors are as expected all override type errors which are unavoidable if we use key: object. (they could be avoided by using key: Any but as we discussed you strongly advise for object in the case of builtins.dict. Btw. is it really approapriate that typeshed still only uses mypy the reference here? Shouldn't we also consider impact with other type checkers? For instance, with ty I noticed this annoyance which I believe should go away with this PR (though there may be some more specific underlying issue) def is_config(obj: object):
if not isinstance(obj, dict):
return False
if not isinstance(obj.get("_target_"), str): # ty: expected Never, found "_target_"
return False
return True |
Sorry, something went wrong.
Copy link
Copy Markdown
Member
Yeah I agree! Sorry, I only got partway through a review last night |
Sorry, something went wrong.
Copy link
Copy Markdown
Member
We've discussed this in the past (#7642) — we could certainly discuss it again now that there are even more type checkers on the scene. Mypy is still the most popular type checker around, so it will continue to have a lot of weight in our decisions here for a while, but as the author of another type checker I do obviously agree that we should consider other type checkers as well when making decisions in this repo 😄 |
Sorry, something went wrong.
This comment has been minimized.
This comment has been minimized.
Copy link
Copy Markdown
Collaborator
|
I don't think this is a good idea. If you use the wrong thing as the key, you will not get an error. After all, the type checker is supposed to check the types. For this PR, mypy_primer is not a very good way to consider the real-world impact of this, because most projects don't contain the mistakes that the key parameter's type is supposed to catch: someone has already noticed and fixed those bugs likely before committing to git, perhaps because their editor complained that the key type was wrong. IMO we should start by allowing None, and reconsider this later if that is not enough in practice. |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
Author
|
@Akuli This has already been discussed in some of the linked issues, the current behavior leads to false positives and doesn't align with runtime semantics. What you want is a linter rule, but such a rule should live as a special case inside a type-checker rather than typeshed itself. There are some very clear cases were the currently behavior hurts, such as literal vs dynamic string, union types, etc. If we get something like @srittau's suggested Overlapping type type, then this may be a good substitute. |
Sorry, something went wrong.
This comment has been minimized.
Copy link
Copy Markdown
Member
|
We could implement something resembling the suggested Overlapping[] type in typeshed, with Overlapping: TypeAlias = T | object. That would at least allow type checkers that want to provide a little more safety to do so; pycroscope already implements Overlapping[]. But in general I'm not enthusiastic about this change. Type checkers exist to find wrong code, and code that passes a type to dict.get() that has nothing to do with the type in the dictionary is almost certainly wrong. |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
Author
I agree with this statement, but just because some key is not a subtype of the dictionary key type does not mean it has nothing to do with the type of the dictionary. Consider:
|
Sorry, something went wrong.
Copy link
Copy Markdown
Collaborator
|
I like the Overlapping[T] idea. It is similar to _typeshed.MaybeNone. How about the following approach?
Notes:
|
Sorry, something went wrong.
Copy link
Copy Markdown
Collaborator
|
I've had a quick read through this discussion. I think allowing None would cover the case that prompted me to open |
Sorry, something went wrong.
randolf-scholz
force-pushed
the
dict_signatures
branch
from
dfe54ff to
d796036
Compare
August 25, 2026 13:52
Copy link
Copy Markdown
Contributor
|
Diff from mypy_primer, showing the effect of this PR on open source code: beartype (https://github.com/beartype/beartype)
+ beartype/_check/convert/_reduce/redmain.py:702: error: Unused "type: ignore" comment [unused-ignore]
+ beartype/_check/convert/_reduce/redmain.py:780: error: Unused "type: ignore" comment [unused-ignore]
+ beartype/_check/code/_pep/pep484585/codepep484585container.py:99: error: Unused "type: ignore" comment [unused-ignore]
+ beartype/door/_cls/pep/pep484585/doorpep484585subscripted.py:50: error: Unused "type: ignore" comment [unused-ignore]
+ beartype/door/_cls/util/doorclsmap.py:91: error: Unused "type: ignore" comment [unused-ignore]
pylint (https://github.com/pycqa/pylint)
+ pylint/config/arguments_provider.py:54: error: Unused "type: ignore" comment [unused-ignore]
+ pylint/config/arguments_provider.py:54: error: Incompatible types in "yield" (actual type "tuple[None, list[tuple[str, dict[str, str | bool | int | Pattern[str] | Iterable[str | int | Pattern[str]] | type[_CallbackAction] | Callable[[Any], Any] | Callable[[Any, Any, Any, Any], Any] | None], Any]]]", expected type "tuple[str, list[tuple[str, dict[str, str | bool | int | Pattern[str] | Iterable[str | int | Pattern[str]] | type[_CallbackAction] | Callable[[Any], Any] | Callable[[Any, Any, Any, Any], Any] | None], Any]]] | tuple[None, dict[str, list[tuple[str, dict[str, str | bool | int | Pattern[str] | Iterable[str | int | Pattern[str]] | type[_CallbackAction] | Callable[[Any], Any] | Callable[[Any, Any, Any, Any], Any] | None], Any]]]]") [misc]
+ pylint/config/arguments_provider.py:54: note: Error code "misc" not covered by "type: ignore[call-overload]" comment
rotki (https://github.com/rotki/rotki)
+ rotkehlchen/tasks/events.py:829: error: Unused "type: ignore" comment [unused-ignore]
+ rotkehlchen/chain/aggregator.py:564: error: Unused "type: ignore" comment [unused-ignore]
+ rotkehlchen/chain/aggregator.py:885: error: Unused "type: ignore" comment [unused-ignore]
+ rotkehlchen/chain/aggregator.py:888: error: Unused "type: ignore" comment [unused-ignore]
+ rotkehlchen/chain/aggregator.py:1097: error: Unused "type: ignore" comment [unused-ignore]
altair (https://github.com/vega/altair)
+ altair/utils/html.py:417: error: Unused "type: ignore" comment [unused-ignore]
+ altair/utils/html.py:422: error: Item "str" of "Literal['standard', 'universal', 'inline', 'olli'] | Template" has no attribute "render" [union-attr]
operator (https://github.com/canonical/operator)
- ops/_private/harness.py:2689: error: No overload variant of "pop" of "dict" matches argument types "str", "None" [call-overload]
- ops/_private/harness.py:2689: note: Possible overload variants:
- ops/_private/harness.py:2689: note: def pop(self, int, /) -> dict[str, Any]
- ops/_private/harness.py:2689: note: def pop(self, int, dict[str, Any], /) -> dict[str, Any]
- ops/_private/harness.py:2689: note: def [_T] pop(self, int, _T, /) -> dict[str, Any] | _T
django-stubs (https://github.com/typeddjango/django-stubs)
+ django-stubs/utils/datastructures.pyi:77: error: Signature of "get" incompatible with supertype "builtins.dict" [override]
+ django-stubs/utils/datastructures.pyi:77: note: Superclass:
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def get(self, object, None = ..., /) -> _V | None
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def get(self, object, _V, /) -> _V
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def [_T] get(self, object, _T, /) -> _V | _T
+ django-stubs/utils/datastructures.pyi:77: note: Subclass:
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def get(self, key: _K, default: None = ...) -> _V | None
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def get(self, key: _K, default: _V) -> _V
+ django-stubs/utils/datastructures.pyi:77: note: @overload
+ django-stubs/utils/datastructures.pyi:77: note: def [_Z] get(self, key: _K, default: _Z) -> _V | _Z
+ django-stubs/contrib/sessions/backends/base.pyi:34: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: Superclass:
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: def pop(self, object, /) -> Any
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: def pop(self, object, Any, /) -> Any
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: def [_T] pop(self, object, _T, /) -> Any | _T
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: Subclass:
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: def pop(self, key: str) -> Any
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: @overload
+ django-stubs/contrib/sessions/backends/base.pyi:34: note: def pop(self, key: str, default: Any) -> Any
+ django-stubs/http/request.pyi:223: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ django-stubs/http/request.pyi:223: note: Superclass:
+ django-stubs/http/request.pyi:223: note: @overload
+ django-stubs/http/request.pyi:223: note: def pop(self, object, /) -> str
+ django-stubs/http/request.pyi:223: note: @overload
+ django-stubs/http/request.pyi:223: note: def pop(self, object, str, /) -> str
+ django-stubs/http/request.pyi:223: note: @overload
+ django-stubs/http/request.pyi:223: note: def [_T] pop(self, object, _T, /) -> str | _T
+ django-stubs/http/request.pyi:223: note: Subclass:
+ django-stubs/http/request.pyi:223: note: @overload
+ django-stubs/http/request.pyi:223: note: def pop(self, str | bytes, /) -> Never
+ django-stubs/http/request.pyi:223: note: @overload
+ django-stubs/http/request.pyi:223: note: def [_Z] pop(self, str | bytes, str | _Z = ..., /) -> Never
discord.py (https://github.com/Rapptz/discord.py)
+ discord/state.py:372: error: Unused "type: ignore" comment [unused-ignore]
+ discord/state.py:437: error: Unused "type: ignore" comment [unused-ignore]
+ discord/state.py:474: error: Unused "type: ignore" comment [unused-ignore]
+ discord/state.py:478: error: Unused "type: ignore" comment [unused-ignore]
+ discord/state.py:497: error: Unused "type: ignore" comment [unused-ignore]
- discord/ui/view.py:959: error: No overload variant of "pop" of "dict" matches argument types "tuple[int, str]", "None" [call-overload]
- discord/ui/view.py:959: note: Possible overload variants:
- discord/ui/view.py:959: note: def pop(self, Pattern[str], /) -> type[DynamicItem[Item[Any]]]
- discord/ui/view.py:959: note: def pop(self, Pattern[str], type[DynamicItem[Item[Any]]], /) -> type[DynamicItem[Item[Any]]]
- discord/ui/view.py:959: note: def [_T] pop(self, Pattern[str], _T, /) -> type[DynamicItem[Item[Any]]] | _T
+ discord/ui/view.py:984: error: Unused "type: ignore" comment [unused-ignore]
- discord/ui/view.py:979: error: No overload variant of "pop" of "dict" matches argument types "tuple[int, str]", "None" [call-overload]
- discord/ui/view.py:979: note: Possible overload variants:
- discord/ui/view.py:979: note: def pop(self, Pattern[str], /) -> type[DynamicItem[Item[Any]]]
- discord/ui/view.py:979: note: def pop(self, Pattern[str], type[DynamicItem[Item[Any]]], /) -> type[DynamicItem[Item[Any]]]
- discord/ui/view.py:979: note: def [_T] pop(self, Pattern[str], _T, /) -> type[DynamicItem[Item[Any]]] | _T
pandera (https://github.com/pandera-dev/pandera)
+ pandera/io/_script_model.py:113: error: Unused "type: ignore" comment [unused-ignore]
rclip (https://github.com/yurijmikhalevich/rclip)
- rclip/utils/tokenizer.py:91: error: No overload variant of "get" of "dict" matches argument types "tuple[str, str]", "float" [call-overload]
- rclip/utils/tokenizer.py:91: note: Possible overload variants:
- rclip/utils/tokenizer.py:91: note: def get(self, str, None = ..., /) -> int | None
- rclip/utils/tokenizer.py:91: note: def get(self, str, int, /) -> int
- rclip/utils/tokenizer.py:91: note: def [_T] get(self, str, _T, /) -> int | _T
kornia (https://github.com/kornia/kornia)
- kornia/io/io.py:90: error: Argument 1 to "get" of "dict" has incompatible type "int | None"; expected "int" [arg-type]
steam.py (https://github.com/Gobot1234/steam.py)
- steam/ext/commands/utils.py:43: note: def get(self, str, None = ..., /) -> _VT | None
+ steam/ext/commands/utils.py:43: note: def get(self, object, None = ..., /) -> _VT | None
- steam/ext/commands/utils.py:43: note: def get(self, str, _VT, /) -> _VT
+ steam/ext/commands/utils.py:43: note: def get(self, object, _VT, /) -> _VT
- steam/ext/commands/utils.py:43: note: def [_T] get(self, str, _T, /) -> _VT | _T
+ steam/ext/commands/utils.py:43: note: def [_T] get(self, object, _T, /) -> _VT | _T
- steam/ext/commands/utils.py:52: note: def pop(self, str, /) -> _VT
+ steam/ext/commands/utils.py:52: note: def pop(self, object, /) -> _VT
- steam/ext/commands/utils.py:52: note: def pop(self, str, _VT, /) -> _VT
+ steam/ext/commands/utils.py:52: note: def pop(self, object, _VT, /) -> _VT
- steam/ext/commands/utils.py:52: note: def [_T] pop(self, str, _T, /) -> _VT | _T
+ steam/ext/commands/utils.py:52: note: def [_T] pop(self, object, _T, /) -> _VT | _T
+ steam/ext/commands/bot.py:537: error: Unused "type: ignore" comment [unused-ignore]
starlette (https://github.com/encode/starlette)
+ starlette/middleware/sessions.py:114: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ starlette/middleware/sessions.py:114: note: Superclass:
+ starlette/middleware/sessions.py:114: note: @overload
+ starlette/middleware/sessions.py:114: note: def pop(self, object, /) -> Any
+ starlette/middleware/sessions.py:114: note: @overload
+ starlette/middleware/sessions.py:114: note: def pop(self, object, Any, /) -> Any
+ starlette/middleware/sessions.py:114: note: @overload
+ starlette/middleware/sessions.py:114: note: def [_T] pop(self, object, _T, /) -> Any | _T
+ starlette/middleware/sessions.py:114: note: Subclass:
+ starlette/middleware/sessions.py:114: note: def pop(self, key: str, *args: Any) -> Any
pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/arrays/arrow/array.py:4443: error: Unused "type: ignore" comment [unused-ignore]
pylox (https://github.com/sco1/pylox)
+ pylox/containers/array.py:83: note: def pop(self, object, /) -> Any
- pylox/containers/array.py:83: note: def pop(self, Any, /) -> Any
+ pylox/containers/array.py:83: note: def pop(self, object, Any, /) -> Any
- pylox/containers/array.py:83: note: def pop(self, Any, Any, /) -> Any
- pylox/containers/array.py:83: note: def [_T] pop(self, Any, _T, /) -> Any | _T
+ pylox/containers/array.py:83: note: def [_T] pop(self, object, _T, /) -> Any | _T
jax (https://github.com/google/jax)
- jax/_src/interpreters/partial_eval.py:1162: error: Argument 1 to "get" of "dict" has incompatible type "Var | Literal | None"; expected "Var" [arg-type]
- jax/_src/interpreters/mlir.py:316: error: Argument 1 to "get" of "dict" has incompatible type "tuple[int, AbstractValue | None]"; expected "tuple[int, AbstractValue]" [arg-type]
mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ bson/son.py:138: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ bson/son.py:138: note: Superclass:
+ bson/son.py:138: note: @overload
+ bson/son.py:138: note: def pop(self, object, /) -> _Value
+ bson/son.py:138: note: @overload
+ bson/son.py:138: note: def pop(self, object, _Value, /) -> _Value
+ bson/son.py:138: note: @overload
+ bson/son.py:138: note: def [_T] pop(self, object, _T, /) -> _Value | _T
+ bson/son.py:138: note: Subclass:
+ bson/son.py:138: note: def [_T] pop(self, key: _Key, *args: _Value | _T) -> _Value | _T
setuptools (https://github.com/pypa/setuptools)
+ setuptools/_static.py:170: error: Unused "type: ignore" comment [unused-ignore]
jinja (https://github.com/pallets/jinja)
+ src/jinja2/compiler.py:1422: error: Unused "type: ignore" comment [unused-ignore]
+ src/jinja2/environment.py:514: error: Unused "type: ignore" comment [unused-ignore]
scrapy (https://github.com/scrapy/scrapy)
+ scrapy/http/headers.py:57: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ scrapy/http/headers.py:57: note: Superclass:
+ scrapy/http/headers.py:57: note: @overload
+ scrapy/http/headers.py:57: note: def pop(self, object, /) -> Any
+ scrapy/http/headers.py:57: note: @overload
+ scrapy/http/headers.py:57: note: def pop(self, object, Any, /) -> Any
+ scrapy/http/headers.py:57: note: @overload
+ scrapy/http/headers.py:57: note: def [_T] pop(self, object, _T, /) -> Any | _T
+ scrapy/http/headers.py:57: note: Subclass:
+ scrapy/http/headers.py:57: note: def pop(self, key: str | bytes, *args: Any) -> Any
+ scrapy/http/headers.py:105: error: Signature of "get" incompatible with supertype "builtins.dict" [override]
+ scrapy/http/headers.py:105: note: Superclass:
+ scrapy/http/headers.py:105: note: @overload
+ scrapy/http/headers.py:105: note: def get(self, object, None = ..., /) -> Any | None
+ scrapy/http/headers.py:105: note: @overload
+ scrapy/http/headers.py:105: note: def get(self, object, Any, /) -> Any
+ scrapy/http/headers.py:105: note: @overload
+ scrapy/http/headers.py:105: note: def [_T] get(self, object, _T, /) -> Any | _T
+ scrapy/http/headers.py:105: note: Subclass:
+ scrapy/http/headers.py:105: note: def get(self, key: str | bytes, def_val: Any = ...) -> bytes | None
werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/datastructures/mixins.py:265: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ src/werkzeug/datastructures/mixins.py:265: note: Superclass:
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def pop(self, object, /) -> V
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def pop(self, object, V, /) -> V
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def [_T] pop(self, object, _T, /) -> V | _T
+ src/werkzeug/datastructures/mixins.py:265: note: Subclass:
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def pop(self, key: K) -> V
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def pop(self, key: K, default: V) -> V
+ src/werkzeug/datastructures/mixins.py:265: note: @overload
+ src/werkzeug/datastructures/mixins.py:265: note: def [T] pop(self, key: K, default: T) -> T
+ src/werkzeug/datastructures/structures.py:471: error: Signature of "pop" incompatible with supertype "builtins.dict" [override]
+ src/werkzeug/datastructures/structures.py:471: note: Superclass:
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def pop(self, object, /) -> V
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def pop(self, object, V, /) -> V
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def [_T] pop(self, object, _T, /) -> V | _T
+ src/werkzeug/datastructures/structures.py:471: note: Subclass:
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def pop(self, key: K) -> V
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def pop(self, key: K, default: V) -> V
+ src/werkzeug/datastructures/structures.py:471: note: @overload
+ src/werkzeug/datastructures/structures.py:471: note: def [T] pop(self, key: K, default: T) -> V | T
+ src/werkzeug/debug/__init__.py:551: error: Unused "type: ignore" comment [unused-ignore]
|
Sorry, something went wrong.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #15271
Supersedes: #15297
This PR only changes dict, but not MutableMapping, and changes the signature of dict.get and dict.pop to allow object as key arguments. (allows these methods to work with partially overlapping types)