Document typeshed better: a mypy upgrade suddenly triggers new errors due to typeshed changes #7735
Description
I just updated MyPy in Jinja, and had to ignore the return type of sum in one of my functions because MyPy started saying it wasn't compatible with the TypeVar I was using. pallets/jinja@a24df26
I see a recent change related to the return type of sum: #7578. Is this a bug in typeshed/mypy, or is there a better way I should be annotating that function in Jinja?
V = t.TypeVar("V")
def sync_do_sum(
environment: "Environment",
iterable: "t.Iterable[V]",
attribute: t.Optional[t.Union[str, int]] = None,
start: V = 0, # type: ignore
) -> V:
if attribute is not None:
iterable = map(make_attrgetter(environment, attribute), iterable)
return sum(iterable, start) # type: ignore[no-any-return, call-overload]src/jinja2/filters.py:1295: error: Returning Any from function declared to return "V" [no-any-return] src/jinja2/filters.py:1295: error: No overload variant of "sum" matches argument types "Iterable[V]", "V" [call-overload] src/jinja2/filters.py:1295: note: Possible overload variants: src/jinja2/filters.py:1295: note: def [_SumT <: _SupportsSum] sum(Iterable[_SumT]) -> Union[_SumT, Literal[0]] src/jinja2/filters.py:1295: note: def [_SumT <: _SupportsSum, _SumS <: _SupportsSum] sum(Iterable[_SumT], _SumS) -> Union[_SumT, _SumS] Found 2 errors in 1 file (checked 25 source files)
Activity
srittau commented on Apr 28, 2022
The "correct" way to do this is to add a your own protocol and constrain V to that:
But we should probably add SupportsSum to _typeshed to make it a bit easier to use (and to not add it to the global namespace).
AlexWaygood commented on Apr 28, 2022
I'd prefer SupportsAdd as a name, but I agree.
davidism commented on Apr 28, 2022
Is there any way to make this type of error more discoverable? I don't think most people would know to look at typeshed for this error. I wouldn't have known to use something from _typeshed unless I had posted here, I regularly forget that typeshed exposes some types, because it's not mentioned in Python's typing documentation or in mypy/typing-extensions/mypy-extensions.
davidism commented on Apr 28, 2022
If anyone involved in typing is going to be at PyCon sprints, maybe some documentation for typeshed would be a good sprint task.
srittau commented on Apr 28, 2022
The typing documentation is distributed over multiple sites, at the moment. There's also a lot of institutional knowledge that's onlyt written down in some issue discussions. https://typing.readthedocs.io is supposed to become a central documentation repository in the future, but there's nothing really there yet, unfortunately.
AlexWaygood commented on Apr 28, 2022
There's actually a nice mypy PR I reviewed the other day that would improve the situation a little bit:
The author doesn't seem to be particularly active at the moment, though, so idk if it has a chance of getting merged.
And I agree with @srittau that it's much better to have this information in a centralised place rather than the mypy docs, and https://typing.readthedocs.io/ seems like the best place for this kind of thing.
JelleZijlstra commented on Apr 28, 2022
It would also be useful to distribute types like SupportsAdd in a .py package, maybe typing-extensions.
srittau commented on Apr 28, 2022
It's been my plan for a long time now to get some of the definition from _typeshed into typing_extensions. (Maybe under typing_extensions.ext?). But time is finite. :(
AlexWaygood commented on Apr 30, 2022
Something like this would also be helpful for guiding people to the likely source of the error in cases like this:
srittau commented on Jul 11, 2026
I'm not sure there's anything actionable left here. (If there it, it should probably opened as a tighter issue.)