gh-82129: Fix `NameError` on `get_type_hints` in `dataclasses` by sobolevn · Pull Request #122232 · python/cpython · GitHub

Ok, here's my attempt at solving this.

When you call typing.get_type_hints you still have to import typing.
There's a good chance that it will already be imported by something else in your sys.modules.

We already have this hack in other places:

cpython/Lib/dataclasses.py

Lines 809 to 815 in e968121

    
           typing = sys.modules.get('typing') 
        
           if typing: 
        
               if (_is_classvar(a_type, typing) 
        
                   or (isinstance(f.type, str) 
        
                       and _is_type(f.type, cls, typing, typing.ClassVar, 
        
                                    _is_classvar))): 
        
                   f._field_type = _FIELD_CLASSVAR

So, this solution can be used to hide the problem for older versions of python (down to 3.12). While we can actually fully solve with annotationlib in 3.14+

I think that this hack + user-space one:

S = make_dataclass('S', ['x'])
get_type_hints(S, {'typing': typing})

is good enough for now.

Issue: get_type_hints fails if there are un-annotated fields in a dataclass #82129

neat!

Could we use "__import__('typing').Any"? Since get_type_hints() calls eval(), that should work regardless of whether the dataclass is defined in a place where typing is imported.

I can provide a more robust solution on 3.14 using __annotate__.

Nikita's current solution feels in keeping with what dataclasses does elsewhere:

cpython/Lib/dataclasses.py

Lines 809 to 815 in e968121

    
           typing = sys.modules.get('typing') 
        
           if typing: 
        
               if (_is_classvar(a_type, typing) 
        
                   or (isinstance(f.type, str) 
        
                       and _is_type(f.type, cls, typing, typing.ClassVar, 
        
                                    _is_classvar))): 
        
                   f._field_type = _FIELD_CLASSVAR

dataclasses in general takes great pains to avoid importing typing if it's not already imported. I think these days typing is actually a faster import than dataclasses, but for now I think it's probably best to stick with the existing general philosophy of dataclasses there.

My solution would not involve importing typing in the dataclasses module.

I can provide a more robust solution on 3.14 using annotate.

I am already working on that for 3.14+ 😉

Could we use "import('typing').Any"?

I feel like this might not be a good idea for older versions. Since right now dataclasses do not import typing directly, this might cause performance regressions. This PR can be easily backported.

Plus, we have __annotate__ for the future versions.

Looks like a win-win to me.

Note that I am suggesting to put the __import__ in a string, so typing would only be imported when the annotations are evaluated.

@AlexWaygood it would look surprising, yes, but it would solve the user-facing issue that got reported.

I agree with @AlexWaygood, this might be a very scary thing to see for annotations without the evaluation 😱

That's fair, but then I don't think we should go with the current solution either. My example below seems like a reasonable scenario for how this could end up getting used in practice: user code does not import typing but calls make_dataclass(), later something like cattrs run and tries to get the type hints for the class.

With your PR, this will work only if typing happened to have been imported at the time the dataclass was created. But that becomes very fragile: it means that working code can break if you refactor your code to reorder imports, or remove an import typing from an unrelated part of the code.

Ok, let's try both: typing.Any if typing exists, and "__import__('typing').Any" if it doesn't 👍

This doesn't necessarily fix the issue.

$ cat dc.py 
import dataclasses
X = dataclasses.make_dataclass("X", ['x'])
$ ./python.exe 
Python 3.14.0a0 (heads/issue-82129:0d1081237a8, Jul 24 2024, 06:15:08) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dc
>>> import typing
>>> typing.get_type_hints(dc.X)
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    typing.get_type_hints(dc.X)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/jelle/py/cpython/Lib/typing.py", line 2416, in get_type_hints
    value = _eval_type(value, base_globals, base_locals, base.__type_params__,
                       format=format, owner=obj)
  File "/Users/jelle/py/cpython/Lib/typing.py", line 477, in _eval_type
    return evaluate_forward_ref(t, globals=globalns, locals=localns,
                                type_params=type_params, owner=owner,
                                _recursive_guard=recursive_guard, format=format)
  File "/Users/jelle/py/cpython/Lib/typing.py", line 1069, in evaluate_forward_ref
    value = forward_ref.evaluate(globals=globals, locals=locals,
                                 type_params=type_params, owner=owner)
  File "/Users/jelle/py/cpython/Lib/annotationlib.py", line 140, in evaluate
    value = eval(code, globals=globals, locals=locals)
  File "<string>", line 1, in <module>
NameError: name 'typing' is not defined. Did you forget to import 'typing'?
>>>

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@JelleZijlstra yeap, I even wrote a test-case for that: https://github.com/python/cpython/pull/122232/files#diff-212e368b34eb9b134f87e765787d6d26b747235a358e13da8922f83861c0d676R4129-R4133

Second PR in this chain: #122262
It will allow us to fix this problem for good in 3.14+

Friendly ping @JelleZijlstra

Changing behavior based on whether typing happens to have been imported when the dataclass is created still feels fragile, and I think there's a risk it causes new problems if we make this change in a bugfix release.

So I think it may be better to leave this bug unsolved for earlier versions. There are many scenarios where get_type_hints() could raise NameError (e.g., when if TYPE_CHECKING is used), and callers have to deal with that. In Python 3.14 we'll have a better foundation for fixing this sort of thing, but in older versions it may be safer to stick with existing behavior.

Ok, I agree: there are no clean ways forward. Thanks for everyone's input!

sobolevn added 2 commits July 24, 2024 15:45

pythongh-82129 : Fix NameError on get_type_hints in dataclasses

a96efef

pythongh-82129 : Fix NameError on get_type_hints in dataclasses

0d10812

sobolevn added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes labels Jul 24, 2024

sobolevn requested a review from AlexWaygood July 24, 2024 12:52

sobolevn requested a review from ericvsmith as a code owner July 24, 2024 12:52

bedevere-app Bot mentioned this pull request Jul 24, 2024

get_type_hints fails if there are un-annotated fields in a dataclass #82129

Closed

bedevere-app Bot added the awaiting core review label Jul 24, 2024

AlexWaygood approved these changes Jul 24, 2024

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels Jul 24, 2024

JelleZijlstra reviewed Jul 24, 2024

View reviewed changes

JelleZijlstra requested changes Jul 24, 2024

View reviewed changes

bedevere-app Bot added awaiting changes and removed awaiting merge labels Jul 24, 2024

Address review

7af17e3

sobolevn requested a review from JelleZijlstra July 24, 2024 13:43

sobolevn mentioned this pull request Jul 25, 2024

gh-82129: Provide __annotate__ method for dataclasses from make_dataclass #122262

Closed

JelleZijlstra reviewed Aug 1, 2024

View reviewed changes

sobolevn closed this Aug 1, 2024

	typing = sys.modules.get('typing')
	if typing:
	if (_is_classvar(a_type, typing)
	or (isinstance(f.type, str)
	and _is_type(f.type, cls, typing, typing.ClassVar,
	_is_classvar))):
	f._field_type = _FIELD_CLASSVAR

gh-82129: Fix `NameError` on `get_type_hints` in `dataclasses` by sobolevn · Pull Request #122232 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-82129: Fix NameError on get_type_hints in dataclasses - #122232

gh-82129: Fix NameError on get_type_hints in dataclasses#122232sobolevn wants to merge 3 commits intopython:mainpython/cpython:mainfrom sobolevn:issue-82129sobolevn/cpython:issue-82129Copy head branch name to clipboard

Conversation

sobolevn commented Jul 24, 2024 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JelleZijlstra left a comment

Choose a reason for hiding this comment

Uh oh!

bedevere-app Bot commented Jul 24, 2024

Uh oh!

sobolevn commented Jul 24, 2024

Uh oh!

sobolevn commented Jul 25, 2024

Uh oh!

sobolevn commented Aug 1, 2024

Uh oh!

JelleZijlstra left a comment

Choose a reason for hiding this comment

Uh oh!

sobolevn commented Aug 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gh-82129: Fix NameError on get_type_hints in dataclasses#122232
sobolevn wants to merge 3 commits into
python:mainfrom
sobolevn:issue-82129

sobolevn commented Jul 24, 2024 •
edited by bedevere-app Bot
Loading