Always use `.enum_members` to find enum members by sobolevn · Pull Request #18675 · python/mypy · GitHub

This fixes the problem with nonmember and member special cases, however, it required me to change one test case.

See testEnumReachabilityPEP484ExampleSingletonWithMethod change, because in runtime token is not a member by default, at least in recent python versions. Proof:

# 3.14
>>> from enum import Enum
>>> class Empty(Enum):
...     token = lambda x: x
...     
>>> Empty.token
<function Empty.<lambda> at 0x101251250>
>>> Empty.token.value

and

# 3.11
>>> from enum import Enum
>>> class Empty(Enum):
...     token = lambda x: x
... 
>>> Empty.token
<function Empty.<lambda> at 0x104757600>
>>> Empty.token.value
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute 'value'

So, I had to add member() there to make the test pass.

I will continue to improve enums support in the future :)
For example, we need to support _ignore_ values and some other corner cases.

for more information, see https://pre-commit.ci

Diff from mypy_primer, showing the effect of this PR on open source code:

freqtrade (https://github.com/freqtrade/freqtrade): 2.00x slower (129.2s -> 257.9s in a single noisy sample)

I can confirm the test change is fully valid, bare lambdas have never been treated as members, similar to any lambda self: ... attributes on plain classes becoming regular methods. Previously it was treated incorrectly.

I'm glad to see it fixes one of the problems known to me. Since it's likely the same place, can we by any chance make mypy recognize @member decorator consistently too? In the snippet below, your patch removes false positives in check_1 (thanks!), but E2 and check_2 are still unsupported. Literals are there just for reference, I know they're checked too early and so are more difficult to support and likely fall outside of this PR scope. https://mypy-play.net/?mypy=master&python=3.12&flags=strict%2Cwarn-unreachable&gist=04981264339d6d34aed4a353e0735098

Ough. Something is wrong with test stubs.

Given the following code:

# flags: --python-version 3.12 --warn-unreachable
from enum import Enum, member, nonmember
from typing import Literal, Never

def assert_never(_: Never) -> Never: ...

class E2(Enum):
    @member
    def C() -> None: ...  # E: Method must have at least one argument. Did you forget the "self" argument?  [misc]

c: Literal[E2.C]  # E: Parameter 1 of Literal[...] is invalid  [valid-type]

def check_2(e: E2) -> None:
    match e:
        case E2.C:
            pass
        case other:
            assert_never(other)  # E: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type]

    if e is E2.C:
        pass
    else:
        assert_never(e)  # E: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "__main__.E2">"; expected "Never"  [arg-type]

If I cat <<EOF >/tmp/c.py it,

$ python -m mypy /tmp/c.py --strict --python-version 3.12 --warn-unreachable --config-file=
/tmp/c.py:5: error: Implicit return in function which does not return  [empty-body]
/tmp/c.py:9: error: Method must have at least one argument. Did you forget the "self" argument?  [misc]
/tmp/c.py:11: error: Parameter 1 of Literal[...] is invalid  [valid-type]
/tmp/c.py:18: error: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type]
/tmp/c.py:23: error: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "c.E2">"; expected "Never"  [arg-type]
Found 5 errors in 1 file (checked 1 source file)

but putting the same in a test with [builtins fixtures/enum.pyi] results in

Expected:
  main:9: error: Method must have at least one argument. Did you forget the "self" argument?  [misc] (diff)
  main:11: error: Parameter 1 of Literal[...] is invalid  [valid-type] (diff)
  main:18: error: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type] (diff)
  main:23: error: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "__main__.E2">"; expected "Never"  [arg-type] (diff)
Actual:
  main:9: error: Method must have at least one argument. Did you forget the "self" argument? (diff)
  main:11: error: Parameter 1 of Literal[...] is invalid (diff)
  main:18: error: Argument 1 to "assert_never" has incompatible type "Union[member[Callable[[], None]], E2]"; expected "Never" (diff)

It's either the stubs or something shady with import resolution, but anyway enum tests we're running now may not represent actual mypy behaviour.

I will fix the stubs in the following PR, thanks for finding this! 👍

Does the stub issue impact this PR, i.e. should we wait for the stubs to be fixed before reviewing this PR?

@sterliakov sorry, I don't quite understand your example: where can I find a test case for check2?

I'm not sure there is already a corresponding test, I just checked out your branch to try it against enum problems I encountered recently, added a test from that code and observed critical discrepancies between bare mypy invocation and running pytest with such a test.

Failing testcase (added to check-enum.test):

[case testDemo]
# flags: --python-version 3.13 --warn-unreachable
from enum import Enum, member, nonmember
from typing import Literal, Never

def assert_never(_: Never) -> Never: ...

class E2(Enum):
    @member
    def C() -> None: ...  # E: Method must have at least one argument. Did you forget the "self" argument?  [misc]

c: Literal[E2.C]  # E: Parameter 1 of Literal[...] is invalid  [valid-type]

def check_2(e: E2) -> None:
    match e:
        case E2.C:
            pass
        case other:
            assert_never(other)  # E: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type]

    if e is E2.C:
        pass
    else:
        assert_never(e)  # E: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "__main__.E2">"; expected "Never"  [arg-type]
[builtins fixtures/enum.pyi]

The same code pasted into a plain file produces mypy output matching the comments (I have current branch installed editable, pip install -e .):

$ pytest mypy/test/testcheck.py::TypeCheckSuite::check-enum.test -k testDemo 
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/stas/Documents/Work/mypy
configfile: pyproject.toml
plugins: cov-5.0.0, xdist-3.6.1
2 workers [1 item]      
F                                                                                                                                                                                                                                    [100%]
================================================================================================================= FAILURES =================================================================================================================
_________________________________________________________________________________________________________________ testDemo _________________________________________________________________________________________________________________
[gw0] linux -- Python 3.12.6 /home/stas/Documents/Work/mypy/.venv12/bin/python3
data: /home/stas/Documents/Work/mypy/test-data/unit/check-enum.test:2516:
Failed: Unexpected type checker output (/home/stas/Documents/Work/mypy/test-data/unit/check-enum.test, line 2516)
----------------------------------------------------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------------------------------------------------
Expected:
  main:9: error: Method must have at least one argument. Did you forget the "self" argument?  [misc] (diff)
  main:11: error: Parameter 1 of Literal[...] is invalid  [valid-type] (diff)
  main:18: error: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type] (diff)
  main:23: error: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "__main__.E2">"; expected "Never"  [arg-type] (diff)
Actual:
  main:9: error: Method must have at least one argument. Did you forget the "self" argument? (diff)
  main:11: error: Parameter 1 of Literal[...] is invalid (diff)
  main:18: error: Argument 1 to "assert_never" has incompatible type "Union[member[Callable[[], None]], E2]"; expected "Never" (diff)

Alignment of first line difference:
  E: ...the "self" argument?  [misc]
  A: ...the "self" argument?
                            ^
Update the test output using --update-data (implies -n0; you can additionally use the -k selector to update only specific tests)
========================================================================================================= short test summary info ==========================================================================================================
FAILED mypy/test/testcheck.py::TypeCheckSuite::check-enum.test::testDemo
============================================================================================================ 1 failed in 0.64s =============================================================================================================

$ cat <<EOF >/tmp/a.py
# flags: --python-version 3.13 --warn-unreachable                                                                                                                                                                                           
from enum import Enum, member, nonmember                                                                                                                                                                                                    
from typing import Literal, Never                                                                                                                                                                                                           
                                                                                                                                                                                                                                            
def assert_never(_: Never) -> Never: ...                                                                                                                                                                                                    
                                                                                                                                                                                                                                            
class E2(Enum):                                                                                                                                                                                                                             
    @member                                                                                                                                                                                                                                 
    def C() -> None: ...  # E: Method must have at least one argument. Did you forget the "self" argument?  [misc]                                                                                                                          
                                                                                                                                                                                                                                            
c: Literal[E2.C]  # E: Parameter 1 of Literal[...] is invalid  [valid-type]                                                                                                                                                                 
                                                                                                                                                                                                                                            
def check_2(e: E2) -> None:                                                                                                                                                                                                                 
    match e:                                                                                                                                                                                                                                
        case E2.C:                                                                                                                                                                                                                          
            pass                                                                                                                                                                                                                            
        case other:                                                                                                                                                                                                                         
            assert_never(other)  # E: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type]                                                                                                                 
                                                                                                                                                                                                                                            
    if e is E2.C:                                                                                                                                                                                                                           
        pass                                                                                                                                                                                                                                
    else:                                                                                                                                                                                                                                   
        assert_never(e)  # E: Argument 1 to "assert_never" has incompatible type "<subclass of "enum.member[Callable[[], None]]" and "__main__.E2">"; expected "Never"  [arg-type]                                                          
EOF
python -m mypy --config-file= --warn-unreachable  --python-version 3.13 /tmp/a.py

/tmp/a.py:5: error: Implicit return in function which does not return  [empty-body]
/tmp/a.py:9: error: Method must have at least one argument. Did you forget the "self" argument?  [misc]
/tmp/a.py:11: error: Parameter 1 of Literal[...] is invalid  [valid-type]
/tmp/a.py:18: error: Argument 1 to "assert_never" has incompatible type "E2"; expected "Never"  [arg-type]
/tmp/a.py:23: error: Argument 1 to "assert_never" has incompatible type "<subclass of "member" and "E2">"; expected "Never"  [arg-type]
Found 5 errors in 1 file (checked 1 source file)

I'm not sure how critical this problem is - in my understanding this means that some enum tests may actually deviate from mypy behaviour and so be not representative?

And the expected mypy output on this is only irrelevant error on line 5: that snippet matches runtime behaviour (and passes if I replace @member with regular enum member C = 1).

("method must have at least one argument" is also troubling, because member prevents self binding, but is clearly outside of this PR scope)

This very issue likely doesn't affect any existing tests as @member is used as a method decorator only in two of them, for some reason with self (testEnumImplicitlyFinalForSubclassingWithCallableMember and testEnumNotFinalWithMethodsAndUninitializedValuesStubMember) and never used in comparison/match statements.

@sterliakov can we create a new issue, as this seems like a different problem?

Opened #18719, #18720 and #18722 for the problems in snippet above and #18721 for the stubs issue.

Thanks, looks good!

Always use .enum_members to find enum members

9daa7cb

sobolevn requested review from JukkaL and hauntsaninja February 14, 2025 08:51

[pre-commit.ci] auto fixes from pre-commit.com hooks

2f8cd67

for more information, see https://pre-commit.ci

This comment has been minimized.

Sign in to view

sobolevn mentioned this pull request Feb 14, 2025

Properly account for member and nonmember in TypeInfo.enum_members #18559

Merged

Refactor

de83781

sobolevn requested a review from ilevkivskyi February 14, 2025 11:19

sterliakov mentioned this pull request Feb 22, 2025

mypy behavior differs when run on a file directly vs a test #18721

Open

JukkaL approved these changes Mar 31, 2025

View reviewed changes

JukkaL merged commit 4fb187f into master Mar 31, 2025

JukkaL deleted the issue-18565 branch March 31, 2025 16:08

hauntsaninja mentioned this pull request May 26, 2025

[1.16 regression] determining exhaustive match on an enum with type members #19151

Closed

brianschubert mentioned this pull request Jun 10, 2025

[1.16 regression] no longer recognizes enum singleton as only possible case #19271

Closed

Always use `.enum_members` to find enum members by sobolevn · Pull Request #18675 · python/mypy · GitHub

Repository navigation

Always use .enum_members to find enum members - #18675

Always use .enum_members to find enum members#18675JukkaL merged 3 commits intomasterpython/mypy:masterfrom issue-18565python/mypy:issue-18565Copy head branch name to clipboard

Conversation

sobolevn commented Feb 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

github-actions Bot commented Feb 14, 2025

Uh oh!

sterliakov commented Feb 15, 2025

Uh oh!

sterliakov commented Feb 15, 2025

Uh oh!

sobolevn commented Feb 17, 2025

Uh oh!

JukkaL commented Feb 21, 2025

Uh oh!

sobolevn commented Feb 21, 2025

Uh oh!

sterliakov commented Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sterliakov commented Feb 21, 2025

Uh oh!

sterliakov commented Feb 21, 2025

Uh oh!

sobolevn commented Feb 21, 2025

Uh oh!

sterliakov commented Feb 22, 2025

Uh oh!

JukkaL left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Always use .enum_members to find enum members#18675
JukkaL merged 3 commits into
masterfrom
issue-18565

sobolevn commented Feb 14, 2025 •
edited
Loading

sterliakov commented Feb 21, 2025 •
edited
Loading