Rework narrowing logic for equality and identity by hauntsaninja · Pull Request #20492 · python/mypy · GitHub

Mypy does not narrow as much as it could, which results in false positives.

We would also like to narrow based on containment. The PR for that was previously reverted due to inconsistencies between narrowing via equality and via containment. This fixes the inconsistency on the equality side and paves the road for adding narrowing via containment. That is, we lay groundwork for fixing #17864 and fixing #17841

Fixes #18524
Fixes #20041
Fixes #17162
Fixes #16830
Fixes #13704
Fixes #7642
Fixes #3964

This is a follow up to #20492 that is pure refactoring. I separated it out to make that change easier to review. We inline `narrow_identity_equality_comparison`, improve comments, etc. There is some further refactoring later in my commit stack too.

Requested in #20492 (comment) Regression for the Never's in #18524 (comment)

This relates to changes made in python#20492

Noticed a minor regression in Home Assistant with --warn-unreachable. Not sure if it's tracked already.

# mypy: warn-unreachable
from enum import IntFlag

class WeatherFeature(IntFlag):
    FORECAST_DAILY = 1
    FORECAST_HOURLY = 2
    FORECAST_TWICE_DAILY = 4

def func(supported_features: int) -> None:
    reveal_type(supported_features)
    if (supported_features & WeatherFeature.FORECAST_DAILY) == 0:
        print("Hello World")  # <-- This should work fine

note: Revealed type is "builtins.int"
error: Statement is unreachable  [unreachable]

Thanks, I have a follow up commit that will fix this case

Fixes some of the primer regressions from #20492 Fixes #19753 This should also happen to fix the case mentioned by cdce8p in #20492 (comment)

This relates to changes made in python#20492

We've wanted to do this for a long time, but previous attempt was reverted due to issues like #17841 , #17864 , #17869 Following #20492 we should now be in a position to do this narrowing Fixes #3229 Fixes #20234 Fixes #18208 Fixes #16774

This relates to changes made in python#20492

Relates to this comment from A5rocks: #20492 (comment) Fixes #20532 Fixes #18029 Fixes #17229 Fixes #16465 Closes #18574

This relates to changes made in python#20492

Maybe a little ad hoc, but avoids some of the undesirable primer regression from #20492 Closes #20636

Fixes #20701 / see discussion there Background: - In #20492 I added code to treat bytes, bytearray and memoryview as having custom `__eq__`, since they can compare equal to values that are of other types. I did this based on primer hits on draft versions of that PR - In #20643 I rewrote the handling of custom equality when narrowing. That fixed many issues, but regressed the equivalent of `testNarrowingOptionalBytes` because we're now more principled about custom `__eq__`. Perhaps interestingly the primer on that PR is small and not controversial, so I didn't feel the need to prioritise unsound things However, we only need one of the operands to have custom equality to model these things as reachable, so I think we can probably just remove `builtins.bytes` from here. This behaviour still isn't ideal. For instance, narrowing here shouldn't depend on whether we promote or not (we shouldn't narrow `v_all` to bytes) The real fix is adding more dedicated special casing so that we have an in-between option between "narrow assuming we can only compare equal to values of the same type" and "assume `__eq__` means we can't conclude anything about the positive case" Other relevant PRs: - #20673 attempts to remove the other custom eq logic for list, dict, set I added in #20492 . It should be close to mergeable - #20660 improves reachability and unfortunately to land it I have to do a rethink for how int, float, complex narrowing works. I actually think the current version has pretty good semantics and the primer is valid (but scary). But I want to think about it more