FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Cache type_object_type() by ilevkivskyi · Pull Request #19514 · python/mypy · GitHub

Repository navigation

Cache type_object_type() - #19514

Merged
ilevkivskyi merged 4 commits into
python:masterfrom
ilevkivskyi:cache-type-obj-type
Jul 27, 2025
Merged

ilevkivskyi merged 4 commits into
python:masterfrom
ilevkivskyi:cache-type-obj-type

Conversation

ilevkivskyi commented Jul 26, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

This gives almost 4% performance boost (Python 3.12, compiled). Note there is an old bug in type_object_type(), we treat not ready types as Any without deferring, I disable caching in this case.

Unfortunately, using this in fine-grained mode is tricky, essentially I have three options:

  • Use some horrible hacks to invalidate cache when needed
  • Add (expensive) class target dependency from __init__/__new__
  • Only allow constructor caching during initial load, but disable it in fine-grained increments

I decided to choose the last option. I think it has the best balance complexity/benefits.

This comment has been minimized.

Copy link
Copy Markdown
Member Author

Non-trivial results in mypy_primer are surprising. I know there is a bug in type_object_type() that when it is called before the constructor is processed we may get wrong result (e.g. because it is decorated) instead of deferring the current node. But I thought the way I implemented caching should not be affected by this bug. I will take a look today or tomorrow.

This comment has been minimized.

Copy link
Copy Markdown
Member Author

Although remaining errors are kind of correct, they are sill weird. It looks like they appear because previously

class C
    @no_type_check
    def __new__(cls): ...

resulted in Any as the type of C class object. But somehow it is now C in case of an import cycle. FWIW I can't reproduce this in a simple test.

Copy link
Copy Markdown
Member Author

OK, so it turns out there was an existing bug (inconsistency) with @no_type_check, where the decorated function was not Any for a brief period between semantic analysis and type checking. I simply fix that.

While looking at this I found second place where we don't handle not-ready types properly (decorated overloads), again I simply don't cache in this case, since a proper fix may be non-trivial (I also update the relevant TODO item). I update the PR description to mention this.

This comment has been minimized.

sterliakov left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LG! The idea makes sense, and the bench on GHA shows 2.6-2.9% win for selfcheck. Nice! I'm a bit worried that it uncovered at least three semi-unrelated issues - can there be others, not revealed by primer runs? It may be too much fun to debug caching-related bugs later...

Comment thread mypy/typeops.py Outdated
else:
builtins_type = named_type("builtins.type")

fallback = info.metaclass_type or builtins_type

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Can we keep it lazy (only call named_type if info.metaclass_type is None)?

Copy link
Copy Markdown
Member Author

@sterliakov Of course there can be more. But those are all real bugs, we would need to fix them sooner or later anyway. In an ideal bug-free world, we should be able to always cache.

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL