Refactor Union, Tuple, and Callable by ilevkivskyi · Pull Request #308 · python/typing · GitHub

Fixes #301 now List[Tuple[T, T]][int] == List[Tuple[int, int]]
Fixes #298 now Table = Tuple[T, List[T]] can be used as generic type alias (as in PEP 484 example)
Fixes #299 now class MyTup(Tuple[int, ...]): ... is allowed
Fixes #156 (well, it does not substitute the annotations, but makes this task simple even in complex cases, see example in tests)

Also this PR fixes some minor things that I have found while working on this:

List[Union] (with bare Union, i.e. without arguments), List[Optional], List[Generic[T]], and List[ClassVar[int]] are not valid and are prohibited now.
Generic did not evaluate forward references when asked, now it does.
__qualname__ was not copied on generic class subscription.
Type was not erased on instantiation of subclasses of concrete containers (List, Set, etc).
There was an obscure bug in Python 2: sometimes _abc_registry was erased on instantiation.

The main idea of this PR is to fix the issues mentioned at the top by reusing the existing code. Namely, I pulled flattening and removing duplicates code from _Union and the tree calculation function _subs_tree from GenericMeta. As well I moved Tuple and Callable after GenericMeta and made them inherit the latter. So that now all types that could be generic store their info in common way using __origin__, __parameters__, __args__.

I tried to polish this, to be sure that nothing was broken in the process of "refactoring" (also to improve speed). There is no recursion, the substitution tree is recalculated only when necessary. Also I added a lot of tests and many comments/docstrings (also for things added in my recent PRs).

@gvanrossum Could you please take a look at this? I would like this to get into 3.6b3, so that it will have a chance to be thoroughly tested.

@gvanrossum
OK, I have copied this to Python 2 and updated the PR description accordingly. Please take a look.

@gvanrossum Is there a chance this PR gets into 3.6b3?

(If this one gets merged, I could make a PR for #306 within an hour (if I will not sleep at that time :-))

Sorry, not much chance. :-( I'm overloaded.

I'm not sure if I'll ever be able to actually review this. It's over 1000
lines. The only way this will ever land is on the basis of trust alone...

@gvanrossum Changes to Python 2 and 3 are practically identical, so this is rather 500 lines (even less if one does not count tests). But in no case I want you to feel pressure because of this PR :-)

The only way this will ever land is on the basis of trust alone...

I would say this is an argument in favour of including this PR in 3.6b3 (it will be more tested by wider community before 3.6 final).

Also note that this PR will enable more features from python/mypy#2378 , namely generic aliases like TInt = Tuple[T, int].

OK, here's what I'll do, right now. I'll merge this right now, sight unseen, so you can fix #306.

I will create a CPython issue representing typing.py upgrades, and mention that when I merge this into CPython.

Hopefully the buildbots will stay green, but if not, we'll find out on that issue.

When you have #306 fixed I will merge that into CPython too, referencing the same issue.

I will also try to decide on the parameters of Coroutine.

OK, those changes were pushed, now let's watch the buildbots: https://www.python.org/dev/buildbot/

Thank you for merging this! It looks like all buildbots are OK :-)

…Typing library, which is fixed in python/typing#308. The fixed typing library ships with Python 3.5.3 and newer.

ilevkivskyi and others added 30 commits October 23, 2016 12:57

Make Tuple[T, S] subcriptable and subclassable

e8c3ad6

Make Callable[..., T] subscriptable and subclassable

cfcf81f

Make Union[T, S] subsctiptable

1368e97

Disable plain Union and Optional as type arguments

7ee4a1a

Implement __eq__ for GenericMeta

5fdceba

Implement __eq__ for Unions

f339586

Refactor common code in __eq__ and __repr__

26f50a3

Remove redundant code

65de907

Fix minor bugs + mini-refactoring

897de4e

Fix repr of Callable subclasses

c687ffa

Don't be too pedantic in GenericMeta._eval_type

582a37f

Simplify __hash__ and factor out orig_chain

02e6b14

Remove outdated tests + be more pedanctic in _Union._eval_type

8d802cc

Add tests

4e2527c

Factor out _subs_tree for Union and Generic

92da67f

Prohibit plain Generic[T] as type argument

7563c92

Correct a comment

f33545b

Erase but preserve type also for Callable

4a996b2

Erase but preserve type for all generics

6564742

Restore original __next_in_mro__ and use more precise bases

184f089

Preserve __qualname__ on subscription

b613325

Add tests and factor common stuff for Union

011b9d8

Add even more tests

52712a6

One more test + minor code simplification

2641151

Polishing code

9f1fb98

Minor changes

279a67e

Precalculate hash + refactor + more tests

f173fde

Simplify repr for Union

819d472

Add big test for __eq__, __repr__, and Any substitution

e6f19a8

Add test to illustrate substitution

deed806

ilevkivskyi added 8 commits October 25, 2016 14:15

Minor change in tests

f6382a7

Add/expand comments and docstrings

9bc7374

Fix the __qualname__ in PY 3.2

c2724ce

Add tests to Python 2

fd3e344

Modify Union in Python 2

cd7bbe2

Update Python 2 tests

068ab5e

Change remaining parts in Python 2

09c4797

Fix final test in PY 2

60303f7

Also call super().__init__ in PY 2 Generic

ea5dd05

ilevkivskyi mentioned this pull request Oct 29, 2016

Generic type aliases python/mypy#2378

Merged

gvanrossum merged commit 50d2b4d into python:master Oct 29, 2016

shimizukawa mentioned this pull request Nov 2, 2016

CI raises error with nightly builds sphinx-doc/sphinx#3111

Closed

ilevkivskyi mentioned this pull request Nov 6, 2016

Prohibit Callable[[...], X] #320

Merged

pavel-kirienko added a commit to OpenCyphal/pydsdl that referenced this pull request May 8, 2019

We no longer support Python 3.5.2 or older due to a problem with its …

ac04d74

…Typing library, which is fixed in python/typing#308. The fixed typing library ships with Python 3.5.3 and newer.

pavel-kirienko mentioned this pull request May 8, 2019

Python compatibility clarifications OpenCyphal/pydsdl#26

Merged

Refactor Union, Tuple, and Callable by ilevkivskyi · Pull Request #308 · python/typing · GitHub

Repository navigation

Refactor Union, Tuple, and Callable - #308

Refactor Union, Tuple, and Callable#308
gvanrossum merged 39 commits into
python:masterfrom
ilevkivskyi:the-big-sync

ilevkivskyi commented Oct 25, 2016 •
edited
Loading

Uh oh!

Uh oh!

ilevkivskyi commented Oct 25, 2016

Uh oh!

ilevkivskyi commented Oct 28, 2016

Uh oh!

gvanrossum commented Oct 28, 2016 via email

Uh oh!

gvanrossum commented Oct 29, 2016

Uh oh!

ilevkivskyi commented Oct 29, 2016

Uh oh!

gvanrossum commented Oct 29, 2016

Uh oh!

gvanrossum commented Oct 29, 2016

Uh oh!

ilevkivskyi commented Oct 29, 2016

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Refactor Union, Tuple, and Callable by ilevkivskyi · Pull Request #308 · python/typing · GitHub

Repository navigation

Refactor Union, Tuple, and Callable - #308

Refactor Union, Tuple, and Callable#308gvanrossum merged 39 commits intopython:masterpython/typing:masterfrom ilevkivskyi:the-big-syncilevkivskyi/typehinting:the-big-syncCopy head branch name to clipboard

Conversation

ilevkivskyi commented Oct 25, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilevkivskyi commented Oct 25, 2016

Uh oh!

ilevkivskyi commented Oct 28, 2016

Uh oh!

gvanrossum commented Oct 28, 2016 via email

Uh oh!

gvanrossum commented Oct 29, 2016

Uh oh!

ilevkivskyi commented Oct 29, 2016

Uh oh!

gvanrossum commented Oct 29, 2016

Uh oh!

gvanrossum commented Oct 29, 2016

Uh oh!

ilevkivskyi commented Oct 29, 2016

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Refactor Union, Tuple, and Callable#308
gvanrossum merged 39 commits into
python:masterfrom
ilevkivskyi:the-big-sync

ilevkivskyi commented Oct 25, 2016 •
edited
Loading