FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-91162: Fix substitution of unpacked tuples in generic aliases by serhiy-storchaka · Pull Request #92335 · python/cpython · GitHub

/ cpython Public

gh-91162: Fix substitution of unpacked tuples in generic aliases - #92335

Merged
serhiy-storchaka merged 9 commits into
python:mainfrom
serhiy-storchaka:typing-subst-unpacked
May 8, 2022
Merged

gh-91162: Fix substitution of unpacked tuples in generic aliases#92335
serhiy-storchaka merged 9 commits into
python:mainfrom
serhiy-storchaka:typing-subst-unpacked

Conversation

Copy link
Copy Markdown
Member

No description provided.

Copy link
Copy Markdown
Member

cc @mrahtz

mrahtz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Brilliant, Serhiy - thank you!

Comment thread Lib/test/test_typing.py
('generic[T1, T2]', '[int, str, bool]', 'TypeError'),
('generic[T1, T2]', '[*tuple_type[int]]', 'TypeError'),
('generic[T1, T2]', '[*tuple_type[int, str]]', 'TypeError'),
('generic[T1, T2]', '[*tuple_type[int, str]]', 'generic[int, str]'),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Based on the tentative spec we'd decided on at #91162 (comment), even though this is valid and easy to evaluate, we should also disallow it so that the rule stays simple: "If the generic accepts a fixed number of arguments, it can't take any unpacked type arguments."

Ditto various cases below.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think the spec should be changed. There is no problem to unpack a tuple of fixed length into the corresponding number of type variables. I think that it is simpler to allow this than add a special exception.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@pradeep90 Do you want to argue the case here? I agreed with you in #32341 (comment) tbh mainly because it didn't seem a huge deal and I didn't want to drag out the discussion there, but I actually agree with Serhiy here that it seems simpler to just allow it. Is there context that Serhiy and I are missing here?

As a starting point, I think the rule here could be something like: "You can only use an unpacked arbitrary-length type (such as an unpacked TypeVarTuple or an unpacked arbitrary-length tuple) in the arguments list if the generic accepts an arbitrary number of type parameters". That seems a simple enough rule to me - it disallows tuple[T][*tuple[int, ...]] and tuple[T][*Ts] while allowing tuple[T][*tuple[int]]. Or are there edge cases it wouldn't cover?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

As a starting point, I think the rule here could be something like: "You can only use an unpacked arbitrary-length type (such as an unpacked TypeVarTuple or an unpacked arbitrary-length tuple) in the arguments list if the generic accepts an arbitrary number of type parameters". That seems a simple enough rule to me

I disagree that it's a simple enough rule. Given an already-complex PEP, this seems like yet another arcane rule for users to keep in mind. The only benefit it gives us is that we allow the fairly useless example of generic[T1, T2][*tuple[int, str]]. Contrast that to the simpler rule that "*<anything> is only for variadic types". That way, users don't have to ever be surprised by weird syntax such as list[T][*tuple[int]].

I don't want to block runtime behavior on this. The runtime can always be more flexible and accept arbitrary cases. Type checkers can forbid this during type checking to keep things simple for users.

In short, for the runtime implementation, I'm ok with whatever is simpler for you and Serhiy. I disagree with changing the spec we agreed on for the sake of this.

mrahtz May 8, 2022
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Oh, yeah, to be clear - this conversation is totally just about the spec for what we allow at runtime.

If you're happy with the runtime being flexible on this, then so am I. You can go ahead and resolve this, Serhiy.

Comment thread Lib/test/test_typing.py Outdated
Comment thread Lib/test/test_typing.py Outdated
('generic[T]', '[int, str]', 'TypeError'),
('generic[T]', '[tuple_type[int, ...]]', 'generic[tuple_type[int, ...]]'),
('generic[T]', '[*tuple_type[int]]', 'generic[int]'),
# Should raise TypeError: a) according to the tentative spec,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

We can get rid of the comment here and a few lines below now that these cases do indeed raise a TypeError.

JelleZijlstra added the needs backport to 3.11 only security fixes label May 8, 2022

Copy link
Copy Markdown
Contributor

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

serhiy-storchaka deleted the typing-subst-unpacked branch May 8, 2022 15:32
bedevere-bot removed the needs backport to 3.11 only security fixes label May 8, 2022

Copy link
Copy Markdown

GH-92484 is a backport of this pull request to the 3.11 branch.

!(PyTuple_GET_SIZE(subargs) &&
PyTuple_GET_ITEM(subargs, PyTuple_GET_SIZE(subargs)-1) == Py_Ellipsis))
{
if (PyList_SetSlice(newargs, PY_SSIZE_T_MAX, PY_SSIZE_T_MAX, subargs) < 0) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Noob question: how does setting it to PY_SSIZE_T_MAX work? I'm surprised it's not something like low=i, high=i+PyTuple_GET_SIZE(subargs) ? I think I missed something.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

It works like:

a = [1, 2, 3]
a[sys.maxsize:sys.maxsize] = [4, 5]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I knew the Python equivalent but my mind is blown. I didn't know that was possible in Python! I'd always assumed list.extend was the only way to do it. Thanks for teaching me something new today Serhiy.

Last question: Why not use _PyList_Extend. Or is it just a personal preference which to use?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

_PyList_Extend is not convenient. It returns Py_None, for which you should call Py_DECREF. Compare:

if (PyList_SetSlice(newargs, PY_SSIZE_T_MAX, PY_SSIZE_T_MAX, subargs) < 0) {
    // error
}

and

PyObject *tmp = _PyList_Extend(newargs, subargs);
if (tmp == NULL) {
    // error
}
else {
    Py_DECREF(tmp);
}

Using PyList_SetSlice() for list.extend() or list.clear() is idiomatic.

pablogsal pushed a commit to miss-islington/cpython that referenced this pull request May 31, 2022
pythonGH-92335)

(cherry picked from commit 9d25db9)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
pablogsal added a commit that referenced this pull request Jun 1, 2022
…es (GH-92335) (#92484)

* gh-91162: Fix substitution of unpacked tuples in generic aliases (GH-92335)
(cherry picked from commit 9d25db9)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

* Regenerate ABI file

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants


Back | FazBrowse Home | New Git URL