Repository navigation
Pure Python operator.index doesn't match the C version. #62912
Description
| BPO | 18712 |
|---|---|
| Nosy | @arigo, @mdickinson, @ericsnowcurrently, @zware, @serhiy-storchaka, @corona10, @iritkatriel |
| Dependencies | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2013-08-12.14:09:38.011>
labels = ['extension-modules', 'type-bug', '3.11']
title = "Pure Python operator.index doesn't match the C version."
updated_at = <Date 2021-10-02.15:33:51.696>
user = 'https://github.com/mdickinson'bugs.python.org fields:
activity = <Date 2021-10-02.15:33:51.696>
actor = 'corona10'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2013-08-12.14:09:38.011>
creator = 'mark.dickinson'
dependencies = ['17576']
files = ['31273', '31334', '31335', '31336', '31344', '31353']
hgrepos = []
issue_num = 18712
keywords = ['patch']
message_count = 21.0
messages = ['194966', '195000', '195006', '195064', '195451', '195454', '195457', '195458', '195474', '195476', '195489', '195490', '195510', '195513', '195545', '195546', '195547', '195548', '195549', '195665', '401603']
nosy_count = 7.0
nosy_names = ['arigo', 'mark.dickinson', 'eric.snow', 'zach.ware', 'serhiy.storchaka', 'corona10', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue18712'
versions = ['Python 3.11']
Activity
mdickinson commented on Aug 12, 2013
Nitpick: the pure Python version of operator.index (new in Python 3.4, introduced in issue bpo-16694) doesn't match the C version, in that it looks up __index__ on the object rather than the class.
serhiy-storchaka commented on Aug 12, 2013
We can use type(a).__index__(a). Should we also correct the documentation for operator.index() and operator.length_hint()?
mdickinson commented on Aug 12, 2013
Yes, I think it would make sense to fix the docs as well, at least for Python 3.4. Probably not worth it for the maintenance releases.
serhiy-storchaka commented on Aug 13, 2013
Here is a patch. Now the code of operator.index() becomes even more complicated. Perhaps you want suggest other wording for documentation?
Some code in stdlib (pyio.py, bz2.py, connection.py) uses a.__index_() instead of type(a).__index__(a) (with replacing AttributeError to TypeError). Is it worth to change?
arigo commented on Aug 17, 2013
Just mentioning it here again, but "type(a).__index__(a)" is still not perfectly correct. Attached is a case where it differs.
I think you get always the correct answer by evaluating "range(a).stop". It's admittedly obscure... For example:
serhiy-storchaka commented on Aug 17, 2013
The difference doesn't look significant. In all cases the TypeError is raised.
But there was other differences between C and Python versions -- when __index__() returns non-integer. Updated patch fixes this.
mdickinson commented on Aug 17, 2013
Hmm. "type(a).__dict__'__index__'" ? (With suitable error checks, as in Serhiy's patch.)
arigo commented on Aug 17, 2013
Ok, so here is another case. (I won't go to great lengths trying to convince you that there is a problem, because the discussion already occurred several times; google for example for "_PyType_Lookup pure Python")
serhiy-storchaka commented on Aug 17, 2013
This variant fails on:
mdickinson commented on Aug 17, 2013
Serhiy: Yep, or even on bool. Thanks.
Armin: I don't think either of us thinks there isn't a problem here. :-)
The Google search you suggested didn't turn up a whole lot of useful information for me. Was there a discussion of this on python-dev at some point? (And if not, should there be?)
For this *particular* issue, it seems we can't exactly reproduce. type(a).__index__(a) seems like the best practical approximation to the true behaviour. I'm guessing that it's fairly rare to have useful definitions of __index__ on a metaclass.
arigo commented on Aug 17, 2013
This may have been the most recent discussion of this idea (as far as I can tell):
http://mail.python.org/pipermail//python-ideas/2012-August/016036.html
Basically, it seems to be still unresolved in the trunk Python; sorry, I thought by now it would have been resolved e.g. by the addition of a method on types or a function in the operator module. In the absence of either, you need either to simulate its behavior by doing this:
Or you can piggyback on an unrelated call that simply causes the C-level PyNumber_Index() to be called:
arigo commented on Aug 17, 2013
Sorry, realized that my pure Python algorithm isn't equivalent to _PyType_Lookup() --- it fails the staticmethod example of Serhiy. A closer one would be:
But it's still not a perfect match. I think right now the only "perfect" answer is given by workarounds like "range(a).stop"...
serhiy-storchaka commented on Aug 17, 2013
Here is updated patch which uses Armin's algorithm for lookup special methods and adds special case for int subclasses in index().
I have no idea how the documentation should look.
5 remaining items
arigo commented on Aug 18, 2013
...but yes, it's very obvious that exposing _PyType_Lookup() to pure Python is the right thing to do here. This is a central part of the way Python works internally, after all.
Moreover, sorry about my previous note: if we started today to write PyPy, then it would be enough to have the pure Python version of operator.index(), based on the newly exposed _PyType_Lookup(). With PyPy's JIT, there is no real performance loss. Some of my confusion came from the fact that there *would* be serious performance loss if we had to work with the pure Python looping-over-mro-and-fishing-in-dict version.
zware commented on Aug 19, 2013
A lot of this discussion has flown a rather unfortunate distance over my head, especially since I've barely had time to follow it. But it looks to me like--given the number of other places that do the same thing as operator.index currently does--there needs to be a simple way to do the right thing somewhere accessible, which probably means a builtin.
On the other hand, it seems to me like 'a.__index__()' *should* be "the right thing" to do, but I get the feeling that making that so would be an astronomically huge change without much real benefit and lots of opportunities to break everything. I suspect I'm also missing something fundamental in why it's not the right thing to do.
iritkatriel commented on Sep 10, 2021
Reproduced on 3.11.
devdanzin commented on Sep 21, 2025
Reproduced on main (3.15).
tanloong commented on Feb 21, 2026
Reproduced on main (3.15)
pythongh-62912: fix pure-Python operator.index()
pythongh-62912: return correct exception in python versions for opera…