Repository navigation
Overflows happened silently in array.array setters for 'e', 'f' and 'Zf' format types #156865
Description
Activity
- changed the title [-]Overflows happen silently in array.array setters for 'f' and 'Zf' format types[/-] [+]Overflows happened silently in array.array setters for 'f' and 'Zf' format types[/+] on Sep 3, 2026
This one is tricky. In the array module we raise OverflowError's for 'e' format type, but not for 'f'. In the struct - in both cases.
In the ctypes - overflows are silent, even for swapped types, except for the Zf type: the f_set_sw() also uses PyFloat_Pack4, but do first type-cast to float for PyFloat_AsDouble(value). Perhaps, behavior for the complex type is a bug, or vice-versa.
What should we do here? CC @encukou, CC @meadori (per experts index)
Edit:
Note, that floating-point types in ctypes raise OverflowError's, when integer value overflows double (this handled by PyFloat_AsDouble(), just like in the memoryview or the struct module, or by PyArg_Parse in the array module):Python 3.16.0a0 (heads/main:ee68f5f46a1, Aug 12 2026, 11:19:39) [GCC 14.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import ctypes >>> ctypes.c_float(10**1000) Traceback (most recent call last): File "<python-input-1>", line 1, in <module> ctypes.c_float(10**1000) ~~~~~~~~~~~~~~^^^^^^^^^^ OverflowError: int too large to convert to float
In the memoryview, overflows also are silent for 'f' type:
>>> array.array('f', [10**10]) array('f', [10000000000.0]) >>> m = memoryview(_) >>> m[0] = 1e300 >>> m[0] inf
But not for 'e':
>>> array.array('e', [10]) array('e', [10.0]) >>> m = memoryview(_) >>> m[0] = 123456.0 # not overflows float or double Traceback (most recent call last): File "<python-input-7>", line 1, in <module> m[0] = 123456.0 ~^^^ ValueError: memoryview: invalid value for format 'e'
Note also, that exceptions raised for integer types in struct, array and memoryview, but not in the ctypes:
Python 3.16.0a0 (heads/main:ee68f5f46a1, Aug 12 2026, 11:19:39) [GCC 14.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import array, ctypes, struct >>> struct.pack('i', 10**100) Traceback (most recent call last): File "<python-input-1>", line 1, in <module> struct.pack('i', 10**100) ~~~~~~~~~~~^^^^^^^^^^^^^^ struct.error: 'i' format requires -2147483648 <= number <= 2147483647 >>> a = array.array('i', [0]) >>> m = memoryview(a) >>> a[0] = 10**100 Traceback (most recent call last): File "<python-input-4>", line 1, in <module> a[0] = 10**100 ~^^^ OverflowError: Python int too large to convert to C long >>> m[0] = 10**100 Traceback (most recent call last): File "<python-input-5>", line 1, in <module> m[0] = 10**100 ~^^^ ValueError: memoryview: invalid value for format 'i' >>> ctypes.c_int(10**100) # just wraps c_int(0)
The last one looks undocumented, but long-standing behavior of the ctypes module.
The evil plan is:
- raise OverflowError's in struct/array/memoryview for floating-point types
- be silent on overflows in the ctypes, except for conversions from ints to floating-point types (i.e. c_double(10**1000).
If that's OK, I'll open separate issues for memoryview and ctypes.
The exception also depends on the byte order...
2026-09-03T08:11:26.876088000+0200 maurycy@gimel /Users/maurycy/src/github.com/maurycy/cpython (main c700121) % ./python.exe Python 3.16.0a0 (heads/main:c700121b15c, Sep 3 2026, 08:11:05) [Clang 21.0.0 (clang-2100.1.1.101)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import struct >>> struct.pack('>Zf', 1e70j) Traceback (most recent call last): File "<python-input-1>", line 1, in <module> struct.pack('>Zf', 1e70j) ~~~~~~~~~~~^^^^^^^^^^^^^^ OverflowError: float too large to pack with f format >>> struct.pack('<Zf', 1e70j) b'\x00\x00\x00\x00\x00\x00\x80\x7f' >>>
Similarly:
2026-09-03T08:14:39.455948815+0200 maurycy@eiger /home/maurycy % python Python 3.14.2 (main, Jan 14 2026, 19:38:07) [Clang 21.1.4 ] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import struct >>> struct.pack('>F', 1e70j) Traceback (most recent call last): File "<python-input-1>", line 1, in <module> struct.pack('>F', 1e70j) ~~~~~~~~~~~^^^^^^^^^^^^^ OverflowError: float too large to pack with f format >>> struct.pack('<F', 1e70j) b'\x00\x00\x00\x00\x00\x00\x80\x7f' >>>
This does not smell right to me:
Lines 793 to 800 in 39a9a47
This issue is similar to issues gh-156864 and gh-156867. I would prefer to merge the 3 issues to solve all issues at once. IMO we should chose a behavior and make sure that all struct/memoryview/array/ctypes functions respect this behavior.
The exception also depends on the byte order...
For example, IMO the behavior should not depend on the byte order :-) If we are able to detect underflow and overflow, we should raise an error for all byte orders.
Reacted by Maurycy Pawłowski-WierońskiThis issue is similar to issues #156864 and #156867. I would prefer to merge the 3 issues to solve all issues at once.
Done.
IMO we should chose a behavior and make sure that all struct/memoryview/array/ctypes functions respect this behavior.
This looks as even more evil plan than mine evil plan. Great!
It's hard to say, but there might be some code, that rely on silent overflows in the ctypes...
- marked struct.pack() doesn't checks overflows for native 'Zf' format type #156867 as a duplicate of this issue on Sep 3, 2026
- marked array.array() setter incorrectly handle overflows for 'e' type code #156864 as a duplicate of this issue on Sep 3, 2026
About backports, I'm worried that changing the behavior can break an unknown number of projects, even if we consider that the current behavior is "wrong". If we change the behavior, I would suggest to only change Python 3.16 and not backport the changes. IMO it's too late to change Python 3.15 as well (release candidate 2 was just released).
- changed the title [-]Overflows happened silently in array.array setters for 'f' and 'Zf' format types[/-] [+]Overflows happened silently in array.array setters for 'e', 'f' and 'Zf' format types[/+] on Sep 3, 2026
If struct/memoryview/array/ctypes is modified to raise an exception instead of silently "truncating" the result of "+/-inf", would it be possible to provide a recipe to get the old behavior on new Python? It doesn't sound easy to do :-( I suppose that the recipe would have to replace float larger than the maximum with 'inf' and float smaller than the minimum with '-inf' before calling modified functions. Problem: we do not expose the minimum or maximum of binary16 and binary32.
If struct/memoryview/array/ctypes is modified to raise an exception instead of silently "truncating" the result of "+/-inf", would it be possible to provide a recipe to get the old behavior on new Python? It doesn't sound easy to do :-(
Yes, we don't have an equivalent of C type cast like
float f = (float)(some_double)
(ctypes.cast is for pointers.)
Well, if we keep old behavior for ctypes - this will be available as ctypes.c_float(python_double).
Problem: we do not expose the minimum or maximum of binary16 and binary32.
This might be a problem for alternative interpreters. Not for CPython, as we require IEEE doubles (then highly likely that C floats are binary32 and so on).
- added a commit that references this issue on Sep 18, 2026
Yes, ctypes should emulate how C casts work on common platforms.
Reacted by Victor StinnerI merged the 4 changes (array, struct, memoryview, ctypes). As I wrote previously, I prefer to not backport these changes to stable branches. I close the issue.
Bug report
Bug description:
Consider this:
(BTW, f_setitem() suffers from same issue as #156864.)
Maybe it's a feature and array() should behave differently wrt the struct module. In either case, this behavior looks inconsistent. If we consider cases for 'f'/'Zf' types being correct - we should clear exceptions also for 'e' type code.
See also struct/array issues (merged to this per @vstinner suggestion):
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Linked PRs