gh-97786: Fix compiler warnings in pytime.c by mdickinson · Pull Request #101826 · python/cpython · GitHub

This PR fixes some compiler warnings in pytime.c, and at the same time fixes our out-of-range double-to-integer checks to properly avoid undefined behaviour.

It's hard to write strictly portable code for this kind of thing, but the new check should work under a set of (very) mild assumptions, that are highly unlikely to be violated on any actual platform that we care about:

That time_t is a two's complement signed integer type with no trap representation. (The weakest part of this is the assumption that time_t is a signed integer type at all, but we're already making that assumption.)
That _PyTime_t is also a two's complement signed integer type with no trap representation. (This is already true with the current typedef int64_t _PyTime_t; declaration.)
That PY_TIME_T_MIN and _PyTime_MIN are within the range of a C double. These days we're assuming IEEE 754 binary64 doubles, so we're safe so long as the integer types _PyTime_t and time_t don't have a width of more than 1024.

Here's the underlying logic for the changes, for the record:

We want to check that a C double value x is within the range of a (potentially unknown) signed integer type I with min and max values IMIN and IMAX. More specifically, we want to be sure that a conversion from x to type I will not invoke undefined behaviour; this means that the integer part of x (discarding the fractional part, if any) must be in [IMIN, IMAX].
Assuming that I uses two's complement with no trap representation, IMIN must be the negation of a power of two, and IMAX == -1 - IMIN.
So IMIN is exactly representable as a double (assuming that it's not larger than 2**1023), and (double)IMIN does not change the value.
So the bottom-end check, (double)IMIN <= x, does exactly what we want it to.
For the top-end check, x <= IMAX is problematic since the implicit conversion of IMAX to type double may change the value. But assuming that x is an integer (which it is for all cases in this PR), x <= IMAX is mathematically equivalent to x < (IMAX + 1), which with our two's complement assumption is equivalent to x < -IMIN, and we can express this as x < -(double)IMIN.

Issue: Compiler warning for _Py_InIntegralTypeRange #97786

🤖 New build scheduled with the buildbot fleet by @mdickinson for commit 32aee25 🤖

If you want to schedule another build, you need to add the :hammer: test-with-buildbots label again.

I've removed the fmod asserts from the code. They were somewhat obscure, and it's not necessary for correctness that the target value is an integer.

The only possible issue with a non-integer intpart is one of false positives: if intpart is a large and negative double in the open interval (PY_TIME_T_MIN - 1, PY_TIME_T_MIN) then:

it's safe to cast intpart to time_t, since it's in range after discarding the fractional part, but
the condition if (!((double)PY_TIME_T_MIN <= intpart && intpart < -(double)PY_TIME_T_MIN)) will be true, so an overflow would be reported

So the failure mode is benign. But in any case, intpart must be an integer (or possibly an infinity or nan), so the above can't happen.

I think this is ready to merge. @gpshead Do you have bandwidth for a quick re-review?

Thanks @mdickinson for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖

Sorry, @mdickinson and @gpshead, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker b1b375e2670a58fc37cb4c2629ed73b045159918 3.10

Fixes compiler warnings in pytime.c. (cherry picked from commit b1b375e) Co-authored-by: Mark Dickinson <dickinsm@gmail.com>

GH-102062 is a backport of this pull request to the 3.11 branch.

Fixes compiler warnings in pytime.c.

* main: (60 commits) pythongh-102056: Fix a few bugs in error handling of exception printing code (python#102078) pythongh-102011: use sys.exception() instead of sys.exc_info() in docs where possible (python#102012) pythongh-101566: Sync with zipp 3.14. (pythonGH-102018) pythonGH-99818: improve the documentation for zipfile.Path and Traversable (pythonGH-101589) pythongh-88233: zipfile: handle extras after a zip64 extra (pythonGH-96161) pythongh-101981: Apply HOMEBREW related environment variables (pythongh-102074) pythongh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (pythonGH-101912) pythongh-101819: Adapt _io types to heap types, batch 1 (pythonGH-101949) pythongh-101981: Build macOS as recommended by the devguide (pythonGH-102070) pythongh-97786: Fix compiler warnings in pytime.c (python#101826) pythongh-101578: Amend PyErr_{Set,Get}RaisedException docs (python#101962) Misc improvements to the float tutorial (pythonGH-102052) pythongh-85417: Clarify behaviour on branch cuts in cmath module (python#102046) pythongh-100425: Update tutorial docs related to sum() accuracy (FH-101854) Add missing 'is' to `cmath.log()` docstring (python#102049) pythongh-100210: Correct the comment link for unescaping HTML (python#100212) pythongh-97930: Also include subdirectory in makefile. (python#102030) pythongh-99735: Use required=True in argparse subparsers example (python#100927) Fix incorrectly documented attribute in csv docs (python#101250) pythonGH-84783: Make the slice object hashable (pythonGH-101264) ...

gh-97786: Fix compiler warnings in pytime.c (GH-101826) Fixes compiler warnings in pytime.c. (cherry picked from commit b1b375e) Co-authored-by: Mark Dickinson <dickinsm@gmail.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>

GH-102150 is a backport of this pull request to the 3.10 branch.

* main: (225 commits) pythongh-102056: Fix a few bugs in error handling of exception printing code (python#102078) pythongh-102011: use sys.exception() instead of sys.exc_info() in docs where possible (python#102012) pythongh-101566: Sync with zipp 3.14. (pythonGH-102018) pythonGH-99818: improve the documentation for zipfile.Path and Traversable (pythonGH-101589) pythongh-88233: zipfile: handle extras after a zip64 extra (pythonGH-96161) pythongh-101981: Apply HOMEBREW related environment variables (pythongh-102074) pythongh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (pythonGH-101912) pythongh-101819: Adapt _io types to heap types, batch 1 (pythonGH-101949) pythongh-101981: Build macOS as recommended by the devguide (pythonGH-102070) pythongh-97786: Fix compiler warnings in pytime.c (python#101826) pythongh-101578: Amend PyErr_{Set,Get}RaisedException docs (python#101962) Misc improvements to the float tutorial (pythonGH-102052) pythongh-85417: Clarify behaviour on branch cuts in cmath module (python#102046) pythongh-100425: Update tutorial docs related to sum() accuracy (FH-101854) Add missing 'is' to `cmath.log()` docstring (python#102049) pythongh-100210: Correct the comment link for unescaping HTML (python#100212) pythongh-97930: Also include subdirectory in makefile. (python#102030) pythongh-99735: Use required=True in argparse subparsers example (python#100927) Fix incorrectly documented attribute in csv docs (python#101250) pythonGH-84783: Make the slice object hashable (pythonGH-101264) ...

* [3.10] gh-97786: Fix compiler warnings in pytime.c (GH-101826) Fixes compiler warnings in pytime.c.. (cherry picked from commit b1b375e) Co-authored-by: Mark Dickinson <dickinsm@gmail.com> * Add comment about the casts --------- Co-authored-by: Gregory P. Smith <greg@krypto.org>

Since #101826 was merged, the internal macro `_Py_InIntegralTypeRange` is unused, as are its supporting macros `_Py_IntegralTypeMax` and `_Py_IntegralTypeMin`. This PR removes them. Note that `_Py_InIntegralTypeRange` doesn't actually work as advertised - it's not a safe way to avoid undefined behaviour in an integer to double conversion.

Since python#101826 was merged, the internal macro `_Py_InIntegralTypeRange` is unused, as are its supporting macros `_Py_IntegralTypeMax` and `_Py_IntegralTypeMin`. This PR removes them. Note that `_Py_InIntegralTypeRange` doesn't actually work as advertised - it's not a safe way to avoid undefined behaviour in an integer to double conversion.

Thanks for fixing this old annoying compiler bug, @mdickinson!

I added math.nextafter() and math.ulp() to Python 3.9 to help me to understand this issue :-) But then I failed to come up with a fix for this warning.

math.nextafter and math.ulp are very good to have, regardless! So thank you for that.

Fix compiler warnings in pytime.c

622ec0a

mdickinson added needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes labels Feb 11, 2023

mdickinson requested review from brettcannon and gpshead February 11, 2023 13:21

mdickinson requested review from abalkin and pganssle as code owners February 11, 2023 13:21

bedevere-bot added the awaiting core review label Feb 11, 2023

bedevere-bot mentioned this pull request Feb 11, 2023

Compiler warning for _Py_InIntegralTypeRange #97786

Closed

Add news

32aee25

mdickinson added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Feb 11, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Feb 11, 2023

mdickinson commented Feb 11, 2023

View reviewed changes

Comment thread Python/pytime.c

mdickinson commented Feb 11, 2023

View reviewed changes

Comment thread Python/pytime.c

mdickinson commented Feb 11, 2023

View reviewed changes

Comment thread Python/pytime.c

Add note on nans; remove unnecessary asserts

c381e4c

gpshead approved these changes Feb 20, 2023

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting core review labels Feb 20, 2023

gpshead merged commit b1b375e into python:main Feb 20, 2023

bedevere-bot removed the awaiting merge label Feb 20, 2023

miss-islington assigned gpshead Feb 20, 2023

bedevere-bot removed the needs backport to 3.11 only security fixes label Feb 20, 2023

gpshead removed the needs backport to 3.10 only security fixes label Feb 20, 2023

jaraco pushed a commit to jaraco/cpython that referenced this pull request Feb 20, 2023

pythongh-97786 : Fix compiler warnings in pytime.c (python#101826 )

c6f8395

Fixes compiler warnings in pytime.c.

jaraco mentioned this pull request Feb 20, 2023

gh-88233: zipfile: refactor _strip_extra #102083

Closed

This was referenced Mar 4, 2023

_PyTime_FromDouble() fails to detect an integer overflow when converting a C double to a C int64_t #83458

Closed

Overflow when casting from double to time_t, and_PyTime_t. #78604

Closed

Remove unused internal macros #102415

Merged

mdickinson deleted the fix-pytime-c-compiler-warnings branch April 6, 2023 15:25

gh-97786: Fix compiler warnings in pytime.c by mdickinson · Pull Request #101826 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-97786: Fix compiler warnings in pytime.c - #101826

gh-97786: Fix compiler warnings in pytime.c#101826
gpshead merged 3 commits into
python:mainfrom
mdickinson:fix-pytime-c-compiler-warnings

mdickinson commented Feb 11, 2023 •
edited
Loading

Uh oh!

Uh oh!

bedevere-bot commented Feb 11, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mdickinson commented Feb 12, 2023 •
edited
Loading

Uh oh!

Uh oh!

mdickinson commented Feb 19, 2023

Uh oh!

miss-islington commented Feb 20, 2023

Uh oh!

miss-islington commented Feb 20, 2023

Uh oh!

bedevere-bot commented Feb 20, 2023

Uh oh!

bedevere-bot commented Feb 22, 2023

Uh oh!

vstinner commented Apr 5, 2023

Uh oh!

vstinner commented Apr 5, 2023

Uh oh!

mdickinson commented Apr 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gh-97786: Fix compiler warnings in pytime.c by mdickinson · Pull Request #101826 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-97786: Fix compiler warnings in pytime.c - #101826

gh-97786: Fix compiler warnings in pytime.c#101826gpshead merged 3 commits intopython:mainpython/cpython:mainfrom mdickinson:fix-pytime-c-compiler-warningsmdickinson/cpython:fix-pytime-c-compiler-warningsCopy head branch name to clipboard

Conversation

mdickinson commented Feb 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Feb 11, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mdickinson commented Feb 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mdickinson commented Feb 19, 2023

Uh oh!

miss-islington commented Feb 20, 2023

Uh oh!

miss-islington commented Feb 20, 2023

Uh oh!

bedevere-bot commented Feb 20, 2023

Uh oh!

bedevere-bot commented Feb 22, 2023

Uh oh!

vstinner commented Apr 5, 2023

Uh oh!

vstinner commented Apr 5, 2023

Uh oh!

mdickinson commented Apr 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gh-97786: Fix compiler warnings in pytime.c#101826
gpshead merged 3 commits into
python:mainfrom
mdickinson:fix-pytime-c-compiler-warnings

mdickinson commented Feb 11, 2023 •
edited
Loading

mdickinson commented Feb 12, 2023 •
edited
Loading