[mypyc] feat: new primitive for `int.to_bytes` by BobTheBuidler · Pull Request #19674 · python/mypy · GitHub

This PR adds a new primitive for all arg combinations of int.to_bytes

for more information, see https://pre-commit.ci

… to-bytes

Nice! I will keep this open for a day or two in case @JukkaL has some comments.

On the second look I think I have some more questions.

On the second thought, all these type checks look unnecessary, normally Python wrappers should do them. You can probably verify this by adding some run tests with Anys in them.

what, like this?

def f(x: Any) -> bytes:
    return int.to_bytes(x)

based on @JukkaL response to a similar question on #19673 I think we can safely remove this check since CPyTagged_StealAsObject guarantees the type

I think CPyTagged_StealAsObject is not correct there, since it will transfer the ownership of the parameter, and this can cause a double free. CPyTagged_AsObject will return a new reference which you can decref at the end of the function.

@BobTheBuidler

what, like this?

First, not just the self, second I think you should try more something like this

def to_bytes(n: int, length: int, byteorder: str = "little", signed: bool = False) -> bytes:
    return n.to_bytes(length, byteorder, signed=signed)

x: Any = "no"
bad: Any = "way"
to_bytes(x, bad)

and check that a TypeError will be given even before getting to your specialized code.

I could implement this test, but wouldn't we then just be testing the standard python-wrapper type validation functionality, as opposed to some specific functionality related to this PR?

I can still add the tests accordingly, I just want to make sure we have the same understanding of things before I proceed.

I could implement this test, but wouldn't we then just be testing the standard python-wrapper type validation functionality, as opposed to some specific functionality related to this PR?

That's exactly my point. This whole thread started from me saying "On the second thought, all these type checks look unnecessary, normally Python wrappers should do them. You can probably verify this by adding some run tests with Anys in them."

To be clear, you don't need to add the tests to the PR, but you can (if you want) to check that I am right by running such a test locally.

Maybe also test calling to_bytes() function from interpreted code.

@ilevkivskyi how would I implement that? Is there a good example I can look from?

for more information, see https://pre-commit.ci

… to-bytes

for more information, see https://pre-commit.ci

… to-bytes

1.18 vs this branch

There was a ~5-8% increase in speed for various permutations of the int_to_big_endian benchmark which includes not only a call to to_bytes but a decent bit of other stuff as well

Okay, with the specializer this becomes a 9% improvement for all cases

… to-bytes

for more information, see https://pre-commit.ci

@BobTheBuidler just in case you didn't notice, some tests are failing now.

@BobTheBuidler please ping me when this is ready for review/merge.

That's odd. I fixed the kw-only arg in to_bytes stub, but doing so breaks the IR. Is there a way for me to cover this case with a method_op?

@ilevkivskyi this open question is my only remaining blocker, unsure how to navigate this situation

I don't know, maybe add some prints to specialize_int_to_bytes. For starter I think self is not included in CallExpr.args for methods, but maybe this is something else.

for more information, see https://pre-commit.ci

* Fix int.to_bytes specialization arg handling * Restore specialize.py for to_bytes specialization fixes

LG, thanks!

BobTheBuidler and others added 22 commits August 16, 2025 15:50

[mypyc] feat: new primitive for int.to_bytes

2058f81

[pre-commit.ci] auto fixes from pre-commit.com hooks

f62bfd2

for more information, see https://pre-commit.ci

add headers

813c510

Merge branch 'to-bytes' of https://github.com/BobTheBuidler/mypy into…

4be77d6

… to-bytes

cover all arg combos

cb93329

CPyLong_ToBytes header

6ff1c9b

;

166b6f4

fix name

be2d2de

fix _PyLong_AsByteArray compile err

9ff3a7c

Update ir.py

8399619

define header

db7b483

Merge branch 'to-bytes' of https://github.com/BobTheBuidler/mypy into…

ed05c00

… to-bytes

fix ir

a0c147b

Update ir.py

5bab265

fix ir

b346bbf

Merge branch 'to-bytes' of https://github.com/BobTheBuidler/mypy into…

8d523a0

… to-bytes

fix ir

85652a2

add ir test

8e97165

fix py 3.10

3660a01

use _PyLong_AsByteArray on all pythons

6a8a83c

fix: py3.13 and 3.14

ec95008

optimize if check

95fa8b0

BobTheBuidler marked this pull request as ready for review August 17, 2025 16:39

ilevkivskyi approved these changes Aug 20, 2025

View reviewed changes

Merge branch 'master' into to-bytes

78f4ca1

JukkaL reviewed Aug 21, 2025

View reviewed changes

Comment thread mypyc/lib-rt/int_ops.c Outdated

Comment thread mypyc/lib-rt/CPy.h Outdated

Comment thread mypyc/lib-rt/int_ops.c Outdated

Comment thread mypyc/test-data/run-integers.test Outdated

ilevkivskyi reviewed Aug 21, 2025

View reviewed changes

BobTheBuidler added 3 commits August 21, 2025 09:46

Update CPy.h

326484b

Update int_ops.c

245a122

Update int_ops.c

d2994e1

pre-commit-ci Bot and others added 13 commits September 10, 2025 08:09

[pre-commit.ci] auto fixes from pre-commit.com hooks

b5a6c06

for more information, see https://pre-commit.ci

fix

ef5b1d7

Merge branch 'to-bytes' of https://github.com/BobTheBuidler/mypy into…

9b629a5

… to-bytes

[pre-commit.ci] auto fixes from pre-commit.com hooks

9b69d58

for more information, see https://pre-commit.ci

fix

6a5de07

Merge branch 'to-bytes' of https://github.com/BobTheBuidler/mypy into…

251986e

… to-bytes

fix

f5f9d30

Update irbuild-int.test

9a27e5d

Update run-integers.test

6c82357

Update irbuild-int.test

c4d9673

maybe fix

3ad1e50

Update int_ops.py

77535d3

Update irbuild-int.test

d291fb1

BobTheBuidler and others added 4 commits September 10, 2025 13:22

Merge branch 'to-bytes' of https://github.com/BobTheBuidler/mypy into…

66bb191

… to-bytes

Merge branch 'master' into to-bytes

420b8e2

Merge branch 'master' into to-bytes

2fc6433

[pre-commit.ci] auto fixes from pre-commit.com hooks

11a6aea

for more information, see https://pre-commit.ci

Merge branch 'master' into to-bytes

2d69419

BobTheBuidler and others added 2 commits January 16, 2026 23:39

Merge branch 'master' into to-bytes

4efde1d

[pre-commit.ci] auto fixes from pre-commit.com hooks

c63862d

for more information, see https://pre-commit.ci

BobTheBuidler mentioned this pull request Jan 17, 2026

Fix int.to_bytes specialization arg handling BobTheBuidler/mypy#13

Merged

BobTheBuidler added 2 commits January 18, 2026 03:55

Merge branch 'master' into to-bytes

5b7cae9

Fix int.to_bytes specialization arg handling (#13 )

c761f6b

* Fix int.to_bytes specialization arg handling * Restore specialize.py for to_bytes specialization fixes

ilevkivskyi approved these changes Jan 18, 2026

View reviewed changes

ilevkivskyi merged commit dd55b66 into python:master Jan 18, 2026
16 checks passed

+              // int.to_bytes(length, byteorder, signed=False)
+              PyObject *CPyTagged_ToBytes(CPyTagged self, Py_ssize_t length, PyObject *byteorder, int signed_flag) {
+                  PyObject *pyint = CPyTagged_StealAsObject(self);
+                  if (!PyLong_Check(pyint)) {

+                  assert to_bytes(255, 2, "big") == b'\x00\xff'
+                  assert to_bytes(255, 2, "little") == b'\xff\x00'
+                  assert to_bytes(-1, 2, "big", True) == b'\xff\xff'
+                  assert to_bytes(0, 1, "big") == b'\x00'

[mypyc] feat: new primitive for `int.to_bytes` by BobTheBuidler · Pull Request #19674 · python/mypy · GitHub

Repository navigation

[mypyc] feat: new primitive for int.to_bytes - #19674

[mypyc] feat: new primitive for int.to_bytes#19674ilevkivskyi merged 62 commits intopython:masterpython/mypy:masterfrom BobTheBuidler:to-bytesBobTheBuidler/mypy:to-bytesCopy head branch name to clipboard

Conversation

BobTheBuidler commented Aug 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

BobTheBuidler commented Sep 10, 2025

Uh oh!

ilevkivskyi commented Oct 10, 2025

Uh oh!

ilevkivskyi commented Oct 15, 2025

Uh oh!

BobTheBuidler commented Oct 15, 2025

Uh oh!

ilevkivskyi commented Oct 15, 2025

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[mypyc] feat: new primitive for int.to_bytes#19674
ilevkivskyi merged 62 commits into
python:masterfrom
BobTheBuidler:to-bytes

BobTheBuidler commented Aug 16, 2025 •
edited
Loading