gh-112532: Fix peg generator build.py for mimalloc build by corona10 · Pull Request #113492 · python/cpython · GitHub

Issue: mimalloc: additional integration and changes for --disable-gil builds #112532

AS-IS

➜  cpython git:(main) ✗ ./python.exe -m test test_peg_generator
Using random seed: 2669052678
Raised RLIMIT_NOFILE: 256 -> 1024
0:00:00 load avg: 3.38 Run 1 test sequentially
0:00:00 load avg: 3.38 [1/1] test_peg_generator
/Users/user/oss/cpython/build/test_python_worker_29863æ/tempcwd/venv/lib/python3.13/site-packages/pip/_internal/metadata/importlib/_envs.py:111: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pip._vendor.pkg_resources import find_distributions
Processing /Users/user/oss/cpython/Lib/test/setuptools-67.6.1-py3-none-any.whl
/Users/user/oss/cpython/build/test_python_worker_29863æ/tempcwd/venv/lib/python3.13/site-packages/pip/_internal/metadata/importlib/_dists.py:73: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
  return cls(files, info_location)
Processing /Users/user/oss/cpython/Lib/test/wheel-0.40.0-py3-none-any.whl
Installing collected packages: wheel, setuptools
/Users/user/oss/cpython/build/test_python_worker_29863æ/tempcwd/venv/lib/python3.13/site-packages/pip/_internal/metadata/importlib/_dists.py:73: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
  return cls(files, info_location)
Successfully installed setuptools-67.6.1 wheel-0.40.0
In file included from /Users/user/oss/cpython/Python/Python-ast.c:6:
In file included from /Users/user/oss/cpython/Include/internal/pycore_ceval.h:13:
In file included from /Users/user/oss/cpython/Include/internal/pycore_interp.h:32:
In file included from /Users/user/oss/cpython/Include/internal/pycore_tstate.h:11:
/Users/user/oss/cpython/Include/internal/pycore_mimalloc.h:32:10: fatal error: 'mimalloc.h' file not found
#include "mimalloc.h"
         ^~~~~~~~~~~~
1 error generated.
In file included from /Users/user/oss/cpython/Python/Python-ast.c:6:
In file included from /Users/user/oss/cpython/Include/internal/pycore_ceval.h:13:
In file included from /Users/user/oss/cpython/Include/internal/pycore_interp.h:32:
In file included from /Users/user/oss/cpython/Include/internal/pycore_tstate.h:11:
/Users/user/oss/cpython/Include/internal/pycore_mimalloc.h:32:10: fatal error: 'mimalloc.h' file not found
#include "mimalloc.h"
         ^~~~~~~~~~~~
1 error generated.
In file included from /Users/user/oss/cpython/Python/Python-ast.c:6:
In file included from /Users/user/oss/cpython/Include/internal/pycore_ceval.h:13:
In file included from /Users/user/oss/cpython/Include/internal/pycore_interp.h:32:
In file included from /Users/user/oss/cpython/Include/internal/pycore_tstate.h:11:
/Users/user/oss/cpython/Include/internal/pycore_mimalloc.h:32:10: fatal error: 'mimalloc.h' file not found
#include "mimalloc.h"
         ^~~~~~~~~~~~
1 error generated.
In file included from /Users/user/oss/cpython/Python/Python-ast.c:6:
In file included from /Users/user/oss/cpython/Include/internal/pycore_ceval.h:13:
In file included from /Users/user/oss/cpython/Include/internal/pycore_interp.h:32:
In file included from /Users/user/oss/cpython/Include/internal/pycore_tstate.h:11:
/Users/user/oss/cpython/Include/internal/pycore_mimalloc.h:32:10: fatal error: 'mimalloc.h' file not found
#include "mimalloc.h"
         ^~~~~~~~~~~~
1 error generated.
In file included from /Users/user/oss/cpython/Python/Python-ast.c:6:
In file included from /Users/user/oss/cpython/Include/internal/pycore_ceval.h:13:
In file included from /Users/user/oss/cpython/Include/internal/pycore_interp.h:32:
In file included from /Users/user/oss/cpython/Include/internal/pycore_tstate.h:11:
/Users/user/oss/cpython/Include/internal/pycore_mimalloc.h:32:10: fatal error: 'mimalloc.h' file not found
#include "mimalloc.h"
         ^~~~~~~~~~~~
1 error generated.
In file included from /Users/user/oss/cpython/Python/Python-ast.c:6:
In file included from /Users/user/oss/cpython/Include/internal/pycore_ceval.h:13:
In file included from /Users/user/oss/cpython/Include/internal/pycore_interp.h:32:
In file included from /Users/user/oss/cpython/Include/internal/pycore_tstate.h:11:
/Users/user/oss/cpython/Include/internal/pycore_mimalloc.h:32:10: fatal error: 'mimalloc.h' file not found
#include "mimalloc.h"
         ^~~~~~~~~~~~
1 error generated.
In file included from /Users/user/oss/cpython/Python/Python-ast.c:6:
In file included from /Users/user/oss/cpython/Include/internal/pycore_ceval.h:13:
In file included from /Users/user/oss/cpython/Include/internal/pycore_interp.h:32:
In file included from /Users/user/oss/cpython/Include/internal/pycore_tstate.h:11:
/Users/user/oss/cpython/Include/internal/pycore_mimalloc.h:32:10: fatal error: 'mimalloc.h' file not found
#include "mimalloc.h"
         ^~~~~~~~~~~~
1 error generated.
In file included from /Users/user/oss/cpython/Python/Python-ast.c:6:
In file included from /Users/user/oss/cpython/Include/internal/pycore_ceval.h:13:
In file included from /Users/user/oss/cpython/Include/internal/pycore_interp.h:32:
In file included from /Users/user/oss/cpython/Include/internal/pycore_tstate.h:11:
/Users/user/oss/cpython/Include/internal/pycore_mimalloc.h:32:10: fatal error: 'mimalloc.h' file not found
#include "mimalloc.h"
         ^~~~~~~~~~~~
1 error generated.
In file included from /Users/user/oss/cpython/Python/Python-ast.c:6:
In file included from /Users/user/oss/cpython/Include/internal/pycore_ceval.h:13:
In file included from /Users/user/oss/cpython/Include/internal/pycore_interp.h:32:
In file included from /Users/user/oss/cpython/Include/internal/pycore_tstate.h:11:
/Users/user/oss/cpython/Include/internal/pycore_mimalloc.h:32:10: fatal error: 'mimalloc.h' file not found
#include "mimalloc.h"

TO-BE

➜  cpython git:(gh-112532-fix) ✗ ./python.exe -m test test_peg_generator
Using random seed: 3314716989
Raised RLIMIT_NOFILE: 256 -> 1024
0:00:00 load avg: 3.00 Run 1 test sequentially
0:00:00 load avg: 3.00 [1/1] test_peg_generator
/Users/user/oss/cpython/build/test_python_worker_29244æ/tempcwd/venv/lib/python3.13/site-packages/pip/_internal/metadata/importlib/_envs.py:111: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pip._vendor.pkg_resources import find_distributions
Processing /Users/user/oss/cpython/Lib/test/setuptools-67.6.1-py3-none-any.whl
/Users/user/oss/cpython/build/test_python_worker_29244æ/tempcwd/venv/lib/python3.13/site-packages/pip/_internal/metadata/importlib/_dists.py:73: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
  return cls(files, info_location)
Processing /Users/user/oss/cpython/Lib/test/wheel-0.40.0-py3-none-any.whl
Installing collected packages: wheel, setuptools
/Users/user/oss/cpython/build/test_python_worker_29244æ/tempcwd/venv/lib/python3.13/site-packages/pip/_internal/metadata/importlib/_dists.py:73: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
  return cls(files, info_location)
Successfully installed setuptools-67.6.1 wheel-0.40.0
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups

== Tests result: SUCCESS ==

1 test OK.

Total duration: 14.3 sec
Total tests: run=92
Total test files: run=1/1
Result: SUCCESS

🤖 New build scheduled with the buildbot fleet by @corona10 for commit 3f91a0b 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

I assume we can do this unconditionally on the mimalloc build no?

I assume we can do this unconditionally on the mimalloc build no?

Or we can use support.with_mimalloc(), but not a good use case for this script.

Nah is fine, just wanted to ensure that the header include is correct independently if we have mimalloc or not

Thanks for fixing this.

Yeah, I think the include path is fine even if we are not compiling with mimalloc support.

We always include mimalloc headers through makefile.pre.in, so it's technically fine.

cpython/Makefile.pre.in

Lines 1795 to 1800 in acf3bcc

    
           $(srcdir)/Include/cpython/warnings.h \ 
        
           $(srcdir)/Include/cpython/weakrefobject.h \ 
        
           \ 
        
           $(MIMALLOC_HEADERS) \ 
        
           \ 
        
           $(srcdir)/Include/internal/pycore_abstract.h \

…ngh-113492) pythongh-112532: Fix peg generator for mimalloc build

pythongh-112532 : Fix peg generator for mimalloc build

3f91a0b

corona10 requested a review from colesbury December 26, 2023 17:28

corona10 requested review from lysnikolaou and pablogsal as code owners December 26, 2023 17:28

bedevere-app Bot added the awaiting core review label Dec 26, 2023

bedevere-app Bot mentioned this pull request Dec 26, 2023

mimalloc: additional integration and changes for --disable-gil builds #112532

Closed

corona10 changed the title gh-112532: Fix peg generator for mimalloc build gh-112532: Fix peg generator build.py for mimalloc build Dec 26, 2023

corona10 added the skip news label Dec 26, 2023

corona10 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Dec 26, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Dec 26, 2023

pablogsal approved these changes Dec 26, 2023

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels Dec 26, 2023

colesbury approved these changes Dec 26, 2023

View reviewed changes

corona10 merged commit e5cce70 into python:main Dec 26, 2023

bedevere-app Bot removed the awaiting merge label Dec 26, 2023

corona10 deleted the gh-112532-fix branch December 27, 2023 02:49

kulikjak pushed a commit to kulikjak/cpython that referenced this pull request Jan 22, 2024

pythongh-112532 : Fix peg generator build.py for mimalloc build (pytho…

a5ab557

…ngh-113492) pythongh-112532: Fix peg generator for mimalloc build

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-112532 : Fix peg generator build.py for mimalloc build (pytho…

aa30318

…ngh-113492) pythongh-112532: Fix peg generator for mimalloc build

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-112532 : Fix peg generator build.py for mimalloc build (pytho…

b3e1a96

…ngh-113492) pythongh-112532: Fix peg generator for mimalloc build

gh-112532: Fix peg generator build.py for mimalloc build by corona10 · Pull Request #113492 · python/cpython · GitHub

Repository navigation

gh-112532: Fix peg generator build.py for mimalloc build - #113492

gh-112532: Fix peg generator build.py for mimalloc build#113492
corona10 merged 1 commit into
python:mainfrom
corona10:gh-112532-fix

corona10 commented Dec 26, 2023 •
edited by bedevere-app Bot
Loading

Uh oh!

Uh oh!

corona10 commented Dec 26, 2023

Uh oh!

bedevere-bot commented Dec 26, 2023

Uh oh!

pablogsal left a comment

Uh oh!

corona10 commented Dec 26, 2023

Uh oh!

pablogsal commented Dec 26, 2023

Uh oh!

colesbury left a comment

Uh oh!

corona10 commented Dec 26, 2023 •
edited
Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gh-112532: Fix peg generator build.py for mimalloc build by corona10 · Pull Request #113492 · python/cpython · GitHub

Repository navigation

gh-112532: Fix peg generator build.py for mimalloc build - #113492

gh-112532: Fix peg generator build.py for mimalloc build#113492corona10 merged 1 commit intopython:mainpython/cpython:mainfrom corona10:gh-112532-fixcorona10/cpython:gh-112532-fixCopy head branch name to clipboard

Conversation

corona10 commented Dec 26, 2023 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corona10 commented Dec 26, 2023

AS-IS

TO-BE

Uh oh!

bedevere-bot commented Dec 26, 2023

Uh oh!

pablogsal left a comment

Choose a reason for hiding this comment

Uh oh!

corona10 commented Dec 26, 2023

Uh oh!

pablogsal commented Dec 26, 2023

Uh oh!

colesbury left a comment

Choose a reason for hiding this comment

Uh oh!

corona10 commented Dec 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gh-112532: Fix peg generator build.py for mimalloc build#113492
corona10 merged 1 commit into
python:mainfrom
corona10:gh-112532-fix

corona10 commented Dec 26, 2023 •
edited by bedevere-app Bot
Loading

corona10 commented Dec 26, 2023 •
edited
Loading