FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-114058: The Tier2 Optimizer by Fidget-Spinner · Pull Request #114059 · python/cpython · GitHub

/ cpython Public

gh-114058: The Tier2 Optimizer - #114059

Closed
Fidget-Spinner wants to merge 116 commits into
python:mainfrom
Fidget-Spinner:tier2_abstract_interpreter
ConversationCommits (116)ChecksFiles changed
Closed

gh-114058: The Tier2 Optimizer#114059
Fidget-Spinner wants to merge 116 commits into
python:mainfrom
Fidget-Spinner:tier2_abstract_interpreter

Conversation

Fidget-Spinner commented Jan 14, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Closes #114058

This PR turns on the optimizer for all uops. The tier 2 uops optimizer contains a few parts: the abstract interpreter, the IR, and the codegen.

The abstract interpreter does the following:

  • Value numbering for types (not on all expressions though, that is too expensive)
  • Type propagation
  • Constant propagation
  • Guard elimination
  • Poor man's loop invariant code motion for guards/loop duplication

Function inlining is left out for a future PR, as it's the most complex.

After analysis of the bytecode and doing all of the above, it emits optimized uops, and passes that to the executor.

When uops is enabled, **this passes the entire CPython test suite **. The significant milestone is that this is able to analyse and abstract interpret all CPython uops that we currently support. The other significant milestone is that this generates code that passes CPython's test suite.

Refleak tests will fail as well, as they need a design overhaul.

The design of this PR is here https://github.com/Fidget-Spinner/cpython_optimization_notes/blob/main/3.13/uops_optimizer.md

High level discussion here faster-cpython/ideas#648.

0-2% faster on Linux, 3% faster on macOS ARM64

Fidget-Spinner marked this pull request as draft January 14, 2024 18:38
Fidget-Spinner changed the title The Tier2 Optimizer's abstract interpreter gh-114058: The Tier2 Optimizer's abstract interpreter Jan 14, 2024
bedevere-app Bot mentioned this pull request Jan 14, 2024
Closed

Fidget-Spinner commented Jan 14, 2024

Copy link
Copy Markdown
Member Author

Please give me some time to write out the proper docs explaining the abstract IR this uses.

Fidget-Spinner changed the title gh-114058: The Tier2 Optimizer's abstract interpreter gh-114058: The Tier2 Optimizer Jan 14, 2024

Fidget-Spinner commented Jan 16, 2024

Copy link
Copy Markdown
Member Author

All tests run with uops on now passes except for the following two:

  1. test_capi - there's one test that counts memory allocation, which obviously fails because my optimizer allocates memory. I think I'm just going to skip this test when -Xuops is detected.
  2. test_ctypes - this doesn't actually fail I just have no clue whether it passes because the linker on my system is broken.

brandtbucher self-requested a review January 16, 2024 21:27

brandtbucher commented Jan 17, 2024

Copy link
Copy Markdown
Member

I think I'm just going to skip this test when -Xuops is detected.

Maybe we should just fix the test? This sort of seems to be kicking the can down the road, since eventually tier 2 will be on by default.

There's a without_optimizer helper in test.support. We also have other ways of getting and setting optimizers in _testinternalcapi if we're in a subprocess or something (check out test_opt.temporary_optimizer for an example).

brandtbucher commented Jan 17, 2024

Copy link
Copy Markdown
Member

1% slower on macOS (other platforms aren't building right now). 8% reduction in traces executed, but 3% increase in uops executed.

PGO failure on Windows:

Details 
Running PGInstrument|x64 interpreter...
Using random seed: 1256678172
0:00:00 Run 44 tests sequentially
0:00:00 [ 1/44] test_array
0:00:05 [ 2/44] test_base64
0:00:07 load avg: 0.41 [ 3/44] test_binascii
0:00:07 load avg: 0.41 [ 4/44] test_binop
0:00:08 load avg: 0.41 [ 5/44] test_bisect
0:00:08 load avg: 0.42 [ 6/44] test_bytes
0:00:34 load avg: 0.50 [ 7/44] test_bz2
0:00:40 load avg: 0.49 [ 8/44] test_cmath
0:00:41 load avg: 0.48 [ 9/44] test_codecs
0:00:52 load avg: 0.46 [10/44] test_collections
0:01:03 load avg: 0.52 [11/44] test_complex
0:01:04 load avg: 0.53 [12/44] test_dataclasses
0:01:08 load avg: 0.56 [13/44] test_datetime
0:01:18 load avg: 0.56 [14/44] test_decimal
0:01:49 load avg: 0.61 [15/44] test_difflib -- test_decimal passed in 31.4 sec
0:01:59 load avg: 0.60 [16/44] test_embed
0:02:42 load avg: 0.33 [17/44] test_float -- test_embed passed in 42.7 sec
0:02:43 load avg: 0.34 [18/44] test_fstring
0:02:57 load avg: 0.34 [19/44] test_functools
Windows fatal exception: stack overflow

Thread 0x00002f90 (most recent call first):
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\libregrtest\win_utils.py", line 43 in _update_load

Current thread 0x000019dc (most recent call first):
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1[875](https://github.com/faster-cpython/benchmarking/actions/runs/7559538614/job/20583550792#step:10:876) in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  File "C:\actions-runner\_work\benchmarking\benchmarking\cpython\Lib\test\test_functools.py", line 1875 in fib
  ...

PGO failure on Linux:

Details 
# Next, run the profile task to generate the profile information.
./python -m test --pgo --timeout=
Using random seed: [1366](https://github.com/faster-cpython/benchmarking/actions/runs/7559538614/job/20583550197#step:10:1367)[1376](https://github.com/faster-cpython/benchmarking/actions/runs/7559538614/job/20583550197#step:10:1377)73
0:00:00 load avg: 1.96 Run 44 tests sequentially
0:00:00 load avg: 1.96 [ 1/44] test_array
0:00:00 load avg: 1.89 [ 2/44] test_base64
0:00:01 load avg: 1.89 [ 3/44] test_binascii
0:00:01 load avg: 1.89 [ 4/44] test_binop
0:00:01 load avg: 1.89 [ 5/44] test_bisect
0:00:01 load avg: 1.89 [ 6/44] test_bytes
0:00:05 load avg: 1.89 [ 7/44] test_bz2
0:00:06 load avg: 1.82 [ 8/44] test_cmath
0:00:06 load avg: 1.82 [ 9/44] test_codecs
Fatal Python error: Segmentation fault

Current thread 0x00007fb926b78740 (most recent call first):
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/encodings/idna.py", line 37 in nameprep
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/encodings/idna.py", line 74 in ToASCII
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/encodings/idna.py", line 142 in ToUnicode
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/encodings/idna.py", line 222 in decode
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/test_codecs.py", line 1561 in test_builtin_decode_length_limit
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/unittest/case.py", line 589 in _callTestMethod
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/unittest/case.py", line 636 in run
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/unittest/case.py", line 692 in __call__
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/testresult.py", line 146 in run
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/single.py", line 57 in _run_suite
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/single.py", line 37 in run_unittest
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/single.py", line 132 in test_func
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/single.py", line 88 in regrtest_runner
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/single.py", line 135 in _load_run_test
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/single.py", line 178 in _runtest_env_changed_exc
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/single.py", line 278 in _runtest
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/single.py", line 306 in run_single_test
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/main.py", line 302 in run_test
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/main.py", line 336 in run_tests_sequentially
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/main.py", line 477 in _run_tests
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/main.py", line 509 in run_tests
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/main.py", line 672 in main
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/libregrtest/main.py", line 680 in main
  File "/home/ddfun/actions-runner/_work/benchmarking/benchmarking/cpython/Lib/test/__main__.py", line 2 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main

Extension modules: _testcapi, _testinternalcapi (total: 2)
Segmentation fault (core dumped)

Both look like recursion-related issues. The Windows one may be fixed on main as of yesterday.

Fidget-Spinner commented Jan 18, 2024

Copy link
Copy Markdown
Member Author

Thanks Brandt. Seems like the slowdown is due to bm_nbody, and there's some serious pessimization there for some reason. The only way such a massive slowdown could happen IMO, is if the executors are constantly being discarded and re-optimized again, since optimization is now not completely free (not too sure, will take a look).

Llike Mark said though, benchmark results aren't too important at the moment. Function inlining would be the most important optimization and that's missing from this PR, to be added in a future PR.

gvanrossum commented Jan 18, 2024

Copy link
Copy Markdown
Member

Nevertheless it would be wise to dig deeper into what goes on with bm_nbody. It may be an important canary. :-)

gvanrossum mentioned this pull request Jan 31, 2024
Merged
Merge remote-tracking branch 'upstream/main' into tier2_abstract_inte…
07ba964 
…rpreter
Merge remote-tracking branch 'upstream/main' into tier2_abstract_inte…
50154ad 
…rpreter
Integrate the constant promoter
a0b58b1
fix test
4573fca
try fix memleak?
2096a8a
Revert "try fix memleak?"
4ad2804 
This reverts commit 2096a8a.
fix memleak for real
224f6b4
add more constant prop tests for promoted globals
b784617
refactor
2983553
Simplify codegen -- either types or consts, not both
6a6b11f
cleanup, reduce memory usage even more
cfe1de0
remove more memory allocations
5ad6211
zero dynamic memory allocation
73daf91
forgot return
cac3616
streamline frame creation
e7df93a
more cleanup
65fae9a
fix bug in return code
3fec959
fix on MSVC
dde7d12
Add back constant evaluation
04c902b
cleanup
05e93c2
add type annotation
55f9bcb
make const zapping more error-proof
0726766
fix stack metadata
a8adada
gvanrossum reviewed Feb 5, 2024
View reviewed changes

gvanrossum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Here are some comments on the latest version which I (mostly) wrote before hearing about plans to merge with Mark's version. (I wonder if it might make sense to start a new PR for that, rather than carry the history of over 100 commits along?)

Comment thread Tools/cases_generator/tier2_abstract_generator.py
Comment on lines +131 to +135
if var.name != MANGLED_NULL:
out.emit(f"{var.name} = sym_init_unknown(ctx);\n")
out.emit(f"if({var.name} == NULL) goto error;\n")
if var.type_prop:
out.emit(f"sym_set_type({var.name}, {var.type_prop[0]}, 0);\n")

gvanrossum Feb 4, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This is an example of something that I think can be streamlined further. Instead of calling sym_init_unknown(ctx) followed by sym_set_type(...) there ought to be a single function you can call to create a symbol with a given type. Basically a bunch of convenience constructors.

Comment thread Include/internal/pycore_optimizer.h
Comment on lines +15 to +16
// This the above + additional working space we need.
#define UOP_MAX_TRACE_WORKING_LENGTH (UOP_MAX_TRACE_LENGTH * 2)

gvanrossum Feb 5, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I don't see this used in the code any more.

Comment thread Python/optimizer.c Outdated
Comment thread Python/optimizer_analysis.c
Comment thread Python/optimizer_analysis.c
Comment thread Python/optimizer_analysis.c
Comment thread Python/optimizer_analysis.c
Comment thread Python/optimizer_analysis.c
Comment thread Python/optimizer_analysis.c
Comment thread Python/optimizer_analysis.c
Comment on lines +381 to 389
sym_set_type(sym, GUARD_TYPE_VERSION_TYPE, tp->tp_version_tag);
}
if (tp->tp_flags & Py_TPFLAGS_MANAGED_DICT) {
PyDictOrValues dorv = *_PyObject_DictOrValuesPointer(obj);
if(_PyDictOrValues_IsValues(dorv)) {
sym_set_type(sym, GUARD_DORV_VALUES_TYPE, 0);
}
}
if (tp == &PyLong_Type) {

gvanrossum Feb 5, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This could also use a longer explanation. What special cases are these two if blocks covering?

Fidget-Spinner commented Feb 6, 2024

Copy link
Copy Markdown
Member Author

Here are some comments on the latest version which I (mostly) wrote before hearing about plans to merge with Mark's version. (I wonder if it might make sense to start a new PR for that, rather than carry the history of over 100 commits along?)

Makes sense. I will close this PR and create a new one with the Mark's new DSL changes. Thanks for all your reviews!

Fidget-Spinner mentioned this pull request Feb 6, 2024
Merged

Fidget-Spinner commented Feb 6, 2024

Copy link
Copy Markdown
Member Author

The successor PR with cleaned up history is at #115085.

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

gvanrossum gvanrossum left review comments
markshannon Awaiting requested review from markshannon markshannon is a code owner
brandtbucher Awaiting requested review from brandtbucher brandtbucher is a code owner
ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner

Assignees

No one assigned

Labels

awaiting core review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Tier 2 optimizer's abstract interpreter

5 participants

Back | FazBrowse Home | New Git URL