FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Optimizing BOLT flags · Issue #128514 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

Optimizing BOLT flags #128514

Open
Open
Optimizing BOLT flags#128514
Assignees
Labels
buildThe build process and cross-buildperformancePerformance or resource usagetype-featureA feature request or enhancement

Description

zanieb
opened on Jan 5, 2025

Feature or enhancement

This is a tracking issue for discussion on determining the optimal flags for BOLT to improve performance.

Tuning the flags is mentioned in #101525, but doesn't feel like a blocker for stabilization.

Linked PRs

Activity

  1. zanieb
    added
    type-featureA feature request or enhancement
    performancePerformance or resource usage
    buildThe build process and cross-build
    on Jan 5, 2025

  2. zanieb commented on Jan 5, 2025

    zanieb
    on Jan 5, 2025
    ContributorAuthor

    There was a talk in March 2024 at the LLVM Performance Workshop; I can't find a copy of the talk online but the slides are available at https://llvm.org/devmtg/2024-03/slides/practical-use-of-bolt.pdf

    It includes the following suggestions:

    • Function splitting: -split-functions, -split-strategy=cdsplit
    • Function reordering: -reorder-functions=cdsort
    • Block reordering: -reorder-blocks=ext-tsp
    • Use THP pages for hot text: -hugify
    • PLT optimization: -plt
    • More aggressive ICF: -icf
    • Indirect Call Promotion: -indirect-call-promotion

    We're currently using:

    cpython/configure.ac

    Lines 2199 to 2212 in b60044b

    -reorder-blocks=ext-tsp
    -reorder-functions=cdsort
    -split-functions
    -icf=1
    -inline-all
    -split-eh
    -reorder-functions-use-hot-size
    -peepholes=none
    -jump-tables=aggressive
    -inline-ap
    -indirect-call-promotion=all
    -dyno-stats
    -use-gnu-stack
    -frame-opt=hot

    Suggesting we should explore -split-strategy=cdsplit, -hugify, and -plt

  3. zanieb commented on Jan 5, 2025

    zanieb
    on Jan 5, 2025
    ContributorAuthor

    There's some commentary in #124948 (comment)

    python-build-standalone recently added -hugify and -split-strategy=cdsplit (astral-sh/python-build-standalone#462), though the performance benefits were not validated.

    My intent is to do some benchmarking for each flag.

  4. liusy58 commented on Jan 7, 2025

    liusy58
    on Jan 7, 2025

    Ask me everything if you need. And I am now working on BOLT, I also want to contribute to Python!

  5. liusy58 commented on Jan 7, 2025

    liusy58
    on Jan 7, 2025

    By the way, I wonder how you collect profiles? By instrumentation or perf ?

  6. liusy58 commented on Jan 7, 2025

    liusy58
    on Jan 7, 2025

    I strongly recommend that --split-all-cold should be added.

  7. zanieb commented on Jan 7, 2025

    zanieb
    on Jan 7, 2025
    ContributorAuthor

    I was going to collect benchmarks with https://github.com/python/pyperformance (i.e., not instrumentation) on my Linux machine.

    I think I can also post branches and ask the faster-cpython team to run benchmarks https://github.com/faster-cpython/benchmarking-public

    I have a few commits ready

  8. liusy58 commented on Jan 8, 2025

    liusy58
    on Jan 8, 2025

    Profiles are key to BOLT. You are on x86, right? I remember cdsplit is not supported on AArch64.

  9. zanieb commented on Jan 8, 2025

    zanieb
    on Jan 8, 2025
    ContributorAuthor

    I have machines with both architectures.

    Are you suggesting an alternative approach to measuring the effect?

  10. liusy58 commented on Jan 8, 2025

    liusy58
    on Jan 8, 2025

    Yeah, maybe aarch64 can get more performance.

  11. zanieb commented on Jan 10, 2025

    zanieb
    on Jan 10, 2025
    ContributorAuthor

    As an update, I set up an x86-64 bare metal machine with LLVM 19 and am running benchmarks for the flags I described above. I'm not including LTO in the baseline, should I?

  12. zanieb commented on Jan 11, 2025

    zanieb
    on Jan 11, 2025
    ContributorAuthor

    Here are my initial results https://gist.github.com/zanieb/eee897b77597f93d0d3e0c2081dbc712
    Script at https://github.com/zanieb/cpython-bench/tree/34680389f94795b7b42d8c8dc784be34f0b14ce2

  13. corona10 commented on Jan 15, 2025

    corona10
    on Jan 15, 2025
    Member

    By the way, I wonder how you collect profiles? By instrumentation or perf ?

    FYI, We are getting BOLTed binary through instrumentation, not perf, when we actually build.

  14. corona10 commented on Jan 15, 2025

    corona10
    on Jan 15, 2025
    Member

    As an update, I set up an x86-64 bare metal machine with LLVM 19 and am running benchmarks for the flags I described above. I'm not including LTO in the baseline, should I?

    I belive that we don't have to, if only the difference between baseline is flag :)

  15. corona10
    assigned
    corona10
    and
    zanieb
    on Jan 15, 2025

  16. corona10 commented on Jan 15, 2025

    corona10
    on Jan 15, 2025
    Member

    (I am adding myself as assignee to catch up)

  17. zanieb commented on Jan 16, 2025

    zanieb
    on Jan 16, 2025
    ContributorAuthor

    A second round of benchmarks with more samples comes out a little different https://gist.github.com/zanieb/8614bcb40b0db24dd678f2983146fb43

    The effect depends on the workload.

  18. zanieb commented on Feb 9, 2025

    zanieb
    on Feb 9, 2025
    ContributorAuthor

    Following up with more analysis here

    hugify

    All benchmarks:
===============

Slower (23):
- sqlglot_normalize: 102 ms +- 1 ms -> 279 ms +- 3 ms: 2.74x slower
- python_startup_no_site: 6.52 ms +- 0.03 ms -> 8.53 ms +- 0.04 ms: 1.31x slower
- python_startup: 9.30 ms +- 0.03 ms -> 11.3 ms +- 0.0 ms: 1.21x slower
- spectral_norm: 93.0 ms +- 0.8 ms -> 98.5 ms +- 1.0 ms: 1.06x slower
- richards_super: 49.7 ms +- 0.4 ms -> 51.2 ms +- 0.4 ms: 1.03x slower
- generators: 25.3 ms +- 0.2 ms -> 26.1 ms +- 0.3 ms: 1.03x slower
- richards: 43.6 ms +- 0.4 ms -> 44.8 ms +- 0.4 ms: 1.03x slower
- coroutines: 20.4 ms +- 0.3 ms -> 21.0 ms +- 0.3 ms: 1.03x slower
- scimark_sor: 114 ms +- 1 ms -> 116 ms +- 1 ms: 1.02x slower
- mako: 10.2 ms +- 0.1 ms -> 10.4 ms +- 0.2 ms: 1.02x slower
- regex_dna: 147 ms +- 3 ms -> 150 ms +- 4 ms: 1.02x slower
- genshi_text: 19.6 ms +- 0.2 ms -> 19.9 ms +- 0.2 ms: 1.02x slower
- comprehensions: 15.3 us +- 0.1 us -> 15.5 us +- 0.2 us: 1.01x slower
- deltablue: 2.91 ms +- 0.03 ms -> 2.95 ms +- 0.03 ms: 1.01x slower
- xml_etree_iterparse: 78.7 ms +- 0.6 ms -> 79.7 ms +- 0.7 ms: 1.01x slower
- xml_etree_parse: 114 ms +- 1 ms -> 115 ms +- 1 ms: 1.01x slower
- raytrace: 256 ms +- 3 ms -> 258 ms +- 3 ms: 1.01x slower
- logging_simple: 5.25 us +- 0.07 us -> 5.29 us +- 0.10 us: 1.01x slower
- asyncio_tcp: 429 ms +- 4 ms -> 432 ms +- 4 ms: 1.01x slower
- pyflate: 380 ms +- 3 ms -> 382 ms +- 3 ms: 1.01x slower
- genshi_xml: 46.1 ms +- 0.6 ms -> 46.4 ms +- 0.6 ms: 1.01x slower
- go: 110 ms +- 1 ms -> 110 ms +- 1 ms: 1.00x slower
- asyncio_tcp_ssl: 1.48 sec +- 0.01 sec -> 1.48 sec +- 0.00 sec: 1.00x slower

Faster (39):
- logging_silent: 107 ns +- 1 ns -> 100 ns +- 1 ns: 1.07x faster
- regex_effbot: 2.47 ms +- 0.03 ms -> 2.33 ms +- 0.06 ms: 1.06x faster
- scimark_sparse_mat_mult: 4.21 ms +- 0.11 ms -> 3.98 ms +- 0.09 ms: 1.06x faster
- json_loads: 22.9 us +- 0.2 us -> 21.9 us +- 0.3 us: 1.05x faster
- json_dumps: 10.1 ms +- 0.2 ms -> 9.65 ms +- 0.16 ms: 1.04x faster
- regex_v8: 22.1 ms +- 0.2 ms -> 21.2 ms +- 0.2 ms: 1.04x faster
- telco: 6.60 ms +- 0.10 ms -> 6.38 ms +- 0.08 ms: 1.03x faster
- unpack_sequence: 37.4 ns +- 1.9 ns -> 36.2 ns +- 0.8 ns: 1.03x faster
- deepcopy: 250 us +- 2 us -> 242 us +- 2 us: 1.03x faster
- deepcopy_reduce: 2.60 us +- 0.03 us -> 2.52 us +- 0.04 us: 1.03x faster
- scimark_monte_carlo: 59.3 ms +- 0.4 ms -> 57.6 ms +- 0.6 ms: 1.03x faster
- deepcopy_memo: 29.6 us +- 0.2 us -> 28.7 us +- 0.3 us: 1.03x faster
- typing_runtime_protocols: 153 us +- 3 us -> 148 us +- 3 us: 1.03x faster
- scimark_fft: 313 ms +- 3 ms -> 305 ms +- 6 ms: 1.03x faster
- nqueens: 74.4 ms +- 0.7 ms -> 72.6 ms +- 0.5 ms: 1.02x faster
- chaos: 58.0 ms +- 0.8 ms -> 56.7 ms +- 0.7 ms: 1.02x faster
- pprint_pformat: 1.38 sec +- 0.01 sec -> 1.35 sec +- 0.02 sec: 1.02x faster
- scimark_lu: 112 ms +- 1 ms -> 110 ms +- 1 ms: 1.02x faster
- tomli_loads: 1.93 sec +- 0.02 sec -> 1.89 sec +- 0.03 sec: 1.02x faster
- pickle: 11.1 us +- 0.4 us -> 10.9 us +- 0.2 us: 1.02x faster
- pathlib: 19.2 ms +- 0.1 ms -> 18.8 ms +- 0.1 ms: 1.02x faster
- unpickle: 12.1 us +- 0.2 us -> 11.9 us +- 0.2 us: 1.02x faster
- logging_format: 5.82 us +- 0.13 us -> 5.74 us +- 0.12 us: 1.01x faster
- pprint_safe_repr: 667 ms +- 6 ms -> 657 ms +- 6 ms: 1.01x faster
- xml_etree_process: 54.6 ms +- 0.4 ms -> 53.8 ms +- 0.4 ms: 1.01x faster
- xml_etree_generate: 78.3 ms +- 0.5 ms -> 77.3 ms +- 0.6 ms: 1.01x faster
- pidigits: 160 ms +- 1 ms -> 158 ms +- 1 ms: 1.01x faster
- regex_compile: 118 ms +- 1 ms -> 117 ms +- 1 ms: 1.01x faster
- pickle_list: 3.84 us +- 0.07 us -> 3.80 us +- 0.16 us: 1.01x faster
- mdp: 2.33 sec +- 0.02 sec -> 2.30 sec +- 0.02 sec: 1.01x faster
- pickle_pure_python: 298 us +- 3 us -> 295 us +- 3 us: 1.01x faster
- meteor_contest: 89.6 ms +- 0.4 ms -> 89.0 ms +- 0.5 ms: 1.01x faster
- gc_traversal: 2.78 ms +- 0.06 ms -> 2.76 ms +- 0.05 ms: 1.01x faster
- html5lib: 60.5 ms +- 0.6 ms -> 60.1 ms +- 0.6 ms: 1.01x faster
- create_gc_cycles: 954 us +- 5 us -> 948 us +- 5 us: 1.01x faster
- dulwich_log: 60.0 ms +- 0.3 ms -> 59.7 ms +- 0.3 ms: 1.01x faster
- fannkuch: 372 ms +- 3 ms -> 370 ms +- 3 ms: 1.01x faster
- float: 66.4 ms +- 1.1 ms -> 66.0 ms +- 1.1 ms: 1.00x faster
- hexiom: 5.63 ms +- 0.06 ms -> 5.61 ms +- 0.05 ms: 1.00x faster

Benchmark hidden because not significant (11): 2to3, async_generators, asyncio_websockets, bench_mp_pool, bench_thread_pool, crypto_pyaes, docutils, nbody, pickle_dict, unpickle_list, unpickle_pure_python

Geometric mean: 1.01x slower

    cdsplit

    All benchmarks:
===============

Slower (41):
- sqlglot_normalize: 102 ms +- 1 ms -> 283 ms +- 2 ms: 2.78x slower
- deltablue: 2.91 ms +- 0.03 ms -> 3.20 ms +- 0.05 ms: 1.10x slower
- pyflate: 380 ms +- 3 ms -> 411 ms +- 4 ms: 1.08x slower
- scimark_fft: 313 ms +- 3 ms -> 329 ms +- 4 ms: 1.05x slower
- generators: 25.3 ms +- 0.2 ms -> 26.4 ms +- 0.2 ms: 1.04x slower
- html5lib: 60.5 ms +- 0.6 ms -> 62.7 ms +- 0.5 ms: 1.04x slower
- mako: 10.2 ms +- 0.1 ms -> 10.6 ms +- 0.1 ms: 1.04x slower
- go: 110 ms +- 1 ms -> 114 ms +- 1 ms: 1.03x slower
- logging_format: 5.82 us +- 0.13 us -> 6.00 us +- 0.13 us: 1.03x slower
- logging_simple: 5.25 us +- 0.07 us -> 5.41 us +- 0.12 us: 1.03x slower
- comprehensions: 15.3 us +- 0.1 us -> 15.8 us +- 0.1 us: 1.03x slower
- async_generators: 376 ms +- 4 ms -> 387 ms +- 4 ms: 1.03x slower
- genshi_xml: 46.1 ms +- 0.6 ms -> 47.3 ms +- 0.5 ms: 1.03x slower
- scimark_sparse_mat_mult: 4.21 ms +- 0.11 ms -> 4.32 ms +- 0.07 ms: 1.03x slower
- chaos: 58.0 ms +- 0.8 ms -> 59.4 ms +- 0.8 ms: 1.02x slower
- spectral_norm: 93.0 ms +- 0.8 ms -> 95.1 ms +- 0.9 ms: 1.02x slower
- scimark_sor: 114 ms +- 1 ms -> 116 ms +- 1 ms: 1.02x slower
- coroutines: 20.4 ms +- 0.3 ms -> 20.8 ms +- 0.3 ms: 1.02x slower
- richards: 43.6 ms +- 0.4 ms -> 44.4 ms +- 0.4 ms: 1.02x slower
- fannkuch: 372 ms +- 3 ms -> 378 ms +- 3 ms: 1.02x slower
- richards_super: 49.7 ms +- 0.4 ms -> 50.6 ms +- 0.5 ms: 1.02x slower
- json_dumps: 10.1 ms +- 0.2 ms -> 10.2 ms +- 0.3 ms: 1.02x slower
- regex_dna: 147 ms +- 3 ms -> 149 ms +- 3 ms: 1.02x slower
- genshi_text: 19.6 ms +- 0.2 ms -> 19.9 ms +- 0.2 ms: 1.02x slower
- asyncio_tcp: 429 ms +- 4 ms -> 436 ms +- 5 ms: 1.01x slower
- meteor_contest: 89.6 ms +- 0.4 ms -> 90.7 ms +- 0.5 ms: 1.01x slower
- float: 66.4 ms +- 1.1 ms -> 67.2 ms +- 1.1 ms: 1.01x slower
- typing_runtime_protocols: 153 us +- 3 us -> 154 us +- 3 us: 1.01x slower
- pprint_safe_repr: 667 ms +- 6 ms -> 674 ms +- 7 ms: 1.01x slower
- tomli_loads: 1.93 sec +- 0.02 sec -> 1.95 sec +- 0.02 sec: 1.01x slower
- hexiom: 5.63 ms +- 0.06 ms -> 5.68 ms +- 0.04 ms: 1.01x slower
- xml_etree_process: 54.6 ms +- 0.4 ms -> 55.0 ms +- 0.4 ms: 1.01x slower
- 2to3: 230 ms +- 1 ms -> 232 ms +- 1 ms: 1.01x slower
- regex_compile: 118 ms +- 1 ms -> 119 ms +- 1 ms: 1.01x slower
- xml_etree_iterparse: 78.7 ms +- 0.6 ms -> 79.2 ms +- 0.7 ms: 1.01x slower
- pprint_pformat: 1.38 sec +- 0.01 sec -> 1.39 sec +- 0.01 sec: 1.00x slower
- mdp: 2.33 sec +- 0.02 sec -> 2.34 sec +- 0.01 sec: 1.00x slower
- raytrace: 256 ms +- 3 ms -> 257 ms +- 3 ms: 1.00x slower
- asyncio_tcp_ssl: 1.48 sec +- 0.01 sec -> 1.48 sec +- 0.01 sec: 1.00x slower
- docutils: 2.22 sec +- 0.01 sec -> 2.23 sec +- 0.01 sec: 1.00x slower
- scimark_monte_carlo: 59.3 ms +- 0.4 ms -> 59.5 ms +- 0.5 ms: 1.00x slower

Faster (22):
- regex_effbot: 2.47 ms +- 0.03 ms -> 2.31 ms +- 0.05 ms: 1.07x faster
- scimark_lu: 112 ms +- 1 ms -> 105 ms +- 1 ms: 1.06x faster
- regex_v8: 22.1 ms +- 0.2 ms -> 20.8 ms +- 0.2 ms: 1.06x faster
- json_loads: 22.9 us +- 0.2 us -> 21.7 us +- 0.2 us: 1.06x faster
- logging_silent: 107 ns +- 1 ns -> 102 ns +- 1 ns: 1.05x faster
- unpack_sequence: 37.4 ns +- 1.9 ns -> 36.4 ns +- 0.8 ns: 1.03x faster
- deepcopy_reduce: 2.60 us +- 0.03 us -> 2.55 us +- 0.03 us: 1.02x faster
- unpickle: 12.1 us +- 0.2 us -> 11.9 us +- 0.1 us: 1.02x faster
- deepcopy_memo: 29.6 us +- 0.2 us -> 29.1 us +- 0.3 us: 1.02x faster
- pidigits: 160 ms +- 1 ms -> 158 ms +- 1 ms: 1.01x faster
- gc_traversal: 2.78 ms +- 0.06 ms -> 2.74 ms +- 0.02 ms: 1.01x faster
- create_gc_cycles: 954 us +- 5 us -> 944 us +- 6 us: 1.01x faster
- pathlib: 19.2 ms +- 0.1 ms -> 19.0 ms +- 0.1 ms: 1.01x faster
- bench_thread_pool: 936 us +- 39 us -> 926 us +- 35 us: 1.01x faster
- deepcopy: 250 us +- 2 us -> 248 us +- 2 us: 1.01x faster
- pickle_pure_python: 298 us +- 3 us -> 295 us +- 2 us: 1.01x faster
- unpickle_pure_python: 210 us +- 1 us -> 209 us +- 2 us: 1.01x faster
- unpickle_list: 4.36 us +- 0.07 us -> 4.33 us +- 0.09 us: 1.01x faster
- nbody: 88.3 ms +- 0.6 ms -> 87.8 ms +- 0.6 ms: 1.01x faster
- python_startup: 9.30 ms +- 0.03 ms -> 9.27 ms +- 0.03 ms: 1.00x faster
- python_startup_no_site: 6.52 ms +- 0.03 ms -> 6.49 ms +- 0.03 ms: 1.00x faster
- nqueens: 74.4 ms +- 0.7 ms -> 74.2 ms +- 0.6 ms: 1.00x faster

Benchmark hidden because not significant (10): asyncio_websockets, bench_mp_pool, crypto_pyaes, dulwich_log, pickle, pickle_dict, pickle_list, telco, xml_etree_parse, xml_etree_generate

Geometric mean: 1.02x slower

    cdsplit

    All benchmarks:
===============

Slower (41):
- sqlglot_normalize: 102 ms +- 1 ms -> 283 ms +- 2 ms: 2.78x slower
- deltablue: 2.91 ms +- 0.03 ms -> 3.20 ms +- 0.05 ms: 1.10x slower
- pyflate: 380 ms +- 3 ms -> 411 ms +- 4 ms: 1.08x slower
- scimark_fft: 313 ms +- 3 ms -> 329 ms +- 4 ms: 1.05x slower
- generators: 25.3 ms +- 0.2 ms -> 26.4 ms +- 0.2 ms: 1.04x slower
- html5lib: 60.5 ms +- 0.6 ms -> 62.7 ms +- 0.5 ms: 1.04x slower
- mako: 10.2 ms +- 0.1 ms -> 10.6 ms +- 0.1 ms: 1.04x slower
- go: 110 ms +- 1 ms -> 114 ms +- 1 ms: 1.03x slower
- logging_format: 5.82 us +- 0.13 us -> 6.00 us +- 0.13 us: 1.03x slower
- logging_simple: 5.25 us +- 0.07 us -> 5.41 us +- 0.12 us: 1.03x slower
- comprehensions: 15.3 us +- 0.1 us -> 15.8 us +- 0.1 us: 1.03x slower
- async_generators: 376 ms +- 4 ms -> 387 ms +- 4 ms: 1.03x slower
- genshi_xml: 46.1 ms +- 0.6 ms -> 47.3 ms +- 0.5 ms: 1.03x slower
- scimark_sparse_mat_mult: 4.21 ms +- 0.11 ms -> 4.32 ms +- 0.07 ms: 1.03x slower
- chaos: 58.0 ms +- 0.8 ms -> 59.4 ms +- 0.8 ms: 1.02x slower
- spectral_norm: 93.0 ms +- 0.8 ms -> 95.1 ms +- 0.9 ms: 1.02x slower
- scimark_sor: 114 ms +- 1 ms -> 116 ms +- 1 ms: 1.02x slower
- coroutines: 20.4 ms +- 0.3 ms -> 20.8 ms +- 0.3 ms: 1.02x slower
- richards: 43.6 ms +- 0.4 ms -> 44.4 ms +- 0.4 ms: 1.02x slower
- fannkuch: 372 ms +- 3 ms -> 378 ms +- 3 ms: 1.02x slower
- richards_super: 49.7 ms +- 0.4 ms -> 50.6 ms +- 0.5 ms: 1.02x slower
- json_dumps: 10.1 ms +- 0.2 ms -> 10.2 ms +- 0.3 ms: 1.02x slower
- regex_dna: 147 ms +- 3 ms -> 149 ms +- 3 ms: 1.02x slower
- genshi_text: 19.6 ms +- 0.2 ms -> 19.9 ms +- 0.2 ms: 1.02x slower
- asyncio_tcp: 429 ms +- 4 ms -> 436 ms +- 5 ms: 1.01x slower
- meteor_contest: 89.6 ms +- 0.4 ms -> 90.7 ms +- 0.5 ms: 1.01x slower
- float: 66.4 ms +- 1.1 ms -> 67.2 ms +- 1.1 ms: 1.01x slower
- typing_runtime_protocols: 153 us +- 3 us -> 154 us +- 3 us: 1.01x slower
- pprint_safe_repr: 667 ms +- 6 ms -> 674 ms +- 7 ms: 1.01x slower
- tomli_loads: 1.93 sec +- 0.02 sec -> 1.95 sec +- 0.02 sec: 1.01x slower
- hexiom: 5.63 ms +- 0.06 ms -> 5.68 ms +- 0.04 ms: 1.01x slower
- xml_etree_process: 54.6 ms +- 0.4 ms -> 55.0 ms +- 0.4 ms: 1.01x slower
- 2to3: 230 ms +- 1 ms -> 232 ms +- 1 ms: 1.01x slower
- regex_compile: 118 ms +- 1 ms -> 119 ms +- 1 ms: 1.01x slower
- xml_etree_iterparse: 78.7 ms +- 0.6 ms -> 79.2 ms +- 0.7 ms: 1.01x slower
- pprint_pformat: 1.38 sec +- 0.01 sec -> 1.39 sec +- 0.01 sec: 1.00x slower
- mdp: 2.33 sec +- 0.02 sec -> 2.34 sec +- 0.01 sec: 1.00x slower
- raytrace: 256 ms +- 3 ms -> 257 ms +- 3 ms: 1.00x slower
- asyncio_tcp_ssl: 1.48 sec +- 0.01 sec -> 1.48 sec +- 0.01 sec: 1.00x slower
- docutils: 2.22 sec +- 0.01 sec -> 2.23 sec +- 0.01 sec: 1.00x slower
- scimark_monte_carlo: 59.3 ms +- 0.4 ms -> 59.5 ms +- 0.5 ms: 1.00x slower

Faster (22):
- regex_effbot: 2.47 ms +- 0.03 ms -> 2.31 ms +- 0.05 ms: 1.07x faster
- scimark_lu: 112 ms +- 1 ms -> 105 ms +- 1 ms: 1.06x faster
- regex_v8: 22.1 ms +- 0.2 ms -> 20.8 ms +- 0.2 ms: 1.06x faster
- json_loads: 22.9 us +- 0.2 us -> 21.7 us +- 0.2 us: 1.06x faster
- logging_silent: 107 ns +- 1 ns -> 102 ns +- 1 ns: 1.05x faster
- unpack_sequence: 37.4 ns +- 1.9 ns -> 36.4 ns +- 0.8 ns: 1.03x faster
- deepcopy_reduce: 2.60 us +- 0.03 us -> 2.55 us +- 0.03 us: 1.02x faster
- unpickle: 12.1 us +- 0.2 us -> 11.9 us +- 0.1 us: 1.02x faster
- deepcopy_memo: 29.6 us +- 0.2 us -> 29.1 us +- 0.3 us: 1.02x faster
- pidigits: 160 ms +- 1 ms -> 158 ms +- 1 ms: 1.01x faster
- gc_traversal: 2.78 ms +- 0.06 ms -> 2.74 ms +- 0.02 ms: 1.01x faster
- create_gc_cycles: 954 us +- 5 us -> 944 us +- 6 us: 1.01x faster
- pathlib: 19.2 ms +- 0.1 ms -> 19.0 ms +- 0.1 ms: 1.01x faster
- bench_thread_pool: 936 us +- 39 us -> 926 us +- 35 us: 1.01x faster
- deepcopy: 250 us +- 2 us -> 248 us +- 2 us: 1.01x faster
- pickle_pure_python: 298 us +- 3 us -> 295 us +- 2 us: 1.01x faster
- unpickle_pure_python: 210 us +- 1 us -> 209 us +- 2 us: 1.01x faster
- unpickle_list: 4.36 us +- 0.07 us -> 4.33 us +- 0.09 us: 1.01x faster
- nbody: 88.3 ms +- 0.6 ms -> 87.8 ms +- 0.6 ms: 1.01x faster
- python_startup: 9.30 ms +- 0.03 ms -> 9.27 ms +- 0.03 ms: 1.00x faster
- python_startup_no_site: 6.52 ms +- 0.03 ms -> 6.49 ms +- 0.03 ms: 1.00x faster
- nqueens: 74.4 ms +- 0.7 ms -> 74.2 ms +- 0.6 ms: 1.00x faster

Benchmark hidden because not significant (10): asyncio_websockets, bench_mp_pool, crypto_pyaes, dulwich_log, pickle, pickle_dict, pickle_list, telco, xml_etree_parse, xml_etree_generate

Geometric mean: 1.02x slower

    split-all-cold

    All benchmarks:
===============

Slower (30):
- sqlglot_normalize: 102 ms +- 1 ms -> 283 ms +- 4 ms: 2.78x slower
- bench_mp_pool: 26.1 ms +- 0.4 ms -> 31.0 ms +- 8.7 ms: 1.19x slower
- mako: 10.2 ms +- 0.1 ms -> 11.0 ms +- 0.2 ms: 1.08x slower
- pyflate: 380 ms +- 3 ms -> 407 ms +- 3 ms: 1.07x slower
- create_gc_cycles: 954 us +- 5 us -> 1.01 ms +- 0.01 ms: 1.06x slower
- unpack_sequence: 37.4 ns +- 1.9 ns -> 39.5 ns +- 2.6 ns: 1.06x slower
- regex_dna: 147 ms +- 3 ms -> 152 ms +- 3 ms: 1.04x slower
- scimark_sparse_mat_mult: 4.21 ms +- 0.11 ms -> 4.36 ms +- 0.08 ms: 1.04x slower
- unpickle_list: 4.36 us +- 0.07 us -> 4.50 us +- 0.10 us: 1.03x slower
- telco: 6.60 ms +- 0.10 ms -> 6.81 ms +- 0.07 ms: 1.03x slower
- scimark_sor: 114 ms +- 1 ms -> 117 ms +- 1 ms: 1.03x slower
- generators: 25.3 ms +- 0.2 ms -> 26.1 ms +- 0.3 ms: 1.03x slower
- gc_traversal: 2.78 ms +- 0.06 ms -> 2.85 ms +- 0.06 ms: 1.03x slower
- raytrace: 256 ms +- 3 ms -> 262 ms +- 2 ms: 1.02x slower
- deltablue: 2.91 ms +- 0.03 ms -> 2.98 ms +- 0.02 ms: 1.02x slower
- xml_etree_iterparse: 78.7 ms +- 0.6 ms -> 80.3 ms +- 0.5 ms: 1.02x slower
- scimark_lu: 112 ms +- 1 ms -> 114 ms +- 2 ms: 1.02x slower
- genshi_text: 19.6 ms +- 0.2 ms -> 20.0 ms +- 0.2 ms: 1.02x slower
- async_generators: 376 ms +- 4 ms -> 383 ms +- 4 ms: 1.02x slower
- comprehensions: 15.3 us +- 0.1 us -> 15.6 us +- 0.1 us: 1.02x slower
- richards_super: 49.7 ms +- 0.4 ms -> 50.4 ms +- 0.5 ms: 1.01x slower
- xml_etree_parse: 114 ms +- 1 ms -> 116 ms +- 1 ms: 1.01x slower
- scimark_fft: 313 ms +- 3 ms -> 317 ms +- 3 ms: 1.01x slower
- xml_etree_generate: 78.3 ms +- 0.5 ms -> 79.2 ms +- 0.6 ms: 1.01x slower
- tomli_loads: 1.93 sec +- 0.02 sec -> 1.94 sec +- 0.02 sec: 1.01x slower
- go: 110 ms +- 1 ms -> 111 ms +- 1 ms: 1.01x slower
- xml_etree_process: 54.6 ms +- 0.4 ms -> 55.0 ms +- 0.5 ms: 1.01x slower
- html5lib: 60.5 ms +- 0.6 ms -> 60.8 ms +- 0.8 ms: 1.01x slower
- richards: 43.6 ms +- 0.4 ms -> 43.8 ms +- 0.4 ms: 1.00x slower
- 2to3: 230 ms +- 1 ms -> 230 ms +- 1 ms: 1.00x slower

Faster (29):
- regex_v8: 22.1 ms +- 0.2 ms -> 20.7 ms +- 0.2 ms: 1.06x faster
- logging_silent: 107 ns +- 1 ns -> 101 ns +- 1 ns: 1.06x faster
- spectral_norm: 93.0 ms +- 0.8 ms -> 88.6 ms +- 1.7 ms: 1.05x faster
- json_loads: 22.9 us +- 0.2 us -> 21.8 us +- 0.2 us: 1.05x faster
- coroutines: 20.4 ms +- 0.3 ms -> 19.5 ms +- 0.4 ms: 1.05x faster
- regex_effbot: 2.47 ms +- 0.03 ms -> 2.36 ms +- 0.05 ms: 1.05x faster
- hexiom: 5.63 ms +- 0.06 ms -> 5.46 ms +- 0.04 ms: 1.03x faster
- deepcopy_memo: 29.6 us +- 0.2 us -> 28.8 us +- 0.3 us: 1.03x faster
- deepcopy_reduce: 2.60 us +- 0.03 us -> 2.54 us +- 0.03 us: 1.02x faster
- pickle: 11.1 us +- 0.4 us -> 10.8 us +- 0.1 us: 1.02x faster
- unpickle: 12.1 us +- 0.2 us -> 11.8 us +- 0.2 us: 1.02x faster
- logging_format: 5.82 us +- 0.13 us -> 5.71 us +- 0.08 us: 1.02x faster
- nbody: 88.3 ms +- 0.6 ms -> 86.6 ms +- 0.6 ms: 1.02x faster
- pathlib: 19.2 ms +- 0.1 ms -> 18.8 ms +- 0.1 ms: 1.02x faster
- bench_thread_pool: 936 us +- 39 us -> 921 us +- 34 us: 1.02x faster
- deepcopy: 250 us +- 2 us -> 247 us +- 3 us: 1.01x faster
- nqueens: 74.4 ms +- 0.7 ms -> 73.4 ms +- 0.5 ms: 1.01x faster
- meteor_contest: 89.6 ms +- 0.4 ms -> 88.5 ms +- 0.5 ms: 1.01x faster
- dulwich_log: 60.0 ms +- 0.3 ms -> 59.2 ms +- 0.3 ms: 1.01x faster
- pprint_pformat: 1.38 sec +- 0.01 sec -> 1.37 sec +- 0.01 sec: 1.01x faster
- fannkuch: 372 ms +- 3 ms -> 368 ms +- 4 ms: 1.01x faster
- chaos: 58.0 ms +- 0.8 ms -> 57.5 ms +- 0.5 ms: 1.01x faster
- pprint_safe_repr: 667 ms +- 6 ms -> 661 ms +- 5 ms: 1.01x faster
- regex_compile: 118 ms +- 1 ms -> 117 ms +- 1 ms: 1.01x faster
- pickle_pure_python: 298 us +- 3 us -> 296 us +- 3 us: 1.00x faster
- crypto_pyaes: 63.6 ms +- 0.6 ms -> 63.3 ms +- 0.4 ms: 1.00x faster
- mdp: 2.33 sec +- 0.02 sec -> 2.32 sec +- 0.01 sec: 1.00x faster
- pidigits: 160 ms +- 1 ms -> 160 ms +- 1 ms: 1.00x faster
- python_startup: 9.30 ms +- 0.03 ms -> 9.29 ms +- 0.03 ms: 1.00x faster

Benchmark hidden because not significant (14): asyncio_tcp, asyncio_tcp_ssl, asyncio_websockets, docutils, float, genshi_xml, json_dumps, logging_simple, pickle_dict, pickle_list, python_startup_no_site, scimark_monte_carlo, typing_runtime_protocols, unpickle_pure_python

Geometric mean: 1.02x slower

  19. zanieb commented on Feb 9, 2025

    zanieb
    on Feb 9, 2025
    ContributorAuthor

    I'm not sure what's up with sqlglot_normalize

  20. zanieb commented on Feb 10, 2025

    zanieb
    on Feb 10, 2025
    ContributorAuthor

    I did another benchmark with all three of the aforementioned flags added. It looks like together, there's a more consistent speedup.

    https://gist.github.com/zanieb/2e043b5f1f415a0b1e98fcf72465c0fa

    All benchmarks:
===============

Slower (10):
- sqlglot_normalize: 102 ms +- 1 ms -> 275 ms +- 3 ms: 2.70x slower
- python_startup_no_site: 6.52 ms +- 0.03 ms -> 8.53 ms +- 0.04 ms: 1.31x slower
- python_startup: 9.30 ms +- 0.03 ms -> 11.2 ms +- 0.0 ms: 1.21x slower
- bench_mp_pool: 26.1 ms +- 0.4 ms -> 28.3 ms +- 0.6 ms: 1.08x slower
- create_gc_cycles: 954 us +- 5 us -> 1.00 ms +- 0.01 ms: 1.05x slower
- regex_dna: 147 ms +- 3 ms -> 153 ms +- 1 ms: 1.04x slower
- gc_traversal: 2.78 ms +- 0.06 ms -> 2.84 ms +- 0.03 ms: 1.02x slower
- json_loads: 22.9 us +- 0.2 us -> 23.1 us +- 0.2 us: 1.01x slower
- xml_etree_iterparse: 78.7 ms +- 0.6 ms -> 79.0 ms +- 0.9 ms: 1.00x slower
- xml_etree_parse: 114 ms +- 1 ms -> 114 ms +- 1 ms: 1.00x slower

Faster (58):
- unpack_sequence: 37.4 ns +- 1.9 ns -> 33.1 ns +- 0.5 ns: 1.13x faster
- deltablue: 2.91 ms +- 0.03 ms -> 2.63 ms +- 0.02 ms: 1.11x faster
- nbody: 88.3 ms +- 0.6 ms -> 80.1 ms +- 0.8 ms: 1.10x faster
- logging_silent: 107 ns +- 1 ns -> 97.3 ns +- 1.8 ns: 1.10x faster
- hexiom: 5.63 ms +- 0.06 ms -> 5.13 ms +- 0.05 ms: 1.10x faster
- async_generators: 376 ms +- 4 ms -> 345 ms +- 3 ms: 1.09x faster
- go: 110 ms +- 1 ms -> 101 ms +- 1 ms: 1.09x faster
- chaos: 58.0 ms +- 0.8 ms -> 53.5 ms +- 0.6 ms: 1.08x faster
- regex_compile: 118 ms +- 1 ms -> 110 ms +- 1 ms: 1.08x faster
- unpickle_pure_python: 210 us +- 1 us -> 195 us +- 1 us: 1.08x faster
- pyflate: 380 ms +- 3 ms -> 354 ms +- 4 ms: 1.07x faster
- genshi_text: 19.6 ms +- 0.2 ms -> 18.5 ms +- 0.2 ms: 1.06x faster
- float: 66.4 ms +- 1.1 ms -> 62.5 ms +- 0.6 ms: 1.06x faster
- comprehensions: 15.3 us +- 0.1 us -> 14.4 us +- 0.3 us: 1.06x faster
- deepcopy: 250 us +- 2 us -> 236 us +- 2 us: 1.06x faster
- regex_effbot: 2.47 ms +- 0.03 ms -> 2.33 ms +- 0.04 ms: 1.06x faster
- scimark_fft: 313 ms +- 3 ms -> 297 ms +- 9 ms: 1.06x faster
- scimark_monte_carlo: 59.3 ms +- 0.4 ms -> 56.3 ms +- 0.9 ms: 1.05x faster
- unpickle_list: 4.36 us +- 0.07 us -> 4.14 us +- 0.10 us: 1.05x faster
- pickle_pure_python: 298 us +- 3 us -> 283 us +- 3 us: 1.05x faster
- richards: 43.6 ms +- 0.4 ms -> 41.5 ms +- 0.6 ms: 1.05x faster
- deepcopy_reduce: 2.60 us +- 0.03 us -> 2.48 us +- 0.03 us: 1.05x faster
- telco: 6.60 ms +- 0.10 ms -> 6.30 ms +- 0.11 ms: 1.05x faster
- json_dumps: 10.1 ms +- 0.2 ms -> 9.60 ms +- 0.10 ms: 1.05x faster
- logging_simple: 5.25 us +- 0.07 us -> 5.02 us +- 0.06 us: 1.05x faster
- nqueens: 74.4 ms +- 0.7 ms -> 71.1 ms +- 0.8 ms: 1.05x faster
- genshi_xml: 46.1 ms +- 0.6 ms -> 44.1 ms +- 0.4 ms: 1.05x faster
- meteor_contest: 89.6 ms +- 0.4 ms -> 85.8 ms +- 0.4 ms: 1.04x faster
- spectral_norm: 93.0 ms +- 0.8 ms -> 89.1 ms +- 0.7 ms: 1.04x faster
- xml_etree_process: 54.6 ms +- 0.4 ms -> 52.3 ms +- 0.4 ms: 1.04x faster
- mdp: 2.33 sec +- 0.02 sec -> 2.23 sec +- 0.03 sec: 1.04x faster
- dulwich_log: 60.0 ms +- 0.3 ms -> 57.6 ms +- 0.4 ms: 1.04x faster
- pprint_pformat: 1.38 sec +- 0.01 sec -> 1.33 sec +- 0.01 sec: 1.04x faster
- scimark_lu: 112 ms +- 1 ms -> 107 ms +- 1 ms: 1.04x faster
- coroutines: 20.4 ms +- 0.3 ms -> 19.6 ms +- 0.4 ms: 1.04x faster
- scimark_sparse_mat_mult: 4.21 ms +- 0.11 ms -> 4.05 ms +- 0.12 ms: 1.04x faster
- tomli_loads: 1.93 sec +- 0.02 sec -> 1.85 sec +- 0.02 sec: 1.04x faster
- scimark_sor: 114 ms +- 1 ms -> 110 ms +- 1 ms: 1.04x faster
- xml_etree_generate: 78.3 ms +- 0.5 ms -> 75.7 ms +- 0.6 ms: 1.04x faster
- pprint_safe_repr: 667 ms +- 6 ms -> 646 ms +- 7 ms: 1.03x faster
- 2to3: 230 ms +- 1 ms -> 223 ms +- 1 ms: 1.03x faster
- regex_v8: 22.1 ms +- 0.2 ms -> 21.4 ms +- 0.1 ms: 1.03x faster
- richards_super: 49.7 ms +- 0.4 ms -> 48.3 ms +- 0.5 ms: 1.03x faster
- mako: 10.2 ms +- 0.1 ms -> 9.92 ms +- 0.07 ms: 1.03x faster
- deepcopy_memo: 29.6 us +- 0.2 us -> 28.8 us +- 0.3 us: 1.03x faster
- bench_thread_pool: 936 us +- 39 us -> 913 us +- 38 us: 1.02x faster
- generators: 25.3 ms +- 0.2 ms -> 24.7 ms +- 0.2 ms: 1.02x faster
- raytrace: 256 ms +- 3 ms -> 250 ms +- 2 ms: 1.02x faster
- asyncio_websockets: 508 ms +- 24 ms -> 496 ms +- 2 ms: 1.02x faster
- logging_format: 5.82 us +- 0.13 us -> 5.69 us +- 0.11 us: 1.02x faster
- pidigits: 160 ms +- 1 ms -> 158 ms +- 1 ms: 1.01x faster
- docutils: 2.22 sec +- 0.01 sec -> 2.19 sec +- 0.02 sec: 1.01x faster
- pickle: 11.1 us +- 0.4 us -> 10.9 us +- 0.1 us: 1.01x faster
- typing_runtime_protocols: 153 us +- 3 us -> 151 us +- 6 us: 1.01x faster
- pathlib: 19.2 ms +- 0.1 ms -> 19.0 ms +- 0.1 ms: 1.01x faster
- pickle_list: 3.84 us +- 0.07 us -> 3.82 us +- 0.06 us: 1.00x faster
- asyncio_tcp: 429 ms +- 4 ms -> 428 ms +- 3 ms: 1.00x faster
- asyncio_tcp_ssl: 1.48 sec +- 0.01 sec -> 1.47 sec +- 0.01 sec: 1.00x faster

Benchmark hidden because not significant (5): crypto_pyaes, fannkuch, html5lib, pickle_dict, unpickle

Geometric mean: 1.01x faster
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

buildThe build process and cross-buildperformancePerformance or resource usagetype-featureA feature request or enhancement

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    Back | FazBrowse Home | New Git URL