Use `uv venv` and `uv pip` by maurycy · Pull Request #425 · python/pyperformance · GitHub

This is a conversation starter now: should we gradually start moving towards uv that handles custom builds, venvs and pinned runs more efficiently?

The code already works, confirmed with:

python dev.py run

Tests are 2x faster:

last main run took 13m57s
3a402551d306ba7b4e80c1b308c426f5f2a08e30, with enable-cache in the workflow, took 7m14s
4b5f47a1292e94802af93873709b688cc0c43de1, without caching in the action, took 8m20s

I didn't want to do too many changes at once, but in my opinion the best direction would be iterating over each benchmark with uv. pyperformance is just a benchmark suite after all, and the whole venv, custom build etc. management makes it unnecessarily complex.

Notably, uv run has --project, --isolated (now it's isolated if the requirements cannot be met), --with-requirements and --python (supporting custom builds), eg:

8:41:50.613123804PM CEST maurycy@gunnbjorn ~/pyperformance (uv) % uv run --isolated --project=benchmarks/bm_sympy --python=/home/maurycy/cpython/python benchmarks/bm_sympy/run_benchmark.py
Installed 4 packages in 12ms
.....................
sympy_expand: Mean +- std dev: 281 ms +- 1 ms
.....................
sympy_integrate: Mean +- std dev: 12.8 ms +- 0.1 ms
.....................
sympy_sum: Mean +- std dev: 92.8 ms +- 0.8 ms
.....................
sympy_str: Mean +- std dev: 161 ms +- 1 ms

This is similar to what happens now, just merging temporary outputs.

From what I measured venv management and dependencies only take ~5% of the run time, so no dramatic speed up should be expected, but the simplification might be worth it. Important to keep in mind that many features are never ever used, if a large chunk of benchmarks stays broken for so long and some obvious issues lurked for a while.

Another elephant in the room is compile that could also be outsourced.

I'm more than open to any suggestions!

This reverts commit 4b5f47a.

Since pyperformance users include not only CPython but also alternative implementations, I don’t think relying on uv would be a good approach for us to take. cc @hugovk

Since pyperformance users include not only CPython but also alternative implementations, I don’t think relying on uv would be a good approach for us to take. cc @hugovk

uv supports CPython, PyPy, GraalPy and Pyodidie

--python accepts binaries, eg custom PyPy3 build:

10:36:09.653252000AM CET maurycy@gimel /Users/maurycy/src/pyperformance % uv run --isolated --project=benchmarks/bm_sympy --python=`which pypy3` benchmarks/bm_sympy/run_benchmark.py
      Built psutil==7.1.2
Installed 4 packages in 68ms
.......
sympy_expand: Mean +- std dev: 73.9 ms +- 6.0 ms
.......
sympy_integrate: Mean +- std dev: 22.5 ms +- 3.8 ms
.......
sympy_sum: Mean +- std dev: 46.7 ms +- 6.6 ms
.......
sympy_str: Mean +- std dev: 51.1 ms +- 11.6 ms

If I understand correctly, the only requirement is that -I, -B and -C needs to be supported by an interpreter. I might be totaly wrong here, though. uv seems to check sys.implementation.name and match it

ref astral-sh/uv#2096

No there are a lot of implementations over you listed. e.g RustPython Jython gpython. Let s not make a big huddle to them.

Unless you have a strong motivation to maintain this project based on uv, let's keep it as is based on standard(pip).
I love uv but this should consider other factors.

No there are a lot of implementations over you listed. e.g RustPython Jython gpython. Let s not make a big huddle to them.

That doesn't matter. Not to diss the other implementations, but they most likely can't even run pip. If you can't even run pip, there's no point supporting these.

RustPython can't run pip Tracking pip support in RustPython RustPython/RustPython#5332
Jython only supports Python 2.7, which clearly can't use pyperformance.
GPython says gpython does not include many python modules as many of the core modules are written in C not python. So that's also not gonna work.

There's a reason to not switch to uv by default though --- that would be a breaking change of pyperformance likely requiring a major version bump.

@maurycy I think this is really great work and I would be willing to accept this PR if the default stayed pip, but you allow an option/command line flag to switch to uv. Then I think @corona10's concerns should be solved.

FWIW, starting a pip subprocess is extremely slow on other implementations like GraalPy. Even on CPython it is slow. Running a benchmarking run on GraalPy might be faster if we switch to uv, so I'm very much looking forward to this change.

@maurycy I think this is really great work and I would be willing to accept this PR if the default stayed pip, but you allow an option/command line flag to switch to uv. Then I think @corona10's concerns should be solved.

It will be fine with switching to uv as an optional feature as Ken Jin commented.

RustPython can't run pip RustPython/RustPython#5332

For clarity, they can run old pip but fail with the latest pip; anyway, they have a chance to run pyperformance using old pip.
RustPython/RustPython#5332 (comment)
so switching to uv by default will be a big hurdle for new challengers :)

* main: Bump mypy from 1.19 to 1.19.1 (python#448) Add a YAML-parsing benchmark (python#342) Add FastAPI HTTP request benchmark (python#440) Add option for rigorous in benchmark.conf.sample (python#446) Do not `venv.ensure_reqs` twice (python#432) Bump mypy from 1.18.2 to 1.19.0 (python#442) Bump actions/checkout from 5 to 6 in the actions group (python#441) Add example scripts to run pyperformance on a generic host (python#436) Prepare for 1.13.0 release (python#435) Reenable xdsl (python#433) Bump actions/setup-python from 5 to 6 (python#429) Group Dependabot updates for GitHub Actions into single PR (python#431) Bump actions/checkout from 4 to 5 (python#430)

* main: Replace pre-commit with prek in CI and add cooldown to Dependabot (python#452) Add bm_base64 covering common base64 module APIs (python#447)

taking a stab...

577a8d3

maurycy marked this pull request as draft October 24, 2025 23:41

maurycy changed the title Use uv venv instead of python venv Use uv venv instead of python -m venv Oct 25, 2025

maurycy added 13 commits October 25, 2025 17:26

run_uv

3c6f1da

lint

1414f87

unbreak ga?

76ad094

seed venv

bb1101f

ensure once

2b7cfa3

use uv pip

c776958

gh action caching

40031e2

system in ga should be ok

976e978

--python

5f1af63

psutil is the source of all evil

164f0c8

use run_uv everywhere

87e58b7

import less

05b1da4

separate module, uv pip

d458ec1

maurycy changed the title Use uv venv instead of python -m venv Use uv venv and uv pip Oct 25, 2025

maurycy added 8 commits October 25, 2025 23:23

copy from _pip

a3ef49c

reduce the diff

3a40255

how fast are we without caching?

4b5f47a

Revert "how fast are we without caching?"

60b9812

This reverts commit 4b5f47a.

smaller diff

2ddd03c

reduce diff even more!

4dfad08

diff...

4d1f1e3

simplify the requirements

ba9fc09

corona10 reviewed Oct 26, 2025

View reviewed changes

Merge branch 'main' into uv

76a6e2e

maurycy added 2 commits January 2, 2026 22:45

Merge branch 'main' into uv

76a2021

* main: Replace pre-commit with prek in CI and add cooldown to Dependabot (python#452) Add bm_base64 covering common base64 module APIs (python#447)

Use `uv venv` and `uv pip` by maurycy · Pull Request #425 · python/pyperformance · GitHub

Repository navigation

Use uv venv and uv pip - #425

Use uv venv and uv pip#425
maurycy wants to merge 25 commits into
python:mainfrom
maurycy:uv

maurycy commented Oct 24, 2025 •
edited
Loading

Uh oh!

Uh oh!

corona10 left a comment

Uh oh!

maurycy commented Oct 26, 2025 •
edited
Loading

Uh oh!

Uh oh!

corona10 commented Oct 26, 2025 •
edited
Loading

Uh oh!

Uh oh!

corona10 commented Oct 26, 2025 •
edited
Loading

Uh oh!

Uh oh!

Fidget-Spinner commented Dec 5, 2025

Uh oh!

Fidget-Spinner commented Dec 5, 2025

Uh oh!

corona10 commented Dec 5, 2025 •
edited
Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Use `uv venv` and `uv pip` by maurycy · Pull Request #425 · python/pyperformance · GitHub

Repository navigation

Use uv venv and uv pip - #425

Use uv venv and uv pip#425maurycy wants to merge 25 commits intopython:mainpython/pyperformance:mainfrom maurycy:uvmaurycy/pyperformance:uvCopy head branch name to clipboard

Conversation

maurycy commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corona10 left a comment

Choose a reason for hiding this comment

Uh oh!

maurycy commented Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corona10 commented Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corona10 commented Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Fidget-Spinner commented Dec 5, 2025

Uh oh!

Fidget-Spinner commented Dec 5, 2025

Uh oh!

corona10 commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Use uv venv and uv pip#425
maurycy wants to merge 25 commits into
python:mainfrom
maurycy:uv

maurycy commented Oct 24, 2025 •
edited
Loading

maurycy commented Oct 26, 2025 •
edited
Loading

corona10 commented Oct 26, 2025 •
edited
Loading

corona10 commented Oct 26, 2025 •
edited
Loading

corona10 commented Dec 5, 2025 •
edited
Loading