Commit c6f63c3
docs(perf): record measured CLI and GixPython repository journeys
Document the new `pyperf` workload and local measurements independently of
full-suite setup/coverage timings. Record all eleven means and standard
deviations, fixture/source revisions and environment details: direct opening
and nested discovery favor GixPython, but the already-open journey is about
13% slower because history metadata and commit statistics offset other gains.
Explain retained `Repo` ownership during operation measurements, separate
open/discovery lifecycle costs, fresh high-level wrappers, parity checks and
the extensible CI artifact workflow. Native handle reuse remains future work;
these measurements expose its potential benefit without implementing it.
Both complete calibrated runs finished on existing CPython 3.12.14 with
GixPython first and CLI second. All eleven result digests match; the comparison
and `pyperf` significance table completed successfully. Record the stability
warnings rather than treating one machine's warm-cache values as universal.
The initial dedicated CI benchmark job also passed. `git diff --check` passed.1 parent 244e418 commit c6f63c3
1 file changed
Lines changed: 48 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -115,6 +115,54 @@ an import error; only an absent top-level `gix` selects CLI mode. | |||
| 115 | 115 | ||
| 116 | 116 | ## Performance work | |
| 117 | 117 | ||
| 118 | + ### Existing-repository benchmark | ||
| 119 | + | ||
| 120 | + [`test/performance/README.md`](../test/performance/README.md) describes the | ||
| 121 | + `pyperf` harness and fixed GitPython 3.1.45 fixture. It measures eight public-API | ||
| 122 | + operations and their complete journey on one already-open `git.Repo`, plus | ||
| 123 | + separate direct opening and discovery from `git/objects`. Fresh high-level | ||
| 124 | + wrappers preserve the cost of actual operations; imports, fixture preparation | ||
| 125 | + and parity preflight are outside timing. Native repository reopening remains | ||
| 126 | + inside operation timing, so retaining native state in the future can improve | ||
| 127 | + these measurements without changing the workload. | ||
| 128 | + | ||
| 129 | + At `244e418da6cc43de129cbe2908d11be4c5ad457a`, using official GixPython | ||
| 130 | + 0.1.0 and the same existing CPython 3.12.14/macOS arm64 interpreter for both | ||
| 131 | + installations, with Git 2.54.0 (Apple Git-157), six worker processes with three values each produced the | ||
| 132 | + following means and standard deviations. GixPython ran first, then CLI; | ||
| 133 | + all eleven result digests matched. The fixture has one branch, one untracked | ||
| 134 | + file and an ignored directory, with ambient Git configuration disabled. | ||
| 135 | + | ||
| 136 | + | Measurement | CLI (ms) | GixPython (ms) | CLI / Gix | | ||
| 137 | + | --- | ---: | ---: | ---: | | ||
| 138 | + | Already-open journey | 230.90 ± 6.87 | 260.14 ± 13.37 | 0.89× | | ||
| 139 | + | Reference inventory | 22.77 ± 1.15 | 1.17 ± 0.26 | 19.52× | | ||
| 140 | + | 25 first-parent commits with metadata | 9.24 ± 0.46 | 73.61 ± 7.24 | 0.13× | | ||
| 141 | + | Root tree and README blob | 16.02 ± 1.07 | 2.40 ± 0.46 | 6.66× | | ||
| 142 | + | Index entries | 8.43 ± 0.57 | 2.00 ± 0.38 | 4.21× | | ||
| 143 | + | Commit count and ancestry queries | 82.28 ± 2.27 | 36.19 ± 0.84 | 2.27× | | ||
| 144 | + | Latest diff and commit statistics | 29.98 ± 1.66 | 60.98 ± 2.45 | 0.49× | | ||
| 145 | + | Patch diff (includes CLI fallback) | 17.87 ± 1.28 | 10.86 ± 1.01 | 1.64× | | ||
| 146 | + | Dirty/untracked/ignored paths | 68.56 ± 3.18 | 64.74 ± 2.00 | 1.06× | | ||
| 147 | + | Direct opening and close | 95.08 ± 4.64 | 35.84 ± 2.02 | 2.65× | | ||
| 148 | + | Nested discovery and close | 112.58 ± 5.38 | 49.20 ± 4.16 | 2.29× | | ||
| 149 | + | ||
| 150 | + Ratios above one favor GixPython. This workload's complete journey is about | ||
| 151 | + 13% slower with GixPython: metadata reads and commit statistics offset gains | ||
| 152 | + elsewhere. These are warm-cache measurements on one machine, with `pyperf` | ||
| 153 | + stability warnings for several samples; they do not establish a general | ||
| 154 | + speedup. Full-suite times below also include setup and coverage. | ||
| 155 | + | ||
| 156 | + The separate `Backend benchmark` CI job runs both installations on one runner, | ||
| 157 | + publishes every measurement and `pyperf` significance reporting, and retains | ||
| 158 | + raw JSON and comparison artifacts. Results include revisions and per-operation | ||
| 159 | + native/fallback decisions. Adding a `MEASUREMENTS` entry extends the journey, | ||
| 160 | + individual timings and parity checks together. Opening/discovery remain | ||
| 161 | + separate lifecycle measurements. CI fails on execution/parity errors; timing | ||
| 162 | + ratios are observational on shared runners. | ||
| 163 | + | ||
| 164 | + ### Test-suite setup measurements | ||
| 165 | + | ||
| 118 | 166 | Native repository reuse remains future work. The fixture optimizations below | |
| 119 | 167 | are implemented as separate commits, each validated with GixPython before CLI. | |
| 120 | 168 | The original local measurements use official GixPython 0.1.0 and existing | |
0 commit comments