@Taeknology Can you share a benchmark script (pyperf based, with the most relevant benchmark(s)) for the PR together with your results?

Sure. I reran this with a pyperf script against the exact PR base
(ef0affbf7c1) and current head (b8711bf36cf).

The three cases keep the input at 1 MiB while varying the number of lines, so
they cover an append-heavy case, a 64-byte medium-line case, and a long-line
control. I built both revisions with Clang 21, PGO, and ThinLTO, for regular-GIL
and free-threaded (--disable-gil) configurations. Measurements were pinned to
one logical CPU. The aggregate below combines base→patched and patched→base
runs.

Regular GIL

Free-threaded

The individual free-threaded forward/reverse runs had geometric means of 1.14x
and 1.15x faster. The regular-GIL runs were both 1.01x faster overall; only the
64-byte case was significant in both orders.

"""pyperf benchmarks for the result-list construction in BytesIO.readlines()."""

from __future__ import annotations

import io

import pyperf


# Keep each input close to 1 MiB while varying the number of result-list
# appends.  The long-line case is a useful control: scanning and copying bytes
# dominates there, so a list-append optimization should have less impact.
CASES = {
    "bytesio_readlines_131072x8": (131_072, 8),
    "bytesio_readlines_16384x64": (16_384, 64),
    "bytesio_readlines_1024x1024": (1_024, 1_024),
}


def make_data(line_count: int, line_size: int) -> bytes:
    line = (b"x" * (line_size - 1)) + b"\n"
    data = line * line_count
    assert len(data) == line_count * line_size
    return data


def main() -> None:
    runner = pyperf.Runner()
    runner.metadata["description"] = (
        "BytesIO.readlines() over equally sized inputs with different line counts"
    )

    for name, (line_count, line_size) in CASES.items():
        stream = io.BytesIO(make_data(line_count, line_size))
        runner.timeit(
            name,
            stmt="stream.seek(0); stream.readlines()",
            globals={"stream": stream},
            metadata={
                "line_count": line_count,
                "line_size": line_size,
                "input_size": line_count * line_size,
            },
        )


if __name__ == "__main__":
    main()

Example command:

python bench_bytesio_readlines.py \
    --affinity=2 --processes=25 --values=5 --warmups=2 \
    -o result.json
python -m pyperf compare_to base.json patched.json \
    --table --table-format md

pyperf check still reports “not enough samples” warnings for several cases.
Separately, pyperf system show reported that this WSL2 host had no isolated
CPUs. The free-threaded result reproduced in both orders; the regular-GIL result
should be treated as a small improvement.

These reruns are lower than the earlier author-reported figures, so I have not
changed the PR description yet. If this benchmark shape looks appropriate, I’ll
replace the Benchmark section with these reproducible results.

Sanitized pyperf JSON files used for the tables above (host-identifying and unrelated runtime-state metadata removed; timing samples, warmups, and retained reproducibility metadata unchanged):

pr-155171-pyperf-results-sanitized.zip
SHA-256: c36df8a6b1fb2dc859b6e063ad992c779d3a7de880165068adc080c7eb25a7e7

Thanks for the reviews! Since it's been a little while, I'm just checking in: is there anything else needed from my side before this can be merged?

@Taeknology Thanks for the PR, if no other comments come in I will merge in a few days.