| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
Speed up `in`, `.count()`, `.index()` and `array.array.remove()` for `array.array` and `memoryview` objects with integer formats by comparing the bytes of the items instead of creating an object for each item. The search is implemented using the buffer protocol to get a contiguous buffer, with a fallback to iteration and rich comparison. The buffer is held for the whole search, so a comparison which resizes the array or releases the memoryview now raises BufferError rather than allowing the resize while iterating. Add a `sq_contains` slot to memoryview so `in` uses the faster search when possible. The buffer protocol adds 7 ns of overhead per call on my M3 MacBook relative to passing native pointers. The overhead makes searches which stop at the first item or have fewer than two items slower. Longer cases get substantially faster; full results are below. `in`, `memoryview.index()` and `.count()` with the 'w' format (Py_UCS4) now work, because searching shares array's support for 'w'; they used to raise NotImplementedError. Other memoryview operations still don't support 'w'. `memoryview.index()` with an unsupported format now raises NotImplementedError even when there are no items to search, as `.count()` and `in` already did; it used to raise ValueError. pyperf, release build (-O3) without PGO or LTO, macOS arm64: | Benchmark | main | patch | |---------------------------------|:-------:|:-----------------------:| | array i[8] in (first) | 9.41 ns | 16.6 ns: 1.76x slower | | array i[8] in (last) | 57.3 ns | 19.0 ns: 3.01x faster | | array i[8] index (first) | 13.9 ns | 21.5 ns: 1.55x slower | | array i[8] count | 69.9 ns | 20.7 ns: 3.37x faster | | array d[8] index (first) | 20.4 ns | 26.3 ns: 1.29x slower | | array d[8] count | 94.4 ns | 96.8 ns: 1.03x slower | | memoryview i[8] index (first) | 14.4 ns | 22.6 ns: 1.57x slower | | memoryview i[8] count | 88.1 ns | 22.4 ns: 3.94x faster | | memoryview d[8] count | 125 ns | 96.7 ns: 1.29x faster | | array i[1k] count | 10.5 us | 136 ns: 77.22x faster | | memoryview i[1k:2] count | 10.0 us | 8.38 us: 1.19x faster | | memoryview d[1k] count | 10.8 us | 10.1 us: 1.07x faster | | array B[100k] in (miss) | 616 us | 1.77 us: 348.22x faster | | array i[100k] index (last) | 1.16 ms | 9.38 us: 123.36x faster | | array q[100k] count | 1.30 ms | 7.46 us: 173.79x faster | | memoryview B[100k] count | 805 us | 13.0 us: 61.90x faster | | memoryview i[100k] index (last) | 1.21 ms | 9.26 us: 131.19x faster | | Geometric mean | (ref) | 5.28x faster | Benchmark hidden because not significant (2): array d[1k] count, array w[1k] count Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| Back | FazBrowse Home | New Git URL |
Speed up in, .count(), .index() and array.array.remove() for
array.array and memoryview objects with integer formats by comparing
the bytes of the items instead of creating an object for each item.
The search is implemented using the buffer protocol to get a contiguous
buffer, with a fallback to iteration and rich comparison. The buffer is
held for the whole search, so a comparison which resizes the array or
releases the memoryview now raises BufferError rather than allowing the
resize while iterating.
Add a sq_contains slot to memoryview so in uses the faster search
when possible.
The buffer protocol adds 7 ns of overhead per call on my M3 MacBook
relative to passing native pointers. The overhead makes searches which
stop at the first item or have fewer than two items slower. Longer cases
get substantially faster; full results are below.
in, memoryview.index() and .count() with the 'w' format (Py_UCS4)
now work, because searching shares array's support for 'w'; they used
to raise NotImplementedError. Other memoryview operations still don't
support 'w'.
memoryview.index() with an unsupported format now raises
NotImplementedError even when there are no items to search, as
.count() and in already did; it used to raise ValueError.
pyperf, release build (-O3) without PGO or LTO, macOS arm64:
Benchmark hidden because not significant (2): array d[1k] count, array w[1k] count
Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com