| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
Eliminates an O(N^2) bottleneck when consuming query results: - Previously, StreamedResultSet yielded rows by calling `iter_rows.pop(0)` in a loop. Because Python lists are contiguous arrays, popping index 0 shifts all remaining elements in memory on every row yielded, causing quadratic overhead for larger result sets. - Replaces `pop(0)` with direct iteration (`for row in iter_rows: yield row`) and detaches the list directly (`iter_rows, self._rows = self._rows, []`), allowing row iteration to run in linear O(N) time at C speed.
There was a problem hiding this comment.
This pull request optimizes row streaming iteration in both the synchronous and asynchronous implementations of StreamedResultSet by replacing an inefficient O(N^2) list-popping loop with a standard O(N) for-loop. It also introduces comprehensive unit tests covering large batches, stepwise consumption, chunk boundaries, early termination, and mid-stream errors. As there are no review comments, I have no feedback to provide.
Sorry, something went wrong.
|
(The system test failures are unrelated) |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Eliminates an O(N^2) bottleneck when consuming query results:
Benchmark Performance Verification
Performance verification of this PR branch (spanner-row-popping) compared against the 7-day nightly baseline of the Spanner Python client running on GCE (n2-standard-2, 2 vCPUs, closed-loop single worker):
Results Summary
read-narrow-result-set
(200,000 rows, narrow columns)
P50
P90
P99
374.98 ms
474.97 ms
497.50 ms
730.31 ms
2,866.85 ms
2,866.85 ms
-48.65%
-83.43%
-82.65%
~1.9x faster
~6.0x faster
~5.8x faster
read-large-result-set
(100,000 rows, wide columns)
P50
P90
P99
2,147.62 ms
2,336.65 ms
2,482.95 ms
2,603.18 ms
2,725.26 ms
2,565.57 ms
-17.50%
-14.26%
-3.22%
~1.21x faster
~1.17x faster
~1.03x faster
Observations