GH-51268: [C++][Parquet] Unpack equal-width DELTA_BINARY_PACKED miniblocks in one call - #51250
Draft
prtkgaur wants to merge 1 commit into
Draft
GH-51268: [C++][Parquet] Unpack equal-width DELTA_BINARY_PACKED miniblocks in one call#51250prtkgaur wants to merge 1 commit into
prtkgaur wants to merge 1 commit into
Conversation
Copy link
Copy Markdown
Sorry, something went wrong.
Copy link
Copy Markdown
|
Thanks for opening a pull request! This pull request has been automatically converted to a draft because its title doesn't match Arrow's required format. If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project. Then could you also rename the pull request title in the following format? GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or MINOR: [${COMPONENT}] ${SUMMARY}
After updating the title, you can mark the pull request as ready for review. See also: |
Sorry, something went wrong.
prtkgaur
force-pushed
the
delta-binary-packed-coalesce-miniblocks
branch
from
c16db5a to
fa243dd
Compare
September 9, 2026 03:07
Copy link
Copy Markdown
|
⚠️ GitHub issue #51268 has been automatically assigned in GitHub to PR creator. |
Sorry, something went wrong.
…ne call The miniblocks of a DELTA_BINARY_PACKED block are packed back to back with no padding between them, so a run of miniblocks that share a bit width is bit-identical to a single longer run at that width. GetInternal called the bit unpacker once per miniblock all the same, which with the default geometry is one call per 32 values - mostly per-call setup. Look ahead over the block's stored bit widths and extend the current call over each following miniblock that has the same width and that the caller has room for in full. A miniblock joins the run only when its width equals the current delta_bit_width_, which InitMiniBlock has already validated, and the run also stops at the end of the block. Add a test over the width patterns that decide where a run starts and stops, and read at a batch size that stops partway through a coalesced run. On the decode benchmarks already in the tree this is 1.17x to 1.33x on top of the previous commit wherever the unpacker's per-call cost is a meaningful share of the work. Decoded values are identical; no encoded byte changes.
prtkgaur
force-pushed
the
delta-binary-packed-coalesce-miniblocks
branch
from
fa243dd to
9174897
Compare
September 10, 2026 19:32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reviewers
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
The miniblocks of a DELTA_BINARY_PACKED block are packed back to back with no padding, so
consecutive miniblocks that share a bit width are bit-identical to one longer run at that
width. The decoder calls the bit unpacker once per miniblock all the same -- at the default
geometry one call per 32 values, which at narrow widths is mostly per-call setup.
What changes are included in this PR?
A look-ahead over the block's stored bit widths reports how many following miniblocks can be
folded into the current unpack call, so a run of four asks for 128 values instead of 32. A
miniblock joins the run only when its stored width equals the current one, which has already
been validated, so coalescing never depends on an unchecked width. The run also stops at the
end of the block and at what the caller has room for.
A zero bit width needs no unpack call, so the caller tests for it before the look-ahead.
That reads redundant, since the look-ahead declines anyway, but is not free to drop: without
it the two zero-width arms lose 17%. Both builds emit the same loops with the same
instruction counts, so this is how the compiler arranges the function rather than work saved
-- but it reproduces well outside build-to-build spread.
Are these changes tested?
A new typed test covers the width patterns that decide where a run starts and stops, and the
fixture gains a read batch size that stops partway through a coalesced run. Three mutations
-- ignoring the neighbour's width, ignoring the caller's room, and failing to advance the
block cursor -- each turn it red on both integer widths.
Benchmark
Graviton4, GCC 11.5, Release, one core, 9 repetitions, medians, 65,536 values. Both points
built twice with the builds interleaved; the two builds agree within 0.6%.
The Fixed arms are the zero bit width path: they gain nothing here and give up 2-3% for
the check that keeps them out, stable across builds rather than noise. The wide arms gain
least, spending their time inside the unpacker rather than around it.
Are there any user-facing changes?
No. No API change, no format change, and decoded values are identical.