FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Fix shredded variant struct cast validity by mikamikasuki · Pull Request #11368 · apache/arrow-rs · GitHub

Fix shredded variant struct cast validity - #11368

Open
mikamikasuki wants to merge 1 commit into
apache:mainfrom
mikamikasuki:codex/issue-11301-shredded-struct
Open

mikamikasuki wants to merge 1 commit into
apache:mainfrom
mikamikasuki:codex/issue-11301-shredded-struct

Conversation

Copy link
Copy Markdown

Which issue does this PR close?

Closes #11301.

Rationale for this change

Casting a shredded Variant null or non-object scalar to an Arrow struct can produce a present struct with null fields, while the equivalent unshredded value produces a null struct in safe mode or a cast error in strict mode. This makes observable results depend on the physical shredding representation.

The struct conversion path extracts child fields when typed_value is available, but previously used only the outer Variant validity for the result's parent validity. It therefore skipped checking whether each value was an object.

What changes are included in this PR?

Preserve recursive field extraction while deriving struct parent validity from the actual shredded value. Non-object scalars and Variant nulls now become null structs in safe mode and raise the standard cast error in strict mode. Missing object-field sentinels remain valid structs with null children.

Are these changes tested?

Regression coverage exercises scalar and null values with primitive and struct shredding schemas, preserves empty and missing-field object behavior, and checks strict-mode errors.

Are there any significant user-facing changes?

This fixes observable null and cast-error behavior for conversions from shredded Variant values to Arrow structs; no API changes are introduced.

OpenAI Codex assisted with investigating the bug and generating the implementation and regression tests; I reviewed the changes and verified them.

github-actions Bot added the parquet-variant parquet-variant* crates label Oct 4, 2026

This branch has not been deployed

No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet-variant parquet-variant* crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Variant] Casting shredded input to structs bypasses parent null and type checks

1 participant


Back | FazBrowse Home | New Git URL