Casting a shredded Variant null or non-object scalar to an Arrow struct can produce a present struct with null fields, while the equivalent unshredded value produces a null struct in safe mode or a cast error in strict mode. This makes observable results depend on the physical shredding representation.
The struct conversion path extracts child fields when typed_value is available, but previously used only the outer Variant validity for the result's parent validity. It therefore skipped checking whether each value was an object.
What changes are included in this PR?
Preserve recursive field extraction while deriving struct parent validity from the actual shredded value. Non-object scalars and Variant nulls now become null structs in safe mode and raise the standard cast error in strict mode. Missing object-field sentinels remain valid structs with null children.
Are these changes tested?
Regression coverage exercises scalar and null values with primitive and struct shredding schemas, preserves empty and missing-field object behavior, and checks strict-mode errors.
Are there any significant user-facing changes?
This fixes observable null and cast-error behavior for conversions from shredded Variant values to Arrow structs; no API changes are introduced.
OpenAI Codex assisted with investigating the bug and generating the implementation and regression tests; I reviewed the changes and verified them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #11301.
Rationale for this change
Casting a shredded Variant null or non-object scalar to an Arrow struct can produce a present struct with null fields, while the equivalent unshredded value produces a null struct in safe mode or a cast error in strict mode. This makes observable results depend on the physical shredding representation.
The struct conversion path extracts child fields when typed_value is available, but previously used only the outer Variant validity for the result's parent validity. It therefore skipped checking whether each value was an object.
What changes are included in this PR?
Preserve recursive field extraction while deriving struct parent validity from the actual shredded value. Non-object scalars and Variant nulls now become null structs in safe mode and raise the standard cast error in strict mode. Missing object-field sentinels remain valid structs with null children.
Are these changes tested?
Regression coverage exercises scalar and null values with primitive and struct shredding schemas, preserves empty and missing-field object behavior, and checks strict-mode errors.
Are there any significant user-facing changes?
This fixes observable null and cast-error behavior for conversions from shredded Variant values to Arrow structs; no API changes are introduced.
OpenAI Codex assisted with investigating the bug and generating the implementation and regression tests; I reviewed the changes and verified them.