| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Add an int rowCount parameter to ScalarFunction.evaluate. JniBridge already receives the value from the native side as expectedRowCount for post-call validation; now it is also forwarded into evaluate. For UDFs with at least one argument the value matches what the body could read from args.get(0).getValueCount(). For nullary UDFs (args is empty), this is the only channel that communicates the batch row count, making it possible to implement Volatility.VOLATILE nullary functions like random() / now().
|
Superseded by upstream #64; nullary row count is now available via ScalarFunctionArgs.rowCount(). |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Which issue does this PR close?
Rationale for this change
ScalarFunction.evaluate(BufferAllocator, List<FieldVector>) (introduced in #46) is the contract every Java-implemented scalar UDF must satisfy. It must return a FieldVector whose getValueCount() matches the batch row count DataFusion is driving through the operator tree.
For UDFs with at least one argument, the body can read args.get(0).getValueCount() to learn how many rows to produce. For nullary UDFs -- zero arguments, e.g. analogs of random(), pi(), now() -- args is the empty list, and the body has no other channel to learn the row count.
The native side already knows the value: ScalarFunctionArgs::number_rows is read at native/src/udf.rs:100, used to materialise scalar arg columns at :106. The Java bridge (JniBridge.invokeScalarUdf) receives it but only uses it after the fact, to validate the returned vector's length. It is never communicated to impl.evaluate(...).
The result: any nullary UDF that DataFusion does not constant-fold (anything declared Volatility.VOLATILE, or STABLE calls in plans the optimizer cannot fold) trips the post-hoc row-count validation as soon as it runs over a batch with more than one row.
What changes are included in this PR?
Are these changes tested?
yes
Are there any user-facing changes?
Yes, a source-breaking signature change to ScalarFunction.evaluate. Implementations of the interface need to add an int rowCount parameter to their evaluate override. Bodies that ignore it remain identical otherwise.
Before:
After: