FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

[AURON #1264] Add BatchExecCalc shadow for Flink BATCH mode Calc by argoyal2212 · Pull Request #2516 · apache/auron · GitHub

/ auron Public

[AURON #1264] Add BatchExecCalc shadow for Flink BATCH mode Calc - #2516

Draft
argoyal2212 wants to merge 1 commit into
apache:masterfrom
argoyal2212:argoyal/flink-batch-calc-shadow
Draft

argoyal2212 wants to merge 1 commit into
apache:masterfrom
argoyal2212:argoyal/flink-batch-calc-shadow

Conversation

argoyal2212 commented Sep 8, 2026
edited
Loading

Copy link
Copy Markdown

Which issue does this PR close?

TBD

Rationale for this change

Flink's BATCH mode currently has no native execution: DefaultPlannerFactory routes it to the stock, unmodified BatchPlanner. NativePlanFusionBuilder and FlinkAuronCalcOperator are already engine-agnostic (no streaming-specific types), so the smallest correct first step is a BatchExecCalc shadow reusing both, mirroring the existing StreamExecCalc shadow.

What changes are included in this PR?

  • New org.apache.flink.table.planner.plan.nodes.exec.batch.BatchExecCalc in auron-flink-planner, matching Flink 1.18.1's stock constructor signature exactly.
    • Native conversion via NativePlanFusionBuilder.buildNativeCalcPlan, falling back to Flink's codegen Calc (or throwing if fallback is disabled), same as streaming.
    • No graph-level source fusion yet (no batch counterpart to AuronOperatorFusionProcessor); supportFusionCodegen() returns false.
    • No @ExecNodeMetadata/@JsonCreator, matching stock batch BatchExecCalc at this Flink version (compiled-plan restoration is streaming-only here).
  • Registered in shadowed-flink-execnodes.txt; added matching shade <exclude> in auron-flink-assembly/pom.xml.

No changes to any streaming file or to DefaultPlannerFactory.

Are there any user-facing changes?

Yes: BATCH queries now get native execution for Calc (projection/filter) when auron-flink-assembly is ahead of stock flink-table-planner on the classpath. Falls back to Flink's codegen Calc when not convertible, matching existing streaming behavior.

How was this patch tested?

  • ShadowedExecNodeRegistryTest and AssemblyJarStructureIT pass with the new class registered and shaded correctly.
  • Verified constructor signature and lack of @ExecNodeMetadata/@JsonCreator against the real flink-table-planner_2.12-1.18.1.jar.
  • Manually verified end-to-end on a local Flink 1.18.1 cluster: a BATCH SQL query's Calc step executes via Auron's native engine (native ProjectExec/FFIReader in task logs).

Draft while a dedicated BatchExecCalcTest (mirroring StreamExecCalcTest) is added.

Was this patch authored or co-authored using generative AI tooling?

  • Yes
  • No

ASF guidance: https://www.apache.org/legal/generative-tooling.html

argoyal2212 force-pushed the argoyal/flink-batch-calc-shadow branch from 6f1d8de to 4efc256 Compare September 8, 2026 20:57
Mirrors the existing StreamExecCalc shadow so native execution also covers BATCH mode's Calc operator, reusing NativePlanFusionBuilder and FlinkAuronCalcOperator unchanged. Fusion codegen and source fusion are out of scope for this first step.

Relates to apache#1264
argoyal2212 force-pushed the argoyal/flink-batch-calc-shadow branch from 4efc256 to ee603ca Compare September 8, 2026 20:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL