Section Navigation

Acero - Streaming Execution Engine#

Warning

Acero is experimental and a stable API is not yet guaranteed.

Acero is a streaming query engine, which allows the computation to be expressed as an execution plan (constructed using the Declaration interface). This enables to create a computation composed of pyarrow.compute functions and to execute this efficiently in a batched manner.

`Declaration`(factory_name,...[,inputs])	Helper class for declaring the nodes of an ExecPlan.
`ExecNodeOptions`	Base class for the node options.
`TableSourceNodeOptions`(table)	A Source node which accepts a table.
`ScanNodeOptions`(dataset,**kwargs)	A Source node which yields batches from a Dataset scan.
`FilterNodeOptions`(filter_expression)	Make a node which excludes some rows from batches passed through it.
`ProjectNodeOptions`(expressions[,names])	Make a node which executes expressions on input batches, producing batches of the same length with new columns.
`AggregateNodeOptions`(aggregates[,keys])	Make a node which aggregates input batches, optionally grouped by keys.
`OrderByNodeOptions`([sort_keys,null_placement])	Make a node which applies a new ordering to the data.
`HashJoinNodeOptions`(join_type,left_keys,...)	Make a node which implements join operation using hash join strategy.