Apache DataFusion SQL Query Engine

the portable Python dataframe library

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

LakeSoul is an end-to-end, realtime cloud-native Lakehouse framework for fast data ingestion, concurrent updates, incremental analytics, multimodal data processing and vector search — powering next-generation BI and AI workloads.

Apache Kafka® compatible broker with S3, PostgreSQL, SQLite, Apache Iceberg and Delta Lake

Apache Auron(Incubating) is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

Apache DataFusion Comet Spark Accelerator

Lakehouse native graph engine with git-style workflows

ClickBench: a Benchmark For Analytical Databases

Learn vector search with Rust and DataFusion; the C++/BusTub track is deprecated.

DuckDB-powered data lake analytics from Postgres

Analytical database for data-driven Web applications 🪶

Pushdown cache for DataFusion

Next-generation decentralized data lakehouse and a multi-party stream processing network

Unofficial rust implementation of Apache Iceberg with integration for Datafusion

Batteries included CLI, TUI, and server implementations for DataFusion.

Blazing-Fast Bioinformatic Operations on Python DataFrames

A Rust-native DuckLake engine built on Apache DataFusion