Explore Topics Trending Collections Events GitHub Sponsors

stream-processing

Here are 1,459 public repositories matching this topic...

Language: All Sort: Most stars

pathwaycom / pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

python rust streaming real-time kafka etl machine-learning-algorithms stream-processing data-analytics dataflow data-processing data-pipelines batch-processing pathway iot-analytics etl-framework time-series-analysis

Updated Aug 19, 2026
Python

google-ai-edge / mediapipe

Star

Cross-platform, customizable ML solutions for live and streaming media.

android c-plus-plus calculator machine-learning framework computer-vision deep-learning inference pipeline-framework stream-processing video-processing perception mobile-development audio-processing graph-framework graph-based mediapipe

Updated Aug 19, 2026
C++

vectordotdev / vector

Star

A high-performance observability data pipeline.

agent events monitoring etl metrics high-performance telemetry logs data-transformation pipelines datadog stream-processing rust-lang cloud-native forwarder hacktoberfest observability traces

Updated Aug 20, 2026
Rust

flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

mysql redis elasticsearch streaming kafka spark influxdb rabbitmq clickhouse hbase stream-processing opentsdb loki flink rocketmq

Updated May 6, 2026
Java

oxnr / awesome-bigdata

Star

A curated list of awesome big data frameworks, ressources and other awesomeness.

data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data

Updated Jul 31, 2026

redpanda-data / redpanda

Star

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

go kubernetes streaming microservices kafka cpp seastar storage-engine containers realtime stream-processing event-driven redpanda

Updated Aug 19, 2026
C++

madd86 / awesome-system-design

Star

A curated list of awesome System Design (A.K.A. Distributed Systems) resources.

distributed-systems microservices nosql interview stream-processing microservices-architecture relational-database message-broker hadoop-ecosystem

Updated Feb 27, 2026

ThreeDotsLabs / watermill

Star

Building event-driven applications the easy way in Go.

go golang events reactive kafka cqrs sagas rabbitmq event-sourcing nats stream-processing event-driven watermill

Updated Aug 6, 2026
Go

risingwavelabs / risingwave

Star

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

rust database kafka webhook postgresql stream-processing data-engineering materialized-view etl-pipeline event-streaming apache-iceberg

Updated Aug 20, 2026
Rust

redpanda-data / connect

Star

Fancy stream processing made operationally mundane

go golang kafka cqrs etl rabbitmq amqp logs message-bus event-sourcing nats stream-processing message-queue data-engineering streaming-data stream-processor data-ops

Updated Aug 19, 2026
Go

fluent / fluent-bit

Star

Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows

c metrics logging logs prometheus data-collector stream-processing fluentd fluent-bit cloudnative sql-queries forwarder traces opentelemetry

Updated Aug 19, 2026
C

robinhood / faust

Star

Python Stream Processing

python distributed-systems kafka stream-processing asyncio kafka-streams

Updated Jul 27, 2024
Python

hazelcast / hazelcast

Star

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

java caching distributed-systems real-time big-data hazelcast scalability distributed-computing distributed stream-processing in-memory low-latency hacktoberfest data-insights data-in-motion

Updated Aug 19, 2026
Java

MaterializeInc / materialize

Star

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

mysql rust distributed-systems streaming sql sql-server database kafka postgresql data-store stream-processing cdc streaming-data materialized-view data-mesh postgresql-dialect operational-data-store

Updated Aug 20, 2026
Rust

apache / hudi

Star

Upserts, Deletes And Incremental Processing on Big Data.

bigdata stream-processing data-integration datalake apachespark hudi apachehudi incremental-processing apacheflink

Updated Aug 19, 2026
Java

online-ml / river

Star

🌊 Online machine learning in Python

python data-science machine-learning streaming stream-processing online-learning streaming-data concept-drift real-time-processing incremental-learning online-machine-learning online-statistics

Updated Aug 12, 2026
Python

ag2ai / faststream

Star

FastStream is an asynchronous Python framework for building event-driven applications. It brings together message broker integration, dependency injection, validation, testing utilities, and AsyncAPI documentation generation in a single toolkit

python redis mqtt distributed-systems kafka rabbitmq nats stream-processing mqtt-broker asyncio asyncapi faststream

Updated Aug 18, 2026
Python

fluvio-community / fluvio

Star

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

rust distributed-systems streaming real-time serverless webassembly data-flow stream-processing data-analytics data-integration cloud-native data-pipelines stateful streaming-data stream-processing-engine event-driven-architecture streaming-analytics streaming-data-processing streaming-data-pipelines

Updated Aug 11, 2026
Rust

javascriptdata / danfojs

Star

Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.