Explore Topics Trending Collections Events GitHub Sponsors

streaming-data

Here are 629 public repositories matching this topic...

Language: All Sort: Most stars

oxnr / awesome-bigdata

A curated list of awesome big data frameworks, ressources and other awesomeness.

data-science data awesome database data-stream bigdata series-database data-visualization data-warehouse stream-processing data-analytics awesome-list distributed-database visualize-data streaming-data

Updated Jul 31, 2026

provectus / kafka-ui

Star

Open-Source Web UI for Apache Kafka Management

opensource kafka big-data web-ui streams kafka-connect apache-kafka kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-ui kafka-brokers

Updated Jul 26, 2024
Java

johnkerl / miller

Star

Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

Updated Aug 17, 2026
Go

redpanda-data / connect

Star

Fancy stream processing made operationally mundane

go golang kafka cqrs etl rabbitmq amqp logs message-bus event-sourcing nats stream-processing message-queue data-engineering streaming-data stream-processor data-ops

Updated Aug 19, 2026
Go

MaterializeInc / materialize

Star

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

mysql rust distributed-systems streaming sql sql-server database kafka postgresql data-store stream-processing cdc streaming-data materialized-view data-mesh postgresql-dialect operational-data-store

Updated Aug 20, 2026
Rust

online-ml / river

Star

🌊 Online machine learning in Python

python data-science machine-learning streaming stream-processing online-learning streaming-data concept-drift real-time-processing incremental-learning online-machine-learning online-statistics

Updated Aug 12, 2026
Python

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

mysql rust caching postgres sql backend cache postgresql databases rust-lang mysql-database postgresql-database streaming-data caching-proxy

Updated Aug 19, 2026
Rust

fluvio-community / fluvio

Star

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

rust distributed-systems streaming real-time serverless webassembly data-flow stream-processing data-analytics data-integration cloud-native data-pipelines stateful streaming-data stream-processing-engine event-driven-architecture streaming-analytics streaming-data-processing streaming-data-pipelines

Updated Aug 11, 2026
Rust

piskvorky / smart_open

Sponsor

Star

Utils for streaming large files (S3, HDFS, gzip, bz2...)

python streaming s3 file hdfs hacktoberfest webhdfs boto gzip-stream bz2 streaming-data

Updated Aug 4, 2026
Python

kafbat / kafka-ui

Sponsor

Star

Open-Source Web UI for managing Apache Kafka clusters

opensource kafka big-data foss web-ui streams kafka-connect apache-kafka kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-ui kafka-brokers

Updated Aug 17, 2026
TypeScript

reugn / go-streams

Star

A lightweight stream processing library for Go

Updated Jan 14, 2026
Go

bytewax / bytewax

Star

Python Stream Processing

python rust data-science machine-learning stream-processing data-engineering dataflow data-processing streaming-data

Updated Jun 20, 2026
Python

pravega / pravega

Star

Pravega - Streaming as a new software defined storage primitive

streaming distributed-storage real-time-data streaming-data data-ingestion

Updated Mar 2, 2025
Java

quixio / quix-streams

Star

Python Streaming DataFrames for Kafka

python data-science machine-learning kafka stream-processing data-engineering streaming-data stream-processor event-driven-architecture streaming-data-processing time-series-data data-intensive-applications streaming-data-pipelines real-time-data-processing

Updated Aug 14, 2026
Python

python-streamz / streamz

Star

Real-time stream processing for python

python real-time async streaming-data

Updated Apr 7, 2026
Python

microsoft / Trill

Star

Trill is a single-node query processor for temporal or streaming data.

streaming-data temporal-data

Updated Jan 8, 2024
C#

DoneDeal0 / superdiff

Sponsor

Star

Superdiff provides a rich and readable diff for arrays, objects, texts and coordinates. It supports stream and file inputs for handling large datasets efficiently, is battle-tested, has zero dependencies, and offers a top-tier performance.