Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with streaming-data

A curated list of projects in awesome lists tagged with streaming-data .

https://github.com/materializeinc/materialize

Real-time Data Integration and Transformation: use SQL to transform, deliver, and act on fast-changing data.

data-store database distributed-systems kafka materialized-view operational-data-store postgresql postgresql-dialect rust sql stream-processing streaming streaming-data

Last synced: 09 Sep 2025

https://github.com/readysettech/readyset

Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

backend cache caching caching-proxy databases mysql mysql-database postgres postgresql postgresql-database rust rust-lang sql streaming-data

Last synced: 27 Feb 2026

https://github.com/piskvorky/smart_open

Utils for streaming large files (S3, HDFS, gzip, bz2...)

boto bz2 file gzip-stream hacktoberfest hdfs python s3 streaming streaming-data webhdfs

Last synced: 11 Dec 2025

https://github.com/RaRe-Technologies/smart_open

Utils for streaming large files (S3, HDFS, gzip, bz2...)

boto bz2 file gzip-stream hacktoberfest hdfs python s3 streaming streaming-data webhdfs

Last synced: 31 Mar 2025

https://github.com/memgraph/memgraph

Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.

cypher graph graph-algorithms graph-analysis graph-database kafka kafka-streams nosql opencypher stream-processing streaming-data

Last synced: 14 May 2025

https://github.com/pravega/pravega

Pravega - Streaming as a new software defined storage primitive

data-ingestion distributed-storage real-time-data streaming streaming-data

Last synced: 13 May 2025

https://github.com/python-streamz/streamz

Real-time stream processing for python

async python real-time streaming-data

Last synced: 12 Dec 2025

https://github.com/Microsoft/trill

Trill is a single-node query processor for temporal or streaming data.

streaming-data temporal-data

Last synced: 27 Mar 2025

https://github.com/microsoft/Trill

Trill is a single-node query processor for temporal or streaming data.

streaming-data temporal-data

Last synced: 04 Apr 2025

https://github.com/microsoft/trill

Trill is a single-node query processor for temporal or streaming data.

streaming-data temporal-data

Last synced: 14 May 2025

https://github.com/DoneDeal0/superdiff

Superdiff provides a complete and readable diff for both arrays and objects. Plus, it supports stream and file inputs for handling large datasets efficiently, is battle-tested, has zero dependencies, and is super fast.

array-comparison comparison comparison-tool deep-diff diff json-diff nodejs object-comparison object-diff objectdiff objectdifference react streaming streaming-data typescript

Last synced: 04 Apr 2025

https://github.com/scikit-multiflow/scikit-multiflow

A machine learning package for streaming data in Python. The other ancestor of River.

machine-learning meka moa scikit scikit-learn stream streaming-data

Last synced: 15 May 2025

https://github.com/hstreamdb/hstream

HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.

data-processing database distributed-database distributed-systems financial-analysis haskell hstreamdb iot iot-database kafka materialized-view real-time realtime-database scale sql stream-processing streaming streaming-data streaming-database

Last synced: 15 May 2025

https://github.com/kLabUM/rrcf

🌲 Implementation of the Robust Random Cut Forest algorithm for anomaly detection on streams

anomaly-detection detect-outliers machine-learning outliers python random-forest robust-random-cut-forest streaming-data tree

Last synced: 14 Mar 2025

https://github.com/guillermo-navas-palencia/optbinning

Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning. Scorecard modelling and counterfactual explanations.

batch-processing binning counterfactual-explanations credit-scoring mdlp optimization python scorecard stream streaming-data woe woebinning

Last synced: 30 Dec 2025

https://github.com/lightbend/cloudflow

Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.

akka cloudflow flink kubernetes microservices-architectures spark streaming-applications streaming-data streaming-runtimes

Last synced: 23 Oct 2025

https://github.com/microsoft/data-accelerator

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

apache-spark azure big-data cosmosdb docker eventhub hdinsight iot iothub kafka kafka-streams nodejs react servicefabric spark spark-sql spark-streaming sparksql streaming streaming-data

Last synced: 15 May 2025

https://github.com/goodboy/tractor

A distributed, structured concurrency runtime for Python (and friends)

actor-model async-await distributed-systems multicore-programming multiprocessing rpc streaming-data structured-concurrency trio

Last synced: 15 May 2025

https://github.com/ast-al/rangeless

c++ LINQ -like library of higher-order functions for data manipulation

cpp cpp11 functional functional-programming itertools lazy-evaluation linq parallel pipeline range streaming-algorithms streaming-data

Last synced: 08 May 2025

https://github.com/maraisr/meros

🪢 A fast utility that makes reading multipart responses simple

defer fetch graphql multipart multipart-mixed relay stream streaming-data

Last synced: 07 Oct 2025

https://github.com/evadne/packmatic

Zipping on the fly — Generate downloadable Zip streams by aggregating File or URL Sources

elixir-lang elixir-library elixir-phoenix elixir-plug phoenix streaming-data zip

Last synced: 05 Apr 2025

https://github.com/wso2/streaming-integrator

A stream processing runtime that allows connecting any streaming data source to any destination and act on it

cloud-native event-driven integration real-time siddhi stream-processing streaming-data streaming-integration wso2

Last synced: 28 Mar 2025

https://github.com/wso2/product-streaming-integrator

A stream processing runtime that allows connecting any streaming data source to any destination and act on it

cloud-native event-driven integration real-time siddhi stream-processing streaming-data streaming-integration wso2

Last synced: 10 Oct 2025

https://github.com/bobbyiliev/materialize-tutorials

Materialize is a streaming database for real-time analytics. This is a collection of Materialize demos and tutorials.

analytics databases materialize postgresql real-time-data sql streaming-data streaming-sql

Last synced: 23 Mar 2025

https://github.com/seznam/euphoria

Euphoria is an open source Java API for creating unified big-data processing flows. It provides an engine independent programming model which can express both batch and stream transformations.

apache-flink apache-spark batch-processing big-data hadoop hdfs java-api kafka streaming-data unified-bigdata-processing

Last synced: 21 Aug 2025

https://github.com/axway-streams/axway-amplify-streams-js

AMPLIFY Streams Javascript package containing SDK, documentation and sample applications

angular js nodejs server-sent-events sse streamdataio streaming-data vuejs

Last synced: 08 Mar 2026

https://github.com/pathwaycom/pathway-benchmarks

Benchmarks for data processing systems: Pathway, Spark, Flink, Kafka Streams

benchmark-framework flink kafka-streams latency pagerank pathway spark-streaming streaming streaming-data wordcount

Last synced: 02 Aug 2025

https://github.com/pravega/pravega-samples

Sample Applications for Pravega.

data-streaming pravega sample-app streaming-data

Last synced: 27 Apr 2025

https://github.com/rxswiftcommunity/rxhttpclient

Simple Http client (Use RxSwift for stream data)

nsurlsession rxswift streaming-data swift

Last synced: 16 Jun 2025

https://github.com/andrewssobral/imtsl

IMTSL - Incremental and Multi-feature Tensor Subspace Learning

background-subtraction foreground-detection matlab streaming-data subspace-learning tensor

Last synced: 13 Apr 2025

https://github.com/memgraph/twitter-network-analysis

Analyzing a network of tweets and retweets using graph algorithms

kafka kafka-streams memgraph online-pagerank pagerank pagerank-algorithm streaming streaming-data twitter

Last synced: 24 Oct 2025

https://github.com/jzo001/webapistreaming

How to get data as a stream from a WebAPI (.NET)

csharp dotnet-core iasyncenumerable streaming streaming-api streaming-data webapi-core

Last synced: 07 May 2025

https://github.com/sdpython/pandas-streaming

Streaming API for pandas applied to big datasets

numpy pandas python3 streaming-data streaming-data-processing

Last synced: 30 Jun 2025

https://github.com/marrow/cinje

A Pythonic and ultra fast template engine DSL.

cpython dsl pypy python python-2 python-3 streaming-data template-engine text-processing

Last synced: 13 Jul 2025

https://github.com/garystafford/streaming-sales-generator

Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python

analytics apache-flink apache-kafka data kafka kafka-streams kstreams python spark-structured-streaming streaming-data

Last synced: 03 Aug 2025

https://github.com/certeu/morio

Connect - Stream - Observe - Respond | Morio provides the plumbing for your observability needs

beats cybersecurity cybersecurity-tools kafka observability stream-processing streaming-data

Last synced: 27 Jan 2026

https://github.com/byte271/6cy

High-performance, streaming-first container format with per-block codec polymorphism and robust data recoverability. Reference implementation in Rust.

codec-polymorphism compression container-format data-integrity lz4 rust specification storage-engine streaming-data zstd

Last synced: 23 Feb 2026

https://github.com/pravahio/go-mesh

Realtime data exchange platform for Smart Cities

data realtime streaming-data

Last synced: 14 Jan 2026

https://github.com/ominibyte/richflow

A Node.js and JavaScript synchronous data pipeline processing, data sharing and stream processing library. Actionable & Transformable Pipeline data processing.

data-flow data-pipeline data-processor data-stream data-transformation flow javascript nodejs pipe-data pipeline-framework streaming-data synchronous

Last synced: 19 Feb 2026

https://github.com/mikejareds/hermiter

Efficient Sequential and Batch Estimation of Univariate and Bivariate Probability Density Functions and Cumulative Distribution Functions along with Quantiles (Univariate) and Nonparametric Correlation (Bivariate)

cumulative-distribution-function kendall-correlation-coefficient online-algorithms probability-density-function quantile spearman-correlation-coefficient statistics streaming-algorithms streaming-data

Last synced: 22 Oct 2025

https://github.com/ni/easyrdma

An easy-to-use, cross-platform, MIT-licensed RDMA library from NI

drivers measurements rdma streaming-data

Last synced: 05 Sep 2025

https://github.com/avriiil/stream-this-dataset

Code to convert static datasets into simulated data streams

dataset-generation streaming-data

Last synced: 06 May 2025

https://github.com/meroxa/turbine-go

Turbine Library for Go

data go golang stream-processing streaming-data

Last synced: 17 Jan 2026

https://github.com/MikeJaredS/hermiter

Efficient Sequential and Batch Estimation of Univariate and Bivariate Probability Density Functions and Cumulative Distribution Functions along with Quantiles (Univariate) and Nonparametric Correlation (Bivariate)

cumulative-distribution-function kendall-correlation-coefficient online-algorithms probability-density-function quantile spearman-correlation-coefficient statistics streaming-algorithms streaming-data

Last synced: 13 Jul 2025

https://github.com/cpacker/graphzip

Mining graph streams using dictionary-based compression

compression graph-algorithms graph-mining streaming-data

Last synced: 09 Aug 2025

https://github.com/fajarnugraha37/turborepo-nestjs

Fullstack multiple service application using turborepo typescript, nestjs, nextjs, prisma, mongodb and rabbitmq.

event-driven event-sourcing message-broker message-bus message-queue messaging microservice mongodb nestjs nextjs noodejs prisma rabbitmq react reactjs streaming-data turborepo typescript vercel

Last synced: 15 Oct 2025

https://github.com/microsoft/fabricrtiworkshop

How to build a Medallion design pattern using Fabric Real-Time Intelligence

analytics batch dashboard intelligence realtime streaming-data

Last synced: 29 Oct 2025

https://github.com/mineur/twitter-stream-api

:baby_chick: Another Twitter stream PHP library to retrieve filtered tweets on hot.

guzzle mineur php71 streaming-api streaming-data twitter-streaming-api

Last synced: 24 Feb 2026

https://github.com/kodi/splex

Streaming Log Multiplexer - combine multiple logs to one

cli javascript node stream streaming-data

Last synced: 20 Sep 2025

https://github.com/propensive/turbulence

Simple tools for working with data streams in LazyLists in Scala

multiplexing scala streaming streaming-api streaming-data

Last synced: 16 Aug 2025

https://github.com/janaom/gcp-de-project-streaming-pubsub-beam-dataflow

This project demonstrates an end-to-end solution for processing and analyzing real-time conversations data from a JSON file using GCP services and infrastructure automation, showcasing data storage, streaming, processing, and analysis at scale.

apache-beam bigquery dataflow de-project gcp pubsub streaming-data

Last synced: 18 Oct 2025

https://github.com/jsa-aerial/aerobio

Extensible full DAG streaming computation server with services and jobs for RNA-Seq, Tn-Seq, WG-Seq and Term-Seq.

clojure genome-sequencing pipeline-framework pipelines rna-seq streaming-data term-seq tn-seq wg-seq

Last synced: 12 May 2025

https://github.com/jmaces/statstream

Statistics for Streaming Data

data-science numpy statistics streaming-data

Last synced: 23 Apr 2025

https://github.com/alexklibisz/meetup-viz

Real-time visualization of streaming data from the Meetup.com open events RSVP API.

meetup reactjs streaming-data visualization

Last synced: 28 Oct 2025