Projects in Awesome Lists tagged with stream-processing

https://github.com/runreveal/kawa

A blazingly fast event stream processing library powering the reveald event processing daemon.

events go golang logs message-queue stream-processing

Last synced: 05 Mar 2026

https://github.com/GridProtectionAlliance/openPDC

Open Source Phasor Data Concentrator

bpa-pdc-stream complex-event-processing hadoop iec61850 ieee-1344 ieee-c37118 naspi openpdc pdc phasor-data-concentrator phasor-measurement-unit pmu stream-processing stream-processing-engine streaming-data synchrophasor time-series

Last synced: 17 Apr 2025

https://github.com/devshawn/kafka-shell

⚡A supercharged, interactive Kafka shell built on top of the existing Kafka CLI tools.

apache cli confluent kafka kafka-cli kafka-tools prompt shell stream-processing streams terminal

Last synced: 05 Apr 2026

https://github.com/expediagroup/stream-registry

Stream Discovery and Stream Orchestration

metadata-management oss-portal-featured stream-processing streams

Last synced: 19 Aug 2025

https://github.com/xerial/streamdb-readings

Readings in Stream Processing

database papers research stream-processing

Last synced: 13 Apr 2025

https://github.com/ExpediaGroup/stream-registry

Stream Discovery and Stream Orchestration

metadata-management oss-portal-featured stream-processing streams

Last synced: 11 May 2025

https://github.com/wso2/product-sp

An open source, cloud-native streaming data integration and analytics product optimized for agile digital businesses

analytics complex-event-processing real-time-processing stream-processing streaming-data-integration

Last synced: 09 Apr 2025

https://github.com/miroslavpejic85/mirotalkbro

📡 MiroTalk WebRTC Live Broadcast allows to broadcast live video, audio and screen stream to all connected users (viewers).

audio-streaming broadcast deep-video jitsi live-streaming messaging mirotalk peer-to-peer screen-capture screen-sharing self-hosted stream-processing video-communication video-conferencing video-recording video-streaming webapp webrtc

Last synced: 05 Apr 2025

https://github.com/streamnative/pulsar-spark

Spark Connector to read and write with Pulsar

apache-pulsar apache-spark batch-processing data-processing data-science flink spark spark-sql stream-processing structured-streaming

Last synced: 06 Feb 2026

https://github.com/n0-computer/quic-rpc

A streaming rpc system based on quic

crate quic rpc-library rust stream-processing

Last synced: 15 May 2025

https://github.com/wso2/product-streaming-integrator

A stream processing runtime that allows connecting any streaming data source to any destination and act on it

cloud-native event-driven integration real-time siddhi stream-processing streaming-data streaming-integration wso2

Last synced: 10 Oct 2025

https://github.com/zefhub/zef

Toolkit for graph-relational data across space and time

data-oriented data-oriented-design data-oriented-programming distributed distributed-database distributed-systems domain-specific-language graph graph-database graphdb graphql immutable in-memory in-memory-database knowledge-graph neo4j stream-processing streams version-control versioning

Last synced: 06 Apr 2025

https://github.com/wso2/streaming-integrator

A stream processing runtime that allows connecting any streaming data source to any destination and act on it

cloud-native event-driven integration real-time siddhi stream-processing streaming-data streaming-integration wso2

Last synced: 28 Mar 2025

https://github.com/hurence/logisland

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

analytics big-data cassandra complex-event-processing elasticsearch influxdb kafka kafka-streams pattern-recognition solr spark stream-processing

Last synced: 07 Apr 2025

https://github.com/aleksandarskrbic/khaos

Kafka data generator and load testing tool - generate fake messages, simulate producers and consumers, test broker failures, and run chaos engineering scenarios

apache-kafka chaos-engineering data-engineering data-generator devops devops-tools fake-data fault-injection kafka load-testing python stream-processing stress-testing

Last synced: 13 Jan 2026

https://github.com/trk54ylmz/kafka-spark-streaming-example

Simple examle for Spark Streaming over Kafka topic

java kafka spark stream-processing

Last synced: 12 May 2025

https://github.com/trK54Ylmz/kafka-spark-streaming-example

Simple examle for Spark Streaming over Kafka topic

java kafka spark stream-processing

Last synced: 02 Apr 2025

https://github.com/bytedance/elkeid-hub

Elkeid HUB is a rule/event processing engine maintained by the Elkeid Team that supports streaming/offline (not yet supported by the community edition) data processing. The original intention is to solve complex data/event processing and external system linkage requirements through standardized rules.

event-engine rule-engine ruleengine soar stream-processing

Last synced: 13 Apr 2025

https://github.com/pravega/flink-connectors

Apache Flink connectors for Pravega.

flink pravega stream-processing

Last synced: 09 Apr 2025

https://github.com/Banno/kafka4s

Functional programming with Kafka and Scala

cats cats-effect fp fs2 functional-programming kafka scala stream-processing

Last synced: 04 Apr 2025

https://github.com/bytedance/Elkeid-HUB

Elkeid HUB is a rule/event processing engine maintained by the Elkeid Team that supports streaming/offline (not yet supported by the community edition) data processing. The original intention is to solve complex data/event processing and external system linkage requirements through standardized rules.

event-engine rule-engine ruleengine soar stream-processing

Last synced: 19 Jul 2025

https://paragroup.github.io/WindFlow/

A C++17 Data Stream Processing Parallel Library for Multicores and GPUs

cuda gpu gpu-acceleration gpu-computing gpu-programming multi-core multicore multithreading parallel-computing parallel-patterns parallel-programming parallelism sliding-windows stream stream-api stream-processing streaming streaming-api streaming-data streams

Last synced: 14 May 2025

https://github.com/blockchain-etl/blockchain-etl-streaming

Streaming Ethereum and Bitcoin blockchain data to Google Pub/Sub or Postgres in Kubernetes

apache-beam bitcoin blockchain blockchain-analytics crypto cryptocurrency data-analytics data-engineering ethereum etl gcp google-bigquery google-cloud-platform google-dataflow google-pubsub on-chain-analysis real-time real-time-analytics stream-processing web3

Last synced: 25 Jun 2025

https://github.com/spinoco/fs2-kafka

Kafka client for functional streams for scala (fs2)

asynchronous fs2-kafka kafka kafka-client kafka-driver scala scodec stream-processing streaming

Last synced: 07 Sep 2025

https://github.com/decodableco/examples

🌟 Examples of use cases that utilize Decodable, as well as demos for related open-source projects such as Apache Flink, Debezium, and Postgres.

datamesh dataproducts etl flink flink-stream-processing kafka pyflink python stream-processing

Last synced: 09 Apr 2026

https://github.com/lsds/LightSaber

Multi-core Window-Based Stream Processing Engine

aggregation compression cpp incremental-computation libaio llvm multi-core numa rdma sliding-windows ssd stream-processing

Last synced: 14 May 2025

https://github.com/lsds/lightsaber

Multi-core Window-Based Stream Processing Engine

aggregation compression cpp incremental-computation libaio llvm multi-core numa rdma sliding-windows ssd stream-processing

Last synced: 21 Mar 2025

https://github.com/rmoff/ksqldb-go

A Golang client for ksqlDB

kafka kafka-streams sql stream-processing

Last synced: 07 Apr 2026

https://github.com/intellistream/streamprocessing_readinglist

stream processing reading list

reading-list stream-processing

Last synced: 16 Feb 2026

https://github.com/scramjetorg/transform-hub

Scramjet Transform Hub (STH) is a runtime supervisor that can run data processing programs called Sequences and manage local resources on any Linux server, Docker on small edge servers, and even large-scale Kubernetes clusters in the cloud or datacenters. It connects to Scramjet Spaces in Scramjet Cloud Platform.

containers data-acquisition docker kubernetes kubernetes-deployment nodejs python python3 real-time runtime-supervisor stream-processing streaming transform-as-a-service

Last synced: 14 Mar 2026

https://github.com/maxim2266/csvplus

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

csv csv-format etl etl-framework etl-pipeline fluent-interface go go-csv stream-processing

Last synced: 14 Apr 2025

https://github.com/blendle/pg2kafka

Ship changes in Postgres 🐘 to Kafka 📖

golang kafka postgresql stream-processing

Last synced: 14 May 2025

https://github.com/sanmai/pipeline

Build streaming pipelines in PHP. The missing pipe operator. General-purpose collection pipeline. Since 2017.

chain collections functional-programming php-library pipeline stream-processing

Last synced: 16 May 2025

https://github.com/project-flogo/stream

Elegant stream processing pipeline written entirely in Golang

edge flogo stream-processing streaming

Last synced: 15 Apr 2025

https://github.com/thibauts/styx

Simple, high-performance event streaming broker

broker event-streaming messaging pubsub stream-processing

Last synced: 28 Aug 2025

https://github.com/kurrent-io/kurrentdb-client-java

Official Asynchronous Java 8+ Client Library for KurrentDB

client-libraries grpc java kurrent kurrentdb stream-processing

Last synced: 10 Apr 2025

https://github.com/djarza/football-events

Event-Driven microservices with Kafka Streams

cqrs demo event-driven event-driven-microservices event-sourcing example kafka kafka-connect kafka-streams microservices microservices-architecture stream-processing

Last synced: 26 Mar 2025

https://github.com/kurrent-io/KurrentDB-Client-Java

Official Asynchronous Java 8+ Client Library for EventStoreDB 20.6+

client-libraries eventstore eventstoredb grpc java stream-processing

Last synced: 16 Jul 2025

https://github.com/lpgauth/swirl

High-Performance Erlang Stream Processor

erlang flow high-performance stream-processing

Last synced: 08 May 2025

https://github.com/online-ml/watermill.rs

🦀 Online statistics in Rust

crate online-statistics rust statistics stream-processing streaming

Last synced: 04 May 2025

https://github.com/rodrigo-arenas/kafkaml-anomaly-detection

Project for real-time anomaly detection using Kafka and python

anomaly-detection apache-karaf confluent-kafka kafka machine-learning python real-time-analytics real-time-processing scikit-learn scikit-learning sklearn stream-processing

Last synced: 30 Apr 2025

https://github.com/mikeroyal/apache-flink-guide

Apache Flink Guide

data-science database flink flink-kafka flink-stream-processing flink-streaming stream-processing streaming

Last synced: 02 Sep 2025

https://github.com/seliumlabs/selium

A software framework and runtime for building scalable, connected applications

runtime stream-processing webassembly

Last synced: 21 Jan 2026

https://github.com/getsentry/arroyo

A library to build streaming applications that consume from and produce to Kafka.

kafka stream-processing streaming tag-production

Last synced: 04 Mar 2026

https://github.com/itsubaki/gostream

Stream Processing library for Go

complex-event-processing stream-processing

Last synced: 20 Oct 2025

https://github.com/unixjunkie/parany

Parallelize _anything_ //

multi-processing ocaml-library parallel-computing stream-processing

Last synced: 22 Mar 2025

https://github.com/twalthr/flink-api-examples

Examples for using Apache Flink® with DataStream API, Table API, Flink SQL and connectors such as MySQL, JDBC, CDC, Kafka.

apache-flink data-engineering flink flink-examples flink-sql stream-processing

Last synced: 04 Apr 2025

https://github.com/systemaccounting/mxfactorial

a payment application intended for deployment by the united states treasury that replaces banking with accounting

banking bivector capital cga combinatorial-game combinatorial-optimization conservation-laws economics federal-reserve finance fintech game-theory geometric-algebra inflation mathematical-physics monetary-inflation physics price-discovery stream-processing streaming-data

Last synced: 22 Jul 2025

https://github.com/nextbreakpoint/flink-controller

Flink Controller implements a Kubernetes Custom Controller (aka Kubernetes Operator) for Apache Flink

flink infrastructure-as-code kotlin kubernetes kubernetes-controller kubernetes-operator stream-processing

Last synced: 26 Sep 2025

https://github.com/cau-se/theodolite

Theodolite is a framework for benchmarking the horizontal and vertical scalability of cloud-native applications.

benchmarking cloud-native event-driven-microservices operators scalability stream-processing

Last synced: 22 Jan 2026

https://github.com/gretelai/gretel-python-client

The Gretel Python Client allows you to interact with the Gretel REST API.

datascience machine-learning privacy privacy-enhancing-technologies stream-processing synthetic-data

Last synced: 04 Apr 2025

https://github.com/point72/csp

csp is a high performance reactive stream processing library, written in C++ and Python

cpp python reactive reactive-programming stream-processing streaming

Last synced: 10 Apr 2025

https://github.com/kantord/headson

head/tail for structured data - summarize/preview JSON/YAML and source code

ai-agents bash cli data-analysis data-presentation json json-parser performance python python-extension rust rust-cli shell-oneliner stream-processing summarization text-processing tree-structure yaml

Last synced: 01 Feb 2026

https://github.com/eschizoid/kpipe

Composable Kafka consumer library for building modular, testable JVM data pipelines.

apache-kafka data-pipelines event-driven functional-programming java kafka stream-processing

Last synced: 20 May 2026

https://github.com/maxhalford/taxi-demo-rp-mz-rv-rd-st

🚕 Self-contained demo using Redpanda, Materialize, River, Redis, and Streamlit to predict taxi trip durations

machine-learning materialize online-learning real-time redpanda river stream-processing streamlit

Last synced: 04 Aug 2025

https://github.com/nastel/tnt4j

Track and Trace 4 Java API, Application event streaming and logging framework for correlation, diagnostics and tracking of application activities.

java logging logging-library stream-processing streaming streaming-api

Last synced: 22 Apr 2025

https://github.com/benthosdev/benthos-captain

A Kubernetes Operator to orchestrate Benthos pipelines

benthos data data-engineering gitops go golang helm kubernetes kustomize pipelines stream-processing

Last synced: 22 Jan 2026

https://github.com/ibmstreams/streamsx.health

(Incubation) Streams Healthcare Analytics Platform - To enable and inspire the next generation healthcare solutions

analytics bokeh fhir healthcare healthcare-demo hl7 ibm-streams jupyter-notebook microservice python stream-processing vital-signs waveform

Last synced: 15 Jul 2025

https://github.com/ibmstreams/samples

This repository contains open-source sample applications for IBM Streams.

database geofence geofencing hdfs healthcare ibm-streams samples stream-processing text-analytics timeseries

Last synced: 15 Jul 2025

https://github.com/tradewelltech/beavers

Python stream processing for analytics

analytics apache-arrow data kafka pandas python realtime stream-processing

Last synced: 14 Jan 2026

https://github.com/maki-nage/makinage

Stream Processing Made Easy

distributed-systems kafka machine-learning python reactive-machine-learning reactive-programming reactive-systems stream-processing streaming streaming-data

Last synced: 14 Dec 2025

https://github.com/lsds/saber

Window-Based Hybrid CPU/GPU Stream Processing Engine

gpu high-throughput hybrid multicore multicore-cpu saber sliding-windows stream stream-processing streaming streaming-data

Last synced: 24 Jun 2025

https://github.com/lsds/Saber

Window-Based Hybrid CPU/GPU Stream Processing Engine

gpu high-throughput hybrid multicore multicore-cpu saber sliding-windows stream stream-processing streaming streaming-data

Last synced: 27 Mar 2025

https://github.com/xtra-computing/briskstream

A Multicore, NUMA Optimised Data Stream Processing System

multicore numa-systems stream-processing

Last synced: 13 Apr 2025

https://github.com/openfun/ralph

:gear: Ralph, the ultimate Learning Record Store (and more!) for your learning analytics

analytics cli data-stream docker elasticsearch fastapi gelf k8s learning-analytics lrs pydantic python stream-processing xapi

Last synced: 22 Apr 2025

https://github.com/alibaba/table-computing

Table-Computing (Simplified as TC) is a high performance and low latency computing framework, 10x faster than Flink for complicated use cases, distributed and light weighted, relational operation, simple to use, write less and do more.

big-data data-analysis java stream-processing table-computing tc

Last synced: 14 Oct 2025

https://github.com/graphform/traffic

Massively real-time traffic streaming application

actor-model actorsystem concurrency concurrent-programming demo-app distributed distributed-systems map mapbox open-source opensource real-time realtime stateful stream-processing streaming traffic-analysis

Last synced: 18 Aug 2025

https://github.com/mp911de/redis-stream-demo

Demo for Redis Streams

java redis spring-boot stream stream-processing

Last synced: 05 Jul 2025

https://github.com/doganulus/reelay

A header-only C++ library for system-level verification and declarative testing of real-time systems with Python bindings.

assertion-library cyber-physical-systems formal-methods header-only runtime-verification stream-processing system-verification temporal-logic verification

Last synced: 05 Apr 2025

https://github.com/numaproj/numalogic-prometheus

AIOps for metrics in Prometheus

aiops machine-learning stream-processing timeseries-analysis

Last synced: 29 Oct 2025

https://github.com/bakdata/streams-bootstrap

Utility functions and base classes for Kafka Streams applications

helm kafka kafka-streams kubernetes stream-processing

Last synced: 10 Apr 2025

https://github.com/asavinov/machine-learning-and-data-processing

A collection of resources on machine learning, data processing and related areas

analytics big-data data-mining data-processing data-science databases machine-learning software stream-processing

Last synced: 02 Mar 2026

https://github.com/xtaci/sp

Stream Processors on Kafka in Golang

golang stream-processing

Last synced: 15 Apr 2025

https://github.com/ibmstreams/streamsx.messaging

This toolkit is focused on interacting with popular messaging systems such as Kafka, JMS, XMS, and MQTT. After release v5.4.2 the complete toolkit will be deprecated. See the README.md file for hints to alternative toolkits.

ibm-streams iot-platform jms kafka mqtt rabbitmq stream-processing websphere-mq xms

Last synced: 15 Jul 2025

https://github.com/ibmstreams/streamsx.topology

Develop streaming applications for IBM Streams in Python, Java & Scala.

ibm-streams java python python-3 scala stream-processing streaming-topologies toolkit

Last synced: 01 Jul 2025

https://github.com/streamreasoning/rsp4j

RSP4J a library for RDF Stream Processing

rsp semantic-web stream-processing stream-reasoning

Last synced: 09 Mar 2026

https://github.com/certeu/morio

Connect - Stream - Observe - Respond | Morio provides the plumbing for your observability needs

beats cybersecurity cybersecurity-tools kafka observability stream-processing streaming-data

Last synced: 27 Jan 2026

https://github.com/sustainable-computing-io/kepler-model-server

Model Server for Kepler

machine-learning model-server online-machine-learning prometheus stream-processing sustainability tensorflow tensorflow-io tensorflow2

Last synced: 08 May 2025

https://github.com/wp-labs/wp-editor

data-collector data-processing etl events metrics observability oml siem stream-processing streaming telemetry warp-parse wp-ed wp-parse wpl

Last synced: 07 Apr 2026

https://github.com/swimos/traffic

Massively real-time traffic streaming application

actor-model actorsystem concurrency concurrent-programming demo-app distributed distributed-systems map mapbox open-source opensource real-time realtime stateful stream-processing streaming traffic-analysis

Last synced: 08 Apr 2025

https://github.com/gigahexhq/jetprobe

🚀 Validation DSL for data pipelines

akka akka-actors bigdata integration-testing scala stream-processing

Last synced: 26 Jun 2025

https://github.com/perfectlysoft/perfect-kafka

An Express Swift Client of Apache Kafka 0.8, the Stream Processing Platform

kafka message-queue stream-processing

Last synced: 07 Mar 2026

https://github.com/thriving-dev/kafka-streams-cassandra-state-store

'Drop-in' Kafka Streams State Store implementation that persists data to Apache Cassandra / ScyllaDB

apache-cassandra cassandra java kafka-streams stream-processing

Last synced: 10 Apr 2025

https://github.com/ranjanrak/clickhouse-tickstore

Go package to store real time streaming websocket data in clickhouse using queuing and bulk insert.

clickhouse go-channel go-routine stream-processing

Last synced: 07 Jul 2025

https://github.com/saidsef/aws-kinesis-local

AWS Kinesis local for building applications with streaming data

amazon-kinesis-streams aws-kinesis aws-kinesis-local aws-kinesis-stream docker-container nodejs stream-processing streaming-data

Last synced: 02 Jun 2026

https://github.com/o2-czech-republic/proxima-platform

The Proxima platform.

analytical-platform apache-beam apache-flink apache-spark batch-processing data-mesh iot-platform stream-processing unified-data-processing

Last synced: 17 Jan 2026

https://github.com/hazelcast/hazelcast-jet-contrib

Extension modules for Hazelcast Jet

big-data hazelcast hazelcast-jet java stream-processing

Last synced: 07 Jul 2025

https://github.com/heyoka/faxe

data-flow edge-computing erlang faxe flowbased iiot iot stream-processing

Last synced: 19 Aug 2025

https://github.com/swimos/ripple

Simple shared surface streaming application

actor-model distributed-computing distributed-systems game game-development low-latency massively-multiplayer mmo open-source opensource real-time realtime stateful stream-processing streaming

Last synced: 08 Apr 2025

https://github.com/ibmstreams/administration

Umbrella project for the IBMStreams organization. This project will be used for the management of the individual projects within the IBMStreams organization.

administration ibm-streams stream-processing

Last synced: 25 Feb 2026