Projects in Awesome Lists tagged with stream-processing

https://github.com/chartbeat-labs/ottla

An opinionated clojure framework for writing kafka machines

Last synced: 12 Apr 2025

https://github.com/thecodemonkey/kafka-microservices

A series of tiny examples around Kafka and event driven microservices using Spring Framework written in Kotlin & Java.

cqrs event-driven event-driven-architecture event-driven-microservices event-sourcing hacktoberfest kafka kafka-api kafka-cqrs kafka-ksqldb kafka-saga kafka-streams microservices microservices-kafka stream-processing

Last synced: 06 Apr 2025

https://github.com/ibmstreams/tutorials

A project for developing tutorials for Streams

ibm-streams stream-processing tutorial tutorial-exercises

Last synced: 15 Jul 2025

https://github.com/meroxa/turbine-go

Turbine Library for Go

data go golang stream-processing streaming-data

Last synced: 17 Jan 2026

https://github.com/qntfy/frizzle

The magic message bus

consumer golang golang-library kafka kinesis message-bus pipeline producer stream-processing streaming-data

Last synced: 12 Jan 2026

https://github.com/robhowley/s3-streaming

stream and (de)serialize s3 streams

aws file-io s3 stream-processing

Last synced: 30 Dec 2025

https://github.com/kumkeehyun/gstream

Kafka Streams DSL inspired, Stream processing library abstracting pipelines pattern using generic.

generic kafka-streams-dsl pipelines stateful stateless stream-processing

Last synced: 23 Mar 2025

https://github.com/innogames/flink-real-time-crm

Demo Flink and Kafka project to show how to react on tracking events in real-time and trigger offer for customer engagement based on campaign configurations. The project also utilizes the Broadcast State Pattern in order to update the rules (campaigns) at runtime without restarting the project, using a dedicated, low-frequency, Kafka topic.

apacheflink apachekafka flink flink-examples flink-stream-processing flink-streaming java kafka stream-processing

Last synced: 18 Jun 2025

https://github.com/bakdata/kpops

Deploy Kafka pipelines to Kubernetes

kafka kafka-connect kafka-streams kubernetes pipelines stream-processing

Last synced: 03 Apr 2025

https://github.com/ibmstreams/streamsx.kafka

Repository for integration with Apache Kafka

apache-spark ibm-streams kafka messaging stream-processing toolkit

Last synced: 24 Jun 2025

https://github.com/usedatabrew/blink

OpenSource data platform to build event-driven systems. It's like Deebezium for golang :)

data-engineering data-processing debezium etl etl-pipeline kafka stream-processing stream-processor streaming

Last synced: 24 Dec 2025

https://github.com/irc-sphere/hyperstream

HyperStream

compute-engine hyperstream sphere stream-processing streaming-data workflow-engine

Last synced: 13 Aug 2025

https://github.com/qntfy/frafka

Frizzle for Apache Kafka

consumer golang golang-library kafka message-bus pipeline producer stream-processing streaming-data

Last synced: 12 Jan 2026

https://github.com/calvinlfer/akka-streams-custom-stream-processing-examples

Demos of how to do custom stream processing using the Akka Streams GraphStages API

akka akka-streams scala stream-processing

Last synced: 23 Jun 2025

https://github.com/maxhalford/tuna

:fish: A streaming ETL for fish

etl feature-extraction go golang large-dataset machine-learning online-algorithms stream stream-processing

Last synced: 07 May 2025

https://github.com/ishaanadarsh/medstream-analytics

The project leverages Apache Flink, Apache Kafka and Python digital Twin to provide real-time insights into healthcare data, enabling timely interventions and proactive patient care.

analytics apache-flink apache-kafka complex-event-processing digital-twin stream-processing streaming

Last synced: 11 Mar 2026

https://github.com/IBMStreams/streamsx.kafka

Repository for integration with Apache Kafka

apache-spark ibm-streams kafka messaging stream-processing toolkit

Last synced: 11 May 2025

https://github.com/abhirockzz/kafka-streams-interactive-queries

Interactive Queries in Kafka Streams

docker jaxrs jersey kafka kafka-streams stream-processing

Last synced: 05 Jul 2025

https://github.com/hpi-epic/espbench

ESPBench - The Enterprise Stream Processing Benchmark

benchmark stream-processing

Last synced: 27 Feb 2025

https://github.com/decodableco/dbt-decodable

A dbt adapter for Decodable

apache-flink dbt flink sql stream-processing

Last synced: 09 Apr 2026

https://github.com/billyrrr/onto

Idealistic and yet usable framework for event-driven backend

architecture backend backend-for-frontend firestore flasgger framework gcloud mobile-backend openapi python3 reactive-programming reactive-services stream-processing websocket

Last synced: 07 May 2025

https://github.com/angelhof/flumina

A parallel programming model for online applications with complex synchronization requirements.

erlang stream-processing

Last synced: 15 May 2025

https://github.com/lromul/rosny

Lightweight library for building concurrent systems

concurrency multiprocessing parallel python stream-processing threading

Last synced: 07 May 2025

https://github.com/vectorhacker/goro

Go client library for EventStore.

event-sourcing eventsourcing eventstore golang golang-library golang-package goro stream-processing streaming

Last synced: 22 Jan 2026

https://github.com/hpgrahsl/wearedevs-2018

Code for my talk "Stateful & Reactive Streaming Applications Without a Database" at WeAreDevelopers 2018

emojis java kafka kafka-connect kafka-streams reactive reactive-programming spring-boot-2 stream-processing tweets

Last synced: 20 Oct 2025

https://github.com/monasca/monasca-aggregator

Near real-time continuous aggregation of Monasca metrics

golang kubernetes monasca monitoring openstack stream-processing

Last synced: 15 Jan 2026

https://github.com/mikeroyal/apache-ignite-guide

Apache Ignite Guide

data-science database hadoop hadoop-cluster ignite nosql nosql-data-storage nosql-databases stream-processing streaming

Last synced: 06 May 2025

https://github.com/uhop/stream-fork

Fork a writable stream into several parallel writable streams.

stream-processing streams

Last synced: 23 Apr 2025

https://github.com/minhtran241/grpc-introduction

Implementation of gRPC with Go

deadlines error-handling evans golang grpc grpc-go mongodb mongodriver protocol-buffers reflection ssl-certificates stream-processing unary-rpc

Last synced: 22 Jun 2025

https://github.com/winkjs/wink-composer

Composable Streaming Intelligence

anomaly-detection composable edge-ai edge-computing event-driven industrial-iot iot javascript nodejs real-time-analytics statistical-analysis statistical-learning stream-processing streaming-analytics wink-composer

Last synced: 04 Oct 2025

https://github.com/moimhossain/event-sourcing

An implementation of Event Sourcing and CQRS based on Azure table storage (EventStore) and Document DB (Materialized Views)

azure-storage cosmosdb documentdb event-sourcing event-stream eventstore stream-processing table-storage

Last synced: 14 Aug 2025

https://github.com/jweinst1/fire

The Reactive Programming Language

programming-language reactive-programming reactive-streams stream-processing

Last synced: 10 Jul 2025

https://github.com/codefeedr/codefeedr

CodeFeedr core infrastructure

flink kafka sbt scala stream-processing

Last synced: 24 Oct 2025

https://github.com/naughtygopher/nibbler

A package for micro batch processing

batch batch-processing event-processing go go-library go-package golang micro-batches microbatch stream stream-processing

Last synced: 07 May 2025

https://github.com/ocha-dap/hdx-python-utilities

A range of helpful utilities for Python developers including streaming tabular data, date parsing, JSON and YAML handling, dictionary and list utilities

datetime hdx json logging stream-processing tabular-data utilities yaml

Last synced: 16 Feb 2026

https://github.com/ibmstreams/streamsx.inet

This toolkit supports common internet protocols, such as HTTP and WebSockets

ftp http ibm-streams inet stream-processing toolkit websockets

Last synced: 15 Jul 2025

https://github.com/questdb/flink-questdb-connector

QuestDB connector for Apache Flink (over InfluxDB Line Protocol)

flink sql stream-processing time-series

Last synced: 27 Jun 2025

https://github.com/correia-jpv/fucking-awesome-bigdata

A curated list of awesome big data frameworks, resources and other awesomeness. With repository stars⭐ and forks🍴

awesome awesome-list bigdata data data-analytics data-science data-stream data-visualization data-warehouse database distributed-database series-database stream-processing streaming-data visualize-data

Last synced: 27 Apr 2025

https://github.com/anicolaspp/minikaf

Minikaf is a minimal implementation of Publish / Subscriber for in memory communication.

kafka reactive reactive-programming scala stream-processing streams

Last synced: 14 Apr 2025

https://github.com/gamussa/streaming-movie-ratings-min-0519

A small version of of https://github.com/confluentinc/demo-scene/tree/master/streams-movie-demo

apache-kafka groovy java kafka kafka-streams stream-processing

Last synced: 02 Aug 2025

https://github.com/segeljakt/swag

Sliding Window Aggregation

rust-lang stream-processing window-aggregation

Last synced: 07 Mar 2026

https://github.com/ibmstreams/streamsx.hdfs

This toolkit provides operators and functions for interacting with Hadoop File System.

hadoop hdfs ibm-streams java stream-processing toolkit

Last synced: 09 Sep 2025

https://github.com/askmeagain/event-meshinery

A new Framework for asynchronous signaling of processes via Kafka/Mysql/anything.

kafka mysql rest spring-boot stream-processing

Last synced: 13 Apr 2025

https://github.com/ibmstreams/streamsx.hbase

Integration of IBM Streams and Apache HBase

hbase ibm-streams stream-processing toolkit

Last synced: 15 Jul 2025

https://github.com/zensey/go-data-routing

Pipelined data routing/trasnformation engine

data-flow dsl golang pipeline stream-processing

Last synced: 22 Apr 2025

https://github.com/glassflow/example-real-time-ai-alerts

Real-Time Alerts with AI, NATs and Streamlit

ai nats python real-time stream-processing

Last synced: 12 Oct 2025

https://github.com/ibmstreams/streamsx.documentation

Streams documentation project for writing development guides and documentation

documentation ibm-streams stream-processing

Last synced: 04 Jan 2026

https://github.com/ibmstreams/streamsx.tcp

(Incubation) TCPServer is a multi-connection, multi-threaded operator that allows for full duplex TCP communication.

duplex-tcp-communication ibm-streams stream-processing tcp tcp-server tcp-socket toolkit

Last synced: 09 Jul 2025

https://github.com/ibmstreams/streamsx.sparkmllib

Toolkit for real-time scoring using Apache Spark MLLib library

ibm-streams spark spark-mllib-library stream-processing toolkit

Last synced: 15 Jul 2025

https://github.com/iravid/stream-processing-with-scala

Stream Processing with Scala - course exercises

functional-programming scala stream-processing zio

Last synced: 25 Oct 2025

https://github.com/bastion-rs/bastion-streams

Reactive stream processing framework

reactive-programming reactive-streams stream-processing

Last synced: 21 Aug 2025

https://github.com/mkuthan/stream-processing

Learn how to develop and test stateful streaming and batch data pipelines

apache-beam scio stream-processing

Last synced: 07 Apr 2025

https://github.com/marklogic/kafka-marklogic-connector

A Kafka connector for subscribing to topics and streaming messages into MarkLogic

kafka kafka-connector marklogic marklogic-data-hub marklogic-database marklogic-server stream-processing streaming topic

Last synced: 03 Mar 2026

https://github.com/sedgecloud/ECSNeTpp

ECSNeT++ is a simulation framework built on OMNeT++ for Distributed Stream Processing applications running on Edge and Cloud computing environments.

edge-computing omnet-simulation omnetpp simulation-framework stream-processing

Last synced: 11 May 2025

https://github.com/glassflow/glassflow-python-sdk

GlassFlow Python SDK to publish and consume data to your pipelines at Glassflow.dev

data data-processing datastreaming python real-time sdk stream-processing

Last synced: 18 Feb 2026

https://github.com/mihaitodor/benthos-playground

Various https://benthos.dev experiments

benthos go golang kafka low-code stream-processing

Last synced: 15 Jun 2025

https://github.com/stephenlb/eon-spark

Developers can create lovely visuals from live data streams. EON Spark is the visual representation of live streaming data, the way you would imagine. iOS, Android and web browsers.

android data-visualization es6 graph ios pubnub stream stream-processing svg visualization

Last synced: 07 May 2025

https://github.com/point72/csp-adapter-symphony

A csp adapter for symphony

chat chatbot cpp python reactive reactive-programming stream-processing streaming symphony

Last synced: 20 Jul 2025

https://github.com/zertyz/reactive-mutiny

async Event-Driven Reactive Library for Rust with optimized Stream executors -- loosely inspired on SmallRye's Mutiny Java Library

async event-driven reactive-programming rust stream-processing

Last synced: 08 Apr 2026

https://github.com/huangyanbin/smartflow

Android streaming response （Android 流式响应）

android events flow stream-processing

Last synced: 12 Apr 2025

https://github.com/asavinov/bistro

A general-purpose data analysis engine radically changing the way batch and stream data is processed

analytics big-data-analytics edge-analytics iot stream-analytics stream-processing

Last synced: 10 Sep 2025

https://github.com/signicode/humanify

Add human touch to otherwise very machined node.js streams.

human-computer-interaction node-js nodejs reactive-programming stream-processing

Last synced: 01 Jun 2026

https://github.com/nextorigin/sseries-of-tubes

Takes Express/Connect routes and creates stream.Writable endpoints for Server Sent Events

eventemitter express-middleware expressjs keep-alive nodejs-streams server-sent-events sse stream-processing streams streams2 streams3

Last synced: 13 Aug 2025

https://github.com/ibmstreams/streamsx.waterconservation.starterkit

Starter kit for smart and connected sprinkler system using Apache Edgent, Streaming Analytics and Insights for Weather

bluemix edgent ibm-streams iot nodejs raspberry-pi raspberrypi reactjs sensor-data smart-home starterkit stream-processing water-resources watson-iot-platform weather-api

Last synced: 28 Aug 2025

https://github.com/lexvicacom/monoblok

monoblok is a NATS core-style messaging broker with a built-in stream processing DSL

data-streaming dsl edge edge-computing edn financial-data industrial-iot iot market-data monoblok nats pubsub stream-processing telemetry ticker-data trading-servers

Last synced: 24 May 2026

https://github.com/trk54ylmz/musik

Musik: Music recognition application

audio-processing cassandra flink kafka stream-processing

Last synced: 03 Jul 2025

https://github.com/ibmstreams/streamsx.iot

(Incubation) Provide the ability to have an IBM Streams application easily interact with IoTF, either in Bluemix (Streaming Analytics Service) or on-premises (IBM Streams).

edgent ibm-streams iot microservice stream-processing toolkit watson-iot-platform

Last synced: 15 Jul 2025

https://github.com/dylanmei/statefun-examples

Dylan's Flink Stateful Functions examples

flink flink-statefun statefun stream-processing

Last synced: 03 Oct 2025

https://github.com/mikbry/vpiper

Nextgen cross-platform solution to transform, transcode and stream medias in Realtime using hardware acceleration and ML.

android audio ffmpeg framework ios javascript machine-learning mobile-development mp4 rtsp rust stream-processing streaming tensorflow typescript video wasm webgpu webm webrtc

Last synced: 15 Apr 2025

https://github.com/point72/csp-adapter-slack

A csp adapter for slack

chat chatbot cpp python reactive reactive-programming slack slack-sdk stream-processing streaming

Last synced: 01 Apr 2026

https://github.com/martin-majlis/py-probstructs

Collection of probabilistic data structures - CM-Sketch, ECM-Sketch, exponential histogram

data-structures probabilistic-programming python stream-processing

Last synced: 04 Oct 2025

https://github.com/glassflow/cli

GlassFlow CLI to create and manage data pipelines

cli data-pipelines data-transformation real-time stream-processing

Last synced: 13 Nov 2025

https://github.com/doganulus/reelay-codegen

A code generator from high-level formal specifications for monitoring and pattern matching sequential/temporal data.

code-generation data-stream formal-specification monitoring monitoring-tool regular-expression runtime-verification stream-processing temporal-data temporal-logic

Last synced: 05 Apr 2025

https://github.com/ibmstreams/streamsx.document

(Incubation) This toolkit allows extract text and metadata from documents in a binary formats such as PDF, Word, Office, etc

extractor ibm-streams stream-processing

Last synced: 11 Oct 2025

https://github.com/elastiflow/pipelines

A lightweight Go framework for building stateful, real-time data pipelines. It processes both streams and batches with a declarative API to create scalable data applications, all in pure Go.

data-pipeline stream-processing

Last synced: 16 Jan 2026

https://github.com/spencermountain/sunday-driver

be cool with large files

large-files stream-processing

Last synced: 14 Jul 2025

https://github.com/soundvibe/reacto

Functional reactive microservices for the JVM. Uses Observables for composing event streams.

flow functional java microservices observable-streams reactive reactive-streams rxjava stream-processing

Last synced: 21 Apr 2025

https://github.com/akhenakh/geo-bento

Benthos plugins to transform geographic coordinates from a stream.

benthos geographic gis h3 pip point-in-polygon redpanda-connect s2 stream-processing

Last synced: 15 Mar 2026

https://github.com/KMax/cqels

A fork of CQELS (Continuous Query Evaluation over Linked Data)

rdf stream-processing

Last synced: 15 Nov 2025

https://github.com/improvetheworld/dataflow.net

Unified sync/async stream processing with category-based filtering. Single API for IEnumerable/IAsyncEnumerable with Cases/SelectCase/ForEachCase pattern. Eliminates Rx complexity, enables elegant pipeline composition for real-time data processing.

async-await async-enumerable category-filtering csharp data-streaming dotnet event-processing fluent-api functional-programming linq-extensions observables-alternative pipeline-composition- reactive-programming stream-processing