Projects in Awesome Lists tagged with redpanda

https://github.com/redpanda-data/redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

containers cpp event-driven go kafka kubernetes microservices realtime redpanda seastar storage-engine stream-processing streaming

Last synced: 22 Apr 2026

https://github.com/twmb/franz-go

franz-go contains a feature complete, pure Go library for interacting with Kafka from 0.8.0 through 3.8+. Producing, consuming, transacting, administrating, etc.

client go golang kafka kafka-client redpanda

Last synced: 11 May 2025

https://github.com/timeplus-io/proton

High-performance, low-footprint SQL database written in C++. Process millions of rows per second from Kafka/Pulsar, Iceberg, or ClickHouse, and seamlessly write results back. Supports powerful features like JOIN, CDC, UPSERT, and LOOKUP, enabling real-time analytics and ETL at scale.

analytics clickhouse confluent cpp flink-alternative high-performance iceberg kakfa ksqldb-alternative redpanda simd single-binary sql stream-processing streaming-sql udf

Last synced: 14 May 2025

https://github.com/silverton-io/buz

Serverless multi-protocol + multi-destination event collection system.

analytics analytics-tracking cloudevents cloudevents-schema contracts data data-collection data-platform eventbridge jsonschema product-analytics redpanda redpanda-console schema-registry schema-validation snowplow-analytics streaming-analytics streaming-data webhook-receiver webhook-server

Last synced: 12 Apr 2025

https://github.com/bogdanp/franz

A desktop client for Apache Kafka.

apache-kafka kafka kafka-client racket redpanda swift swiftui

Last synced: 05 Apr 2025

https://github.com/redpanda-data/deployment-automation

Cluster configuration best practices

ansible redpanda

Last synced: 22 Jun 2025

https://github.com/paulescu/incremental-ml-training-and-serving

Incremental ML learning in the real-world

ml online-ml python redpanda stream

Last synced: 27 Mar 2025

https://github.com/maxhalford/taxi-demo-rp-mz-rv-rd-st

🚕 Self-contained demo using Redpanda, Materialize, River, Redis, and Streamlit to predict taxi trip durations

machine-learning materialize online-learning real-time redpanda river stream-processing streamlit

Last synced: 04 Aug 2025

https://github.com/singlestore-labs/singlestore-logistics-sim

Scalable package delivery logistics simulator built using SingleStore and Vectorized Redpanda

avro demo kafka logistics redpanda simulation singlestore vectorized

Last synced: 06 Feb 2026

https://github.com/sha3sha3/UE-EasyKafka

EasyKafka is a Kafka/Redpanda client sub-system for unreal engine. It supports producing and consuming records through blueprint and C++.

cloud kafka microservice microservices networking redpanda unreal unreal-engine unreal-engine-4 unreal-engine-5 unrealengine

Last synced: 08 Apr 2025

https://github.com/marketcalls/sketchmaker

Self Hostable - Digital Creative Machine for your Teams

chatgpt4o daisyui flask flux flux-dev flux-realism fluxpro fluxpro-ultra lora python recraftv3 redpanda sqlitedb tailwindcss

Last synced: 13 Aug 2025

https://github.com/redpanda-data/redpanda-edge-agent

Lightweight internet of things agent that forwards events from the edge

arm edge iot kafka redpanda

Last synced: 22 Jun 2025

https://github.com/flolu/kafka

Realtime BTC and ETH Wallet Tracker with Node.js and Kafka

bitcoin consumer ethereum kafka message-queue nodejs partitioning producer publish-subscribe realtime redpanda topics typescript websockets

Last synced: 03 Oct 2025

https://github.com/singlestore-labs/singlestore-workshop-data-intensive-app

This repo provides a starting point for building applications using SingleStore, Redpanda (by Vectorized), and the Go language. SingleStore is a scale-out relational database built for data-intensive workloads. Redpanda is a Kafka API compatible streaming platform for mission-critical workloads created by the team at Vectorized.

database docker getting-started go golang memsql quickstart redpanda singlestore sql tutorial vectorized workshop

Last synced: 06 Feb 2026

https://github.com/mneedham/pinot-wiki

kafka pinot pulsar redpanda wikipedia

Last synced: 29 Jun 2025

https://github.com/hardiksinghbehl/just-another-testcontainer-integration

Backend application using Testcontainers for Integration Testing AWS S3, Azure Blob Storage, Kafka, MySQL, Egress calls including Chaos testing and Redis.

azurite chaos-testing java kafka-testing localstack mockserver mysql-test redis-test redpanda spring-boot testcontainers toxiproxy

Last synced: 09 Apr 2025

https://github.com/iamazy/kafkas

async kafka client in pure Rust

async-std kafka redpanda rust tokio

Last synced: 12 Apr 2025

https://github.com/redpanda-data/redpanda-examples

A collection of examples to demonstrate how to interact with Redpanda from various clients and languages.

example kafka redpanda

Last synced: 22 Jun 2025

https://github.com/kafclaw/kafclaw

🦾 Ultra-compact OpenClaw in Go + Kafka enabled. Operate a distributed swarm.

ai ai-agents ai-assistant amazon-msk confluent-kafka kafka kafscale openclaw own-your-data redpanda

Last synced: 15 Mar 2026

https://github.com/redpanda-data/docs

Open source content for the Redpanda documentation

antora docs redpanda streaming streaming-data

Last synced: 02 Feb 2026

https://github.com/sdairs/bluebird

Pushes the Bluesky Firehose to various data services

bluesky kafka redpanda tinybird

Last synced: 18 Jul 2025

https://github.com/xdgrulez/streampunk

Polyglot Programmable Shell for Kafka

confluent debugging event-streaming graalvm java javascript kafka oracle python r redpanda repl ruby scripting wasm

Last synced: 12 Oct 2025

https://github.com/milenkovicm/testcontainers-redpanda-rs

Rust Redpanda Testcontainer

docker kafka redpanda rust testcontainer testcontainers testcontainers-rust

Last synced: 28 Oct 2025

https://github.com/quackscience/duckdb-extension-kafquack

Experimental DuckDB Kafka Consumer Extension

duckdb kafka kafka-consumer redpanda

Last synced: 26 Jul 2025

https://github.com/kameshsampath/grpc-todo-app

A demo to show how to get started with building Data Streaming API using https://grpc.de) and https://redpanda.com, and how to add analytics using https://snowflake.com e.g. a https://streamlit.io dashboard.

demo-app grpc-client grpc-go grpc-golang grpc-server-streaming kafka-connector protobuf3 redpanda snowflake

Last synced: 22 Apr 2025

https://github.com/ren294/covid-data-process

This project integrates real-time data processing and analytics using Apache NiFi, Kafka, Spark, Hive, and AWS services for comprehensive COVID-19 data insights.

airflow aws aws-ec2 aws-quicksight big-data big-data-analytics covid19-data docker docker-compose hadoop-hdfs hdfs hive kafka nifi pipeline redpanda spark spark-sql spark-streaming sparksql

Last synced: 29 Oct 2025

https://github.com/redpanda-data/cloud-docs

Redpanda Cloud documentation

antora docs redpanda streaming streaming-data

Last synced: 25 Oct 2025

https://github.com/voutilad/redpanda-pytorch-demo

Putting a PyTorch ML model into production with Redpanda Connect

huggingface ml python pytorch redpanda redpanda-connect sentiment-analysis

Last synced: 09 Feb 2026

https://github.com/memgraph/ecommerce-recommender-demo

E-Commerce recommender demo with real-time data and a graph database

apache-pulsar memgraph pulsar real-time recommendation-engine recommendation-system redpanda

Last synced: 25 Jun 2025

https://github.com/kameshsampath/rp-k3d

Scripts to provision Kubernetes(k3s) cluster using https://k3d.io and deploy https://redpanda.com cluster on to it

k3d redpanda

Last synced: 27 Jun 2025

https://github.com/rafaelrok/ms-order-pattern-saga-orchestrator

Projeto de microservices no padrão Saga de maneira de gerenciar a consistência de dados entre microservices em cenários de transação distribuída1. Uma saga é uma sequência de transações que atualiza cada serviço e publica uma mensagem ou evento para disparar a próxima etapa de transação utilizando o kafka.

api-rest docker docker-compose java kafka kafka-consumer kafka-producer micro postgresql redpanda spring-boot

Last synced: 03 Aug 2025

https://github.com/ivpal/cdc-realtime-search

Project demonstrates realtime index updates via change data capture

change-data-capture debezium elasticsearch kafka-connect redpanda spring-boot webflux

Last synced: 30 Apr 2026

https://github.com/danthelion/twitter-trending

A small example visualizing live Twitter trending data using Redpanda and Materialize

kafka materialize metabase redpanda stream-processing

Last synced: 13 Mar 2026

https://github.com/s0rg/clickhouse-kafka-compose

Example docker-compose with clickhouse and kafka or redpanda as event sources

clickhouse docker docker-compose example kafka redpanda

Last synced: 03 May 2026

https://github.com/nenodias/redpanda-docker-compose

Redpanda docker compose example

docker-compose kafka redpanda

Last synced: 29 Apr 2026

https://github.com/gordonmurray/debezium_redpanda

Docker compose for trying out Debezium, RedPanda and RedPanda Console UI

debezium mariadb redpanda redpanda-console

Last synced: 26 Mar 2025

https://github.com/dipjyotimetia/eventshark

kafka event streaming

k6 kafka kafka-go kafka-producer redpanda redpanda-console

Last synced: 27 Jul 2025

https://github.com/marvin-hansen/iggy-streaming-system

Sample streaming project for iggy.rs

kafka messaging mqtt redpanda rust streaming

Last synced: 02 Aug 2025

https://github.com/chukwuemekaaham/data-engineering-zoomcamp

Datatalks Club Free Data Engineering Zoomcamp Project

bigquery dbt docker-compose duckdb gcp gcp-cloud-storage github-actions jupyter-notebook kafka linux looker-studio mageai pandas postgresql prefect python redpanda risingwave spark terraform

Last synced: 24 Feb 2026

https://github.com/kuro337/scalamono

Scala Monorepo Tooling for Kafka, Opensearch, Spark, Redpanda, Hadoop - and Lang Reference.

data database duckdb hadoop kafka redpanda sdala spark

Last synced: 13 Apr 2026

https://github.com/hrbrmstr/skygrep

A real-time Bluesky Jetstream firehose consumer that filters and forwards posts to Kafka topics based on configurable rules.

bluesky bluesky-firehose bluesky-jetstream deno docker docker-compose kafka redpanda

Last synced: 13 Apr 2026

https://github.com/polyseam/deno-kafka-demo

integrating Deno and KafkaJS for educational purposes

deno kafka kafkajs redpanda typescript

Last synced: 19 May 2026

https://github.com/cludden/benthos-plugin-temporal

a Temporal integration for benthos/redpanda-connect/bento

benthos benthos-plugin bento redpanda temporal

Last synced: 08 Apr 2025

https://github.com/deemkeen/clusterlite

A lightweight distributed database system combining SQLite and Kafka for event-driven replication across multiple nodes

clustering cqrs event-sourcing golang kafka redpanda replication sqlite

Last synced: 11 Feb 2026

https://github.com/nightmarcher/redis-df

Aim to define schema and read redis to DataFrame, inspired by sqlalchemy.

dataframe orm-like pandas python3 redis redpanda sqlalchemy

Last synced: 03 May 2026

https://github.com/voutilad/redpanda-flight-rs

An Apache Arrow Flight proxy for Redpanda

apache-arrow kafka redpanda

Last synced: 09 May 2026

https://github.com/bobbyiliev/materialize-emulator-subscribe-poc

docker kafka materialize python redpanda sql streaming-data

Last synced: 09 May 2026

https://github.com/hy0g0/crypto-data

This project aim to build a reliable application that can handle a lot of crypto data then display it in dashboard

clickhouse grafana kafka python redpanda

Last synced: 29 Apr 2026

https://github.com/ngunyigachie/real_time_energy_logger

docker-compose flask kafka redis redpanda

Last synced: 08 Apr 2026

https://github.com/smhmayboudi/cdc-example

cdc changedatacapture cockroach cockroachdb connectors console debezium devcontainer go golang golang-migrate materialize migrate redpanda

Last synced: 08 Oct 2025

https://github.com/kameshsampath/go-datastreaming-quickstart

A demo quickstart to build an simple datastreaming API application with https://echo.labstack.com and https://readpanda.com

data-streaming-examples demo golang quickstart redpanda

Last synced: 29 Mar 2025

https://github.com/kameshsampath/rp-dev-gke

Terraform scripts to setup Redpanda dev cluster setup on GKE

development-environment gke-cluster redpanda terraform

Last synced: 29 Mar 2025

https://github.com/hassonor/nrt-e2e-data-warehouses

apache-airflow apache-kafka apache-pinot apache-superset celery financial-institutions postgresql redpanda

Last synced: 18 May 2026

https://github.com/ivpal/prefixy

Project demonstrates how to get and update autocomplete suggestions

autocomplete kafka quarkus redis redpanda suggestions

Last synced: 19 Jan 2026

https://github.com/4thel00z/task-manager

Bash task worker written in Go. Uses redpanda for queues and sqlite for storage. Very fast, and low mem footprint.

cli fast go low-memory low-memory-footprint redpanda sqlite task task-queue

Last synced: 01 Feb 2026

https://github.com/abdelilahou/redpanda-go

Set up mysql cdc with debezium connect and redpanda

cdc-data debezium-connector golang kafka mysql redpanda redpanda-console

Last synced: 13 May 2026

https://github.com/abombalemba/cpaums

Cross-platform application update management system

api crud docker java jenkins jwt maven minio mvc postgresql redpanda spring-boot stateless swagger telegrambot

Last synced: 15 Apr 2026

https://github.com/datarocks-ag/redpanda-provisioner

A Go CLI tool that idempotently provisions Redpanda/Kafka resources from a YAML config file. Designed as a Docker Compose init container or Kubernetes Job.

docker go kafka privisioner redpanda

Last synced: 11 Jun 2026

https://github.com/dipakdotyadav/redpanda-proxy

High-performance C# proxy for RedPanda/Kafka clusters with load balancing, health monitoring, and single endpoint access for clients

csharp distributed-systems dotnet dotnet-core kafka load-balancer microservices proxy redpanda tcp-proxy

Last synced: 18 May 2026

https://github.com/shiningflash/kafka-python-messaging-engine

A Python-based Kafka messaging engine with Avro serialization, Schema Registry integration, and Redpanda support for real-time data streaming.

avro confluent-kafka data-pipeline data-streaming event-driven event-driven-architecture kafka kafka-python python real-time-data real-time-messaging redpanda schema-registry

Last synced: 15 May 2026

https://github.com/relaxe111/cdc-pipeline-generator

Reusable library for generating Redpanda Connect CDC pipelines.

cdc-pipeline-console-generator py-script redpanda redpanda-connect

Last synced: 10 May 2026

https://github.com/hhromic/kadumper

Kafka Avro dumping utility written in Go.

avro dumper golang json kafka redpanda schema-registry

Last synced: 12 Feb 2026

https://github.com/oragazz0/nexus-event-stream

This repository demonstrates concepts and the implementation of Event-Driven Architecture (EDA), Polyglot Microservices (Python + Go) and high-performance Message Streaming.

django django-rest-framework docker docker-compose event-driven go golang microservices postgresql python redpanda

Last synced: 14 Feb 2026

https://github.com/mukeshmithrakumar/missing-semester

The Missing Semester of My CS Education

beam distributed-systems microservice redpanda rust streaming

Last synced: 17 Apr 2026

https://github.com/tsnsoft/mysettinglocale

Программа с локализацией на нужный язык для wxWidgets и RedPanda-CPP

cpp redpanda wxwidgets

Last synced: 04 Apr 2026

https://github.com/eddumelendez/spring-boot-redpanda-testcontainers-reusable-mode

redpanda spring-boot testcontainers

Last synced: 24 Apr 2026

https://github.com/impossibleforge/pfc-kafka-consumer

Kafka consumer that compresses log messages directly to PFC format

compression confluent kafka log-management logs observability pfc-jsonl python redpanda s3

Last synced: 25 Apr 2026

https://github.com/voutilad/rpcn-cdc

A demo of using Debezium CDC with Redpanda Connect.

cdc debezium redpanda redpanda-connect

Last synced: 07 Jun 2026

https://github.com/gordonmurray/terraform_rds_debezium_kafka_ksqldb

Create an environment on AWS using Terraform and Ansible to show a Change Data Capture (CDC) process working end to end from MariaDB on RDS to Redis on Elasticache, using Debezium and ksqlDB.

aws debezium kafka ksqldb msk redpanda terraform warpstream