Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

Projects in Awesome Lists tagged with kafka

A curated list of projects in awesome lists tagged with kafka .

https://github.com/adyliu/jafka

a fast and simple distributed publish-subscribe messaging system (mq)

jafka-mq jms kafka message-queue mq

Last synced: 31 Jul 2024

https://github.com/dromara/mendmix-cloud

Mendmix定位是一站式分布式开发架构开源解决方案及云原生架构技术底座。Mendmix提供了数据库、缓存、消息中间件、分布式定时任务、安全框架、网关以及主流产商云服务快速集成能力。基于Mendmix可以不用关注技术细节快速搭建高并发高可用基于微服务的分布式架构。

cloud-native distributed framework kafka mybatis redis security spring-web springboot springcloud

Last synced: 27 Sep 2024

https://github.com/hstreamdb/hstream

HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.

data-processing database distributed-database distributed-systems financial-analysis haskell hstreamdb iot iot-database kafka materialized-view real-time realtime-database scale sql stream-processing streaming streaming-data streaming-database

Last synced: 28 Sep 2024

https://github.com/dushixiang/kafka-map

A beautiful, concise and powerful kafka web management tool. 一个美观简洁且强大的kafka web管理工具。

kafka kafka-manager kafka-monitor kafka-ui kafka-viewer kafka-web-ui

Last synced: 01 Aug 2024

https://github.com/chenjiabing666/javafamily

【Java面试+Java学习指南】 一份涵盖大部分Java程序员所需要掌握的核心知识。

java kafka mybatis mysql nacos rabbitmq redis rocketmq sentinel skywalking spring spring-cloud spring-mvc spring-security springboot springboot2

Last synced: 28 Sep 2024

https://github.com/veal98/echo

🦄 论坛:基于 SpringBoot + MyBatis + MySQL + Redis + Kafka + Elasticsearch + Spring Security + ... 并提供详细的开发文档和配套教程。包含帖子、评论、私信、系统通知、点赞、关注、搜索、用户设置、数据统计等模块。

caffeine elasticsearch java kafka mybatis mysql redis spring springboot springmvc springsecurity

Last synced: 28 Sep 2024

https://github.com/ydf0509/funboost

pip install funboost,python全功能分布式函数调度框架,。支持python所有类型的并发模式和一切知名消息队列中间件,支持如 celery dramatiq等框架整体作为funboost中间件,python函数加速器,框架包罗万象,用户能想到的控制功能全都有。一统编程思维,兼容50% python业务场景,适用范围广。只需要一行代码即可分布式执行python一切函数,99%用过funboost的pythoner 感受是 简易 方便 强劲 强大,相见恨晚 。

ack asyncio celery concurrent distributed eventlet gevent kafka multiprocessing nameko nsq python qps rabbitmq redis rocketmq sqlite tcp threading zeromq

Last synced: 31 Jul 2024

https://github.com/kafka4beam/brod

Apache Kafka client library for Erlang/Elixir

client elixir erlang kafka klarna-featured

Last synced: 29 Sep 2024

https://github.com/seglo/kafka-lag-exporter

Monitor Kafka Consumer Group Latency with Kafka Lag Exporter

kafka monitoring prometheus prometheus-exporter

Last synced: 01 Aug 2024

https://github.com/fengzhizi715/netdiscovery

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

coroutines crawler disruptor dsl htmlunit kafka kotlin lettuce middleware redis rxjava2 selenium spider vertx3

Last synced: 28 Sep 2024

https://github.com/fengzhizi715/NetDiscovery

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

coroutines crawler disruptor dsl htmlunit kafka kotlin lettuce middleware redis rxjava2 selenium spider vertx3

Last synced: 02 Aug 2024

https://github.com/pinterest/DoctorK

DoctorK is a service for Kafka cluster auto healing and workload balancing

kafka

Last synced: 02 Aug 2024

https://github.com/gangly/datafaker

Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具

bigdata datafaker fakedata faker hbase hive kafka mysql oracle postgresql python testing

Last synced: 01 Aug 2024

https://github.com/ddd-by-examples/all-things-cqrs

Comprehensive guide to a couple of possible ways of synchronizing two states with Spring tools. Synchronization is shown by separating command and queries in a simple CQRS application.

cqrs ddd debezium domain-driven-design events kafka kafka-connect mongodb spring spring-boot

Last synced: 01 Aug 2024

https://github.com/kafkaex/kafka_ex

Kafka client library for Elixir

elixir erlang kafka

Last synced: 28 Sep 2024

https://github.com/artie-labs/transfer

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.

apache-kafka bigquery cdc change-data-capture data-integration data-pipelines database debezium elt golang kafka redshift snowflake

Last synced: 29 Sep 2024

https://github.com/redpanda-data/kminion

KMinion is a feature-rich Prometheus exporter for Apache Kafka written in Go. It is lightweight and highly configurable so that it will meet your requirements.

consumer consumer-group exporter group kafka lag prometheus prometheus-exporter

Last synced: 01 Aug 2024

https://github.com/Farfetch/kafkaflow

Apache Kafka .NET Framework to create applications simple to use and extend.

apache-kafka c-sharp consumer csharp dotnet dotnetcore farfetch hacktoberfest kafka kafka-client kafka-clients kaflaflow middlewares nuget workers

Last synced: 02 Aug 2024

https://github.com/farfetch/kafkaflow

Apache Kafka .NET Framework to create applications simple to use and extend.

apache-kafka c-sharp consumer csharp dotnet dotnetcore farfetch hacktoberfest kafka kafka-client kafka-clients kaflaflow middlewares nuget workers

Last synced: 30 Sep 2024

https://github.com/TurboWay/spiderman

基于 scrapy-redis 的通用分布式爬虫框架

hbase hive kafka rdbm scapy-redis scrapy spiderman

Last synced: 31 Jul 2024

https://github.com/hemin1003/spring-boot-study

SpringBoot框架源码实战(已更新到springboot2版本实现)~基本用法,Rest,Controller,事件监听,连接数据库MySQL,jpa,redis集成,mybatis集成(声明式与xml两种方式~对应的添删查改功能),日志处理,devtools配置,拦截器用法,资源配置读取,测试集成,Web层实现请求映射,security安全验证,rabbitMq集成,kafka集成,分布式id生成器等。项目实战:https://github.com/hemin1003/yfax-parent 已投入生产线上使用

kafka mybatis oauth2 redis restful-api spring-boot springsecurity

Last synced: 04 Aug 2024

https://github.com/team-ide/teamide

Team IDE 集成MySql、Oracle、金仓、达梦、神通等数据库、SSH、FTP、Redis、Zookeeper、Kafka、Elasticsearch、Mongodb、小工具等管理工具

database elasticsearch file-manager kafka mongodb redis ssh terminal tools zookeeper

Last synced: 01 Aug 2024

https://github.com/thingsboard/tbmq

Open-source, scalable, and fault-tolerant MQTT broker able to handle 4M+ concurrent client connections, supporting at least 3M messages per second throughput per single cluster node with low latency delivery. The cluster mode supports more than 100M concurrently connected clients.

broker iot java kafka mqtt mqtt-broker netty platform thingsboard

Last synced: 30 Sep 2024

https://github.com/dapr/components-contrib

Community driven, reusable components for distributed apps

blob-storage cosmosdb event-hubs kafka mongodb redis sns sqlserver sqs

Last synced: 28 Sep 2024

https://github.com/scality/zenko

Zenko is the open source multi-cloud data controller: own and keep control of your data on any cloud.

artesca aws-s3 azure-storage cloud-native docker-swarm google-cloud grafana helm hybrid-cloud kafka kubernetes mongodb multi-cloud object-storage prometheus redis scality zenko zookeeper

Last synced: 27 Sep 2024

https://github.com/scality/Zenko

Zenko is the open source multi-cloud data controller: own and keep control of your data on any cloud.

artesca aws-s3 azure-storage cloud-native docker-swarm google-cloud grafana helm hybrid-cloud kafka kubernetes mongodb multi-cloud object-storage prometheus redis scality zenko zookeeper

Last synced: 01 Aug 2024

https://github.com/Mongey/terraform-provider-kafka

Terraform provider for managing Apache Kafka Topics + ACLs

kafka kafka-acl kafka-topic terraform terraform-provider

Last synced: 10 Aug 2024

https://github.com/housepower/clickhouse_sinker

Easily load data from kafka to ClickHouse

clickhouse clickhouse-bulk clickhouse-server kafka

Last synced: 02 Aug 2024

https://github.com/eventeum/eventeum

A resilient Ethereum event listener that bridges your smart contract events and backend microservices

blockchain ethereum ethereum-blockchain event-broadcasting event-driven event-listener kafka smart-contracts

Last synced: 03 Aug 2024

https://github.com/DataDog/kafka-kit

Kafka storage rebalancing, automated replication throttle, cluster API and more

golang kafka kafka-kit kafka-topic-management rack-aware rebalance storage

Last synced: 03 Aug 2024

https://github.com/aklivity/zilla

🦎 A multi-protocol, event-native proxy. Securely interface web apps, IoT clients, & microservices to Apache Kafka® via declaratively defined, stateless APIs.

api-gateway asyncapi event-driven-architecture event-stream-proxy event-streaming grpc iot kafka kafka-proxy mqtt openapi proxy rest server-sent-events streaming-api

Last synced: 01 Aug 2024

https://github.com/hong52ni/javaadvance-interview

2022java面试题、JVM面试题、多线程面试题、并发编程、Redis面试题、MySQL面试题、Java2022面试题、Netty面试题、Elasticsearch面试题、Tomcat面试题、Dubbo面试题、Kafka面试题、Linux面试题、2021面试题、java面试、2022面试题,Java面试题、设计模式面试题、Spring面试题、MyBatis面试题、Memcached面试题、MongoDB面试题、ZooKeepe面试题、RabbitMQ面试题、HTML面试题、CSS面试题、Vue面试题、React面试题、JavaScript面试题、2021java面试题、2021java面试、Java2021面试题、2022java面试

css elasticsearch html java js jvm kafka mysql netty rabbitmq redis rocketmq springcloud zookeeper

Last synced: 30 Sep 2024

https://github.com/ankurchavda/streamify

A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

airflow data-engineering dbt gcp kafka python spark

Last synced: 07 Aug 2024

https://github.com/apache/dubbo-go-pixiu

Based on the proxy gateway service of dubbo-go, it solves the problem that the external protocol calls the internal Dubbo cluster. At present, it supports HTTP and gRPC[developing].

api-gateway api-rest cloud-native cloudnative dubbo dubbo-cluster dubbo-go dubbogo gateway gateways grpc grpc-go http-proxies kafka microservice microservices protocols proxy rocketmq upstream-services

Last synced: 30 Sep 2024

https://github.com/grepplabs/kafka-proxy

Proxy connections to Kafka cluster. Connect through SOCKS Proxy, HTTP Proxy or to cluster running in Kubernetes.

kafka kafka-gateway kafka-proxy oauthbearer proxy sasl socks5

Last synced: 03 Aug 2024

https://github.com/zendesk/racecar

Racecar: a simple framework for Kafka consumers in Ruby

kafka kafka-consumer rails ruby

Last synced: 06 Aug 2024

https://github.com/Lancetnik/Propan

Propan is a powerful and easy-to-use Python framework for building event-driven applications that interact with any MQ Broker

amqp asyncapi event-driven kafka messaging nats propan python python-types rabbitmq redis sqs

Last synced: 01 Aug 2024

https://github.com/confluentinc/cp-ansible

Ansible playbooks for the Confluent Platform

ansible confluent kafka kafka-connect ksql playbook rest-proxy schema-registry

Last synced: 03 Aug 2024

https://github.com/linkedin/venice

Venice, Derived Data Platform for Planet-Scale Workloads.

ai database hadoop kafka ml

Last synced: 02 Aug 2024

https://github.com/mhausenblas/dnpipes

Distributed Named Pipes

distributed-systems ipc kafka named-pipes pipe

Last synced: 31 Jul 2024

https://github.com/daydreamdev/seconds-kill

基于 Springboot + Redis + Kafka 的秒杀系统,乐观锁 + 缓存 + 限流 + 异步,TPS 从 500 优化到 3000

druid kafka lombok maven mybatis mysql nginx postman redis springboot

Last synced: 28 Sep 2024

https://github.com/streamnative/kop

Kafka-on-Pulsar - A protocol handler that brings native Kafka protocol to Apache Pulsar

apache-kafka apache-pulsar dotnet event-streaming kafka kafka-on-pulsar kop pub-sub pubsub pulsar

Last synced: 31 Jul 2024

https://github.com/movio/kasper

Kasper is a lightweight library for processing Kafka topics.

golang-library kafka stream-processing

Last synced: 28 Aug 2024

https://github.com/nmaquet/kasper

Kasper is a lightweight library for processing Kafka topics.

golang-library kafka stream-processing

Last synced: 02 Aug 2024

https://github.com/netsampler/goflow2

High performance sFlow/IPFIX/NetFlow Collector

collector go ipfix kafka netflow network protobuf sflow

Last synced: 01 Aug 2024

https://github.com/SuperCowPowers/zat

Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark

bro data-analysis kafka networking pandas python scikit-learn security spark zeek zeek-analysis

Last synced: 07 Aug 2024

https://github.com/landoop/schema-registry-ui

Web tool for Avro Schema Registry |

avro kafka schema-registry

Last synced: 28 Aug 2024

https://github.com/lensesio/schema-registry-ui

Web tool for Avro Schema Registry |

avro kafka schema-registry

Last synced: 02 Aug 2024

https://github.com/pingcap/tiflow

This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)

cdc dm kafka mysql ticdc tidb

Last synced: 01 Aug 2024

https://github.com/AliwareMQ/aliware-kafka-demos

提供各种客户端接入阿里云 消息队列 Kafka 的demo工程(Provide a demo project for various clients to access Alibaba Cloud message queue Kafka)

aliware filebeat go java kafka logstash nodejs php python spring-cloud

Last synced: 31 Jul 2024

https://github.com/Aiven-Open/karapace

Karapace - Your Apache Kafka® essentials in one tool

kafka rest-proxy schema-registry

Last synced: 01 Aug 2024

https://github.com/SourceLabOrg/kafka-webview

Full-featured web-based Apache Kafka consumer UI

admin admin-panel kafka kafka-consumer

Last synced: 02 Aug 2024

https://github.com/langstream/langstream

LangStream. Event-Driven Developer Platform for Building and Running LLM AI Apps. Powered by Kubernetes and Kafka.

genai gpt-4 huggingface kafka kafka-connect kubernetes langchain llamaindex llm openai python vectordb

Last synced: 01 Oct 2024

https://github.com/openmessaging/benchmark

OpenMessaging Benchmark Framework

activemq-artemis kafka nats pulsar rabbitmq rocketmq

Last synced: 01 Aug 2024

https://github.com/LangStream/langstream

LangStream. Event-Driven Developer Platform for Building and Running LLM AI Apps. Powered by Kubernetes and Kafka.

genai gpt-4 huggingface kafka kafka-connect kubernetes langchain llamaindex llm openai python vectordb

Last synced: 01 Aug 2024

https://github.com/minghsu0107/go-random-chat

Modern real-time chat in scalable architecture.

cassandra chat go kafka messaging microservices random-chat real-time redis

Last synced: 29 Sep 2024

https://github.com/Telefonica/prometheus-kafka-adapter

Use Kafka as a remote storage database for Prometheus (remote write only)

cdco kafka kafka-producer prometheus prometheus-adapter

Last synced: 01 Aug 2024

https://github.com/long2ice/synch

Sync data from the other DB to ClickHouse(cluster)

clickhouse data-etl increment-etl kafka mysql postgresql replication

Last synced: 02 Aug 2024

https://github.com/karafka/rdkafka-ruby

Modern and performant Kafka client library for Ruby based on librdkafka

apache-kafka apache-kafka-consumer apache-kafka-producer kafka librdkafka ruby

Last synced: 06 Aug 2024

https://github.com/miguno/wirbelsturm

[PROJECT IS NO LONGER MAINTAINED] Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

apache-kafka apache-spark apache-storm kafka puppet spark storm vagrant

Last synced: 28 Sep 2024

https://github.com/line/decaton

High throughput asynchronous task processing on Apache Kafka

java kafka kafka-consumer

Last synced: 03 Aug 2024

https://github.com/streamthoughts/kafka-connect-file-pulse

🔗 A multipurpose Kafka Connect connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka

amazon-s3 avro azure-storage csv etl file-streaming google-cloud grok-filters kafka kafka-connect kafka-connector kafka-producer xml

Last synced: 02 Aug 2024

https://github.com/amazingwujun/mqttx

MQTTX Project 完整实现 mqttv3.1.1 协议,旨在提供易于使用且性能优异的 mqtt broker

bridge broker cluster docker iot-application kafka mqtt mqtt-broker mqttx netty redis share-topic springboot topic websocket

Last synced: 30 Sep 2024

https://github.com/sderosiaux/every-single-day-i-tldr

A daily digest of the articles or videos I've found interesting, that I want to share with you.

akka architecture bigdata category-theory data-engineering ddd googlecloudplatform java javascript kafka kubernetes microservices reactjs scala spark technology watch

Last synced: 04 Sep 2024

https://github.com/devshawn/kafka-gitops

🚀Manage Apache Kafka topics and generate ACLs through a desired state file.

acl-management acls gitops kafka kafka-acls kafka-cluster kafka-dsf kafka-gitops kafka-manager kafka-topics

Last synced: 03 Aug 2024

https://github.com/raystack/firehose

Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems.

apache-kafka bigquery dataops firehose influxdb kafka postgresql prometheus sink streaming

Last synced: 29 Sep 2024

https://github.com/Amazingwujun/mqttx

MQTTX Project 完整实现 mqttv3.1.1 协议,旨在提供易于使用且性能优异的 mqtt broker

bridge broker cluster docker iot-application kafka mqtt mqtt-broker mqttx netty redis share-topic springboot topic websocket

Last synced: 01 Aug 2024

https://github.com/NHadi/Pos

Sample Application DDD, Reactive Microservices, CQRS Event Sourcing Powered by DERMAYON LIBRARY

cqrs cqrs-es ddd-architecture event-driven event-sourcing generic-repository kafka microservices microservices-architecture netcore reactive

Last synced: 02 Aug 2024

https://github.com/fhussonnois/kafkastreams-cep

Complex Event Processing on top of Kafka Streams

cep complex-event-processing kafka kafka-streams

Last synced: 02 Aug 2024

https://github.com/lensesio/kafka-cheat-sheet

Curated by Lenses.io

kafka kafka-commands

Last synced: 02 Aug 2024

https://github.com/capitalone/cqrs-manager-for-distributed-reactive-services

Experimental CQRS and Event Sourcing service

cqrs event-sourcing kafka

Last synced: 02 Aug 2024

https://github.com/fluent/fluent-plugin-kafka

Kafka input and output plugin for Fluentd

fluentd fluentd-plugin kafka

Last synced: 31 Jul 2024

https://github.com/krallistic/kafka-operator

A Kafka Operator for Kubernetes

kafka kubernetes kubernetes-operator

Last synced: 02 Aug 2024

https://github.com/microsoft/data-accelerator

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

apache-spark azure big-data cosmosdb docker eventhub hdinsight iot iothub kafka kafka-streams nodejs react servicefabric spark spark-sql spark-streaming sparksql streaming streaming-data

Last synced: 28 Sep 2024

https://github.com/nodefluent/node-sinek

:tophat: Most advanced high level Node.js Kafka client

backpressure consumer easy kafka kafka-client kerberos nodejs producer sasl ssl

Last synced: 01 Aug 2024

https://github.com/ozontech/file.d

A blazing fast tool for building data pipelines: read, process and output events. Our community: https://t.me/file_d_community

actions clickhouse elasticsearch events file gelf go http input json kafka logs observability output pipeline processing reading sre throttle tracing

Last synced: 01 Aug 2024

https://github.com/debezium/debezium-ui

A web UI for Debezium; Please log issues at https://issues.redhat.com/browse/DBZ.

change-data-capture debezium kafka

Last synced: 01 Aug 2024

https://github.com/jzyong/gameserver4j

Distributed Java game server, including login, gateway, game demo

game game-server grpc kafka mmo mmorpg-game mmorpg-server netty rpg server spring-boot zookeeper

Last synced: 30 Sep 2024

https://github.com/adevinta/zoe

The Kafka CLI for humans

cli kafka kotlin kubernetes

Last synced: 04 Aug 2024