awesome-kafka

Everything about Apache Kafka
https://github.com/dharmeshkakadia/awesome-kafka

Last synced: about 18 hours ago
JSON representation

Development
- Client libraries
  - Node
  - others
  - Php
  - Python (kafka-python)
  - Python (asyncio)
  - Karafka - Ruby and Rails framework
  - Elixir
  - Node Kafka streams
  - Node
  - Go Kafka streaming
  - Php
  - Go
  - Go
  - Node
  - Node librdkafka
  - Perl
  - Python (pykafka)
  - Ruby
  - Phobos - Ruby App framework
  - Dart
  - JRuby
  - Erlang
  - LinkedIn wrapper library
  - Go
  - Rust
  - Node
  - Phobos - Ruby App framework
  - Erlang
- Testing
- Kafka Streams Libraries
- Producers
  - Maxwell's daemon - a mysql-to-json kafka producer
- Consumers
  - Spark
  - Mapreduce
  - Divolte
  - Azure Data Explorer/Kusto
  - Firehose - Firehose is an extensible, no-code, and cloud-native service to load real-time streaming data from Kafka to data stores, data lakes, and analytical storage systems.
  - Raccoon - Raccoon is a high-throughput, low-latency service to collect events in real-time from your web, mobile apps, and services using multiple network protocols and publish to Kafka.
- KSQL
  - KSQL Addon - User Defined Function (UDF) for Machine Learning
  - KSQL JDBC Driver
- Connectors
- Transformations
  - Dagger - Dagger is an easy-to-use, SQL framework to process built on top of Apache Flink for stateful processing of Kafka data.
Operations
- Mirroring
- UI
- Schema Management
  - Schema registry
  - Node Schema registry
  - Schema Registry Transfer SMT
  - Expedia Stream Registry
  - Stencil - Stencil is a Protobuf schema registry for Kafka that enables you to create, maintain and consume protobuf messages and APIs dynamically, efficiently, and reliably.
  - Schema Registry Transfer SMT
- Operational Utilities
- Monitoring
- Tools
- Performance tools
- CI/CD
  - kafka-gitops
- Security
  - Kafka security playbook
- Backup
  - Backup and Restore topics & offsets
- Metadata Management
  - Go library for Kafka metadata in Zookeeper
  - Meteor - Meteor is a scalable, easy-to-use, extensible metadata collection framework from the different cloud providers and on-prem sources and publish to Kafka.
Deployment
- Integrations
  - Ecosystem
  - Brooklin
  - hermes
  - IBMStreams
- Kubernetes
- Docker
  - fast-data-dev
Alternative Implementations
- Integrations
  - Go
  - C/C++
Projects built on top of Kafka
- Integrations
  - In-Memory Cache Backed by Kafka
  - ransactional Metadata Store Backed by Kafka
Unsorted
- Integrations
  - Azkarra Streams - lightweight java framework which makes easy to create production-ready Kafka Streams applications

Programming Languages

Java 51 Go 17 Scala 10 JavaScript 8 Python 7 Ruby 6 Shell 3 Erlang 3 TypeScript 2 HTML 2

Categories

Development 71 Operations 66 Deployment 8 Projects built on top of Kafka 2 Alternative Implementations 2 Unsorted 1

Sub Categories

Client libraries 28 Tools 20 Connectors 20 UI 10 Testing 10 Integrations 9 Operational Utilities 9 Monitoring 8 Consumers 6 Schema Management 6 Mirroring 5 Kafka Streams Libraries 3 Performance tools 3 Kubernetes 3 KSQL 2 Metadata Management 2 Backup 1 Security 1 Producers 1 Transformations 1 Docker 1 CI/CD 1

Keywords

kafka 70 kafka-connect 18 apache-kafka 11 confluent 8 kafka-streams 8 kafka-client 8 kafka-connector 7 kubernetes 6 nodejs 5 messaging 5 java 5 schema-registry 5 big-data 4 etl 4 stream-processing 4 python 4 distributed-systems 4 golang 4 kafka-consumer 4 json 4 kafka-cluster 4 avro 4 topic 3 kubernetes-operator 3 ruby 3 kafka-manager 3 streaming 3 kafka-producer 3 go 3 scala 3 streams 3 event-streaming 3 protobuf 3 schema 3 cli 3 monitoring 3 google-cloud 2 driver 2 jdbc 2 metrics 2 microservices 2 erlang 2 elixir 2 openshift 2 klarna-featured 2 lag 2 data-streaming 2 hadoop 2 hdfs 2 zookeeper 2