{"id":13476742,"url":"https://github.com/semantalytics/awesome-kafka","last_synced_at":"2025-03-27T04:30:49.777Z","repository":{"id":41225972,"uuid":"135755010","full_name":"semantalytics/awesome-kafka","owner":"semantalytics","description":"A curated list of awesome things related to Apache Kafka","archived":false,"fork":false,"pushed_at":"2024-02-27T15:41:14.000Z","size":171,"stargazers_count":214,"open_issues_count":9,"forks_count":38,"subscribers_count":20,"default_branch":"master","last_synced_at":"2024-05-23T06:22:38.446Z","etag":null,"topics":["awesome","awesome-list","kafka"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semantalytics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"contributing.md","funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2018-06-01T19:17:18.000Z","updated_at":"2024-05-16T05:25:05.000Z","dependencies_parsed_at":"2024-01-14T21:21:15.312Z","dependency_job_id":"1e7b7de4-d3e0-4e8f-895e-bd9594115338","html_url":"https://github.com/semantalytics/awesome-kafka","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semantalytics%2Fawesome-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semantalytics%2Fawesome-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semantalytics%2Fawesome-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semantalytics%2Fawesome-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semantalytics","download_url":"https://codeload.github.com/semantalytics/awesome-kafka/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245784764,"owners_count":20671612,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["awesome","awesome-list","kafka"],"created_at":"2024-07-31T16:01:34.062Z","updated_at":"2025-03-27T04:30:49.769Z","avatar_url":"https://github.com/semantalytics.png","language":null,"funding_links":[],"categories":["Others","Programming Language Lists","Misc"],"sub_categories":["Java Lists"],"readme":"\u003cp align=\"center\"\u003e\n  \u003cbr\u003e\n  \u003cimg width=\"400\" src=\"./awesome%20kafka.png\" alt=\"logo of kafka-awesome repository\"\u003e\n  \u003cbr\u003e\n  \u003cbr\u003e\n\u003c/p\u003e\n\n# Awesome Kafka [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome)\n\n\u003e A curated list of awesome things related to Apache Kafka.\n\n## Contents\n\n- [Libraries](#libraries)\n  - [Kafka](#kafka)\n  - [Kafka UI](#kafka-ui)\n  - [Kafka Streams](#kafka-streams)\n  - [Kafka Connect](#kafka-connect)\n  - [REST Proxy](#rest-proxy)\n  - [KSQL](#ksql)\n  - [Schema Registry](#schema-registry)\n  - [Other Awesome Kafka](#other-awesome-kafka)\n  - [Kafkaesque](#kafkaesque)\n- [Resources](#resources)\n  - [Learning](#learning)\n  - [Blogs](#blogs)\n  - [Books](#books)\n\n\u003c!-- md-parser-start --\u003e\n\n## Libraries\n\n### Kafka\n\n- [topicctl](https://github.com/segmentio/topicctl) - A tool for easy, declarative management of Kafka topics. Includes the ability to \"apply\" topic changes from YAML as well as a repl for interactive exploration of brokers, topics, consumer groups, messages, and more.\n- [sarama](https://github.com/Shopify/sarama) - Sarama is an MIT-licensed Go client library for Apache Kafka version 0.8 (and later).\n- [kadeck](https://www.xeotek.com/kadeck/) - Apache Kafka Monitoring, Kafka UI and data platform for Desktop or Web.\n- [conduktor](https://www.conduktor.io/)\n- [mirus](https://github.com/salesforce/mirus) - Mirus is a cross data-center data replication tool for Apache Kafka.\n- [kareldb](https://github.com/rayokota/kareldb) - A Relational Database Backed by Apache Kafka.\n- [kcache](https://github.com/rayokota/kcache) - An In-Memory Cache Backed by Apache Kafka.\n- [kafka-lag-based-assignor](https://github.com/grantneale/kafka-lag-based-assignor) - Kafka partition assignor that distributes lag evenly across a consumer group.\n- [kafka-pixy](https://github.com/mailgun/kafka-pixy) - Kafka-Pixy is a dual API (gRPC and REST) proxy for Kafka with automatic consumer group control.\n- [Burrow](https://github.com/linkedin/Burrow) - Kafka Consumer Lag Checking.\n- [kafka-tools](https://github.com/linkedin/kafka-tools) - A collection of tools for working with Apache Kafka.\n- [schema-registry-ui](https://github.com/landoop/schema-registry-ui) - Web tool for Avro Schema Registry.\n- [kafkacat](https://github.com/edenhill/kafkacat) - Generic command line non-JVM Apache Kafka producer and consumer.\n- [kafka-jackson](https://github.com/jcustenborder/kafka-jackson) - Kafka Serializer, Deserializer, and Serde for Jackson JSON.\n- [kafkahq](https://github.com/tchiotludo/kafkahq) - Kafka GUI for topics, topics data, consumers group, schema registry, connect and more.\n- [secor](https://github.com/pinterest/secor) - Secor is a service implementing Kafka log persistence.\n- [reactive-kafka](https://github.com/akka/reactive-kafka) - Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.\n- [kafka-monitor](https://github.com/linkedin/kafka-monitor)\n- [chaperone](https://github.com/uber/chaperone) - A Kafka audit system.\n- [uReplicator](https://github.com/uber/uReplicator) - Improvement of Apache Kafka Mirrormaker.\n- [kafka-websocket](https://github.com/b/kafka-websocket)\n- [kafka-unit](https://github.com/chbatey/kafka-unit)\n- [cruise-control](https://github.com/linkedin/cruise-control) - Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a kafka cluster.\n- [jocko](https://github.com/travisjeffery/jocko) - Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native).\n- [hermes](https://github.com/allegro/hermes) - Fast and reliable message broker built on top of Kafka.\n- [kt](https://github.com/fgeller/kt) - Kafka command line tool.\n- [kafka-eagle](https://github.com/smartloli/kafka-eagle) - Used to monitor the consumer status of Kafka clusters, as well as offsets, metadata and other information.\n- [dockerkafka](https://github.com/pinterest/doctorkafka) - DoctorKafka is a service for Kafka cluster auto healing and workload balancing.\n- [kasper](https://github.com/movio/kasper) - Kasper is a lightweight library for processing Kafka topics.\n- [Yelp kafka-utils](https://github.com/Yelp/kafka-utils)\n- [kafka-spark-consumer](https://github.com/dibbhatt/kafka-spark-consumer)\n- [kafka-streams-cep](https://github.com/fhussonnois/kafkastreams-cep) - Complex Event Processing on top of Kafka Streams.\n- [spring-kafka](https://github.com/spring-projects/spring-kafka)\n- [ksql-machine-learning-udf](https://github.com/kaiwaehner/ksql-machine-learning-udf)\n- [kafkabeat](https://github.com/dearcode/kafkabeat) - Elasticsearch Beats for kafka.\n- [sangrenel](https://github.com/jamiealquiza/sangrenel)\n- [Strimzi](https://github.com/strimzi/strimzi-kafka-operator) - Operator for deploying and running Apache Kafka on Kubernetes and OpenShift.\n- [Kafka Manager](https://github.com/yahoo/kafka-manager) - Web-based tool for managing a Kafka cluster.\n- [Strimzi Kafka CLI](https://github.com/systemcraftsman/strimzi-kafka-cli) - A CLI for Strimzi Kafka Operator.\n- [Kafka Cluster Kraft Mode - DockerCompose](https://github.com/minhhungit/kafka-kraft-cluster-docker-compose) - Workable kafka cluster with kraft mode using docker-compose.\n- [Decaton](https://github.com/line/decaton) - High throughput asynchronous task processing framework\n\n### Kafka UI\n\n- [KafkaTrail](https://github.com/imkrishnaagrawal/KafkaTrail)\n- [kafka-topics-ui](https://github.com/lensesio/kafka-topics-ui)\n- [kafka-connect-ui](https://github.com/landoop/kafka-connect-ui) - Web tool for Kafka Connect.\n- [Kafdrop](https://github.com/obsidiandynamics/kafdrop) - Web UI for browsing Kafka topics and consumer groups.\n- [kafka-webview](https://github.com/SourceLabOrg/kafka-webview)\n- [kafka-view](https://github.com/fede1024/kafka-view)\n- [cruise-control-ui](https://github.com/linkedin/cruise-control-ui)\n- [real-time-ui-with-kafka-streams](https://github.com/lucasjellema/real-time-ui-with-kafka-streams)\n- [tsujun](https://github.com/matsumana/tsujun) - Yet another Web UI for KSQL.\n- [kowl](https://github.com/cloudhut/kowl)\n- [akhq](https://akhq.io)\n\n### Kafka Streams\n\n- [schema-registry-ui](https://github.com/lensesio/schema-registry-ui)\n- [kafka-streams-consumer-offsets-to-json](https://github.com/sderosiaux/kafka-streams-consumer-offsets-to-json) - A Kafka Streams process to convert consumer_offsets to a JSON-readable topic.\n- [kafka-operator](https://github.com/krallistic/kafka-operator)\n- [kafkabalancer](https://github.com/CAFxX/kafkabalancer)\n- [mockedstreams](https://github.com/jpzk/mockedstreams)\n- [winton-kafka-streams](https://github.com/wintoncode/winton-kafka-streams) - A Python implementation of Apache Kafka Streams.\n\n### Kafka Connect\n\n- [Apache Camel Kafka Connect](https://camel.apache.org/camel-kafka-connector/latest/index.html) - 340+ Apache Camel components as Kafka Connect connectors.\n- [kafka-connect-client](https://github.com/SourceLabOrg/kafka-connect-client) - A kafka-connect REST api client for java.\n- [Confluent Connector Hub](https://www.confluent.io/product/connectors/)\n- [kafka-connect-protobuf-converter](https://github.com/blueapron/kafka-connect-protobuf-converter) - Protobuf converter plugin for Kafka Connect.\n- [Maxwell](https://github.com/zendesk/maxwell)\n- [kafka-connect-transformers](https://github.com/Landoop/kafka-connect-transformers)\n- [kafka-connect-mq-source](https://github.com/ibm-messaging/kafka-connect-mq-source)\n- [kafka-connect-jenkins](https://github.com/yaravind/kafka-connect-jenkins)\n- [toketi-kafka-connect-iohub](https://github.com/Azure/toketi-kafka-connect-iothub)\n- [connectctl](https://github.com/90poe/connectctl) - Manage kafka connect connectors easily.\n- [kafka-connect-transform-common](https://github.com/jcustenborder/kafka-connect-transform-common) - Common Transforms for Kafka Connect.\n- [aiven-kafka-connect-transforms](https://github.com/aiven/aiven-kafka-connect-transforms) - A collection of Single Message Transformations (SMTs) for Kafka Connect.\n- [kafka-connect-file-pulse](https://github.com/streamthoughts/kafka-connect-file-pulse) - A polyvalent, scalable and reliable, Kafka Connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka.\n- [kafka-connect-http](https://github.com/thomaskwscott/kafka-connect-http)\n- [snowflake-kafka-connector](https://github.com/snowflakedb/snowflake-kafka-connector)\n\n### REST Proxy\n\n- [kafka-rest](https://github.com/confluentinc/kafka-rest) - Confluent REST Proxy.\n- [strimzi-http-bridge](https://github.com/strimzi/strimzi-kafka-bridge) - Strimzi Kafka Bridge (AMQP \u0026 HTTP).\n- [Zilla](https://github.com/aklivity/zilla) - An API gateway built for event-driven architectures and streaming that supports standard protocols such as HTTP, SSE, gRPC, MQTT and the native Kafka protocol.\n\n### KSQL\n\n### Schema Registry\n\n- [apicurio-registry](https://github.com/Apicurio/apicurio-registry) - Apicurio API/schema registry (includes UI).\n- [schema-registry](https://github.com/confluentinc/schema-registry) - Confluent Schema registry for Kafka.\n- [ksql-jdbc-driver](https://github.com/mmolimar/ksql-jdbc-driver)\n- [ballerina-schema-registry](https://github.com/ballerina-platform/module-ballerinax-confluent.cregistry) - Ballerina Confluent Schema Registry\n\n### Other Awesome Kafka\n\n- [ApacheKafka](https://github.com/jsroyal/ApacheKafka)\n- [dharmeshkakadia/awesome-kafka](https://github.com/dharmeshkakadia/awesome-kafka)\n- [infoslack/awesome-kafka](https://github.com/infoslack/awesome-kafka)\n- [awesome-kafka-playground](https://github.com/anascotti/awesome-kafka-playground)\n- [ballerina-avro-kafka-serializer](https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes) - Ballerina Confluent Avro Serializer/Deserializer\n\n### Kafkaesque\n\n- [Pulsar](https://github.com/apache/incubator-pulsar)\n- [Flink](https://flink.apache.org/)\n- [Spark](https://spark.apache.org/)\n- [Heron](https://github.com/apache/incubator-heron)\n- [Beam](https://beam.apache.org/)\n- [Arrow](https://arrow.apache.org/)\n- [Samza](https://samza.apache.org/)\n- [Nats](https://nats.io/)\n- [Chronicle-Queue](https://github.com/OpenHFT/Chronicle-Queue)\n- [CorfuDB](https://github.com/CorfuDB/CorfuDB)\n- [GenStage](https://hexdocs.pm/gen_stage/GenStage.html)\n- [faust](https://github.com/robinhood/faust) - Python Stream Processing.\n- [redpanda](https://vectorized.io/)\n- [Meteor](https://github.com/obsidiandynamics/meteor) - Lightweight, broker-less alternative to Kafka for message streaming.\n\n## Resources\n\n### Learning\n\n- [Passionate Developer: Kafka Streams DSL vs processor API](https://mkuthan.github.io/blog/2017/11/02/kafka-streams-dsl-vs-processor-api/)\n- [Mastering Kafka Streams](https://jaceklaskowski.gitbooks.io/mastering-kafka-streams)\n- [Spark Streaming + Kafka Integration Guide](https://spark.apache.org/docs/latest/streaming-kafka-integration.html)\n\n### Blogs\n\n- [CodingJunkie](http://codingjunkie.net/) - Random Thoughts on Coding by Bill Bejeck.\n- [Understanding Kafka with Factorio]([https://ruurtjan.com/articles/understanding-kafka-with-factorio)\n\n### Books\n\n- [Kafka Streams in Action](https://www.manning.com/books/kafka-streams-in-action)\n- [Kafka in Action](https://www.manning.com/books/kafka-in-action)\n- [Kafka the Definitive Guide](https://shop.oreilly.com/product/0636920044123.do)\n- [Streaming Data Pipelines with Kafka](https://www.manning.com/books/streaming-data-pipelines-with-kafka)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemantalytics%2Fawesome-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemantalytics%2Fawesome-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemantalytics%2Fawesome-kafka/lists"}