{"id":17790223,"url":"https://github.com/sudheerj/kafka-cheatsheet","last_synced_at":"2026-03-01T16:01:49.241Z","repository":{"id":254466272,"uuid":"846592015","full_name":"sudheerj/kafka-cheatsheet","owner":"sudheerj","description":null,"archived":false,"fork":false,"pushed_at":"2024-08-24T09:27:35.000Z","size":3,"stargazers_count":19,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-24T09:46:38.011Z","etag":null,"topics":["interview","kafka","kafka-cluster","kafka-topic"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sudheerj.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-23T14:31:15.000Z","updated_at":"2025-09-06T05:02:02.000Z","dependencies_parsed_at":"2024-10-27T10:52:50.408Z","dependency_job_id":"43012bb1-843a-4a25-8209-01ad91c4ce7e","html_url":"https://github.com/sudheerj/kafka-cheatsheet","commit_stats":null,"previous_names":["sudheerj/kafka-cheatsheet"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sudheerj/kafka-cheatsheet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sudheerj%2Fkafka-cheatsheet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sudheerj%2Fkafka-cheatsheet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sudheerj%2Fkafka-cheatsheet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sudheerj%2Fkafka-cheatsheet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sudheerj","download_url":"https://codeload.github.com/sudheerj/kafka-cheatsheet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sudheerj%2Fkafka-cheatsheet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29974321,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-01T15:41:30.362Z","status":"ssl_error","status_checked_at":"2026-03-01T15:37:07.343Z","response_time":124,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["interview","kafka","kafka-cluster","kafka-topic"],"created_at":"2024-10-27T10:41:43.053Z","updated_at":"2026-03-01T16:01:49.218Z","avatar_url":"https://github.com/sudheerj.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# kafka-cheatsheet\n\n## Install Kafka with Zookeeper on Windows\n1. Install WSL2 using `wsl --install`\n\n2. Install Java JDK version 11\n\n3. Download Apache Kafka from https://kafka.apache.org/downloads under Binary Downloads section or use wget command\n\n    ```bash\n    wget https://archive.apache.org/dist/kafka/3.8.0/kafka_2.13-3.8.0.tgz\n    ```\n    Add below system run command in `.bashrc`\n\n    ```bash\n    PATH=\"$PATH:~/kafka_2.13-3.8.0/bin\"\n    ```\n\n4. Extract the contents on WSL2 and setup the $PATH environment variables to easy access the Kafka binaries\n   ```bash\n   tar xzf kafka_2.13-3.0.0.tgz\n   mv kafka_2.13-3.0.0 ~\n   ```\n\n5. Start Zookeeper using the binaries in WSL2\n   ```bash\n   zookeeper-server-start.sh ~/kafka_2.13-3.8.0/config/zookeeper.properties\n   ```\n\n6. Start Kafka using the binaries in an another process in WSL2\n\n   ```bash\n   kafka-server-start.sh ~/kafka_2.13-3.8.0/config/server.properties\n   ```\n\n## Kafka Topics commands\n\n 1. Create:\n\n      ```bash\n      kafka-topics.sh --bootstrap-server localhost:9092 --topic first_topic --create\n\n      kafka-topics.sh --bootstrap-server localhost:9092 --topic second_topic --create --partitions 3\n\n      kafka-topics.sh --bootstrap-server localhost:9092 --topic third_topic --create --partitions 3 --replication-factor 1\n      ```\n\n 2. List:\n   \n      ```bash\n      kafka-topics.sh --bootstrap-server localhost:9092 --list \n      ```\n 3. Describe:\n   \n      ```bash\n      kafka-topics.sh --bootstrap-server localhost:9092 --topic first_topic --describe \n      ```\n 4. Delete:\n   \n      ```bash\n      kafka-topics.sh --bootstrap-server localhost:9092 --topic first_topic --delete // works only when delete.topic.enable=true\n      ```\n\n## Kafka Console Producer commands\n\n   1. Basic producing:\n   \n      ```bash\n      kafka-console-producer.sh --bootstrap-server localhost:9092 --topic first_topic\n      \u003eHello\n      \u003eMay name is Sudheer\n      \u003eThis is a kafka demo\n      ```\n\n   2. Producer with properties:\n\n      ```bash\n       kafka-console-producer.sh --bootstrap-server localhost:9092 --topic first_topic --producer-property acks=all\n      \u003eThe producer supports properties through arguments\n      \u003eLet's broker acknowledge the message\n      ```\n\n   3. Producting to non-existing topic:\n\n      ```bash\n      kafka-console-producer.sh --bootstrap-server localhost:9092 --topic new_topic\n      \u003e hello new topic!\n      ```\n   4. Producting with keys:\n   \n      ```bash\n      kafka-console-producer.sh --bootstrap-server localhost:9092 --topic first_topic --property parse.key=true --property key.separator=*\n      \u003eveggies: potato\n      \u003efruits: banana\n      ```\n\n## Kafka Console Consumer commands\n\n1. Consuming messages:\n\n   ```bash\n   kafka-topics.sh --bootstrap-server localhost:9092 --topic fourth_topic --create --partitions 3\n   kafka-console-producer.sh --bootstrap-server localhost:9092 --producer-property partitioner.class=org.apache.kafka.clients.producer.RoundRobinPartitioner --topic fourth_topic\n   \u003eone\n   \u003etwo\n   \u003ethree\n\n   kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fourth_topic\n   one\n   two\n   three\n   ```\n\n2. Consuming from beginning:\n\n   ```bash\n   kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fourth_topic --from-beginning\n   one\n   three\n   two\n   ```\n\n3. Message formatting with key, value, partitioning and timestamp info:\n\n   ```bash\n   kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fourth_topic --formatter kafka.tools.DefaultMessageFormatter --property print.timestamp=true --property print.key=true --property print.value=true --property print.partition=true --from-beginning\n\n   CreateTime:1724466964735        Partition:1     null    one\n   CreateTime:1724466968330        Partition:0     null    three\n   CreateTime:1724466966554        Partition:2     null    two\n   ``` \n\n## Kafka Console Consumer group commands\n\n**Prerequisites:** Create a new topic with 3 partitions\n\n```bash\nkafka-topics.sh --bootstrap-server localhost:9092 --topic fourth_topic --create --partitions 3\n```\n\n1. Create a first consumer in consumer group:\n```bash\nroot@Sonu:~# kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fourth_topic --group my-first-group\n```\n\n2. Create a producer to produce messages:\n```bash\nkafka-console-producer.sh --bootstrap-server localhost:9092 --producer-property partitioner.class=org.apache.kafka.clients.producer.RoundRobinPartitioner --topic fourth_topic\n\u003eone\n\u003etwo\n\u003ethree\n\u003efour\n```\n\n3. Create more consumers in a group and message spread:\n```bash\nkafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fourth_topic --group my-first-group\none\nfour\nkafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fourth_topic --group my-first-group\nthree\nkafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fourth_topic --group my-first-group\ntwo\n```\n\n4. Create a new group with from beginning option:\n```bash\nkafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fourth_topic --group my-fourth-group --from-beginning\ntwo\none\nfour\nthree\n```\n\n## Kafka Consumer Group commands\n1. List consumer groups\n   \n```bash\nkafka-consumer-groups.sh --bootstrap-server localhost:9092 --list\nmy-fourth-group\nmy-first-group\nmy-third-group\nmy-second-group\n```\n\n2. Describe a consumer group\n\n```bash\nkafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-first-group\n\nGROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                           HOST            CLIENT-ID\nmy-first-group  fourth_topic    0          3               3               0               console-consumer-e6eb8120-3613-4b28-b722-6917474e8d46 /127.0.0.1      console-consumer\nmy-first-group  fourth_topic    1          3               3               0               console-consumer-e6eb8120-3613-4b28-b722-6917474e8d46 /127.0.0.1      console-consumer\nmy-first-group  fourth_topic    2          2               2               0               console-consumer-e91f28d3-403b-45b4-89a7-1fb60566fe95 /127.0.0.1      console-consumer\n```\n\n3. Reset offsets\n\n```bash\nkafka-consumer-groups.sh --bootstrap-server localhost:9092 --group my-first-group --reset-offsets --to-earliest --topic fourth_topic --execute\n\nWARN [AdminClient clientId=adminclient-1] The DescribeTopicPartitions API is not supported, using Metadata API to describe topics. (org.apache.kafka.clients.admin.KafkaAdminClient)\n\nGROUP                          TOPIC                          PARTITION  NEW-OFFSET     \nmy-first-group                 fourth_topic                   0          0              \nmy-first-group                 fourth_topic                   1          0              \nmy-first-group                 fourth_topic                   2          0             \n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsudheerj%2Fkafka-cheatsheet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsudheerj%2Fkafka-cheatsheet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsudheerj%2Fkafka-cheatsheet/lists"}