{"id":17305821,"url":"https://github.com/s8sg/kafka-examples","last_synced_at":"2025-06-25T04:43:50.134Z","repository":{"id":83014176,"uuid":"275527350","full_name":"s8sg/kafka-examples","owner":"s8sg","description":null,"archived":false,"fork":false,"pushed_at":"2020-06-28T14:33:40.000Z","size":5,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-13T19:05:25.903Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/s8sg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-06-28T07:03:21.000Z","updated_at":"2023-03-05T05:06:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"4479e776-27d7-47ac-9fa6-07ce1fa5a217","html_url":"https://github.com/s8sg/kafka-examples","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/s8sg/kafka-examples","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s8sg%2Fkafka-examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s8sg%2Fkafka-examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s8sg%2Fkafka-examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s8sg%2Fkafka-examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/s8sg","download_url":"https://codeload.github.com/s8sg/kafka-examples/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/s8sg%2Fkafka-examples/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261807681,"owners_count":23212660,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-15T11:56:50.841Z","updated_at":"2025-06-25T04:43:49.833Z","avatar_url":"https://github.com/s8sg.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kafka-examples\n\n\n## Consumer Group\n\nSimple example to understand kafka consumer group \n\n### Getting Started \nBuild the docker images for producer and consumer \n```sh\ndocker-compose build \n```\nNow run the kafka \n```sh\ndocker-compose up -d kafka\n```\nThis kafka is configured to auto create the Topics, \nalthough to set the custom partition number we wil manually create it\n\n\nCreate topic `my_topic` with **3** partitions\n```sh\ndocker exec kafka_kafka_1 /opt/kafka_2.11-0.10.1.0/bin/kafka-topics.sh \\\n   --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 \\\n   --topic my_topic\n```\n\n#### Run Consumer \n```sh\ndocker-compose run kafka-consumer\n```\nThis will spin off the 3 consumer  \n\n#### Run Producer\n```sh\ndocker-compose run kafka-producer\n```\nThis will provice an interactive prompt to send messages \n\n### Configuration \nThis example demonstrate how consumer-group can be used to broadcast or equally distributed topics accross multiple consumer \n```yaml\n- \"BROADCAST=false\"\n```\nThis `BROADCAST` flag controls how we assign the consumer group in the example \n#### `BROADCAST=false`\nThis assigns all the consumer in a same consumer group \nwhich will allocate one partiion to each consumer.\n\nIn this case each consumer will only receives message for the parition its asssigned with \n```sh\nmessage received by 2 at topic/partition/offset my_topic/0/2: 1 = pantu\n\nmessage received by 1 at topic/partition/offset my_topic/1/2: 2 = test2\n\nmessage received by 2 at topic/partition/offset my_topic/0/3: 1 = jaja\n\nmessage received by 3 at topic/partition/offset my_topic/2/2: 3 = test3 \n\nmessage received by 2 at topic/partition/offset my_topic/0/4: 4 = test4\n\nmessage received by 3 at topic/partition/offset my_topic/2/3: 6 = test5\n\nmessage received by 1 at topic/partition/offset my_topic/1/3: 5 = empty\n\nmessage received by 1 at topic/partition/offset my_topic/1/4: 7 = test6\n\nmessage received by 2 at topic/partition/offset my_topic/0/5: 8 = test7\n```\n\n#### `BROADCAST=false`\nThis assignes all the consumer in seperate consumer group\nwhich will assign all the partiion to each of the consumer.\n\nIn this case each consumer will receive all messages\n```sh\nmessage received by 3 at topic/partition/offset my_topic/0/3: 1 = jaja\n\nmessage received by 1 at topic/partition/offset my_topic/1/2: 2 = test2\n\nmessage received by 3 at topic/partition/offset my_topic/0/4: 4 = test4\n\nmessage received by 2 at topic/partition/offset my_topic/1/2: 2 = test2\n\nmessage received by 3 at topic/partition/offset my_topic/0/5: 8 = test7\n\nmessage received by 2 at topic/partition/offset my_topic/1/3: 5 = empty\n\nmessage received by 3 at topic/partition/offset my_topic/2/2: 3 = test3 \n\nmessage received by 3 at topic/partition/offset my_topic/2/3: 6 = test5\n\nmessage received by 2 at topic/partition/offset my_topic/1/4: 7 = test6\n\nmessage received by 1 at topic/partition/offset my_topic/1/3: 5 = empty\n\nmessage received by 3 at topic/partition/offset my_topic/1/2: 2 = test2\n\nmessage received by 1 at topic/partition/offset my_topic/1/4: 7 = test6\n\nmessage received by 2 at topic/partition/offset my_topic/0/3: 1 = jaja\n\nmessage received by 1 at topic/partition/offset my_topic/0/3: 1 = jaja\n\nmessage received by 3 at topic/partition/offset my_topic/1/3: 5 = empty\n\nmessage received by 1 at topic/partition/offset my_topic/0/4: 4 = test4\n\nmessage received by 3 at topic/partition/offset my_topic/1/4: 7 = test6\n\nmessage received by 1 at topic/partition/offset my_topic/0/5: 8 = test7\n\nmessage received by 2 at topic/partition/offset my_topic/0/4: 4 = test4\n\nmessage received by 1 at topic/partition/offset my_topic/2/2: 3 = test3 \n\nmessage received by 2 at topic/partition/offset my_topic/0/5: 8 = test7\n\nmessage received by 1 at topic/partition/offset my_topic/2/3: 6 = test5\n\nmessage received by 2 at topic/partition/offset my_topic/2/2: 3 = test3 \n\nmessage received by 2 at topic/partition/offset my_topic/2/3: 6 = test5\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fs8sg%2Fkafka-examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fs8sg%2Fkafka-examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fs8sg%2Fkafka-examples/lists"}