{"id":16970147,"url":"https://github.com/erikjiang/kafka_cluster_example","last_synced_at":"2025-03-22T14:31:16.052Z","repository":{"id":89927568,"uuid":"167295549","full_name":"ErikJiang/kafka_cluster_example","owner":"ErikJiang","description":":beetle: kafka cluster example","archived":false,"fork":false,"pushed_at":"2019-02-15T03:32:26.000Z","size":1841,"stargazers_count":40,"open_issues_count":0,"forks_count":9,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-18T12:02:26.551Z","etag":null,"topics":["cluster","docker","kafka","sarama"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ErikJiang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-24T03:21:27.000Z","updated_at":"2025-01-03T21:47:48.000Z","dependencies_parsed_at":"2023-05-30T13:30:33.489Z","dependency_job_id":null,"html_url":"https://github.com/ErikJiang/kafka_cluster_example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ErikJiang%2Fkafka_cluster_example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ErikJiang%2Fkafka_cluster_example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ErikJiang%2Fkafka_cluster_example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ErikJiang%2Fkafka_cluster_example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ErikJiang","download_url":"https://codeload.github.com/ErikJiang/kafka_cluster_example/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244971905,"owners_count":20540881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster","docker","kafka","sarama"],"created_at":"2024-10-14T00:27:31.009Z","updated_at":"2025-03-22T14:31:16.047Z","avatar_url":"https://github.com/ErikJiang.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# :beetle: kafka_cluster_example\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./gopher.png\" /\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Go%20version-1.11-brightgreen.svg\" /\u003e\n  \u003cimg src=\"https://img.shields.io/badge/License-MIT-blue.svg\" /\u003e\n\u003c/p\u003e\n\n---\n\n### 项目获取：\n``` sh\n$ git clone --depth=1 https://github.com/ErikJiang/kafka_cluster_example.git\n```\n\n### 项目依赖工具：\n* [Make](https://www.gnu.org/software/make/)\n* [Kafkacat](https://github.com/edenhill/kafkacat)\n* [ApacheBench](https://httpd.apache.org/docs/2.4/programs/ab.html)\n\n### 支持 make 构建：\n```\n$ make\n\nChoose a command run in kafka_cluster_example:\n\nUsage: make [target]\n\nValid target values are:\n\nvendor                  Auto generate go vendor dir.\nup                      Docker compose up for src.\ndown                    Docker compose down for src.\nps                      Docker compose ps for src.\nlogs                    Docker compose logs for src.\nclean                   Clean up docker images for src.\ntest                    Apache benchmark test for src.\nkafka-up                Docker compose up for kafka services.\nkafka-down              Docker compose down for kafka services.\nkafka-clean             Clean up log and data files for kafka services.\nkafka-test              Check running state of the kafka service.\nhelp                    print this help message and exit.\n```\n\n---\n\n### 1. 配置 hosts 域名\n使用 `ifconfig -a` 查看本地IP地址；\n\n配置 /etc/hosts 文件，将域名 kfk1、kfk2、kfk3 映射到当前本地 IP 地址，例如；\n\n``` sh\n# 假设本地IP为: 192.168.0.166\n192.168.0.166 kfk1 kfk2 kfk3\n```\n\n### 2. 构建 Kafka 集群\n\n``` sh\n# docker compose 构建方式：\n$ docker-compose -f kafka/docker-compose.yml up -d\n\n# 或使用 make 构建方式：\n$ make kafka-up\n```\n\n如若在构建下载过程中，出现等待连接超时，可尝试在 docker 的 `daemon.json` 中添加注册镜像：\n``` json\n{\n    \"registry-mirrors\":[\"https://docker.mirrors.ustc.edu.cn\"]\n}\n```\n\n若构建完成，可使用 kafkacat 检测服务是否正常运行：\n``` sh\n# 直接进行检测验证：\n$ kafkacat -L -b kfk1:19092\n\n# 或使用 make 方式验证：\n$ make kafka-test\n```\n\n### 3. 构建 Produce \u0026 Consume 服务\n\n为 produce 和 consume 生成 vendor 依赖：\n``` sh\n$ make vendor\n```\n\n构建 Produce \u0026 Consume Docker 服务\n``` sh\n# 使用 docker compose 直接构建方式：\n$ docker-compose -f src/docker-compose.yml up -d\n\n# 或者使用 make 构建方式：\n$ make up\n```\n\n### 4. 最终测试\n\n使用 ApacheBench 进行并发测试（并发数为10，总计100个请求）：\n``` sh\n# 直接使用 ab 命令进行测试：\n$ ab -n100 -c10 -T application/json -p test/ab_post_test.json http://127.0.0.1:9000/api/v1/data\n\n# 或者使用 make 方式测试：\n$ make test\n```\n\n---\n\n\u003e 文档详见：[Wiki](https://github.com/ErikJiang/kafka_cluster_example/wiki)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferikjiang%2Fkafka_cluster_example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ferikjiang%2Fkafka_cluster_example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferikjiang%2Fkafka_cluster_example/lists"}