{"id":24801470,"url":"https://github.com/zkfmapf123/kafka-configure","last_synced_at":"2025-03-25T03:55:43.518Z","repository":{"id":271221347,"uuid":"910692682","full_name":"zkfmapf123/Kafka-Configure","owner":"zkfmapf123","description":"kafka-configure","archived":false,"fork":false,"pushed_at":"2025-01-18T16:44:58.000Z","size":882,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-25T03:55:39.125Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zkfmapf123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-01T04:59:07.000Z","updated_at":"2025-01-19T02:34:02.000Z","dependencies_parsed_at":"2025-01-06T11:33:56.804Z","dependency_job_id":"f508b828-8204-4d45-b6c8-997316c4eaa9","html_url":"https://github.com/zkfmapf123/Kafka-Configure","commit_stats":null,"previous_names":["zkfmapf123/content-delivery","zkfmapf123/kafka-configure"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkfmapf123%2FKafka-Configure","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkfmapf123%2FKafka-Configure/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkfmapf123%2FKafka-Configure/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkfmapf123%2FKafka-Configure/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zkfmapf123","download_url":"https://codeload.github.com/zkfmapf123/Kafka-Configure/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245394770,"owners_count":20608123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-30T04:28:48.480Z","updated_at":"2025-03-25T03:55:43.498Z","avatar_url":"https://github.com/zkfmapf123.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Content-Delivery\n\n- [x] API\n- [x] Kafka\n- [x] Consumer Server\n- [x] Database\n- [x] fluentbit\n- [x] prometheus\n- [x] loki\n- [ ] tempo\n- [x] grafana\n- [ ] grafana -\u003e metric dashboard\n- [ ] grafana -\u003e trace dashboard\n- [ ] grafana -\u003e log dashboard\n\n## Docker-Compose\n\n- [docker-compose.yml](./docker-compose.yml) api 서버\n- [docker-compose.kafka.yml](./docker-compose.kafka.yml) kafka + ui\n- [docker-compose.log.yml](./docker-compose.log.yml) prometheus + loki + grafana + open telemetry\n\n# Huge Traffic Architecture Use Kafka\n\n![asis](./public/asis.png)\n\n- 대용량에 트래픽이 발생하게 될때 -\u003e \u003cb\u003eDB 모든 부하가 발생함\u003c/b\u003e\n- DB 자체에 장애가 발생한다면 모든시스템에 장애가 발생할 수 있음 (SPOF)\n\n![tobe](./public/tobe.png)\n\n## LGTM Stack\n\n![ff](./public/fluentbit.png)\n\n### 로그수집\n\n- fluentbit + loki\n\n### 메트릭 수집\n\n- fluentbit + prometheus + mimir\n\n### 분산 Tracing\n\n- application + tempo\n\n## 참고\n\n- \u003ca href=\"https://medium.com/@dudwls96/opentelemetry-grafana-loki-tempo-prometheus%EB%A5%BC-%ED%99%9C%EC%9A%A9%ED%95%9C-golang-observability-%EA%B5%AC%EC%84%B1%ED%95%98%EA%B8%B0-3cf148039c1c\"\u003e Golang Tempo 구성하기 \u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzkfmapf123%2Fkafka-configure","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzkfmapf123%2Fkafka-configure","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzkfmapf123%2Fkafka-configure/lists"}