{"id":18489751,"url":"https://github.com/abicky/fluentd-daemon-service-example","last_synced_at":"2026-01-24T14:44:56.148Z","repository":{"id":139218276,"uuid":"350152560","full_name":"abicky/fluentd-daemon-service-example","owner":"abicky","description":"This is a sample application to send docker container logs to fluentd aggregators via NLB or fluentd forwarders","archived":false,"fork":false,"pushed_at":"2021-03-24T19:32:22.000Z","size":8,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-14T00:38:44.209Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/abicky.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-22T00:03:25.000Z","updated_at":"2021-03-24T19:32:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"6e95ed00-a1ea-467e-a841-b6b5dffdcca7","html_url":"https://github.com/abicky/fluentd-daemon-service-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/abicky/fluentd-daemon-service-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abicky%2Ffluentd-daemon-service-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abicky%2Ffluentd-daemon-service-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abicky%2Ffluentd-daemon-service-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abicky%2Ffluentd-daemon-service-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/abicky","download_url":"https://codeload.github.com/abicky/fluentd-daemon-service-example/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abicky%2Ffluentd-daemon-service-example/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28730175,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-24T10:24:43.181Z","status":"ssl_error","status_checked_at":"2026-01-24T10:24:36.112Z","response_time":89,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T12:57:44.579Z","updated_at":"2026-01-24T14:44:56.132Z","avatar_url":"https://github.com/abicky.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fluentd-daemon-service-example\n\nThis is a sample application to send docker container logs to fluentd aggregators via NLB or fluentd forwarders.\n\n\n## How to use\n\n### Create fluentd services and task definition for application to send logs\n\n```\n./deploy.sh\n```\n\n### Send container logs\n\nWith the following command, you can send docker container logs via NLB for about 3 minutes:\n\n```\n./run.sh aggregator\n```\n\nYou can also send logs via fluentd forwarders with the following command:\n\n```\n./run.sh forwarder\n```\n\n\n## Example result\n\n### Send logs via NLB\n\nThe following are the logs of the fluentd aggregators after `./run.sh aggregator` was executed:\n\n```\nfluentd/fluentd/e21167dd1ed144d5909b2e4adb5b5dc3 2021-03-21 21:46:00.895331357 +0000 flowcount: {\"count\":2624,\"count_rate\":43.37,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/e68726eff1da443f84948d8525ab148c 2021-03-21 21:46:03.873949662 +0000 flowcount: {\"count\":1160,\"count_rate\":19.17,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/a55d1d27fbd9469c81dc2b25a79e984a 2021-03-21 21:46:04.160516079 +0000 flowcount: {\"count\":5828,\"count_rate\":96.33,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/777cb168dd3e4f8eb2dc8cd4c517be8b 2021-03-21 21:46:05.556735127 +0000 flowcount: {\"count\":1306,\"count_rate\":21.58,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/e21167dd1ed144d5909b2e4adb5b5dc3 2021-03-21 21:47:00.895819993 +0000 flowcount: {\"count\":17799,\"count_rate\":296.64,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/e68726eff1da443f84948d8525ab148c 2021-03-21 21:47:03.874672300 +0000 flowcount: {\"count\":5932,\"count_rate\":98.86,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/a55d1d27fbd9469c81dc2b25a79e984a 2021-03-21 21:47:04.660120893 +0000 flowcount: {\"count\":29910,\"count_rate\":494.38,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/777cb168dd3e4f8eb2dc8cd4c517be8b 2021-03-21 21:47:06.056316981 +0000 flowcount: {\"count\":5982,\"count_rate\":98.87,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/e21167dd1ed144d5909b2e4adb5b5dc3 2021-03-21 21:48:01.395749103 +0000 flowcount: {\"count\":17950,\"count_rate\":296.69,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/e68726eff1da443f84948d8525ab148c 2021-03-21 21:48:04.373710915 +0000 flowcount: {\"count\":5984,\"count_rate\":98.91,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/a55d1d27fbd9469c81dc2b25a79e984a 2021-03-21 21:48:05.160610260 +0000 flowcount: {\"count\":29917,\"count_rate\":494.49,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/777cb168dd3e4f8eb2dc8cd4c517be8b 2021-03-21 21:48:06.556881706 +0000 flowcount: {\"count\":5983,\"count_rate\":98.89,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/e21167dd1ed144d5909b2e4adb5b5dc3 2021-03-21 21:49:01.894901215 +0000 flowcount: {\"count\":15633,\"count_rate\":258.4,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/e68726eff1da443f84948d8525ab148c 2021-03-21 21:49:04.374282396 +0000 flowcount: {\"count\":4926,\"count_rate\":82.09,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/a55d1d27fbd9469c81dc2b25a79e984a 2021-03-21 21:49:05.660895237 +0000 flowcount: {\"count\":24355,\"count_rate\":402.56,\"tag\":\"docker.application-using-aggregator\"}\nfluentd/fluentd/777cb168dd3e4f8eb2dc8cd4c517be8b 2021-03-21 21:49:07.056829769 +0000 flowcount: {\"count\":4731,\"count_rate\":78.19,\"tag\":\"docker.application-using-aggregator\"}\n```\n\nThe number of logs each task processed is summarized as follow:\n\nTask ID | Count\n----------|----------------\n777cb168dd3e4f8eb2dc8cd4c517be8b | 18002\na55d1d27fbd9469c81dc2b25a79e984a | 90010\ne21167dd1ed144d5909b2e4adb5b5dc3 | 54006\ne68726eff1da443f84948d8525ab148c | 18002\n\n\nOn the other hand, the following are the logs after the command `./run.sh forwarder`:\n\n```\nfluentd/fluentd/e21167dd1ed144d5909b2e4adb5b5dc3 2021-03-21 21:39:58.395912823 +0000 flowcount: {\"count\":2969,\"count_rate\":49.07,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/e68726eff1da443f84948d8525ab148c 2021-03-21 21:40:00.874669417 +0000 flowcount: {\"count\":2973,\"count_rate\":49.54,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/a55d1d27fbd9469c81dc2b25a79e984a 2021-03-21 21:40:01.660638178 +0000 flowcount: {\"count\":5360,\"count_rate\":89.33,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/777cb168dd3e4f8eb2dc8cd4c517be8b 2021-03-21 21:40:02.556869094 +0000 flowcount: {\"count\":4946,\"count_rate\":81.75,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/e21167dd1ed144d5909b2e4adb5b5dc3 2021-03-21 21:40:58.895504102 +0000 flowcount: {\"count\":11869,\"count_rate\":196.18,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/e68726eff1da443f84948d8525ab148c 2021-03-21 21:41:01.374179512 +0000 flowcount: {\"count\":18801,\"count_rate\":310.76,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/a55d1d27fbd9469c81dc2b25a79e984a 2021-03-21 21:41:02.160881563 +0000 flowcount: {\"count\":10889,\"count_rate\":179.98,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/777cb168dd3e4f8eb2dc8cd4c517be8b 2021-03-21 21:41:03.056295483 +0000 flowcount: {\"count\":16822,\"count_rate\":278.05,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/e21167dd1ed144d5909b2e4adb5b5dc3 2021-03-21 21:41:59.395674810 +0000 flowcount: {\"count\":14827,\"count_rate\":245.07,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/e68726eff1da443f84948d8525ab148c 2021-03-21 21:42:01.873815546 +0000 flowcount: {\"count\":13841,\"count_rate\":228.77,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/a55d1d27fbd9469c81dc2b25a79e984a 2021-03-21 21:42:02.660226027 +0000 flowcount: {\"count\":19797,\"count_rate\":327.22,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/777cb168dd3e4f8eb2dc8cd4c517be8b 2021-03-21 21:42:03.556922470 +0000 flowcount: {\"count\":10880,\"count_rate\":179.83,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/e21167dd1ed144d5909b2e4adb5b5dc3 2021-03-21 21:42:59.895220205 +0000 flowcount: {\"count\":10887,\"count_rate\":179.95,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/e68726eff1da443f84948d8525ab148c 2021-03-21 21:43:02.374503816 +0000 flowcount: {\"count\":6462,\"count_rate\":106.8,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/a55d1d27fbd9469c81dc2b25a79e984a 2021-03-21 21:43:02.660722801 +0000 flowcount: {\"count\":11877,\"count_rate\":197.94,\"tag\":\"docker.application-using-forwarder\"}\nfluentd/fluentd/777cb168dd3e4f8eb2dc8cd4c517be8b 2021-03-21 21:43:04.057112091 +0000 flowcount: {\"count\":16820,\"count_rate\":278.01,\"tag\":\"docker.application-using-forwarder\"}\n```\n\nAs you can see below, each task processed logs more evenly:\n\nTask ID | Count\n----------|----------------\n777cb168dd3e4f8eb2dc8cd4c517be8b | 49468\na55d1d27fbd9469c81dc2b25a79e984a | 47923\ne21167dd1ed144d5909b2e4adb5b5dc3 | 40552\ne68726eff1da443f84948d8525ab148c | 42077\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabicky%2Ffluentd-daemon-service-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabicky%2Ffluentd-daemon-service-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabicky%2Ffluentd-daemon-service-example/lists"}