{"id":15875981,"url":"https://github.com/max-lobur/centralized-container-logging","last_synced_at":"2025-04-01T23:22:59.303Z","repository":{"id":93833998,"uuid":"87780425","full_name":"max-lobur/centralized-container-logging","owner":"max-lobur","description":"A demo of centralized container logging via Logspout","archived":false,"fork":false,"pushed_at":"2017-04-10T07:42:18.000Z","size":4,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-27T09:04:09.104Z","etag":null,"topics":["containers","docker","logging","logspout","rsyslog"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/max-lobur.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-10T07:32:04.000Z","updated_at":"2017-04-20T23:14:32.000Z","dependencies_parsed_at":"2023-06-15T07:15:28.383Z","dependency_job_id":null,"html_url":"https://github.com/max-lobur/centralized-container-logging","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/max-lobur%2Fcentralized-container-logging","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/max-lobur%2Fcentralized-container-logging/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/max-lobur%2Fcentralized-container-logging/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/max-lobur%2Fcentralized-container-logging/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/max-lobur","download_url":"https://codeload.github.com/max-lobur/centralized-container-logging/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246725710,"owners_count":20823676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["containers","docker","logging","logspout","rsyslog"],"created_at":"2024-10-06T02:00:48.786Z","updated_at":"2025-04-01T23:22:59.279Z","avatar_url":"https://github.com/max-lobur.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Inputs\nUsing docker, setup three instances that communicate to each other.\n\n* a docker instance running rsyslog (centralized logging server)\n* a docker instance running rsyslog which forwards to the centralized logging server (logging agent)\n* a docker instance running nginx serving static content (app server)\n\nEnsure that the app server's access logs are forwarded to the centralized logging server via the logging agent.\n\nThe ideal solution here should be re-usable. In production we would want to run one logging agent and many app server instances together on any host. This would provide us with an easy way to ship logs for each container from each docker host to the centralized logging service.\n\n# Requirements\n1. Include the ability to easily configure the address of the centralized logging server (it could change so just re-configuring the logging agent should be easy).\n2. Be able to easily choose what files we want to ship to the logging service.\n\n# Running the solution\nCurrently 3 services are merged into one yaml for convenience. It has comments about how it would be split in a real-world scenario.\n\n1. Start: `docker-compose -f rsyslog.yml up -d`\n2. Generate web logs: `curl localhost:8080`\n3. Observe log server logs: `docker logs centralizedcontainerlogging_log-server_1`\n4. Observe locally saved logs: `cat ./logs/*`\n5. Stop `docker-compose -f rsyslog.yml down`\n\n# Adressing Requirements\n1. Log server is linked via [docker built-in service discovery](https://docs.docker.com/docker-cloud/apps/service-links/), e.g. referred by\nname \"log-server\".\n2. Logspout used as a log-agent is only capturing container's stdout/stderr, which\nI believe is correct because a container should run no more than 1 process. Unlike\na \"Sidecar\" approach you cannot list specific files here. However multiple message-level filters are available:\n * [Logspout filters](https://github.com/gliderlabs/logspout#including-specific-containers) (which I used to select \\_web\\_)\n * [Logspout ignore](https://github.com/gliderlabs/logspout#ignoring-specific-containers)\n * [Rsyslog selectors]( http://www.rsyslog.com/doc/v8-stable/configuration/filters.html#selectors)\n\n\n # Extra credit\nCentralized logging is available out-of-box in kubernetes: [fluentd + elasticsearch + grafana/kibana](https://deis.com/blog/2016/kubernetes-logging-with-elasticsearch-and-kibana/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmax-lobur%2Fcentralized-container-logging","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmax-lobur%2Fcentralized-container-logging","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmax-lobur%2Fcentralized-container-logging/lists"}