{"id":13509969,"url":"https://github.com/AliyunContainerService/log-pilot","last_synced_at":"2025-03-30T15:30:29.156Z","repository":{"id":39905952,"uuid":"81317754","full_name":"AliyunContainerService/log-pilot","owner":"AliyunContainerService","description":"Collect logs for docker containers","archived":true,"fork":false,"pushed_at":"2022-08-21T15:12:09.000Z","size":13223,"stargazers_count":1432,"open_issues_count":117,"forks_count":402,"subscribers_count":56,"default_branch":"master","last_synced_at":"2025-03-30T12:34:38.808Z","etag":null,"topics":["container","docker","filebeat","fluentd","kubernetes","log-pilot","logging"],"latest_commit_sha":null,"homepage":"https://yq.aliyun.com/articles/674327","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AliyunContainerService.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-02-08T10:28:54.000Z","updated_at":"2025-03-20T07:00:40.000Z","dependencies_parsed_at":"2022-07-15T06:47:20.307Z","dependency_job_id":null,"html_url":"https://github.com/AliyunContainerService/log-pilot","commit_stats":null,"previous_names":["aliyuncontainerservice/fluentd-pilot","jzwlqx/fluentd-pilot"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AliyunContainerService%2Flog-pilot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AliyunContainerService%2Flog-pilot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AliyunContainerService%2Flog-pilot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AliyunContainerService%2Flog-pilot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AliyunContainerService","download_url":"https://codeload.github.com/AliyunContainerService/log-pilot/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246338564,"owners_count":20761402,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["container","docker","filebeat","fluentd","kubernetes","log-pilot","logging"],"created_at":"2024-08-01T02:01:19.278Z","updated_at":"2025-03-30T15:30:24.145Z","avatar_url":"https://github.com/AliyunContainerService.png","language":"Go","funding_links":[],"categories":["Misc","Logging","Go","Operation System","Interaction","kubernetes"],"sub_categories":["Server Infrastructure and High Performance Computing","High Performance Computing"],"readme":"log-pilot\n=========\n\n[![CircleCI](https://circleci.com/gh/AliyunContainerService/log-pilot.svg?style=svg)](https://circleci.com/gh/AliyunContainerService/log-pilot)\n[![Go Report Card](https://goreportcard.com/badge/github.com/AliyunContainerService/log-pilot)](https://goreportcard.com/report/github.com/AliyunContainerService/log-pilot)\n\n`log-pilot` is an awesome docker log tool. With `log-pilot` you can collect logs from docker hosts and send them to your centralized log system such as elasticsearch, graylog2, awsog and etc. `log-pilot` can collect not only docker stdout but also log file that inside docker containers.\n\nTry it\n======\n\nPrerequisites:\n\n- docker-compose \u003e= 1.6\n- Docker Engine \u003e= 1.10\n\n```\n# download log-pilot project\ngit clone git@github.com:AliyunContainerService/log-pilot.git\n# build log-pilot image\ncd log-pilot/ \u0026\u0026 ./build-image.sh\n# quick start\ncd quickstart/ \u0026\u0026 ./run\n```\n\nThen access kibana under the tips. You will find that tomcat's has been collected and sended to kibana.\n\nCreate index:\n![kibana](quickstart/Kibana.png)\n\nQuery the logs:\n![kibana](quickstart/Kibana2.png)\n\nQuickstart\n==========\n\n### Run pilot\n\n```\ndocker run --rm -it \\\n    -v /var/run/docker.sock:/var/run/docker.sock \\\n    -v /etc/localtime:/etc/localtime \\\n    -v /:/host:ro \\\n    --cap-add SYS_ADMIN \\\n    registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.5-filebeat\n```\n\n### Run applications whose logs need to be collected\n\nOpen a new terminal, run the application. With tomcat for example:\n\n```\ndocker run -it --rm  -p 10080:8080 \\\n    -v /usr/local/tomcat/logs \\\n    --label aliyun.logs.catalina=stdout \\\n    --label aliyun.logs.access=/usr/local/tomcat/logs/localhost_access_log.*.txt \\\n    tomcat\n```\n\nNow watch the output of log-pilot. You will find that log-pilot get all tomcat's startup logs. If you access tomcat with your broswer, access logs in `/usr/local/tomcat/logs/localhost_access_log.\\*.txt` will also be displayed in log-pilot's output.\n\nMore Info: [Fluentd Plugin](docs/fluentd/docs.md) and [Filebeat Plugin](docs/filebeat/docs.md)\n\nFeature\n========\n\n- Support both [fluentd plugin](docs/fluentd/docs.md) and [filebeat plugin](docs/filebeat/docs.md). You don't need to create new fluentd or filebeat process for every docker container.\n- Support both stdout and log files. Either docker log driver or logspout can only collect stdout.\n- Declarative configuration. You need do nothing but declare the logs you want to collect.\n- Support many log management: elastichsearch, graylog2, awslogs and more.\n- Tags. You could add tags on the logs collected, and later filter by tags in log management.\n\nBuild log-pilot\n===================\n\nPrerequisites:\n\n- Go \u003e= 1.6\n\n```\ngo get github.com/AliyunContainerService/log-pilot\ncd $GOPATH/github.com/AliyunContainerService/log-pilot\n# This will create a new docker image named log-pilot:latest\n./build-image.sh\n```\n\nContribute\n==========\n\nYou are welcome to make new issues and pull reuqests.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAliyunContainerService%2Flog-pilot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAliyunContainerService%2Flog-pilot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAliyunContainerService%2Flog-pilot/lists"}