{"id":13990205,"url":"https://github.com/ypereirareis/docker-metricbeat-example","last_synced_at":"2025-03-21T23:34:04.546Z","repository":{"id":46988967,"uuid":"111900018","full_name":"ypereirareis/docker-metricbeat-example","owner":"ypereirareis","description":"Elasticsearch Metricbeat example configuration to monitor Host and Services with docker","archived":false,"fork":false,"pushed_at":"2017-12-01T14:32:24.000Z","size":488,"stargazers_count":83,"open_issues_count":0,"forks_count":30,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-18T06:11:42.786Z","etag":null,"topics":["beats","docker","elasticsearch","kibana","metricbeat","monitoring"],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ypereirareis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-24T09:26:11.000Z","updated_at":"2024-09-06T06:08:44.000Z","dependencies_parsed_at":"2022-09-08T16:52:47.210Z","dependency_job_id":null,"html_url":"https://github.com/ypereirareis/docker-metricbeat-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ypereirareis%2Fdocker-metricbeat-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ypereirareis%2Fdocker-metricbeat-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ypereirareis%2Fdocker-metricbeat-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ypereirareis%2Fdocker-metricbeat-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ypereirareis","download_url":"https://codeload.github.com/ypereirareis/docker-metricbeat-example/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244885399,"owners_count":20526291,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beats","docker","elasticsearch","kibana","metricbeat","monitoring"],"created_at":"2024-08-09T13:02:27.691Z","updated_at":"2025-03-21T23:34:04.184Z","avatar_url":"https://github.com/ypereirareis.png","language":"Makefile","funding_links":[],"categories":["Makefile"],"sub_categories":[],"readme":"# docker-metricbeat-example\n\nDocker and Metricbeat configurations examples.\nLearn how to use Docker and Metricbeat to monitor you host system, all your nodes, your docker containers and all your (distributed) services.\n\n**Warning**\n\u003e This project is built for example and testing.  \n\u003e DO NOT use any part of it in production unless you know exactly what you are doing.\n\n## Software versions\n\n| Name  | Version |\n| ------------- | ------------- |\n| Docker  | 17.10.0-ce (f4ffd25) (Tue Oct 17 19:05:05 2017)  |\n| Docker-compose  | 1.17.0, build ac53b73 |\n| Elasticsearch  | 6.0.0 |\n| Metricbeat  | 6.0.0 |\n| Kibana  | 6.0.0 |\n| Apache | 2.4.29-alpine |\n| MongoDB | 3.5.13-jessie |\n| MySQL | 8.0 |\n| Nginx | 1.13.7-alpine |\n| RabbitMQ | 3.6.14-management-alpine |\n| Redis | 3.2.11-alpine |\n\n\n## Install\n\nClone the repository\n\n```bash\ngit clone git@github.com:ypereirareis/docker-metricbeat-example.git \u0026\u0026 cd docker-metricbeat-example\n```\n\nSetup your host (**sudo/root needed**).\nYou shoud have something like this.\n\n```bash\n$ make setup\n=\u003e ACLs on /var/run/docker.sock OK\nvm.max_map_count = 262144\n=\u003e vm.max_map_count=262144 OK\n2105e36d174f6ad3d0d23cac5769454d60a51fa8998fc0fbe946c4e713f1493c\nBuilding metricbeat-host\nStep 1/6 : FROM docker.elastic.co/beats/metricbeat:6.0.0\n ---\u003e bc8ff400feac\nStep 2/6 : ARG METRICBEAT_FILE=metricbeat.yml\n ---\u003e Using cache\n ---\u003e ebd68e798d93\nStep 3/6 : COPY ${METRICBEAT_FILE} /usr/share/metricbeat/metricbeat.yml\n ---\u003e Using cache\n ---\u003e e24c78b4b7e8\nStep 4/6 : USER root\n ---\u003e Using cache\n ---\u003e 84b29d7e9635\nStep 5/6 : RUN mkdir /var/log/metricbeat     \u0026\u0026 chown metricbeat /usr/share/metricbeat/metricbeat.yml     \u0026\u0026 chmod go-w /usr/share/metricbeat/metricbeat.yml     \u0026\u0026 chown metricbeat /var/log/metricbeat\n ---\u003e Using cache\n ---\u003e 0d1485e2e4c1\nStep 6/6 : USER metricbeat\n ---\u003e Using cache\n ---\u003e 7c5172b2912c\nSuccessfully built 7c5172b2912c\nSuccessfully tagged dockermetricbeatexample_metricbeat-host:latest\nelasticsearch uses an image, skipping\nBuilding metricbeat\nStep 1/6 : FROM docker.elastic.co/beats/metricbeat:6.0.0\n ---\u003e bc8ff400feac\nStep 2/6 : ARG METRICBEAT_FILE=metricbeat.yml\n ---\u003e Using cache\n ---\u003e ebd68e798d93\nStep 3/6 : COPY ${METRICBEAT_FILE} /usr/share/metricbeat/metricbeat.yml\n ---\u003e Using cache\n ---\u003e 19c48873ed8e\nStep 4/6 : USER root\n ---\u003e Using cache\n ---\u003e fa6d8c6138d6\nStep 5/6 : RUN mkdir /var/log/metricbeat     \u0026\u0026 chown metricbeat /usr/share/metricbeat/metricbeat.yml     \u0026\u0026 chmod go-w /usr/share/metricbeat/metricbeat.yml     \u0026\u0026 chown metricbeat /var/log/metricbeat\n ---\u003e Using cache\n ---\u003e b691cc7eb5bf\nStep 6/6 : USER metricbeat\n ---\u003e Using cache\n ---\u003e 5e02571e6fb2\nSuccessfully built 5e02571e6fb2\nSuccessfully tagged dockermetricbeatexample_metricbeat:latest\nkibana uses an image, skipping\n```\n\n## Host Monitoring\n\nWe are assuming your `docker0` interface IP is: `inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0`\nIf not the case, please adjust the configuration in `docker-compose.yml` file for service `metricbeat-host`.\n\n```yaml\nmetricbeat-host:\n  ...\n  environment:\n    - HOST_ELASTICSEARCH=elasticsearch:9222\n    - HOST_KIBANA=kibana:5666\n  extra_hosts:\n    - \"elasticsearch:172.17.0.1\" # The IP of docker0 interface to access host from container\n    - \"kibana:172.17.0.1\" # The IP of docker0 interface to access host from container\n  network_mode: host # Mandatory to monitor host filesystem, memory, processes,...\n```\n\nStart monitoring your host.\n\n```bash\n$ make start-monitoring-host \nCreating metricbeat-elasticsearch ... \nCreating metricbeat-elasticsearch ... done\nmetricbeat-elasticsearch is up-to-date\nCreating metricbeat-kibana ... \nCreating metricbeat-kibana ... done\nWaiting for elasticsearch...\nCreating metricbeat-metricbeat-host ... \nCreating metricbeat-metricbeat-host ... done\n```\n\n* You can check everything is OK, and you should have 3 containers running...\n* Be careful Elasticsearch and Kibana ports are exposed on 0.0.0.0 network (every IP address).\n* Default Metricbeat dashboard are automatically loaded into Kibana (`setup.dashboards.enabled: true`)\n\n```bash\n$ docker ps                 \nCONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                              NAMES\ne25a76b4f1e4        dockermetricbeatexample_metricbeat-host               \"/usr/local/bin/do...\"   2 minutes ago       Up 2 seconds                                           metricbeat-metricbeat-host\n27668d971ddf        docker.elastic.co/kibana/kibana:6.0.0                 \"/bin/bash /usr/lo...\"   2 minutes ago       Up 2 minutes        0.0.0.0:5666-\u003e5601/tcp             metricbeat-kibana\naf93d8214167        docker.elastic.co/elasticsearch/elasticsearch:6.0.0   \"/usr/local/bin/do...\"   2 minutes ago       Up 2 minutes        9300/tcp, 0.0.0.0:9222-\u003e9200/tcp   metricbeat-elasticsearch\n```\n\nIf everything is fine, you should be able to access Kibana, and Monitoring dashboard:\n\n* Kibana =\u003e [http://127.0.0.1:5666](http://127.0.0.1:5666/app/kibana)\n* Dashboard list =\u003e [http://127.0.0.1:5666/app/kibana#/dashboards?_g=()](http://127.0.0.1:5666/app/kibana#/dashboards?_g=())\n* System Overview =\u003e [http://127.0.0.1:5666/app/kibana#/dashboard/Metricbeat-system-overview?_g=()](http://127.0.0.1:5666/app/kibana#/dashboard/Metricbeat-system-overview?_g=())\n\n**Host Dashboard**\n\n![Host dashboard](./img/host.png)\n\n**Docker Dashboard**\n\n![Docker dashboard](./img/docker.png)\n\n## Services Monitoring\n\nFirst of all you need to start the Metricbeat container to monitor services.\n\n```bash\nmake start-monitoring\n```\n\nYou should now have an extra Metricbeat container for services (called `metricbeat-metricbeat-services`).\n\n```bash\n$ docker ps\nCONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                              NAMES\nd934ee72db19        dockermetricbeatexample_metricbeat                    \"/usr/local/bin/do...\"   4 minutes ago       Up 4 minutes                                           metricbeat-metricbeat-services\n...\n```\n\n### Start/Stop all services\n\n```bash\nmake start-all\nmake stop-all\n```\n\n### Start/Stop services one by one\n\n#### Apache\n\n* Compose file: [./docker-compose.apache.yml](./docker-compose.apache.yml)\n* Start: `make start-apache`\n* Stop: `make stop-apache`\n\n```bash\n4f4cb6e72a39    httpd:2.4   \"httpd-foreground\"  11 minutes ago  Up About a minute   80/tcp  metricbeat-apache\n```\n\n#### MongoDB\n\n* Compose file: [./docker-compose.mongodb.yml](./docker-compose.mongodb.yml)\n* Start: `make start-mongodb`\n* Stop: `make stop-mongodb`\n\n```bash\na379163bc90d    mongo   \"docker-entrypoint...\"   11 minutes ago    Up About a minute   27017/tcp    metricbeat-mongodb\n```\n\n#### MySQL\n\n* Compose file: [./docker-compose.mysql.yml](./docker-compose.mysql.yml)\n* Start: `make start-mysql`\n* Stop: `make stop-mysql`\n\n```bash\n3ae8e0c7c1e3    mysql:8.0   \"docker-entrypoint...\"   11 minutes ago   Up About a minute   3306/tcp metricbeat-mysql\n```\n\n#### Nginx\n\n* Compose file: [./docker-compose.nginx.yml](./docker-compose.nginx.yml)\n* Start: `make start-nginx`\n* Stop: `make stop-nginx`\n\n```bash\nda1a01c36c4c    nginx:1.13.7-alpine \"nginx -g 'daemon ...\"   11 minutes ago   Up About a minute   80/tcp  metricbeat-nginx\n```\n\n#### RabbitMq\n\n* Compose file: [./docker-compose.rabbitmq.yml](./docker-compose.rabbitmq.yml)\n* Start: `make start-rabbitmq`\n* Stop: `make stop-rabbitmq`\n\n```bash\n6c072660a008    bijukunjummen/rabbitmq-server:3.6.5 \"/bin/sh -c /opt/r...\"   14 minutes ago      Up 2 seconds   4369/tcp, 5672/tcp, 9100-9105/tcp, 25672/tcp, 127.0.0.1:1234-\u003e15672/tcp   metricbeat-rabbitmq\n```\n\n#### Redis\n\n* Compose file: [./docker-compose.redis.yml](./docker-compose.redis.yml)\n* Start: `make start-redis`\n* Stop: `make stop-redis`\n\n```bash\n52be9d662f8f    redis:3.2.11-alpine \"docker-entrypoint...\"   11 minutes ago Up About a minute   6379/tcp    metricbeat-redis\n```\n\n#### Others\n\nMore to come\n\n## Clean everything\n\nA simple command to remove all containers related to Metricbeat\n\n```bash\n$ make clean\nd934ee72db19\n3ae8e0c7c1e3\na379163bc90d\n6c072660a008\n52be9d662f8f\nda1a01c36c4c\n4f4cb6e72a39\na400ee0b914a\n43d22a57fde4\nc1ca7e0d7e6a\nAll METRICBEAT containers removed !\n```\n\n## (Re)start everything\n\n```bash\n$ make install\n```\n\n## LICENSE\n\nMIT License\n\nCopyright (c) 2017 Yannick Pereira-Reis\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fypereirareis%2Fdocker-metricbeat-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fypereirareis%2Fdocker-metricbeat-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fypereirareis%2Fdocker-metricbeat-example/lists"}