{"id":20678519,"url":"https://github.com/codecentric/elk_docker","last_synced_at":"2025-04-19T21:29:23.891Z","repository":{"id":24923880,"uuid":"28340952","full_name":"codecentric/elk_docker","owner":"codecentric","description":"ELK (Elasticsearch, Logstash, Kibana) as Docker Container","archived":false,"fork":false,"pushed_at":"2015-12-06T16:52:38.000Z","size":34,"stargazers_count":8,"open_issues_count":0,"forks_count":7,"subscribers_count":166,"default_branch":"master","last_synced_at":"2025-03-29T13:35:13.590Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codecentric.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-12-22T14:05:16.000Z","updated_at":"2022-04-13T17:51:36.000Z","dependencies_parsed_at":"2022-08-23T03:50:51.875Z","dependency_job_id":null,"html_url":"https://github.com/codecentric/elk_docker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codecentric%2Felk_docker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codecentric%2Felk_docker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codecentric%2Felk_docker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codecentric%2Felk_docker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codecentric","download_url":"https://codeload.github.com/codecentric/elk_docker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249809755,"owners_count":21328472,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T21:20:40.475Z","updated_at":"2025-04-19T21:29:23.869Z","avatar_url":"https://github.com/codecentric.png","language":"Shell","readme":"elk_docker\n==========\n\nELK (Elasticsearch, Logstash, Kibana) as Docker Container.\nYou can use it for demos or as a development environment e.g.\n\nVersion\n==========\n\nelasticsearch 2.1\n\nlogstash 2.1\n\nkibana 4.3\n\nThis setup allows for watching files in a local directory and log event forwarding over a TLS-secured connection.\n\n## Automatic Container Setup\n\nThis repository contains a Vagrantfile which allows for automated provisioning of a virtual machine and running all defined docker containers inside it.\n\nJust three steps are required:\n\n1. Install [Vagrant](https://www.vagrantup.com/)\n2. Run `vagrant up` in the directory containing the Vagrantfile\n3. Access http://localhost:5601 in your web browser to open kibana\n\nTo monitor and restart docker containers enter the virtual machine via `vagrant ssh`.\n\n## Manual Container Setup\n\nFirst of all, build images from all the Dockerfiles. Therefore, change dir into the subfolders and run\n\n```sh\n  $ cd elasticsearch\n  $ sudo docker build -t elasticsearch .\n  $ cd ../logstash\n  $ sudo docker build -t logstash .\n  $ cd ../kibana\n  $ sudo docker build -t kibana .\n```\n\nThen start a container for Elasticsearch, Logstash, and Kibana, respectively.\n\n```sh\n  $ cd ..\n  $ sudo docker run --name elasticsearch -d -p=9200:9200 elasticsearch\n  $ mkdir logs\n  $ sudo docker run --name logstash -d -p=5000:5000 --link elasticsearch:elasticsearch \\\n      -v `pwd`/logstash/config:/conf -v `pwd`/logs:/var/logstash/logs logstash\n  $ sudo docker run --name kibana -d -p=5601:5601 --link elasticsearch:elasticsearch kibana\n```\n\nFinally, go to http://localhost:5601 to access kibana.\n\n**Container Parameter Details:**\n\n* all containers are named appropriately (`--name`) and run in the background (`-d`).\n* elasticsearch exposes its HTTP service port 9200.\n* logstash exposes port 5000 for (TLS-secured) log forwarding.\n* kibana is accessible on port 5601.\n* the logstash configuration is not embedded in the docker image but mounted as a data volume under '/conf' when the container is started. Thus, the logstash configuration can be updated easily and a restart of the logstash container is sufficient to apply the changes (`docker restart logstash`).\n* a local log file directory 'logs/' is created and mounted in the logstash container. It is watched for any added or updated log files. \n\n**Container Inspection:**\n\nVerify that all containers are running.\n\n```sh\n  $ sudo docker ps -a\n```\n\nCheck for any errors in the logstash container log.\n\n```sh\n  $ sudo docker logs logstash\n```\n\n## Adding Log Messages\n\nThe logstash configuration (logstash/logstash.conf) defines two input options for processing log events:\n\n* via log files in the 'logs/' directory.\n* via TLS-secured log forwarding on port 5000.\n\n### Watching Log Files\n\nThe easiest options for adding log messages is to\n\n* copy a line-based log file to the local 'logs/' directory, e.g.  \n  `sudo cp /var/log/dmesg logs/`\n* append lines of log messages to a log file in the local 'logs/' directory, e.g.  \n  ```\n  sudo chown `whoami` logs \u0026\u0026 echo \"Hello World\" \u003e\u003elogs/test.log\n  ```\n\n### Forwarding Log Events Via TLS\n\nFirst of all, the security certificate needs to be obtained from the logstash docker image.\n\n```sh\nsudo docker cp logstash:/etc/pki/tls/certs/logstash-ca.crt .\n```\n\nOpenSSL can be used for testing. Type following command to open the connection and then enter any lines of log messages.\n\n```sh\nopenssl s_client -quiet -CAfile logstash-ca.crt -connect localhost:5000\n```\n\n[NXlog](http://nxlog.co/products/nxlog-community-edition) is a full-featured log processor which can also forward logs via TLS. A basic NXlog setup is provided in the nxlog container.\n\n```sh\n$ cp logstash-ca.crt nxlog/\n$ cd nxlog\n$ sudo docker build -t nxlog .\n$ cd ..\n$ sudo docker run --name nxlog -d --link logstash:logstash -v `pwd`/nxlog/config:/conf nxlog\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodecentric%2Felk_docker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodecentric%2Felk_docker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodecentric%2Felk_docker/lists"}