{"id":15065987,"url":"https://github.com/azhinu/elastic-stack","last_synced_at":"2026-02-18T16:31:42.129Z","repository":{"id":65253392,"uuid":"439050759","full_name":"azhinu/elastic-stack","owner":"azhinu","description":"Production-ready docker compose project for Elastic Stack (Elasticsearch, Kibana, Logstash)","archived":false,"fork":false,"pushed_at":"2023-02-10T09:39:42.000Z","size":2212,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-26T21:03:25.472Z","etag":null,"topics":["docker-compose","elasticsearch","elasticstack","elk"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/azhinu.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-16T16:08:23.000Z","updated_at":"2022-12-14T10:09:47.000Z","dependencies_parsed_at":"2024-06-20T11:15:04.794Z","dependency_job_id":"ace091e0-4802-4d3d-ad48-94ff46ab0134","html_url":"https://github.com/azhinu/elastic-stack","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":"deviantony/docker-elk","purl":"pkg:github/azhinu/elastic-stack","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azhinu%2Felastic-stack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azhinu%2Felastic-stack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azhinu%2Felastic-stack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azhinu%2Felastic-stack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/azhinu","download_url":"https://codeload.github.com/azhinu/elastic-stack/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azhinu%2Felastic-stack/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29585548,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T13:56:48.962Z","status":"ssl_error","status_checked_at":"2026-02-18T13:54:34.145Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker-compose","elasticsearch","elasticstack","elk"],"created_at":"2024-09-25T00:59:03.745Z","updated_at":"2026-02-18T16:31:42.107Z","avatar_url":"https://github.com/azhinu.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Elastic stack on Docker\n\n[![Elastic Stack version](https://img.shields.io/badge/Elastic%20Stack-8.2.0-00bfb3?style=flat\u0026logo=elastic-stack)](https://www.elastic.co/blog/category/releases)\n\nRun the [Elastic stack](https://www.elastic.co/what-is/elk-stack) with Docker Compose.\n\nIt gives you the ability to analyze any data set by using the searching/aggregation capabilities of Elasticsearch and\nthe visualization power of Kibana.\n\n![Animated demo](https://user-images.githubusercontent.com/3299086/140641708-cea70d17-cc04-459f-89d9-3fcb5c58bc35.gif)\n\nUses the official Docker images from Elastic:\n\n* [Elasticsearch](https://github.com/elastic/elasticsearch/tree/master/distribution/docker)\n* [Logstash](https://github.com/elastic/logstash/tree/main/docker)\n* [Kibana](https://github.com/elastic/kibana/tree/master/src/dev/build/tasks/os_packages/docker_generator)\n* [Beats](https://github.com/elastic/beats/tree/main/deploy/docker)\n* [Elastic Agent](https://github.com/elastic/elastic-agent)\n\nAnd built from sources\n* [Elastic package registry](https://github.com/elastic/package-registry)\n---\n\n\n## Contents\n\n1. [Features](#features)\n1. [Requirements](#requirements)\n   * [Host setup](#host-setup)\n   * [Docker Desktop](#docker-desktop)\n      * [Windows](#windows)\n      * [macOS](#macos)\n1. [Usage](#usage)\n    * [Prepare docker host](#prepare-docker-host)\n    * [Initial setup](#initial-setup)\n    * [Docker network driver](#docker-network-driver)\n    * [Cleanup](#cleanup)\n    * [Access Kibana](#access-kibana)\n    * [Default Kibana index pattern creation](#default-kibana-index-pattern-creation)\n      * [Via the Kibana web UI](#via-the-kibana-web-ui)\n      * [On the command line](#on-the-command-line)\n1. [Configuration](#configuration)\n    * [How to configure Elasticsearch](#how-to-configure-elasticsearch)\n    * [How to configure Kibana](#how-to-configure-kibana)\n      * [Kibana TLS](#kibana-tls)\n    * [How to configure Logstash](#how-to-configure-logstash)\n    * [How to configure Beats](#how-to-configure-beats)\n    * [How to configure Fleet server](#how-to-configure-fleet-server)\n    * [How to Elastic registry](#elastic-registry)\n    * [How to scale out the Elasticsearch cluster](#how-to-scale-out-the-elasticsearch-cluster)\n    * [Healthcheck](#healthcheck)\n2. [Extensibility](#extensibility)\n    * [How to add plugins](#how-to-add-plugins)\n3. [JVM tuning](#jvm-tuning)\n    * [How to specify the amount of memory used by a service](#how-to-specify-the-amount-of-memory-used-by-a-service)\n    * [How to enable a remote JMX connection to a service](#how-to-enable-a-remote-jmx-connection-to-a-service)\n7. [Going further](#going-further)\n    * [Swarm mode](#swarm-mode)\n\n\n## Features\n\nThis repository based at [deviantony/docker-elk](https://github.com/deviantony/docker-elk/), but adapted to my own requirements. The main goal of this project is running production-ready single node Elasticsearch instance.\n\n**Comparing to original repo:**\n\n1. Using original container images. This time I don't use plugins and see no point to build custom images.\n2. Using `basic` license by default.  \n3. Enabled bootstrap checks.\n4. Enabled TLS and X-Pack security features.\n5. Configured container memory ulimits according to Elasticsearch documentation.\n6. Added healthcheck scripts.\n7. Added Logstash pipelines config file binding.\n\n\n## Requirements\n### Host setup\n\n* [Docker Engine](https://docs.docker.com/install/) version **18.06** or newer\n* [Docker Compose](https://docs.docker.com/compose/install/) version **1.26.0** or newer\n*:information_source: Following instructions assumes that you are using Docker compose V2. If you use legacy docker-compose, use `docker-compose` instead of `docker compose`.*\n* 3 GB of RAM\n\n*:information_source: Especially on Linux, make sure your user has the [required permissions](https://docs.docker.com/install/linux/linux-postinstall/) to\ninteract with the Docker daemon.*\n*:information_source: [Change Java heap](#how-to-specify-the-amount-of-memory-used-by-a-service) with your requirements.*\n**:warning: Docker-compose commands below assume using docker-compose v2.**\n\n### Docker Desktop\n#### Windows\n\nIf you are using the legacy Hyper-V mode of _Docker Desktop for Windows_, ensure [File Sharing](https://docs.docker.com/desktop/windows/#file-sharing) is\nenabled for the `C:` drive.\n\n#### macOS\n\nThe default configuration of _Docker Desktop for Mac_ allows mounting files from `/Users/`, `/Volume/`, `/private/`,\n`/tmp` and `/var/folders` exclusively. Make sure the repository is cloned in one of those locations, or follow the\ninstructions from the [documentation](https://docs.docker.com/desktop/mac/#file-sharing) to add more locations.\n\n## Usage\n### Prepare docker host\n\nIncrease virtual memory map:\n```console\n$ sudo sysctl -w vm.max_map_count=262144\n```\n**:warning: This is not persisted change. Make a file in `/etc/sysctl.d/` dir with this setting.**\n\nPlease, check [Elastic docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/system-config.html) for more information.\n\n### Initial setup\n**:warning: This project prepared to use Elasticsearch with enabled TLS for Elasticsearch. You can disable TLS in services configs and healthcheck scripts if you don't need TLS.**\n\n1. Clone this repository onto the Docker host.\n2. Follow [TLS setup settings](./tls/)\n3. Enable built-in system accounts:\n   1. Start Elasticsearch with `docker compose up -d elastic`\n   2. After a few seconds run\n         ```shell\n         docker compose exec elastic bin/elasticsearch-setup-passwords auto --batch -u https://localhost:9200\n         ```\n         That will generate passwords for system accounts.\n   3. Add `logstash_writer` role and `logstash_internal` user if needed with POST request\n      *:information_source: Replace variables below with your values*\n      ```shell\n      # Create role\n      curl --insecure \\\n        --user elastic:${ELASTIC_PASSWORD} \\\n        --request POST \\\n        --header \"Content-Type: application/json\" \\\n        --data '{\"cluster\":[\"manage_index_templates\",\"monitor\",\"manage_ilm\"],\"indices\":[{\"names\":[\"logs-generic-default\",\"logstash-*\",\"ecs-logstash-*\"],\"privileges\":[\"write\",\"create\",\"create_index\",\"manage\",\"manage_ilm\"]},{\"names\":[\"logstash\",\"ecs-logstash\"],\"privileges\":[\"write\",\"manage\"]}]}' \\\n        https://localhost:9200/_security/role/logstash_writer\n      # Create iser\n      curl --insecure \\\n        --user elastic:${ELASTIC_PASSWORD} \\\n        --request POST \\\n        --header \"Content-Type: application/json\" \\\n        --data '{\"password\":\"${LOGSTASH_INTERNAL_PASSWD}\",\"roles\":[\"logstash_writer\"]}' \\\n        https://localhost:9200/_security/user/logstash_internal\n        ```\n   4. Add `remote_logging_agent` role and `beats_writer` user if needed with POST request\n      *:information_source: Replace variables below with your values*\n      ```shell\n      # Create role\n      curl --insecure \\\n        --user elastic:${ELASTIC_PASSWORD} \\\n        --request POST \\\n        --header \"Content-Type: application/json\" \\\n        --data '{\"cluster\":[\"manage_index_templates\",\"manage_ingest_pipelines\",\"monitor\",\"manage_ilm\",\"manage_pipeline\"],\"indices\":[{\"names\":[\"logs-*\",\"filebeat-*\",\"metrics-*\",\"metricbeat-*\"],\"privileges\":[\"write\",\"create\",\"create_index\",\"manage\",\"manage_ilm\"]}]}' \\\n        https://localhost:9200/_security/role/remote_logging_agent\n      # Create iser\n      curl --insecure \\\n        --user elastic:${ELASTIC_PASSWORD} \\\n        --request POST \\\n        --header \"Content-Type: application/json\" \\\n        --data '{\"password\":\"${BEATS_WRITER_PASSWD}\",\"roles\":[\"remote_logging_agent\",\"remote_monitoring_agent\"]}' \\\n        https://localhost:9200/_security/user/beats_writer\n        ```\n   5. Fill passwords with generated ones in following files:\n        \u0026emsp;`.env`\n        \u0026emsp;`logstash/pipeline/main.conf`\n4. Fill `.env` file.\n5. Load Filebeat and Metricbeat Kibana settings with\n    ```shell\n    docker compose run filebeat setup -E output.elasticsearch.username=elastic -E output.elasticsearch.password=${your_elastic_root_password} -c config/filebeat.docker.yml --strict.perms=false\n    docker compose run metricbeat setup -E output.elasticsearch.username=elastic -E output.elasticsearch.password=${your_elastic_root_password} -c config/metricbeat.docker.yml --strict.perms=false\n    ```\n6.   Start services  with:`docker compose up`\n    You can also run all services in the background (detached mode) by adding the `-d` flag to the above command.\n\n### Docker network driver\n\nThere are two network drivers that can be used with docker-compose: `bridge` and `host`.\n\n**bridge:** Add virtual network and pass-through selected ports. Also provide ability to use internal domain names (`elastic`, `kibana`, etc). Unfortunately, brings some routing overhead.\n\n**host:** Just use host network.  No network isolation, no internal domains, no overhead.\n\nAccording to [Rally](https://github.com/elastic/rally) testing with `metricbeat` race, there is no significant difference.\n\n**Using host network:**\nTo use host network for Elastic stack, remove `network` and `ports` sections from `docker-compose.yml` file and add `network_mode: host` key to services you want to use host network driver. You can use all services with host network mode.\nWhen Elasticsearch set to use host network, change `elasticsearch.hosts` to `localhost` both in Kibana and Logstash configs.\n\nCheck [docker compose reference](https://docs.docker.com/compose/compose-file/compose-file-v3/#network-configuration-reference) for more information.\n\n### Custom changes in `docker-compose.yml`\n\nTo stay synced with remote repo it's recommended to add all local changes to `docker-compose.override.yml`.\n\nOverride file is a same as docker-compose file, but not required all section specified. Just overrides.\n\nMore info at [docker docs](https://docs.docker.com/compose/extends/)\n\n### Cleanup\n\nElasticsearch data is persisted inside a volume by default.\n\nIn order to entirely shutdown the stack and remove all persisted data, use the following Docker Compose command:\n\n```console\n$ docker compose down -v\n```\n\n### Access Kibana\n\nGive Kibana about a minute to initialize, then access the Kibana web UI by opening \u003chttp://localhost:5601\u003e in a web\nbrowser and use the following credentials to log in:\n\n* user: *elastic*\n* password: *\\\u003cyour generated elastic password\u003e*\n\n### Default Kibana index pattern creation\n\nWhen Kibana launches for the first time, it is not configured with any index pattern.\n\n#### Via the Kibana web UI\n\n*:information_source: You need to inject data into Logstash before being able to configure a Logstash index pattern via\nthe Kibana web UI.*\n\nNavigate to the _Discover_ view of Kibana from the left sidebar. You will be prompted to create an index pattern. Enter\n`logstash-*` to match Logstash indices then, on the next page, select `@timestamp` as the time filter field. Finally,\nclick _Create index pattern_ and return to the _Discover_ view to inspect your log entries.\n\nRefer to [Connect Kibana with Elasticsearch](https://www.elastic.co/guide/en/kibana/current/connect-to-elasticsearch.html) and [Creating an index pattern](https://www.elastic.co/guide/en/kibana/current/index-patterns.html) for detailed\ninstructions about the index pattern configuration.\n\n#### On the command line\n\nCreate an index pattern via the Kibana API:\n\n```console\n$ curl -XPOST -D- 'http://localhost:5601/api/saved_objects/index-pattern' \\\n    -H 'Content-Type: application/json' \\\n    -H 'kbn-version: 8.1.2' \\\n    -u elastic:\u003cyour generated elastic password\u003e \\\n    -d '{\"attributes\":{\"title\":\"logstash-*\",\"timeFieldName\":\"@timestamp\"}}'\n```\n\nThe created pattern will automatically be marked as the default index pattern as soon as the Kibana UI is opened for the\nfirst time.\n\n## Configuration\n\n*:information_source: Configuration is not dynamically reloaded, you will need to restart individual components after any configuration change.*\n\n### How to configure Elasticsearch\n\nLearn more about the security of the Elastic stack at [Secure the Elastic Stack](https://www.elastic.co/guide/en/elasticsearch/reference/current/secure-cluster.html).\n\nThe Elasticsearch configuration is stored in [`elastic/elasticsearch.yml`](./elastic/elasticsearch.yml).\n\nYou can also specify the options you want to override by setting environment variables inside the Compose file:\n\n```yml\nelastic:\n\n  environment:\n    network.host: _non_loopback_\n    cluster.name: my-cluster\n```\n\nPlease refer to the following documentation page for more details about how to configure Elasticsearch inside Docker\ncontainers: [Install Elasticsearch with Docker](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html).\n\n### How to configure Kibana\n\nThe Kibana default configuration is stored in [`kibana/config/kibana.yml`](./kibana/kibana.yml).\n\n#### Kibana TLS\n\nIt's highly recommended to use Kibana with secure TLS connection. There is two ways to achieve that:\n\n* Setup reverse proxy (like Nginx).\n* Setup Kibana using TLS itself.\n\nYou can find Kibana TLS setup instructions in [`tls/README.md`](./tls/)\n\nPlease refer to the following documentation page for more details about how to configure Kibana inside Docker\ncontainers: [Install Kibana with Docker](https://www.elastic.co/guide/en/kibana/current/docker.html).\n\n### How to configure Logstash\n\n*:information_source: Do not use the `logstash_system` user inside the Logstash **pipeline** file, it does not have sufficient permissions to create indices. Follow the instructions at [Configuring Security in Logstash](https://www.elastic.co/guide/en/logstash/current/ls-security.html) to create a user with suitable roles.*\n\nThe Logstash configuration is stored in [`logstash/logstash.yml`](./logstash/logstash.yml), Logstash pipelines configuration is in [`logstash/pipelines.yml`](./logstash/pipelines.yml)\n\nPlease refer to the following documentation page for more details about how to configure Logstash inside Docker\ncontainers: [Configuring Logstash for Docker](https://www.elastic.co/guide/en/logstash/current/docker-config.html).\n\n### How to configure Beats\n\nFilebeat and Metricbeat are using for Elastic stack monitoring. Refered docs:\n  - [Collecting Elasticsearch monitoring data with Metricbeat](https://www.elastic.co/guide/en/elasticsearch/reference/8.1/configuring-metricbeat.html)\n  - [Collecting Elasticsearch log data with Filebeat](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-filebeat.html)\n  - [Good unofficial article](https://keepgrowing.in/tools/monitoring-elastic-stack/)\n\nBeats can be configured with `beats/filebeat.docker.yml` file or with docker labels. But for some reason X-Pack monitoring configured with labels doesn't works.\n\nPlease refer to the following documentation page for more details about how to configure Filebeat inside Docker\ncontainers:\n  - [Configuring Filebeat](https://www.elastic.co/guide/en/beats/filebeat/current/configuring-howto-filebeat.html)\n  - [Filebeat Autodiscover](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html#_docker_2)\n\n### How to configure Fleet server\n\nFleet is a new way to manage log shippers. Instead of bundle of beats now we can use only one service, called `Elastic Agent`. And Fleet is a management server for Elastic Agent.\n\nIn order to impossibility of preconfigure Kibana for Fleet server with environment variables, use web UI to configure Fleet and then fill `FLEET_SERVER_POLICY_ID` and `FLEET_SERVER_SERVICE_TOKEN` with your values.\n\n### Elastic registry\n\nElastic package registry is service which Kibana and Fleet system uses to get integration packages. Usually it's optional but required when using Fleet system isolated from official elastic registry.\n\n### How to scale out the Elasticsearch cluster\n\nFollow the instructions from the Wiki: [Scaling out Elasticsearch](https://github.com/deviantony/docker-elk/wiki/Elasticsearch-cluster)\n\n### Healthcheck\n\nRepo contains healthcheck bash scripts and utility buit with Go. You can choose one oh them or don't use service healthcheck.\n\n#### Healthcheck Go utility\n\n**Usage:** healthcheck [options] [elastic | kibana | logstash] [host]\n\nBy default tool configurated for default repo settings (https for elastic, default ports, ignoring invalid certs).\n\n*:warning: Flags should be defore service type and host!*\n* To use basic auth, add `-u \u003cusername`(Default remote_monitoring_user) and `-p \u003cpassword\u003e` flags.\n* Trigger status can be setted with RegExp by `-s` flag, e.g: `healthcheck -s 'green|yellow' elastic`\n* Accept non default hostname/scheme, e.g: `healthcheck elastic http://elastic`\n\n#### Healthcheck scripts\n\n1. Add mount point for each script to corresponding service.\n2. Change **`healthcheck: test: \"CMD\"`** to service healthcheck script.\n3. Change checking endpoint and username/password.\n\n## Extensibility\n### How to add plugins\n\nTo add plugins to any Elastic stack component, you have to:\n\n1. Create Dockerfile for service you want to apply plugin.\n2. Add a `RUN` statement to the corresponding `Dockerfile` (e.g. `RUN logstash-plugin install logstash-filter-json`)\n```dockerfile\n# https://www.docker.elastic.co/\nFROM docker.elastic.co/logstash/logstash:${LOGSTASH_VERSION}\n\n# Add your logstash plugins setup here\nRUN logstash-plugin install logstash-filter-json\n```\n3. Add the associated plugin code configuration to the service configuration (eg. Logstash input/output)\n4. Add following to docker compose service section you want to apply plugin (e.g. Logstash):\n```yaml\nbuild:\n      context: logstash/\n```\n5. (Re)Build the images using the `docker compose build` command\n\n## JVM tuning\n### How to specify the amount of memory used by a service\n\nBy default, both Elasticsearch and Logstash start with [1/4 of the total host memory](https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/parallel.html#default_heap_size) allocated to the JVM Heap Size.\n\nThe startup scripts for Elasticsearch and Logstash can append extra JVM options from the value of an environment\nvariable, allowing the user to adjust the amount of memory that can be used by each component:\n\n| Service       | Environment variable |\n|---------------|----------------------|\n| Elasticsearch | ES_JAVA_OPTS         |\n| Logstash      | LS_JAVA_OPTS         |\n\n\nFor example, to increase the maximum JVM Heap Size for Logstash:\n\n```yml\nlogstash:\n\n  environment:\n    LS_JAVA_OPTS: -Xmx1g -Xms1g\n```\n\n### How to enable a remote JMX connection to a service\n\nAs for the Java Heap memory (see above), you can specify JVM options to enable JMX and map the JMX port on the Docker host.\n\nUpdate the `{ES,LS}_JAVA_OPTS` environment variable with the following content (I've mapped the JMX service on the port 18080, you can change that). Do not forget to update the `-Djava.rmi.server.hostname` option with the IP address of your Docker host (replace **DOCKER_HOST_IP**):\n\n```yml\nlogstash:\n\n  environment:\n    LS_JAVA_OPTS: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=18080 -Dcom.sun.management.jmxremote.rmi.port=18080 -Djava.rmi.server.hostname=DOCKER_HOST_IP -Dcom.sun.management.jmxremote.local.only=false\n```\n\n## Going further\n### Swarm mode\n\nThis time, there are no plans on support for Docker [Swarm mode](https://docs.docker.com/engine/swarm/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazhinu%2Felastic-stack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fazhinu%2Felastic-stack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazhinu%2Felastic-stack/lists"}