{"id":13463709,"url":"https://github.com/pires/kubernetes-elasticsearch-cluster","last_synced_at":"2026-01-10T19:36:39.617Z","repository":{"id":26542940,"uuid":"29996379","full_name":"pires/kubernetes-elasticsearch-cluster","owner":"pires","description":"Elasticsearch cluster on top of Kubernetes made easy.","archived":true,"fork":false,"pushed_at":"2018-11-07T10:15:41.000Z","size":272,"stargazers_count":1511,"open_issues_count":55,"forks_count":689,"subscribers_count":88,"default_branch":"master","last_synced_at":"2024-09-27T03:41:34.601Z","etag":null,"topics":["document-database","elasticsearch","kubernetes"],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pires.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-01-29T00:55:05.000Z","updated_at":"2024-09-27T02:46:47.000Z","dependencies_parsed_at":"2022-07-25T15:32:10.962Z","dependency_job_id":null,"html_url":"https://github.com/pires/kubernetes-elasticsearch-cluster","commit_stats":null,"previous_names":[],"tags_count":29,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pires%2Fkubernetes-elasticsearch-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pires%2Fkubernetes-elasticsearch-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pires%2Fkubernetes-elasticsearch-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pires%2Fkubernetes-elasticsearch-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pires","download_url":"https://codeload.github.com/pires/kubernetes-elasticsearch-cluster/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234569763,"owners_count":18854133,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["document-database","elasticsearch","kubernetes"],"created_at":"2024-07-31T14:00:26.983Z","updated_at":"2025-09-28T23:30:25.960Z","avatar_url":"https://github.com/pires.png","language":null,"funding_links":[],"categories":["Others","kubernetes","Performance"],"sub_categories":[],"readme":"# This project is no longer maintained\n\nAs of November 7th, 2018, I've decided to end my commitment to maintaining this repo and related.\n\nIt's been 3 years since I last used Elasticsearch, so I no longer have the motivation it takes to maintain and evolve this project. Also, other projects need all the attention I can give.\n\nIt was a great run, **thank you all**.\n\n# kubernetes-elasticsearch-cluster\nElasticsearch cluster on top of Kubernetes made easy.\n\n### Table of Contents\n\n* [(Very) Important Notes](#important-notes)\n* [Pre-Requisites](#pre-requisites)\n* [Build container image (optional)](#build-images)\n* [Test](#test)\n  * [Deploy](#deploy)\n  * [Access the service](#access-the-service)\n* [Pod anti-affinity](#pod-anti-affinity)\n* [Availability](#availability)\n* [Deploy with Helm](#helm)\n* [Install plug-ins](#plugins)\n* [Clean-up with Curator](#curator)\n* [Kibana](#kibana)\n* [FAQ](#faq)\n* [Troubleshooting](#troubleshooting)\n\n## Abstract\n\n[Elasticsearch best-practices recommend to separate nodes in three roles](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-node.html):\n\n* `Master` nodes - intended for clustering management only, no data, no HTTP API\n* `Data` nodes - intended for client usage and data\n* `Ingest` nodes - intended for document pre-processing during ingestion\n\nGiven this, I'm going to demonstrate how to provision a production grade scenario consisting of 3 master, 2 data and 2 ingest nodes.\n\n\u003ca id=\"important-notes\"\u003e\n\n## (Very) Important notes\n\n* Elasticsearch pods need for an init-container to run in privileged mode, so it can set some VM options.\n  For that to happen, the `kubelet` should be running with args `--allow-privileged`, otherwise the init-container will fail to run.\n\n* By default, `ES_JAVA_OPTS` is set to `-Xms256m -Xmx256m`. This is a *very low* value but many users, i.e. `minikube` users,\n  were having issues with pods getting killed because hosts were out of memory.\n  One can change this in the deployment descriptors available in this repository.\n\n* As of the moment, Kubernetes pod descriptors use an `emptyDir` for storing data in each data node container.\n  This is meant to be for the sake of simplicity and should be adapted according to one's storage needs.\n\n* The [stateful](stateful) directory contains an example which deploys the data pods as a `StatefulSet`.\n  These use a `volumeClaimTemplates` to provision persistent storage for each pod.\n\n* By default, `PROCESSORS` is set to `1`. This may not be enough for some deployments, especially at startup time.\n  Adjust `resources.limits.cpu` and/or `livenessProbe` accordingly if required. Note that `resources.limits.cpu` must be an integer.\n\n\u003ca id=\"pre-requisites\"\u003e\n\n## Pre-requisites\n\n* Kubernetes 1.11.x (tested with v1.11.2 on top of [Vagrant + CoreOS](https://github.com/pires/kubernetes-vagrant-coreos-cluster)).\n* `kubectl` configured to access the Kubernetes API.\n\n\u003ca id=\"build-images\"\u003e\n\n## Build images (optional)\n\nProviding one's own version of [the images automatically built from this repository](https://github.com/pires/docker-elasticsearch-kubernetes) will not be supported. This is an *optional* step. One has been warned.\n\n## Test\n\n### Deploy\n\n```shell\nkubectl create -f es-discovery-svc.yaml\nkubectl create -f es-svc.yaml\nkubectl create -f es-master.yaml\nkubectl rollout status -f es-master.yaml\n\nkubectl create -f es-ingest-svc.yaml\nkubectl create -f es-ingest.yaml\nkubectl rollout status -f es-ingest.yaml\n\nkubectl create -f es-data.yaml\nkubectl rollout status -f es-data.yaml\n```\n\nLet's check if everything is working properly:\n\n```shell\nkubectl get svc,deployment,pods -l component=elasticsearch\nNAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE\nservice/elasticsearch             ClusterIP   10.100.243.196   \u003cnone\u003e        9200/TCP   3m\nservice/elasticsearch-discovery   ClusterIP   None             \u003cnone\u003e        9300/TCP   3m\nservice/elasticsearch-ingest      ClusterIP   10.100.76.74     \u003cnone\u003e        9200/TCP   2m\n\nNAME                              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE\ndeployment.extensions/es-data     2         2         2            2           1m\ndeployment.extensions/es-ingest   2         2         2            2           2m\ndeployment.extensions/es-master   3         3         3            3           3m\n\nNAME                             READY     STATUS    RESTARTS   AGE\npod/es-data-56f8ff8c97-642bq     1/1       Running   0          1m\npod/es-data-56f8ff8c97-h6hpc     1/1       Running   0          1m\npod/es-ingest-6ddd5fc689-b4s94   1/1       Running   0          2m\npod/es-ingest-6ddd5fc689-d8rtj   1/1       Running   0          2m\npod/es-master-68bf8f86c4-bsfrx   1/1       Running   0          3m\npod/es-master-68bf8f86c4-g8nph   1/1       Running   0          3m\npod/es-master-68bf8f86c4-q5khn   1/1       Running   0          3m\n```\n\nAs we can assert, the cluster seems to be up and running. Easy, wasn't it?\n\n### Access the service\n\n*Don't forget* that services in Kubernetes are only acessible from containers in the cluster. For different behavior one should [configure the creation of an external load-balancer](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer). While it's supported within this example service descriptor, its usage is out of scope of this document, for now.\n\n*Note:* if you are using one of the cloud providers which support external load balancers, setting the type field to \"LoadBalancer\" will provision a load balancer for your Service. You can uncomment the field in [es-svc.yaml](https://github.com/pires/kubernetes-elasticsearch-cluster/blob/master/es-svc.yaml).\n\n```shell\nkubectl get svc elasticsearch\nNAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE\nelasticsearch   ClusterIP   10.100.243.196   \u003cnone\u003e        9200/TCP   3m\n```\n\nFrom any host on the Kubernetes cluster (that's running `kube-proxy` or similar), run:\n\n```shell\ncurl http://10.100.243.196:9200\n```\n\nOne should see something similar to the following:\n\n```json\n{\n  \"name\" : \"es-data-56f8ff8c97-642bq\",\n  \"cluster_name\" : \"myesdb\",\n  \"cluster_uuid\" : \"RkRkTl26TDOE7o0FhCcW_g\",\n  \"version\" : {\n    \"number\" : \"6.3.2\",\n    \"build_flavor\" : \"default\",\n    \"build_type\" : \"tar\",\n    \"build_hash\" : \"053779d\",\n    \"build_date\" : \"2018-07-20T05:20:23.451332Z\",\n    \"build_snapshot\" : false,\n    \"lucene_version\" : \"7.3.1\",\n    \"minimum_wire_compatibility_version\" : \"5.6.0\",\n    \"minimum_index_compatibility_version\" : \"5.0.0\"\n  },\n  \"tagline\" : \"You Know, for Search\"\n}\n```\n\nOr if one wants to see cluster information:\n\n```shell\ncurl http://10.100.243.196:9200/_cluster/health?pretty\n```\n\nOne should see something similar to the following:\n\n```json\n{\n  \"cluster_name\" : \"myesdb\",\n  \"status\" : \"green\",\n  \"timed_out\" : false,\n  \"number_of_nodes\" : 7,\n  \"number_of_data_nodes\" : 2,\n  \"active_primary_shards\" : 0,\n  \"active_shards\" : 0,\n  \"relocating_shards\" : 0,\n  \"initializing_shards\" : 0,\n  \"unassigned_shards\" : 0,\n  \"delayed_unassigned_shards\" : 0,\n  \"number_of_pending_tasks\" : 0,\n  \"number_of_in_flight_fetch\" : 0,\n  \"task_max_waiting_in_queue_millis\" : 0,\n  \"active_shards_percent_as_number\" : 100.0\n}\n```\n\n\u003ca id=\"pod-anti-affinity\"\u003e\n\n## Pod anti-affinity\n\nOne of the main advantages of running Elasticsearch on top of Kubernetes is how resilient the cluster becomes, particularly during\nnode restarts. However if all data pods are scheduled onto the same node(s), this advantage decreases significantly and may even\nresult in no data pods being available.\n\nIt is then **highly recommended**, in the context of the solution described in this repository, that one adopts [pod anti-affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity-beta-feature)\nin order to guarantee that two data pods will never run on the same node.\n\nHere's an example:\n\n```yaml\nspec:\n  affinity:\n    podAntiAffinity:\n      preferredDuringSchedulingIgnoredDuringExecution:\n      - weight: 100\n        podAffinityTerm:\n          labelSelector:\n            matchExpressions:\n            - key: component\n              operator: In\n              values:\n              - elasticsearch\n            - key: role\n              operator: In\n              values:\n              - data\n          topologyKey: kubernetes.io/hostname\n  containers:\n  - (...)\n```\n\n\u003ca id=\"availability\"\u003e\n\n## Availability\n\nIf one wants to ensure that no more than `n` Elasticsearch nodes will be unavailable at a time, one can optionally (change and) apply the following manifests:\n\n```shell\nkubectl create -f es-master-pdb.yaml\nkubectl create -f es-data-pdb.yaml\n```\n\n**Note:** This is an advanced subject and one should only put it in practice if one understands clearly what it means both in the Kubernetes and Elasticsearch contexts. For more information, please consult [Pod Disruptions](https://kubernetes.io/docs/concepts/workloads/pods/disruptions).\n\n\u003ca id=\"helm\"\u003e\n\n## Deploy with Helm\n\n**WARNING:** The Helm chart is maintained by someone else in the community and may not up-to-date with this repo.\n\n[Helm](https://github.com/kubernetes/helm) charts for a basic (non-stateful) ElasticSearch deployment are maintained at https://github.com/clockworksoul/helm-elasticsearch. With Helm properly installed and configured, standing up a complete cluster is almost trivial:\n\n```shell\ngit clone https://github.com/clockworksoul/helm-elasticsearch.git\nhelm install helm-elasticsearch\n```\n\nVarious parameters of the cluster, including replica count and memory allocations, can be adjusted by editing the `helm-elasticsearch/values.yaml` file. For information about Helm, please consult the [complete Helm documentation](https://github.com/kubernetes/helm/blob/master/docs/index.md).\n\n\u003ca id=\"plugins\"\u003e\n\n## Install plug-ins\n\nThe image used in this repo is very minimalist. However, one can install additional plug-ins at will by simply specifying the `ES_PLUGINS_INSTALL` environment variable in the desired pod descriptors. For instance, to install [Google Cloud Storage](https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-gcs.html) and [S3](https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3.html) plug-ins it would be like follows:\n\n```yaml\n- name: \"ES_PLUGINS_INSTALL\"\n  value: \"repository-gcs,repository-s3\"\n```\n\n**Note:** The X-Pack plugin does not currently work with the `quay.io/pires/docker-elasticsearch-kubernetes` image. See Issue #102\n\n\u003ca id=\"curator\"\u003e\n\n## Clean-up with Curator\n\nAdditionally, one can run a [CronJob](http://kubernetes.io/docs/user-guide/cron-jobs/) that will periodically run [Curator](https://github.com/elastic/curator) to clean up indices (or do other actions on the Elasticsearch cluster).\n\n```shell\nkubectl create -f es-curator-config.yaml\nkubectl create -f es-curator.yaml\n```\n\nPlease, confirm the job has been created.\n\n```shell\nkubectl get cronjobs\nNAME      SCHEDULE    SUSPEND   ACTIVE    LAST-SCHEDULE\ncurator   1 0 * * *   False     0         \u003cnone\u003e\n```\n\nThe job is configured to run once a day at _1 minute past midnight and delete indices that are older than 3 days_.\n\n**Notes**\n\n* One can change the schedule by editing the cron notation in `es-curator.yaml`.\n* One can change the action (e.g. delete older than 3 days) by editing the `es-curator-config.yaml`.\n* The definition of the `action_file.yaml` is quite self-explaining for simple set-ups. For more advanced configuration options, please consult the [Curator Documentation](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html).\n\nIf one wants to remove the curator job, just run:\n\n```shell\nkubectl delete cronjob curator\nkubectl delete configmap curator-config\n```\n\n\u003ca id=\"kibana\"\u003e\n\n## Kibana\n\n**WARNING:** The Kibana section is maintained by someone else in the community and may not up-to-date with this repo.\n\n### Deploy\n\nIf Kibana defaults are not enough, one may want to customize `kibana.yaml` through a `ConfigMap`.\nPlease refer to [Configuring Kibana](https://www.elastic.co/guide/en/kibana/current/settings.html) for all available attributes.\n\n```shell\nkubectl create -f kibana-cm.yaml\nkubectl create -f kibana-svc.yaml\nkubectl create -f kibana.yaml\n```\n\nKibana will become available through service `kibana`, and one will be able to access it from within the cluster, or proxy it through the Kubernetes API as follows:\n\n```shell\ncurl https://\u003cAPI_SERVER_URL\u003e/api/v1/namespaces/default/services/kibana:http/proxy\n```\n\nOne can also create an Ingress to expose the service publicly or simply use the service nodeport.\nIn the case one proceeds to do so, one must change the environment variable `SERVER_BASEPATH` to the match their environment.\n\n## FAQ\n\n### Why does `NUMBER_OF_MASTERS` differ from number of master-replicas?\n\nThe default value for this environment variable is 2, meaning a cluster will need a minimum of 2 master nodes to operate. If a cluster has 3 masters and one dies, the cluster still works. Minimum master nodes are usually `n/2 + 1`, where `n` is the number of master nodes in a cluster. If a cluster has 5 master nodes, one should have a minimum of 3, less than that and the cluster _stops_. If one scales the number of masters, make sure to update the minimum number of master nodes through the Elasticsearch API as setting environment variable will only work on cluster setup. More info: https://www.elastic.co/guide/en/elasticsearch/guide/1.x/_important_configuration_changes.html#_minimum_master_nodes\n\n\n### How can I customize `elasticsearch.yaml`?\n\nRead a different config file by settings env var `ES_PATH_CONF=/path/to/my/config/` [(see the Elasticsearch docs for more)](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html#config-files-location). Another option would be to build one's own image from  [this repository](https://github.com/pires/docker-elasticsearch-kubernetes)\n\n## Troubleshooting\n\n### No up-and-running site-local\n\nOne of the errors one may come across when running the setup is the following error:\n\n```\n[2016-11-29T01:28:36,515][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]\norg.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: No up-and-running site-local (private) addresses found, got [name:lo (lo), name:eth0 (eth0)]\n\tat org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:116) ~[elasticsearch-5.0.1.jar:5.0.1]\n\tat org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:103) ~[elasticsearch-5.0.1.jar:5.0.1]\n\tat org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.0.1.jar:5.0.1]\n\tat org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:96) ~[elasticsearch-5.0.1.jar:5.0.1]\n\tat org.elasticsearch.cli.Command.main(Command.java:62) ~[elasticsearch-5.0.1.jar:5.0.1]\n\tat org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) ~[elasticsearch-5.0.1.jar:5.0.1]\n\tat org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:73) ~[elasticsearch-5.0.1.jar:5.0.1]\nCaused by: java.lang.IllegalArgumentException: No up-and-running site-local (private) addresses found, got [name:lo (lo), name:eth0 (eth0)]\n\tat org.elasticsearch.common.network.NetworkUtils.getSiteLocalAddresses(NetworkUtils.java:187) ~[elasticsearch-5.0.1.jar:5.0.1]\n\tat org.elasticsearch.common.network.NetworkService.resolveInternal(NetworkService.java:246) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.common.network.NetworkService.resolveInetAddresses(NetworkService.java:220) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.common.network.NetworkService.resolveBindHostAddresses(NetworkService.java:130) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.transport.TcpTransport.bindServer(TcpTransport.java:575) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.transport.netty4.Netty4Transport.doStart(Netty4Transport.java:182) ~[?:?]\n \tat org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.transport.TransportService.doStart(TransportService.java:182) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.node.Node.start(Node.java:525) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:211) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:288) ~[elasticsearch-5.0.1.jar:5.0.1]\n \tat org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:112) ~[elasticsearch-5.0.1.jar:5.0.1]\n \t... 6 more\n[2016-11-29T01:28:37,448][INFO ][o.e.n.Node               ] [kIEYQSE] stopping ...\n[2016-11-29T01:28:37,451][INFO ][o.e.n.Node               ] [kIEYQSE] stopped\n[2016-11-29T01:28:37,452][INFO ][o.e.n.Node               ] [kIEYQSE] closing ...\n[2016-11-29T01:28:37,464][INFO ][o.e.n.Node               ] [kIEYQSE] closed\n```\n\nThis is related to how the container binds to network ports (defaults to ``_local_``). It will need to match the actual node network interface name, which depends on what OS and infrastructure provider one uses. For instance, if the primary interface on the node is `p1p1` then that is the value that needs to be set for the `NETWORK_HOST` environment variable.\nPlease see [the documentation](https://github.com/pires/docker-elasticsearch#environment-variables) for reference of options.\n\nIn order to workaround this, set `NETWORK_HOST` environment variable in the pod descriptors as follows:\n\n```yaml\n- name: \"NETWORK_HOST\"\n  value: \"_eth0_\" #_p1p1_ if interface name is p1p1, _ens4_ if interface name is ens4, and so on.\n```\n\n### (IPv6) org.elasticsearch.bootstrap.StartupException: BindTransportException\n\nIntermittent failures occur when the local network interface has both IPv4 and IPv6 addresses, and Elasticsearch tries to bind to the IPv6 address first.\nIf the IPv4 address is chosen first, Elasticsearch starts correctly.\n\nIn order to workaround this, set `NETWORK_HOST` environment variable in the pod descriptors as follows:\n\n```yaml\n- name: \"NETWORK_HOST\"\n  value: \"_eth0:ipv4_\" #_p1p1:ipv4_ if interface name is p1p1, _ens4:ipv4_ if interface name is ens4, and so on.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpires%2Fkubernetes-elasticsearch-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpires%2Fkubernetes-elasticsearch-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpires%2Fkubernetes-elasticsearch-cluster/lists"}