{"id":16655030,"url":"https://github.com/ycliuhw/k8s-elasticsearch-cluster","last_synced_at":"2026-02-27T12:04:57.084Z","repository":{"id":98948728,"uuid":"105031000","full_name":"ycliuhw/k8s-elasticsearch-cluster","owner":"ycliuhw","description":"setup elasticsearch-cluster on k8s - (Note: this is a copy from https://github.com/kubernetes/examples/tree/master/staging/elasticsearch/production_cluster with enhancement for RBAC support)","archived":false,"fork":false,"pushed_at":"2017-09-27T14:50:49.000Z","size":4,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-12T18:35:57.177Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ycliuhw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-27T14:47:47.000Z","updated_at":"2018-10-26T14:49:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"4095c617-fc23-4330-b454-e37d8d8251c0","html_url":"https://github.com/ycliuhw/k8s-elasticsearch-cluster","commit_stats":{"total_commits":1,"total_committers":1,"mean_commits":1.0,"dds":0.0,"last_synced_commit":"94eb896c347dc8deb46422168d1bda7ac6b6537c"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ycliuhw/k8s-elasticsearch-cluster","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ycliuhw%2Fk8s-elasticsearch-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ycliuhw%2Fk8s-elasticsearch-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ycliuhw%2Fk8s-elasticsearch-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ycliuhw%2Fk8s-elasticsearch-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ycliuhw","download_url":"https://codeload.github.com/ycliuhw/k8s-elasticsearch-cluster/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ycliuhw%2Fk8s-elasticsearch-cluster/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001290,"owners_count":26083058,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T09:51:46.045Z","updated_at":"2025-10-09T11:10:40.367Z","avatar_url":"https://github.com/ycliuhw.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Elasticsearch for Kubernetes\n\nKubernetes makes it trivial for anyone to easily build and scale [Elasticsearch](http://www.elasticsearch.org/) clusters. Here, you'll find how to do so.\nCurrent Elasticsearch version is `1.7.1`.\n\nBefore we start, one needs to know that Elasticsearch best-practices recommend to separate nodes in three roles:\n* `Master` nodes - intended for clustering management only, no data, no HTTP API\n* `Client` nodes - intended for client usage, no data, with HTTP API\n* `Data` nodes - intended for storing and indexing your data, no HTTP API\n\nThis is enforced throughout this document.\n\n\u003cimg src=\"http://kubernetes.io/kubernetes/img/warning.png\" alt=\"WARNING\" width=\"25\" height=\"25\"\u003e Current pod descriptors use an `emptyDir` for storing data in each data node container. This is meant to be for the sake of simplicity and [should be adapted according to your storage needs](../../../docs/design/persistent-storage.md).\n\n## Docker image\n\nThis example uses [this pre-built image](https://github.com/pires/docker-elasticsearch-kubernetes). Feel free to fork and update it to fit your own needs, but keep in mind that you will need to change Kubernetes descriptors accordingly.\n\n## Deploy\n\n```\nkubectl create -f examples/elasticsearch/production_cluster/service-account.yaml\nkubectl create -f examples/elasticsearch/production_cluster/es-discovery-svc.yaml\nkubectl create -f examples/elasticsearch/production_cluster/es-svc.yaml\nkubectl create -f examples/elasticsearch/production_cluster/es-master-rc.yaml\n```\n\nWait until `es-master` is provisioned, and\n\n```\nkubectl create -f examples/elasticsearch/production_cluster/es-client-rc.yaml\n```\n\nWait until `es-client` is provisioned, and\n\n```\nkubectl create -f examples/elasticsearch/production_cluster/es-data-rc.yaml\n```\n\nWait until `es-data` is provisioned.\n\nNow, I leave up to you how to validate the cluster, but a first step is to wait for containers to be in ```RUNNING``` state and check the Elasticsearch master logs:\n\n```\n$ kubectl get pods\nNAME              READY     STATUS    RESTARTS   AGE\nes-client-2ep9o   1/1       Running   0          2m\nes-data-r9tgv     1/1       Running   0          1m\nes-master-vxl6c   1/1       Running   0          6m\n```\n\n```\n$ kubectl logs es-master-vxl6c\nlog4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.\nlog4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.\nlog4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.\n[2015-08-21 10:58:51,324][INFO ][node                     ] [Arc] version[1.7.1], pid[8], build[b88f43f/2015-07-29T09:54:16Z]\n[2015-08-21 10:58:51,328][INFO ][node                     ] [Arc] initializing ...\n[2015-08-21 10:58:51,542][INFO ][plugins                  ] [Arc] loaded [cloud-kubernetes], sites []\n[2015-08-21 10:58:51,624][INFO ][env                      ] [Arc] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.4gb], net total_space [15.5gb], types [ext4]\n[2015-08-21 10:58:57,439][INFO ][node                     ] [Arc] initialized\n[2015-08-21 10:58:57,439][INFO ][node                     ] [Arc] starting ...\n[2015-08-21 10:58:57,782][INFO ][transport                ] [Arc] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.244.15.2:9300]}\n[2015-08-21 10:58:57,847][INFO ][discovery                ] [Arc] myesdb/-x16XFUzTCC8xYqWoeEOYQ\n[2015-08-21 10:59:05,167][INFO ][cluster.service          ] [Arc] new_master [Arc][-x16XFUzTCC8xYqWoeEOYQ][es-master-vxl6c][inet[/10.244.15.2:9300]]{data=false, master=true}, reason: zen-disco-join (elected_as_master)\n[2015-08-21 10:59:05,202][INFO ][node                     ] [Arc] started\n[2015-08-21 10:59:05,238][INFO ][gateway                  ] [Arc] recovered [0] indices into cluster_state\n[2015-08-21 11:02:28,797][INFO ][cluster.service          ] [Arc] added {[Gideon][4EfhWSqaTqikbK4tI7bODA][es-data-r9tgv][inet[/10.244.59.4:9300]]{master=false},}, reason: zen-disco-receive(join from node[[Gideon][4EfhWSqaTqikbK4tI7bODA][es-data-r9tgv][inet[/10.244.59.4:9300]]{master=false}])\n[2015-08-21 11:03:16,822][INFO ][cluster.service          ] [Arc] added {[Venomm][tFYxwgqGSpOejHLG4umRqg][es-client-2ep9o][inet[/10.244.53.2:9300]]{data=false, master=false},}, reason: zen-disco-receive(join from node[[Venomm][tFYxwgqGSpOejHLG4umRqg][es-client-2ep9o][inet[/10.244.53.2:9300]]{data=false, master=false}])\n```\n\nAs you can assert, the cluster is up and running. Easy, wasn't it?\n\n## Scale\n\nScaling each type of node to handle your cluster is as easy as:\n\n```\nkubectl scale --replicas=3 rc es-master\nkubectl scale --replicas=2 rc es-client\nkubectl scale --replicas=2 rc es-data\n```\n\nDid it work?\n\n```\n$ kubectl get pods\nNAME              READY     STATUS    RESTARTS   AGE\nes-client-2ep9o   1/1       Running   0          4m\nes-client-ye5s1   1/1       Running   0          50s\nes-data-8az22     1/1       Running   0          47s\nes-data-r9tgv     1/1       Running   0          3m\nes-master-57h7k   1/1       Running   0          52s\nes-master-kuwse   1/1       Running   0          52s\nes-master-vxl6c   1/1       Running   0          8m\n```\n\nLet's take another look of the Elasticsearch master logs:\n\n```\n$ kubectl logs es-master-vxl6c\nlog4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.\nlog4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.\nlog4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.\n[2015-08-21 10:58:51,324][INFO ][node                     ] [Arc] version[1.7.1], pid[8], build[b88f43f/2015-07-29T09:54:16Z]\n[2015-08-21 10:58:51,328][INFO ][node                     ] [Arc] initializing ...\n[2015-08-21 10:58:51,542][INFO ][plugins                  ] [Arc] loaded [cloud-kubernetes], sites []\n[2015-08-21 10:58:51,624][INFO ][env                      ] [Arc] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.4gb], net total_space [15.5gb], types [ext4]\n[2015-08-21 10:58:57,439][INFO ][node                     ] [Arc] initialized\n[2015-08-21 10:58:57,439][INFO ][node                     ] [Arc] starting ...\n[2015-08-21 10:58:57,782][INFO ][transport                ] [Arc] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.244.15.2:9300]}\n[2015-08-21 10:58:57,847][INFO ][discovery                ] [Arc] myesdb/-x16XFUzTCC8xYqWoeEOYQ\n[2015-08-21 10:59:05,167][INFO ][cluster.service          ] [Arc] new_master [Arc][-x16XFUzTCC8xYqWoeEOYQ][es-master-vxl6c][inet[/10.244.15.2:9300]]{data=false, master=true}, reason: zen-disco-join (elected_as_master)\n[2015-08-21 10:59:05,202][INFO ][node                     ] [Arc] started\n[2015-08-21 10:59:05,238][INFO ][gateway                  ] [Arc] recovered [0] indices into cluster_state\n[2015-08-21 11:02:28,797][INFO ][cluster.service          ] [Arc] added {[Gideon][4EfhWSqaTqikbK4tI7bODA][es-data-r9tgv][inet[/10.244.59.4:9300]]{master=false},}, reason: zen-disco-receive(join from node[[Gideon][4EfhWSqaTqikbK4tI7bODA][es-data-r9tgv][inet[/10.244.59.4:9300]]{master=false}])\n[2015-08-21 11:03:16,822][INFO ][cluster.service          ] [Arc] added {[Venomm][tFYxwgqGSpOejHLG4umRqg][es-client-2ep9o][inet[/10.244.53.2:9300]]{data=false, master=false},}, reason: zen-disco-receive(join from node[[Venomm][tFYxwgqGSpOejHLG4umRqg][es-client-2ep9o][inet[/10.244.53.2:9300]]{data=false, master=false}])\n[2015-08-21 11:04:40,781][INFO ][cluster.service          ] [Arc] added {[Erik Josten][QUJlahfLTi-MsxzM6_Da0g][es-master-kuwse][inet[/10.244.59.5:9300]]{data=false, master=true},}, reason: zen-disco-receive(join from node[[Erik Josten][QUJlahfLTi-MsxzM6_Da0g][es-master-kuwse][inet[/10.244.59.5:9300]]{data=false, master=true}])\n[2015-08-21 11:04:41,076][INFO ][cluster.service          ] [Arc] added {[Power Princess][V4qnR-6jQOS5ovXQsPgo7g][es-master-57h7k][inet[/10.244.53.3:9300]]{data=false, master=true},}, reason: zen-disco-receive(join from node[[Power Princess][V4qnR-6jQOS5ovXQsPgo7g][es-master-57h7k][inet[/10.244.53.3:9300]]{data=false, master=true}])\n[2015-08-21 11:04:53,966][INFO ][cluster.service          ] [Arc] added {[Cagliostro][Wpfx5fkBRiG2qCEWd8laaQ][es-client-ye5s1][inet[/10.244.15.3:9300]]{data=false, master=false},}, reason: zen-disco-receive(join from node[[Cagliostro][Wpfx5fkBRiG2qCEWd8laaQ][es-client-ye5s1][inet[/10.244.15.3:9300]]{data=false, master=false}])\n[2015-08-21 11:04:56,803][INFO ][cluster.service          ] [Arc] added {[Thog][vkdEtX3ESfWmhXXf-Wi0_Q][es-data-8az22][inet[/10.244.15.4:9300]]{master=false},}, reason: zen-disco-receive(join from node[[Thog][vkdEtX3ESfWmhXXf-Wi0_Q][es-data-8az22][inet[/10.244.15.4:9300]]{master=false}])\n```\n\n## Access the service\n\n*Don't forget* that services in Kubernetes are only accessible from containers in the cluster. For different behavior you should [configure the creation of an external load-balancer](http://kubernetes.io/v1.0/docs/user-guide/services.html#type-loadbalancer). While it's supported within this example service descriptor, its usage is out of scope of this document, for now.\n\n```\n$ kubectl get service elasticsearch\nNAME            LABELS                                SELECTOR                              IP(S)          PORT(S)\nelasticsearch   component=elasticsearch,role=client   component=elasticsearch,role=client   10.100.134.2   9200/TCP\n```\n\nFrom any host on your cluster (that's running `kube-proxy`), run:\n\n```\ncurl http://10.100.134.2:9200\n```\n\nYou should see something similar to the following:\n\n\n```json\n{\n  \"status\" : 200,\n  \"name\" : \"Cagliostro\",\n  \"cluster_name\" : \"myesdb\",\n  \"version\" : {\n    \"number\" : \"1.7.1\",\n    \"build_hash\" : \"b88f43fc40b0bcd7f173a1f9ee2e97816de80b19\",\n    \"build_timestamp\" : \"2015-07-29T09:54:16Z\",\n    \"build_snapshot\" : false,\n    \"lucene_version\" : \"4.10.4\"\n  },\n  \"tagline\" : \"You Know, for Search\"\n}\n```\n\nOr if you want to check cluster information:\n\n\n```\ncurl http://10.100.134.2:9200/_cluster/health?pretty\n```\n\nYou should see something similar to the following:\n\n```json\n{\n  \"cluster_name\" : \"myesdb\",\n  \"status\" : \"green\",\n  \"timed_out\" : false,\n  \"number_of_nodes\" : 7,\n  \"number_of_data_nodes\" : 2,\n  \"active_primary_shards\" : 0,\n  \"active_shards\" : 0,\n  \"relocating_shards\" : 0,\n  \"initializing_shards\" : 0,\n  \"unassigned_shards\" : 0,\n  \"delayed_unassigned_shards\" : 0,\n  \"number_of_pending_tasks\" : 0,\n  \"number_of_in_flight_fetch\" : 0\n}\n```\n\n\n\u003c!-- BEGIN MUNGE: GENERATED_ANALYTICS --\u003e\n[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/examples/elasticsearch/production_cluster/README.md?pixel)]()\n\u003c!-- END MUNGE: GENERATED_ANALYTICS --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fycliuhw%2Fk8s-elasticsearch-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fycliuhw%2Fk8s-elasticsearch-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fycliuhw%2Fk8s-elasticsearch-cluster/lists"}