{"id":24629577,"url":"https://github.com/framsouza/eck-ready-for-production","last_synced_at":"2025-05-07T22:20:52.421Z","repository":{"id":57799509,"uuid":"460815995","full_name":"framsouza/eck-ready-for-production","owner":"framsouza","description":"If you want to spin up ECK with production feature like, SAML cert-manager, this guide is for you.","archived":false,"fork":false,"pushed_at":"2022-04-26T15:23:26.000Z","size":101546,"stargazers_count":36,"open_issues_count":1,"forks_count":17,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-31T14:43:01.908Z","etag":null,"topics":["cert-manager","cloudflare","eck","elasticsearch","esrally","external-dns","ingress-nginx","kibana","kubernetes","monitoring"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/framsouza.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"license.yaml","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-02-18T10:54:47.000Z","updated_at":"2025-03-25T07:47:19.000Z","dependencies_parsed_at":"2022-08-26T02:30:33.529Z","dependency_job_id":null,"html_url":"https://github.com/framsouza/eck-ready-for-production","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/framsouza%2Feck-ready-for-production","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/framsouza%2Feck-ready-for-production/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/framsouza%2Feck-ready-for-production/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/framsouza%2Feck-ready-for-production/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/framsouza","download_url":"https://codeload.github.com/framsouza/eck-ready-for-production/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252962577,"owners_count":21832336,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cert-manager","cloudflare","eck","elasticsearch","esrally","external-dns","ingress-nginx","kibana","kubernetes","monitoring"],"created_at":"2025-01-25T06:13:18.350Z","updated_at":"2025-05-07T22:20:52.399Z","avatar_url":"https://github.com/framsouza.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# ECK in production environment\n\nThis articles will guide you on how to spin up an ECK environment ready for production which contains,\n\n- dedicated stack monitoring,\n- fleet-server \u0026 elastic-agent (with kubernetes integration),\n- elasticsearch autoscaling,\n- nodeAffinity \u0026 nodeSelector,\n- SAML with auth0,\n- hot, warm, cold, frozen architecture,\n- heartbeat monitor SSL certificate expiration,\n- [external-dns](https://github.com/kubernetes-sigs/external-dns) (with CloudFlare integration),\n- [cert-manager](https://github.com/cert-manager/cert-manager) -with let's encrypt integration),\n- [ingress controller](https://kubernetes.github.io/ingress-nginx/) (using Cloudflare to register the endpoints)\n- _Optional_: [esrally](https://esrally.readthedocs.io/en/stable/) to validate autoscaling and run benchmark against es cluster\n\n_Remember you must have `basic` or `enterprise` license to run ECK._\n\n## Demo\n\nhttps://user-images.githubusercontent.com/16880741/165335636-94099118-e880-4004-83f1-040ea21c7a5f.mov\n\n\n### cert-manager\ncert-manager adds certificates and  certificate issuers as resource types in Kubernetes, and simplifies the process of obtaining, renewing and using those certificates. It's very command Kubernetes administrator use cert-manager to handle certificate, and on this exemple we are going to use cert-manager with let's encrypt to access Kibana. Remember that, TLS certificates for the Elasticsearch transport layer that are used for internal communications between Elasticsearch nodes are managed by ECK and **cannot** be changed.\n\n### ingress\nIngress controller is specialized load balancer for Kubernetes, qhich accepts traffic from outside the Kubernetes cluster and balances it to pods.\n\n### external-dns\nIt's a addon that configures public DNS servers about exposed Kubernetes services, on this examples we are integrating external-dns with Cloudflare. For each Ingress/Service resource you us, a DNS entry will created on Cloudflare with the respective IP address, on external-dns logs you should be able to see the following\n\n```\nlevel=info msg=\"Using inCluster-config based on serviceaccount-token\"\nlevel=info msg=\"Created Kubernetes client https://10.76.0.1:443\"\nlevel=info msg=\"Changing record.\" action=CREATE record=kibana.framsouza.co ttl=1 type=A zone=4cd4c7c1cb8f7bf3a7482749654ae6fb\nlevel=info msg=\"Changing record.\" action=CREATE record=monitoring.framsouza.co ttl=1 type=TXT zone=4cd4c7c1cb8f7bf3a7482749654ae6fb\n```\n\n### How-to setup\n\n_Make sure to respect the commands execution order_\n\n1. Create GKE cluster with Kubernetes `type` hot, warm, cold,  frozen for each dedicated node pool, make sure you will have enough resouce to run the pods in the nodes. [Here](https://github.com/framsouza/terraform), there's a terraform example that will spin up it for you,\n2. Create a cluster role mapping that gives you permission to install ECK operator\n\t- `kubectl create clusterrolebinding cluster-admin-binding --cluster-role=cluster-admin --user=\u003cUSERNAME\u003e`\n3. Install ECK operator\n\t- `helm repo add elastic https://helm.elastic.co \u0026\u0026 helm repo update \u0026\u0026 helm install elastic-operator elastic/eck-operator -n elastic-system --create-namespace`\n4. Create dedicated storage class by applying files **storageclass-hot.yaml** and **storageclass-warm.yaml**\n5. Download your license and apply it via secret (or apply the [license.yaml](https://github.com/framsouza/eck-ready-for-production/blob/main/license.yaml))\n\t- `kubectl create secret generic eck-license --from-file \u003cLICENSE-PATH\u003e -n elastic-system \u0026\u0026 kubectl label secret eck-license \"license.k8s.elastic.co/scope\"=operator -n elastic-system`\n6. Create the monitoring cluster (it will create a `ns` call *monitoring*) by applying [monitoring-es.yaml](https://github.com/framsouza/eck-ready-for-production/blob/main/monitoring-es.yaml),\n7. Create elasticsearch resource, [elasticsearch.yaml](https://github.com/framsouza/eck-ready-for-production/blob/main/elasticsearch.yaml)\n8. Create kibana resource, [kibana.yaml](https://github.com/framsouza/eck-ready-for-production/blob/main/kibana.yaml)\n9. Create fleet resource, [fleet.yaml](https://github.com/framsouza/eck-ready-for-production/blob/main/fleet.yaml)\n10. Create heartbeat, [heartbeat.yaml](https://github.com/framsouza/eck-ready-for-production/blob/main/heartbeat.yaml)\n11. Install external-dns\n\t- `kubectl apply -f external-dns.yml`\n12. Install cert-manager\n\t- `kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.7.1/cert-manager.yaml`\n13. Install ingress-nginx ,\n\t- `helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace`\n16. Create ingress resource, [ingress.yaml](https://github.com/framsouza/eck-ready-for-production/blob/main/ingress.yaml)\n14. Create cluster issuer apply [clusterissuer.yaml](https://github.com/framsouza/eck-ready-for-production/blob/main/clusterissuer.yaml)\n15. Create let's encrypt certificate [certificate.yaml](https://github.com/framsouza/eck-ready-for-production/blob/main/certificate.yaml)\n\n### Accessing\n\nFor this example, I am using a domain call **framsouza.co** and as I am using external-dns, the DNS entry will be automatically added to Cloudflare, **https://kibana.framsouza.co** and **https://monitoring.framsouza.co**.\nYou can check the connection is safe and we are using a valid certificate by let's encrypt.\n\nIf you want to login using SAML, make sure to adjust the `saml` session on `elasticsearch.yml` according to your environment.\n\n\n### Autoscaling validation\n(_Optional_)\n\nTo confirm if autoscaling is working as expected, you can use [esrally](https://esrally.readthedocs.io/en/stable/) to test that and run benchmark against your cluster. Make sure to have `params-file.json` with the following content:\n```\n{\n    \"number_of_replicas\": 1,\n    \"number_of_shards\": 2\n}\n```\n\nthen you just need to run and wait a couple of hours until the test is finished.\n\n```\ndocker run -v /tmp/params-file.json:/tmp/params-file.json elastic/rally race --track=http_logs --target-hosts=${IP}:9200 --pipeline=benchmark-only --client-options=\"timeout:60,use_ssl:true,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'${PASSWORD}'\"  --track-params=/tmp/params-file.json\n```\n\nHave a look at [esrally-result.txt](https://github.com/framsouza/eck-ready-for-production/blob/main/esrally-result.txt).\n\n\nSeya.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fframsouza%2Feck-ready-for-production","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fframsouza%2Feck-ready-for-production","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fframsouza%2Feck-ready-for-production/lists"}