{"id":16195284,"url":"https://github.com/tethik/terraform-aws-elasticsearch","last_synced_at":"2026-01-20T04:32:22.378Z","repository":{"id":146621964,"uuid":"172888829","full_name":"Tethik/terraform-aws-elasticsearch","owner":"Tethik","description":null,"archived":false,"fork":false,"pushed_at":"2019-03-12T13:00:04.000Z","size":74,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-07T15:55:18.663Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Tethik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-27T09:53:08.000Z","updated_at":"2019-03-12T13:00:06.000Z","dependencies_parsed_at":"2023-04-19T02:28:13.494Z","dependency_job_id":null,"html_url":"https://github.com/Tethik/terraform-aws-elasticsearch","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/Tethik/terraform-aws-elasticsearch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tethik%2Fterraform-aws-elasticsearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tethik%2Fterraform-aws-elasticsearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tethik%2Fterraform-aws-elasticsearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tethik%2Fterraform-aws-elasticsearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Tethik","download_url":"https://codeload.github.com/Tethik/terraform-aws-elasticsearch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tethik%2Fterraform-aws-elasticsearch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28596079,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T02:08:49.799Z","status":"ssl_error","status_checked_at":"2026-01-20T02:08:44.148Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-10T08:26:56.911Z","updated_at":"2026-01-20T04:32:22.365Z","avatar_url":"https://github.com/Tethik.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003e ⚠️ This project was created as part of a technical test as part of a job interview. It has never ran in production, and my ability to support this project will be limited since I'm not currently using it myself.\n\n# terraform-aws-elasticsearch\n\n![version 0.1.1](version.svg)\n\nThis is a terraform module to easily provision an AWS ElasticSearch domain inside a VPC for smaller clusters. It is structured into three separate modules.\n\n- `aws-elasticsearch` is the main module which provisions the elasticsearch domain inside your VPC. This is the basic requirement.\n- `aws-elasticsearch-cloudwatch-dashboard` which adds a simple CloudWatch dashboard to your elasticsearch domain.\n- `aws-elasticsearch-cloudwatch-sns-alerting` which adds a single alarm to send an email if your elasticsearch domain status is no longer green. **Currently this does not work though :(**\n\n## Usage\n\nHere's some code by example, just fill in the gaps.\n\n```t\nmodule \"aws-elasticsearch\" {\n  # Replace the version number at the end to use a newer (or older) version of the module.\n  source = \"git::git@github.com:Tethik/terraform-aws-elasticsearch.git//aws-elasticsearch?ref=0.1.1\"\n\n  domain     = \"\" # The domain name of the ES cluster\n  vpc_id     = \"\" # The VPC to run the ES cluster inside. This will limit the access to only hosts inside that VPC.\n  subnet_ids = [\"\"] # The subnet(s) inside the VPC to place the cluster in.\n  elasticsearch_version = \"\" # Defaults to 6.4\n  elasticsearch_instance_type = \"\" # Defaults to t2.small.elasticsearch\n  elasticsearch_instance_count = \"\" # Defaults to 1\n  elasticsearch_volume_size = \"\" # Size in GB of the disk allocated to the ES Instance(s). Defaults to 10GB\n}\n\nmodule \"aws-elasticsearch-cloudwatch-dashboard\" {\n  source = \"git::git@github.com:Tethik/terraform-aws-elasticsearch.git//aws-elasticsearch-cloudwatch-dashboard?ref=0.1.1\"\n\n  domain = \"\" # The domain name of the ES cluster. Use the same as above.\n}\n\nmodule \"aws-elasticsearch-cloudwatch-sns-alerting\" {\n  source = \"git::git@github.com:Tethik/terraform-aws-elasticsearch.git//aws-elasticsearch-cloudwatch-sns-alerting?ref=0.1.1\"\n\n  domain       = \"\" # The domain name of the ES cluster. Use the same as above.\n  alarms_email = \"\" # The email to send alarms to\n}\n```\n\n## Testing\n\nIn the top level of this repository I have provided a `sample.tf` file which showcases how to use these modules.\nSimply `terraform plan/apply` with your AWS credentials to try it out.\n\nBy default the sample will set an [access policy](https://github.com/Tethik/terraform-aws-elasticsearch/blob/master/sample.tf#L45-L63) to allow for anything coming from AWS (inside the VPC).\n\nUse an EC2 instance inside the VPC as a proxy, using SSH to tunnel the connection. E.g.\n\n```\nssh \u003cuser@ip\u003e -N -L 9200:vpc-example-n2mczfaqfo25w65nd4afx23yim.eu-central-1.es.amazonaws.com:443\n```\n\nNow you should be able to access the cluster on `https://localhost:9200` and kibana on `https://localhost:9200/_plugin/kibana`\n\n## Monitoring\n\nIncluded is a CloudWatch dashboard with some minimal stats displayed. You'll find it on your AWS account under `CloudWatch -\u003e Dashboards`.\n![A nice picture of the dashboard](./dashboard.png)\n\nBy default the AWS ES domains already publish metrics to CloudWatch. These can then be used elsewhere too.\nhttps://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains.html#es-managedomains-cloudwatchmetrics\n\n## Further Improvements\n\nThese are some improvements I would make before considering this module production ready. In the interest of time\nI won't delve too much into them, but I want to note them down to show my awareness of the problems that exist.\n\n## Scaling\n\nI did not test anything besides the very smallest instance size to save costs. Not sure how this would work for larger clusters.\nThere's a lot of configuration available that you may want to enable if you need to run a larger cluster that I do not expose here (e.g. master nodes?).\n\n### Longer term storage\n\nWhat happens when the cluster runs out of space? This could eventually happen, it will definitely happen\nif you log e.g. requests.\n\nIdeally you would not have to keep that data. In which case you could just rotate out old data.\n\nOtherwise you might want to also ship the raw logs somewhere, e.g. S3. Not sure how this would work.\n\n### Version Upgrades\n\nNot sure how upgrades would work, but there should be a strategy and it should be tested. i.e. How do you upgrade to a newer\nversion of ES without losing data? For now I've set up some basic versioning on the project using [git flow](https://github.com/nvie/gitflow). I would maintain it using [semantic versioning](https://semver.org/) to organize the version numbers and communicate breaking changes.\n\n### Request Signing\n\nBy default AWS ElasticSearch requires any requests to it to be signed by an IAM credential that has the correct authorization.\nI still need to figure out how would this actually work in practice with Logstash/Beats.\n\n### Better dashboard/alarms based on ES Domain Knowledge\n\nI don't have much experience using ES long term, and most of my experience has been as a developer. I don't know\nwhich metrics are important to keep track of. Therefore I would have liked to invest more time into reading up on\nthis, or asking someone more knowledgeable on hosting ES clusters as to what is important to keep track of.\n\n### Automated Testing\n\nThis is code after all. We could code tests for it to e.g. try connecting to the cluster and logging some data, or ensuring that the cloudwatch alarm works. This would be helpful during development.\n\n### Multiple Subnets\n\nI don't think it currently works to give the module multiple subnets. It would have been nice to only specify the VPC. This might\nhave to do with the fact that I only run a single instance though. Maybe if I ran more AWS would be smart enough to distribute them\naccross the different subnets.\n\n## TODO\n\nMy own little todolist before I sign off on this task:\n\n1. Fix Alerting\n\n## Sources / References\n\nVPC ES setup\nhttps://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-vpc.html#es-prerequisites-vpc-endpoints\n\nES CloudWatch\nhttps://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains.html#es-managedomains-cloudwatchmetrics\n\nAlerting + Monitoring example, especially helpful for the email part.\nhttps://stephenmann.io/post/setting-up-monitoring-and-alerting-on-amazon-aws-with-terraform/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftethik%2Fterraform-aws-elasticsearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftethik%2Fterraform-aws-elasticsearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftethik%2Fterraform-aws-elasticsearch/lists"}