{"id":18284636,"url":"https://github.com/pplu/cfn-elasticsearch","last_synced_at":"2026-04-11T20:46:57.041Z","repository":{"id":66753246,"uuid":"320001716","full_name":"pplu/cfn-elasticsearch","owner":"pplu","description":"Article guiding you how to automate Elasticsearch deployments with CloudFormation","archived":false,"fork":false,"pushed_at":"2020-12-15T11:06:38.000Z","size":6,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-15T00:26:10.311Z","etag":null,"topics":["aws","aws-lambda","cloudformation","cloudformation-custom-resource","cloudformation-helpers","cloudformation-templates","elasticsearch","lambda"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pplu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-09T15:38:59.000Z","updated_at":"2020-12-15T11:06:40.000Z","dependencies_parsed_at":"2023-02-24T16:15:12.370Z","dependency_job_id":null,"html_url":"https://github.com/pplu/cfn-elasticsearch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pplu%2Fcfn-elasticsearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pplu%2Fcfn-elasticsearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pplu%2Fcfn-elasticsearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pplu%2Fcfn-elasticsearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pplu","download_url":"https://codeload.github.com/pplu/cfn-elasticsearch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247987107,"owners_count":21028891,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-lambda","cloudformation","cloudformation-custom-resource","cloudformation-helpers","cloudformation-templates","elasticsearch","lambda"],"created_at":"2024-11-05T13:14:15.783Z","updated_at":"2026-04-11T20:46:52.003Z","avatar_url":"https://github.com/pplu.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Automating ElasticSearch with CloudFormation\n\nGetting to provision ElasticSearch clusters with CloudFormation can be a daunting task. You will look at the CloudFormation User Guide, and see that [Elasticsearch](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticsearch-domain.html) is supported, and start merrily on your way to automation heaven.\n\nMy experience getting everything to work as desired has been more of a hassle than I expected, so I'm going to document what I found along the way:\n\n## ElasticSearch VPC needs a ServiceRole to provision correctly\n\nWhen you create an Elasticsearch cluster inside a VPC, you have to create an [IAM Service Linked Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html) so that \nthe Elasticsearch service can see what your VPC looks like, create Network Interfaces, etc.\n\nWhen you create your first Elasticsearch Cluster from the console, AWS does this for you. If you are using an account where you have never created an Elasticsearch cluster from\nthe console, CloudFormation will fail to provision the cluster with the following error:\n\n```\nBefore you can proceed, you must enable a service-linked role to give Amazon ES permissions to access your VPC. \n(Service: AWSElasticsearch; Status Code: 400; Error Code: ValidationException; Request ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx; Proxy: null)\n```\n\nThere is a [CloudFormation Resource for ServiceLinkedRoles](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-servicelinkedrole.html). You happily put that \nin your template, and it works. Great! (for a while)\n\n```\nResources:\n  BasicSLR:\n    Type: 'AWS::IAM::ServiceLinkedRole'\n    Properties:\n      AWSServiceName: es.amazonaws.com\n```\n\nIf you call create-stack again, to instance the template again, you get an error because the Elasticsearch ServiceLinkedRole already exists in IAM!\n\nWorse even: if you do this in an account which already had an Elasticsearch Cluster provisioned, it will also fail (because the role was already created).\n\nSince you read the whole documentation for the resource, you remember the [CustomSuffix](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-servicelinkedrole.html#cfn-iam-servicelinkedrole-customsuffix) property of the ServiceLinkedRole, which says:\n\n\n\u003e A string that you provide, which is combined with the service-provided prefix to form the complete role name. \nIf you make multiple requests for the same service, then you must supply a different CustomSuffix for each request\n\nYay! They have thought of this for me! I can put a `!Ref AWS::StackName` in the suffix!\n\nThe next error in CloudFormation is desolating:\n\n```\nCustom suffix is not allowed for es.amazonaws.com\n```\n\n## Writing a Custom Resource to automate ServiceLinkedRoles\n\nYou always have the option to resort to CloudFormation [Custom Resources](https://docs.aws.amazon.com/en_us/AWSCloudFormation/latest/UserGuide/template-custom-resources-lambda.html) and Lambda functions to get CloudFormations' small gaps filled in, but I really dread doing this:\n\n1. It always introduces the overhead of testing and getting things right. Even if you find the code for the Lambda Function on some blog post, you have to modify it.\n   I've never found a Custom Resource Lambda Function online that completely works for me. They normally are oblivious of updates, don't handle exceptions, don't always return responses\n   (so your stacks get \"stuck\" for hours). Sometimes they swallow an incoming update, reporting success, but not doing any action underneath. Don't get me wrong, I thank the people that \n   share their experience and code! I write this article as a way of giving back, but you have to know that when you \"adopt\" someone elses Custom Resource Lambda (even mine), you very \n   probably have to tailor it to work for your use cases.\n1. You always have the impression that you're doing lots of work that AWS should have already done (I think AWS CloudFormation should be a first-class citizen) and you always have the\n   impression that you're working hard, when \"some time later\" (days? weeks?) AWS will just render your effort obsolete.\n\nSince I needed stuff to work now, and I thought that a resource that ***ensures*** that a Service Linked Role is present, instead of creating it, would be easy enough to build, I\nwent ahead. Updates would be handled by creating a new role, and deletes would be handled by not doing anything (AWS Service Linked Roles are left around all the time, since lots of services\nautomatically create the Role on first use, and they never get deleted).\n\nHere is [my take](servicelinkedrole.yaml) of a Cloudformation Custom Resource for Service Linked Roles that don't support Custom Prefixes.\n\nYou can inline the resources from the template in your stack, or provision the template as a stack, and then use the Lambdas' ARN. You choose.\n\nYou use the resource in your template like this:\n```\n  ESServiceRole:\n    Type: Custom::EnsuredServiceRole\n    Version: '1.0'\n    Properties:\n      ServiceToken: !GetAtt EnsuredServiceRole.Arn\n      Service: es.amazonaws.com\n```\n\nSo now my trip was seemingly going to be pleasant (or so I thought).\n\n## Enabling ElasticSearch to log to CloudWatch Logs\n\nThe Elasticsearch CloudFormation resource lets you configure logs coming from the ElasticSearch server to CloudWatch Logs.\n\nYou innocently create some log groups in the template for each type of log you want configured:\n\n```\nResources:\n  ElasticSearch:\n    Type: AWS::Elasticsearch::Domain\n    Properties:\n      [...]\n      LogPublishingOptions:\n        SEARCH_SLOW_LOGS:\n          CloudWatchLogsLogGroupArn: !GetAtt LogGroupSearchSlow.Arn\n          Enabled: true\n        INDEX_SLOW_LOGS:\n          CloudWatchLogsLogGroupArn: !GetAtt LogGroupIndexSlow.Arn\n          Enabled: true\n        ES_APPLICATION_LOGS:\n          CloudWatchLogsLogGroupArn: !GetAtt LogGroupApplication.Arn\n          Enabled: true\n      [...]\n  LogGroupSearchSlow:\n    Type: AWS::Logs::LogGroup\n    Properties:\n      LogGroupName: !Join [ '', [ '/', { Ref: 'AWS::StackName' }, '/es/search-slow' ] ] \n      RetentionInDays: 7\n  LogGroupIndexSlow:\n    Type: AWS::Logs::LogGroup\n    Properties:\n      LogGroupName: !Join [ '', [ '/', { Ref: 'AWS::StackName' }, '/es/index-slow' ] ] \n      RetentionInDays: 7\n  LogGroupApplication:\n    Type: AWS::Logs::LogGroup\n    Properties:\n      LogGroupName: !Join [ '', [ '/', { Ref: 'AWS::StackName' }, '/es/application' ] ] \n      RetentionInDays: 7\n```\n\nWhat is your surprise when you get an error in CloudFormation:\n```\nThe Resource Access Policy specified for the CloudWatch Logs log group /xxxxxxxxx/es/search-slow does not grant \nsufficient permissions for Amazon Elasticsearch Service to create a log stream. Please check the Resource Access Policy. \n(Service: AWSElasticsearch; Status Code: 400; Error Code: ValidationException; Request ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx; Proxy: null)\n```\n\nThis is not expected, because you can't find any reference or documentation in the CloudFormation User Guide advising you of such thing.\n\nI even got confused, and tried to attach a AWS::IAM::Policy resource to the ElasticSearch Service role so that the ElasticSearch service could\nwrite to the Log groups. This failed pretty badly, as you cannot modify Service Roles.\n\nI finally [found a poor soul in my same situation](https://stackoverflow.com/questions/62912027/cloudwatch-resource-access-policy-error-while-creating-amazon-elasticsearch-serv)\n\nIt results that you have to tell CloudWatch logs to let the ElasticSearch service write to the LogGroups. [There is no resource in CloudFormation to do this](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_Logs.html). This is done by calling IAM `PutResourcePolicy` on the Logs service:\n\n```\nSERVICE=es.amazonaws.com\nARN=arn:aws:logs:us-east-1:xxxxxxxxxxxx:log-group:/xxxxxxxxx/es/application:*\naws logs put-resource-policy --policy-name POLICY_NAME --policy-document \"{\\\"Version\\\":\\\"2012-10-17\\\",\\\"Statement\\\":[{\\\"Effect\\\":\\\"Allow\\\",\\\"Principal\\\":{\\\"Service\\\":\\\"$SERVICE\\\"},\\\"Action\\\":[\\\"logs:CreateLogStream\\\",\\\"logs:PutLogEvents\\\"],\\\"Resource\\\":\\\"$ARN\\\"}]}\"\n```\n\nThere's no way of doing this from CloudFormation :(\n\n## Writing a Custom Resource to automate PutResourcePolicy\n\nI've already stated how unconfortable it feels to be writing a CloudFormation Custom Resource...\n\nBut since I need this stuff to work now, and I found someone with a similar problem, and some [code](https://gist.github.com/sudharsans/cf9c52d7c78a81818a4a47872982bd76]), \nI went ahead and solved this issue too.\n\nYou can find the code for my take on the CloudWatch Logs PutResourcePolicy Resource [here](permission_customresource.yaml). Note that the `service` is hardcoded to `es.amazonaws.com`. Changing it, or making it a parameter in the stack is left as ani exercise to the reader (PRs welcome).\n\nYou can inline the resources from the template in your stack, or provision the template as a stack, and then use the Lambdas' ARN. Your choice.\n\nTo use the Custom Resource:\n```\n  ResourcePolicyForSearchSlow:\n    Type: Custom::AddResourcePolicy\n    Version: '1.0'\n    Properties:\n      ServiceToken: !GetAtt GiveLogsWritePermissionTo.Arn\n      LogGroupArn: !GetAtt LogGroupSearchSlow.Arn\n  ResourcePolicyForIndexSlow:\n    Type: Custom::AddResourcePolicy\n    Version: '1.0'\n    Properties:\n      ServiceToken: !GetAtt GiveLogsWritePermissionTo.Arn\n      LogGroupArn: !GetAtt LogGroupIndexSlow.Arn\n  ResourcePolicyForApplication:\n    Type: Custom::AddResourcePolicy\n    Version: '1.0'\n    Properties:\n      ServiceToken: !GetAtt GiveLogsWritePermissionTo.Arn\n      LogGroupArn: !GetAtt LogGroupApplication.Arn\n```\n\nNote that on your ElasticSearch cluster, you must `DependOn` these resources, to assure they are created before CloudFormation creates the Elasticsearch cluster:\n\n```\nResources:\n  ElasticSearch:\n    Type: AWS::Elasticsearch::Domain\n    DependsOn:\n    - AddResourcePolicyForIndexSlow\n    - AddResourcePolicyForIndexSlow\n    - AddResourcePolicyForApplication\n    Properties:\n      [...]\n```\n\n# Conclusions\n\nI feel AWS could do a lot to improve the user experience for automating ElasticSearch clusters, as there are some sharp edges:\n - AWS could put a new property on the `AWS::IAM::ServiceLinkedRole` object that assures that the role exists (and doesn't delete it on stack delete).\n - AWS should document in the CloudFormation User Guide that you have to call PutResourcePolicy on CloudWatch logs for ElasticSearch to be able to write to the appropiate Log Groups (note that Route 53 has a configuration with the same problem). This would save a lot of misunderstanding, second guessing, etc.\n - AWS should really write a CloudWatch Logs resource for calling PutResourcePolicy on CloudWatch Logs. This is kind-of a big oversight, since there are people struggling with this. You have two pieces available in CloudFormation, but you don't have the glue to stick them together.\n\n# Author, Copyright and License\n\nThis article was authored by Jose Luis Martinez Torres.\n\nThis article is (c) 2020 Jose Luis Martinez Torres, Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).\n\nThe canonical, up-to-date source is [GitHub](https://github.com/pplu/cfn-elasticsearch). Feel free to contribute back.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpplu%2Fcfn-elasticsearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpplu%2Fcfn-elasticsearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpplu%2Fcfn-elasticsearch/lists"}