{"id":13510006,"url":"https://github.com/ozbillwang/terraform-best-practices","last_synced_at":"2025-05-14T07:08:42.002Z","repository":{"id":42569249,"uuid":"94737529","full_name":"ozbillwang/terraform-best-practices","owner":"ozbillwang","description":"Terraform Best Practices for AWS users","archived":false,"fork":false,"pushed_at":"2024-12-20T00:40:37.000Z","size":158,"stargazers_count":1778,"open_issues_count":2,"forks_count":355,"subscribers_count":64,"default_branch":"master","last_synced_at":"2025-04-11T02:51:33.763Z","etag":null,"topics":["best-practices","best-practises","hashicorp","infrastructure-as-code","terraform"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ozbillwang.png","metadata":{"files":{"readme":"README.0.11.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["ozbillwang"],"custom":["https://www.buymeacoffee.com/ozbillwang","https://github.com/sponsors/ozbillwang"]}},"created_at":"2017-06-19T04:49:23.000Z","updated_at":"2025-04-07T12:35:19.000Z","dependencies_parsed_at":"2023-01-20T04:33:59.709Z","dependency_job_id":"aaeb03c8-289e-4c4f-a5f7-d1c7a3bbe327","html_url":"https://github.com/ozbillwang/terraform-best-practices","commit_stats":{"total_commits":144,"total_committers":18,"mean_commits":8.0,"dds":0.5277777777777778,"last_synced_commit":"6602587b89f9c06ec79dc48cf31d1f0553318111"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ozbillwang%2Fterraform-best-practices","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ozbillwang%2Fterraform-best-practices/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ozbillwang%2Fterraform-best-practices/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ozbillwang%2Fterraform-best-practices/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ozbillwang","download_url":"https://codeload.github.com/ozbillwang/terraform-best-practices/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254092775,"owners_count":22013290,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["best-practices","best-practises","hashicorp","infrastructure-as-code","terraform"],"created_at":"2024-08-01T02:01:20.941Z","updated_at":"2025-05-14T07:08:36.991Z","avatar_url":"https://github.com/ozbillwang.png","language":"HCL","funding_links":["https://github.com/sponsors/ozbillwang","https://www.buymeacoffee.com/ozbillwang"],"categories":["HCL","DevOps","best-practices"],"sub_categories":["Terraform"],"readme":"# terraform-best-practices\n\nTerraform Best Practices for AWS users.\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n\n- [Run terraform command with var-file](#run-terraform-command-with-var-file)\n- [Manage s3 backend for tfstate files](#manage-s3-backend-for-tfstate-files)\n  - [Notes](#notes)\n- [Manage multiple Terraform modules and environments easily with Terragrunt](#manage-multiple-terraform-modules-and-environments-easily-with-terragrunt)\n- [Retrieve state meta data from a remote backend](#retrieve-state-meta-data-from-a-remote-backend)\n- [Turn on debug when you need to do troubleshooting.](#turn-on-debug-when-you-need-to-do-troubleshooting)\n- [Use shared modules](#use-shared-modules)\n  - [Notes](#notes-1)\n- [Isolate environment](#isolate-environment)\n- [Use terraform import to include as many resources as you can](#use-terraform-import-to-include-as-many-resources-as-you-can)\n- [Avoid hard coding the resources](#avoid-hard-coding-the-resources)\n- [Format terraform code](#format-terraform-code)\n- [Enable version control on terraform state files bucket](#enable-version-control-on-terraform-state-files-bucket)\n- [Generate README for each module with input and output variables](#generate-readme-for-each-module-with-input-and-output-variables)\n- [Update terraform version](#update-terraform-version)\n- [Run terraform from docker container](#run-terraform-from-docker-container)\n- [Troubleshooting with messy output - (Decommissioned)](#troubleshooting-with-messy-output---decommissioned)\n- [Run test](#run-test)\n  - [Quick start](#quick-start)\n  - [Run test within docker container](#run-test-within-docker-container)\n- [Minimum AWS permissions necessary for a Terraform run](#minimum-aws-permissions-necessary-for-a-terraform-run)\n- [Tips to deal with lambda functions](#tips-to-deal-with-lambda-functions)\n  - [Explanation](#explanation)\n  - [Notes](#notes-2)\n- [Usage of variable \"self\"](#usage-of-variable-self)\n  - [One more use case](#one-more-use-case)\n- [Use pre-installed Terraform plugins](#use-pre-installed-terraform-plugins)\n- [Useful documents you should read](#useful-documents-you-should-read)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n## Run terraform command with var-file\n\n```\n$ cat config/dev.tfvars\n\nname = \"dev-stack\"\ns3_terraform_bucket = \"dev-stack-terraform\"\ntag_team_name = \"hello-world\"\n\n$ terraform plan -var-file=config/dev.tfvars\n```\n\nWith `var-file`, you can easily manage environment (dev/stag/uat/prod) variables.\n\nWith `var-file`, you avoid running terraform with long list of key-value pairs ( `-var foo=bar` )\n\n## Manage s3 backend for tfstate files\n\nTerraform doesn't support [Interpolated variables in terraform backend config](https://github.com/hashicorp/terraform/pull/12067), normally you write a separate script to define s3 backend bucket name for different environments, but I recommend to hard code it directly as below.\n\nAdd below code in terraform configuration files.\n\n```\n$ cat main.tf\n\nterraform {\n  required_version = \"~\u003e 0.10\"\n\n  backend \"s3\" {\n    encrypt = true\n  }\n}\n```\n\nDefine backend variables for particular environment\n\n```\n$ cat config/backend-dev.conf\nbucket  = \"\u003cunique_bucket_name\u003e-terraform-development\"\nkey     = \"development/service-1.tfstate\"\nencrypt = true\nregion  = \"ap-southeast-2\"\nkms_key_id = \"alias/terraform\"\ndynamodb_table = \"terraform-lock\"\n```\n\n### Notes\n\n- bucket - s3 bucket name, has to be globally unique.\n- key - Set some meaningful names for different services and applications, such as vpc.tfstate, application_name.tfstate, etc\n- dynamodb_table - optional when you want to enable [State Locking](https://www.terraform.io/docs/state/locking.html)\n\nAfter you set `config/backend-dev.conf` and `config/dev.tfvars` properly (for each environment). You can easily run terraform as below:\n\n```bash\nenv=dev\nterraform get -update=true\nterraform init -backend-config=config/backend-${env}.conf\nterraform plan -var-file=config/${env}.tfvars\nterraform apply -var-file=config/${env}.tfvars\n```\n\n## Manage multiple Terraform modules and environments easily with Terragrunt\n\nTerragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules. https://www.gruntwork.io\n\nSample for reference: https://github.com/gruntwork-io/terragrunt-infrastructure-live-example\n\nIts README is too long, if you need a quick start, follow below steps:\n\n```bash\n# Install terraform and terragrunt\n# Make sure you are in right aws account\n$ aws s3 ls\n# use terragrunt to deploy\n$ git clone https://github.com/gruntwork-io/terragrunt-infrastructure-live-example.git\n$ cd terragrunt-infrastructure-live-example\n# for example, you want to deploy mysql in stage non-prod at region us-east-1\n$ cd non-prod/us-east-1/stage/mysql\n$ terragrunt plan\n# Confirm everything works\n$ terragrunt apply\n```\n\nSo if you followed the setting in terragrunt properly, you don't need to care about the backend state files and variable file path in different environments, even more, you can run `terragrunt plan-all` to plan all modules together.\n\n## Retrieve state meta data from a remote backend\n\nNormally we have several layers to manage terraform resources, such as network, database, and application layers. After you create the basic network resources, such as vpc, security group, subnets, nat gateway in vpc stack. Your database layer and applications layer should always refer the resource from vpc layer directly via `terraform_remote_state` data source.\n\n```terraform\ndata \"terraform_remote_state\" \"vpc\" {\n  backend = \"s3\"\n  config{\n    bucket = \"${var.s3_terraform_bucket}\"\n    key = \"${var.environment}/vpc.tfstate\"\n    region=\"${var.aws_region}\"\n  }\n}\n\n# Retrieves the vpc_id and subnet_ids directly from remote backend state files.\nresource \"aws_xx_xxxx\" \"main\" {\n  # ...\n  subnet_ids = \"${split(\",\", data.terraform_remote_state.vpc.data_subnets)}\"\n  vpc_id     = \"${data.terraform_remote_state.vpc.vpc_id}\"\n}\n```\n\n## Turn on debug when you need to do troubleshooting.\n\n```bash\nTF_LOG=DEBUG terraform \u003ccommand\u003e\n\n# or if you run with terragrunt\nTF_LOG=DEBUG terragrunt \u003ccommand\u003e\n```\n\n## Use shared modules\n\nManage terraform resource with shared modules, this will save a lot of coding time. No need to re-invent the wheel!\n\nYou can start from below links:\n\n[terraform module usage](https://www.terraform.io/docs/modules/usage.html)\n\n[Terraform Module Registry](https://registry.terraform.io/)\n\n[Terraform aws modules](https://github.com/terraform-aws-modules)\n\n### Notes\n\nterraform modules don't support `count` parameter currently. You can follow up this ticket for updates: https://github.com/hashicorp/terraform/issues/953\n\n## Isolate environment\n\nSometimes, developers like to create a security group and share it to all non-prod (dev/qa) environments. Don't do that, create resources with different name for each environment and each resource.\n\n```terraform\nvariable \"application\" {\n  description = \"application name\"\n  default = \"\u003creplace_with_your_project_or_application_name\u003e\"\n}\n\nvariable \"environment\" {\n  description = \"environment name\"\n}\n\nlocals {\n  name_prefix    = \"${var.application}-${var.environment}\"\n}\n\nresource \"\u003cany_resource\u003e\" {\n  name = \"${local.name_prefix}-\u003cresource_name\u003e\"\n  # ...\n}\n```\n\nWith that, you will easily define the resource with a meaningful and unique name, and you can build more of the same application stack for different developers without change a lot. For example, you update the environment to dev, staging, uat, prod, etc.\n\n\u003e Tips: some aws resource names have length limits, such as less than 24 characters, so when you define variables of application and environment name, use short name.\n\n## Use terraform import to include as many resources as you can\n\nSometimes developers manually created resources. You need to mark these resource and use `terraform import` to include them in codes.\n\n[terraform import](https://www.terraform.io/docs/import/usage.html)\n\n## Avoid hard coding the resources\n\nA sample:\n\n```\naccount_number=\"123456789012\"\naccount_alias=\"mycompany\"\nregion=\"us-east-2\"\n```\n\nThe current aws account id, account alias and current region can be input directly via [data sources](https://www.terraform.io/docs/providers/aws/).\n\n```terraform\n# The attribute `${data.aws_caller_identity.current.account_id}` will be current account number.\ndata \"aws_caller_identity\" \"current\" {}\n\n# The attribue `${data.aws_iam_account_alias.current.account_alias}` will be current account alias\ndata \"aws_iam_account_alias\" \"current\" {}\n\n# The attribute `${data.aws_region.current.name}` will be current region\ndata \"aws_region\" \"current\" {}\n\n# Set as [local values](https://www.terraform.io/docs/configuration/locals.html)\nlocals {\n  account_id    = \"${data.aws_caller_identity.current.account_id}\"\n  account_alias = \"${data.aws_iam_account_alias.current.account_alias}\"\n  region        = \"${data.aws_region.current.name}\"\n}\n```\n\n## Format terraform code\n\nAlways run `terraform fmt` to format terraform configuration files and make them neat.\n\nI used below code in Travis CI pipeline (you can re-use it in any pipelines) to validate and format check the codes before you can merge it to master branch.\n\n      - find . -type f -name \"*.tf\" -exec dirname {} \\;|sort -u | while read m; do (terraform validate -check-variables=false \"$m\" \u0026\u0026 echo \"√ $m\") || exit 1 ; done\n      - terraform fmt -check=true -write=false -diff=true\n\n## Enable version control on terraform state files bucket\n\nAlways set backend to s3 and enable version control on this bucket.\n\nIf you'd like to manage terraform state bucket as well, I recommend using this repostory I wrote [tf_aws_tfstate_bucket](https://github.com/BWITS/tf_aws_tfstate_bucket) to create the bucket and replicate to other regions automatically.\n\n## Generate README for each module with input and output variables\n\nYou needn't manually manage `USAGE` about input variables and outputs. [terraform-docs](https://github.com/segmentio/terraform-docs) can do this job automatically.\n\n```bash\n$ brew install terraform-docs\n$ cd terraform/modules/vpc\n$ terraform-docs md . \u003e README.md\n```\n\nFor details on how to run `terraform-docs`, check this repository: https://github.com/segmentio/terraform-docs\n\nThere is a simple sample for you to start [tf_aws_acme](https://github.com/BWITS/tf_aws_acme), the README is generatd by `terraform-docs`\n\n## Update terraform version\n\nHashicorp doesn't have a good qa/build/release process for their software and does not follow semantic versioning rules.\n\nFor example, `terraform init` isn't compatible between 0.9 and 0.8. Now they are going to split providers and use \"init\" to install providers as plugin in coming version 0.10\n\nSo recommend to keep updating to latest terraform version\n\n## Run terraform from docker container\n\nTerraform releases official docker containers that you can easily control which version you can run.\n\nRecommend to run terraform docker container, when you set your build job in CI/CD pipeline.\n\n```bash\nTERRAFORM_IMAGE=hashicorp/terraform:0.9.8\nTERRAFORM_CMD=\"docker run -ti --rm -w /app -v ${HOME}/.aws:/root/.aws -v ${HOME}/.ssh:/root/.ssh -v `pwd`:/app -w /app ${TERRAFORM_IMAGE}\"\n${TERRAFORM_CMD} init\n${TERRAFORM_CMD} plan\n```\n\n## Troubleshooting with messy output - (Decommissioned)\n\n\u003e (Decommissioned) after terraform v0.12.x, we needn't run with `terraform-landscape` any more. The new terraform output looks nice already.\n\nSometime, you applied the changes several times, the plan output always prompts there are some changes, essepecially in iam and s3 policy. It is hard to troubleshooting the problem with messy json output in one line.\n\nWith the tool [terraform-landscape](https://github.com/coinbase/terraform-landscape), it improves Terraform plan output to be easier for reading, you can easily find out where is the problem. For details, please go through the project at https://github.com/coinbase/terraform-landscape\n\n    # Install terraform_landscape\n    gem install terraform_landscape\n    # On MacOS, you can install with brew\n    brew install terraform_landscape\n\n    terraform plan -var-file=${env}/${env}.tfvars -input=false -out=plan -lock=false |tee report\n    landscape \u003c report\n\n    # run terraform-landscape container as command\n    alias landscape=\"docker run -i --rm -v $(pwd):/apps alpine/landscape:0.1.18\"\n    landscape --help\n    terraform plan |tee report\n    landscape - \u003c report\n    # Or\n    terraform plan | landscape -\n\nAnother quick way to handle the messy output is to run below command, if you manually copy the part of messy output to a file\n\n    cat output.txt | grep -Ev '\"([^\"]*)\" =\u003e \"\\1\"'\n\n## Run test\n\nRecommend to add [awspec](https://github.com/k1LoW/awspec) tests through [kitchen](https://kitchen.ci/) and [kitchen-terraform](https://newcontext-oss.github.io/kitchen-terraform/).\n\n### Quick start\n\nReference: repo [terraform-aws-modules/terraform-aws-eks](https://github.com/terraform-aws-modules/terraform-aws-eks#testing)\n\n### Run test within docker container\n\nReference: [README for terraform awspec container](https://github.com/alpine-docker/bundle-terraform-awspec)\n\n## Minimum AWS permissions necessary for a Terraform run\n\nThere will be no answer for this. But with below iam policy you can easily get started.\n\n```json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"AllowSpecifics\",\n      \"Action\": [\n        \"lambda:*\",\n        \"apigateway:*\",\n        \"ec2:*\",\n        \"rds:*\",\n        \"s3:*\",\n        \"sns:*\",\n        \"states:*\",\n        \"ssm:*\",\n        \"sqs:*\",\n        \"iam:*\",\n        \"elasticloadbalancing:*\",\n        \"autoscaling:*\",\n        \"cloudwatch:*\",\n        \"cloudfront:*\",\n        \"route53:*\",\n        \"ecr:*\",\n        \"logs:*\",\n        \"ecs:*\",\n        \"application-autoscaling:*\",\n        \"logs:*\",\n        \"events:*\",\n        \"elasticache:*\",\n        \"es:*\",\n        \"kms:*\",\n        \"dynamodb:*\"\n      ],\n      \"Effect\": \"Allow\",\n      \"Resource\": \"*\"\n    },\n    {\n      \"Sid\": \"DenySpecifics\",\n      \"Action\": [\n        \"iam:*User*\",\n        \"iam:*Login*\",\n        \"iam:*Group*\",\n        \"iam:*Provider*\",\n        \"aws-portal:*\",\n        \"budgets:*\",\n        \"config:*\",\n        \"directconnect:*\",\n        \"aws-marketplace:*\",\n        \"aws-marketplace-management:*\",\n        \"ec2:*ReservedInstances*\"\n      ],\n      \"Effect\": \"Deny\",\n      \"Resource\": \"*\"\n    }\n  ]\n}\n```\n\nDepending on your company or project requirement, you can easily update the resources in `Allow` section which terraform commands should have, and add deny policies in `Deny` section if some of permissions are not required.\n\n## Tips to deal with lambda functions\n\nHeadache to save python packages from `pip install` into source codes and generate lambda zip file manually? Here is full codes with solution.\n\nThe folder [lambda](./lambda) includes all codes, here is the explanation.\n\n```\n$ tree\n.\n├── lambda.tf              # terraform HCL to deal with lambda\n├── pip.sh                 # script to install python packages with pip.\n└── source\n    ├── .gitignore         # Ignore all other files\n    ├── main.py            # Lambda function, replace with yours\n    ├── requirements.txt   # python package list, replace with yours.\n    └── setup.cfg          # Useful for mac users who installed python using Homebrew\n```\n\nReplace `main.py` and `requirements.txt` with your applications.\n\n### Explanation\n\nAfter you run `terraform apply`, it will:\n\n1. install all pip packages into source folder\n2. zip the source folder to `source.zip`\n3. deploy lambda function with `source.zip`\n4. because of `source/.gitignore`, it will ignore all new installed pip packages in git source codes.\n\nThis solution is reference from the comments in [Ability to zip AWS Lambda function on the fly](https://github.com/hashicorp/terraform/issues/8344#issuecomment-345807204))\n\nYou should be fine to do the same for lambda functions using nodejs (`npm install`) or other languages with this tip.\n\n### Notes\n\nYou need have python/pip installed when run terraform commands, if you run in terraform container, make sure you install python/pip in it.\n\n## Usage of variable \"self\"\n\nQuote from terraform documents:\n\n\u003e Attributes of your own resource\n\n\u003e The syntax is self.ATTRIBUTE. For example ${self.private_ip} will interpolate that resource's private IP address.\n\n\u003e Note: The self.ATTRIBUTE syntax is only allowed and valid within provisioners.\n\n### One more use case\n\n```terraform\nresource \"aws_ecr_repository\" \"jenkins\" {\n  name = \"${var.image_name}\"\n  provisioner \"local-exec\" {\n    command = \"./deploy-image.sh ${self.repository_url} ${var.jenkins_image_name}\"\n  }\n}\n\nvariable \"jenkins_image_name\" {\n  default = \"mycompany/jenkins\"\n  description = \"Jenkins image name.\"\n}\n```\n\nYou can easily define ecr image url (`\u003caccount_id\u003e.dkr.ecr.\u003caws_region\u003e.amazonaws.com/\u003cimage_name\u003e`) with ${self.repository_url}\n\nAny attributes in this resource can be self referenced by this way.\n\nReference: https://github.com/shuaibiyy/terraform-ecs-jenkins/blob/master/docker/main.tf\n\n## Use pre-installed Terraform plugins\n\nThere is a way to use pre-installed Terraform plugins instead of downloading them with `terraform init`, the accepted answer below gives the detail:\n\n[Use pre-installed Terraform plugins instead of downloading them with terraform init](https://stackoverflow.com/questions/50944395/use-pre-installed-terraform-plugins-instead-of-downloading-them-with-terraform-i?rq=1)\n\n## Useful documents you should read\n\n[terraform tips \u0026 tricks: loops, if-statements, and gotchas](https://blog.gruntwork.io/terraform-tips-tricks-loops-if-statements-and-gotchas-f739bbae55f9)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fozbillwang%2Fterraform-best-practices","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fozbillwang%2Fterraform-best-practices","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fozbillwang%2Fterraform-best-practices/lists"}