{"id":16442398,"url":"https://github.com/richardknop/pinglist-aws-terraform","last_synced_at":"2026-05-17T05:31:27.183Z","repository":{"id":148613138,"uuid":"54318781","full_name":"RichardKnop/pinglist-aws-terraform","owner":"RichardKnop","description":"Terraform manifests to build Pinglist AWS infrastructure.","archived":false,"fork":false,"pushed_at":"2017-05-31T22:04:48.000Z","size":288,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-01-08T17:22:17.653Z","etag":null,"topics":["aws","terraform"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RichardKnop.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-03-20T13:24:33.000Z","updated_at":"2017-06-05T08:04:58.000Z","dependencies_parsed_at":"2023-05-20T17:15:32.252Z","dependency_job_id":null,"html_url":"https://github.com/RichardKnop/pinglist-aws-terraform","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RichardKnop%2Fpinglist-aws-terraform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RichardKnop%2Fpinglist-aws-terraform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RichardKnop%2Fpinglist-aws-terraform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RichardKnop%2Fpinglist-aws-terraform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RichardKnop","download_url":"https://codeload.github.com/RichardKnop/pinglist-aws-terraform/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240788735,"owners_count":19857691,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","terraform"],"created_at":"2024-10-11T09:17:17.577Z","updated_at":"2026-05-17T05:31:22.166Z","avatar_url":"https://github.com/RichardKnop.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"[1]: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#having-ec2-create-your-key-pair\n[2]: http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingHostedZone.html\n\n# Pinglist AWS Terraform\n\nTerraform manifests to build Pinglist AWS infrastructure.\n\nSee also:\n- [pinglist-aws-ansible](https://github.com/RichardKnop/pinglist-aws-ansible)\n- [pinglist-api](https://github.com/RichardKnop/pinglist-api)\n- [pinglist-app](https://github.com/RichardKnop/pinglist-app)\n- [pinglist-ios-app](https://github.com/RichardKnop/pinglist-ios-app)\n\n# Index\n\n* [Pinglist AWS Terraform](#pinglist-aws-terraform)\n* [Index](#index)\n* [Requirements](#requirements)\n  * [Git Submodules](#git-submodules)\n  * [Requirements For AWS Provisioning](#requirements-for-aws-provisioning)\n    * [Hosted DNS Zone](#hosted-dns-zone)\n    * [S3 Bucket For Releases](#s3-bucket-for-releases)\n    * [Environment Variables](#environment-variables)\n    * [SSL Certificate](#ssl-certificate)\n    * [Register Your Mobile App With AWS](#register-your-mobile-app-with-aws)\n      * [iOS](#ios)\n* [Provisioning](#provisioning)\n  * [Variables](#variables)\n  * [Apply Execution Plan](#apply-execution-plan)\n  * [Connecting To Instances](#connecting-to-instances)\n  * [Debugging](#debugging)\n  * [Recreate Database](#recreate-database)\n* [Resources](#resources)\n\n# Requirements\n\nYou need Terraform \u003e= 0.6.9, e.g.:\n\n```\nbrew install terraform\n```\n\nYou need an SSH key. The private key needs to be chmod to 600.\n\nYou need the cloud provider credentials. These will be entered on the command line.\n\n## Git Submodules\n\nSome resources created by Terraform are provisioned using Ansible. The `github.com/RichardKnop/pinglist-ansible` repository has been added as a git submodule. Make sure to initialise git submodiles.\n\n```\ngit submodule init\ngit submodule update\n```\n\n## Requirements For AWS Provisioning\n\n### Hosted DNS Zone\n\nA hosted DNS zone on AWS has been manually created and its ID stored in `variables.tf`.\n\n### S3 Bucket For Releases\n\nAn S3 bucket has been manually created to store releases (Docker builds). This bucket is common for all environments and is configurable via `release_bucket` variable.\n\n### Environment Variables\n\nThe terraform provider for AWS will read the standard AWS credentials environment variables. You must have these variables exported:\n\n- `AWS_ACCESS_KEY_ID`\n- `AWS_SECRET_ACCESS_KEY`\n- `AWS_DEFAULT_REGION`\n\nYou can get the credentials from the AWS console.\n\nTerraform will look for a deployment key in `~/.ssh` directory when creating a NAT instance. Add the deployment key to the ssh-agent, e.g.:\n\n```\nssh-add ~/.ssh/stage-pinglist-deployer\n```\n\n### SSL Certificate\n\nAn SSL certificate for `*.pingli.st` has been purchased from [Comodo](https://www.comodo.com/).\n\nRichard Knop has the private key. Keep it secure, only transfer GPG encrypted if needed.\n\nComodo has then provided the certificate itself in a ZIP file which consists of:\n\n- certificate (`STAR_pingli_st.crt`)\n- intermediate and root Comodo certificates\n\nAWS only supports PEM format of certificates so first, we needed to convert the private key and all certificates to the correct format:\n\n```\nopenssl x509 -inform PEM -in pinglist.key \u003e pinglist-key.pem\nopenssl x509 -inform PEM -in STAR_pingli_st.crt \u003e STAR_pingli_st.pem\nopenssl x509 -inform PEM -in COMODORSADomainValidationSecureServerCA.crt \u003e COMODORSADomainValidationSecureServerCA.pem\nopenssl x509 -inform PEM -in COMODORSAAddTrustCA.crt \u003e COMODORSAAddTrustCA.pem\nopenssl x509 -inform PEM -in AddTrustExternalCARoot.crt \u003e AddTrustExternalCARoot.pem\n```\n\nSecondly, we generated a certificate chain from intermediate and root certificates one by one using `cat` command:\n\n```\ncat COMODORSADomainValidationSecureServerCA.pem COMODORSAAddTrustCA.pem AddTrustExternalCARoot.pem \u003e pinglist-certificate-chain.pem\n```\n\nFinally, we uploaded the certificate to AWS IAM:\n\n```\naws iam upload-server-certificate --server-certificate-name pinglist-certificate \\\n--certificate-body file://STAR_pingli_st.pem --private-key file://pinglist-key.pem \\\n--certificate-chain file://pinglist-certificate-chain.pem\n```\n\nResult:\n\n```json\n{\n  \"ServerCertificateMetadata\": {\n    \"ServerCertificateId\": \"ASCAIRNGM2AA5Z4HA5TUQ\",\n    \"ServerCertificateName\": \"pinglist-certificate\",\n    \"Expiration\": \"2017-04-23T23:59:59Z\",\n    \"Path\": \"/\",\n    \"Arn\": \"arn:aws:iam::800222191807:server-certificate/pinglist-certificate\",\n    \"UploadDate\": \"2016-04-23T17:58:23.272Z\"\n  }\n}\n```\n\n`Arn` attribute is used as `ssl_certificate_id` for HTTPS listener in the ELB.\n\n### Register Your Mobile App With AWS\n\nFor Amazon `SNS` to send notification messages to mobile endpoints, whether it is direct or with subscriptions to a topic, you first need to [register the app with AWS](http://docs.aws.amazon.com/sns/latest/dg/mobile-push-send-register.html).\n\n#### iOS\n\nFor iOS, you will need:\n\n- APNS SSL certificate\n- app private key\n\nTo convert the APNS SSL certificate from `.cer` format to `.pem` format, type the following command. Replace `myapnsappcert.cer` with the name of the certificate you downloaded from the Apple Developer web site.\n\n```\nopenssl x509 -in myapnsappcert.cer -inform DER -out myapnsappcert.pem\n```\n\nTo convert the app private key from `.p12` format to `.pem` format, type the following command. Replace `myapnsappprivatekey.p12` with the name of the private key you exported from Keychain Access.\n\n```\nopenssl pkcs12 -in myapnsappprivatekey.p12 -out myapnsappprivatekey.pem -nodes -clcerts\n```\n\nOpenSSL will ask you for a passphrase.\n\nThen, you can go to AWS console, navigate to SNS home and create a new platform application for Apple. You will need both the APNS certificate and private key created above as well as the original `.p12` private key and the passphrase.\n\nSave the application ARN, for example:\n\n```\narn:aws:sns:us-west-2:800222191807:app/APNS_SANDBOX/pinglist-ios-stage\n```\n\nOr:\n\n```\narn:aws:sns:us-west-2:800222191807:app/APNS/pinglist-ios-prod\n```\n\nThe ARN is needed to create endpoints.\n\n# Provisioning\n\nMake sure submodules are up-to-date:\n\n```\ngit submodule update --remote\n```\n\nRender an SSH configuration file, i.e.:\n\n```\n./render-ssh-config.sh \u003cenv-name-prefix\u003e\n```\n\nCreate virtual Python environment for `pinglist-ansible` submodule:\n\n```\nvirtualenv .venv\nsource .venv/bin/activate\npip install -r ansible/requirements.txt\n```\n\n## Variables\n\nYou will need to export couple of needed environment variables.\n\nMost importantly, define an environment name, e.g.:\n\n```\nexport TF_VAR_env=stage\n```\n\nOr:\n\n```\nexport TF_VAR_env=prod\n```\n\nSetup ETCD discovery URL:\n\n```\nexport TF_VAR_etcd_discovery_url=$(./get-etcd-discovery-url-from-state-file.sh $TF_VAR_env)\n```\n\nOr if you are deploying for the first time, generate a new ETCD discovery URL:\n\n```\nexport TF_VAR_etcd_discovery_url=`curl https://discovery.etcd.io/new?size=1`\n```\n\nSetup APNS platform application ARN:\n\n```\nexport TF_VAR_apns_platform_application_arn=$(cd ansible ; ./get-vault-variable.sh $TF_VAR_env apns_platform_application_arn)\n```\n\nFor test environments, it's useful to disable final DB snapshot:\n\n```\nexport TF_VAR_rds_skip_final_snapshot=true\n```\n\nExport the main DB password from the `ansible-vault`:\n\n```\nexport TF_VAR_db_password=$(cd ansible ; ./get-vault-variable.sh $TF_VAR_env api_database_password)\n```\n\nDefine which API release you want to use:\n\n```\nexport TF_VAR_api_release=v0.0.0\n```\n\nDefine which web app release you want to use:\n\n```\nexport TF_VAR_app_release=v0.0.0\n```\n\nSee `variables.tf` for the full list of variables you can set.\n\n## State Files\n\nWe need to support multiple environments (`stage`, `prod` etc) and share state files across the team. Therefor state files include environment suffix and their encrypted versions are stored in git.\n\nFirst, decrypt a state file you want to use:\n\n```\n./decrypt-state-file.sh $TF_VAR_env\n```\n\nThe above script would decrypt `$TF_VAR_env.tfstate.gpg` to `$TF_VAR_env.tfstate`.\n\nAfter running Terraform, don't forget to update the encrypted state file:\n\n```\n./encrypt-state-file.sh $TF_VAR_env\n```\n\n## Apply Execution Plan\n\n\nFirst, check the Terraform execution plan:\n\n```\nterraform plan -state=$TF_VAR_env.tfstate\n```\n\nNow you can provision the environment:\n\n```\nterraform apply -state=$TF_VAR_env.tfstate\n```\n\n**IMPORTANT**: The option `-var force_destroy=true` will mark all the resources, including S3 buckets to be deleted when destroying the environment. This is fine in test environments, but dangerous in production.\n\nSo, you could provision a test environment like this:\n\n```\nterraform apply -state=$TF_VAR_env.tfstate -var force_destroy=true\n```\n\n## Connecting To Instances\n\nNow you can SSH to instances in a private subnet of the VPS via the NAT instance, e.g.:\n\n```\nssh -F ssh.config \u003cprivate_ip\u003e\n```\n\n## Debugging\n\nOnce you have connected to a CoreOS node, you can use `systemctl` to check status of services:\n\n```\nsystemctl status pinglist-api.service\n```\n\nTo view logs specific to a service running as a Docker container, use `journalctl`:\n\n```\njournalctl -u pinglist-api.service -n 100 -f\n```\n\n## Recreate Database\n\nNEVER do this against production environment.\n\nSometimes during development it is useful or needed to destroy and recreate the database. You can use `taint` command to mark the database instance for destruction. Next time you run the `apply` command the database will be destroyed and a new once created:\n\n```\nterraform taint -module=rds -state=$TF_VAR_env.tfstate aws_db_instance.rds\n```\n\n# Resources\n\n- [Creating EC2 Key Pairs][1]\n- [Create a Public Hosted Zone][2]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frichardknop%2Fpinglist-aws-terraform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frichardknop%2Fpinglist-aws-terraform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frichardknop%2Fpinglist-aws-terraform/lists"}