{"id":23388719,"url":"https://github.com/friz-zy/aws-high-availability-web-worker-setup-example","last_synced_at":"2026-04-29T23:32:01.622Z","repository":{"id":145735771,"uuid":"157095882","full_name":"Friz-zy/aws-high-availability-web-worker-setup-example","owner":"Friz-zy","description":"Example how to set up web services in modern way :)","archived":false,"fork":false,"pushed_at":"2018-11-19T12:11:47.000Z","size":122,"stargazers_count":0,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-08T13:43:17.073Z","etag":null,"topics":["ansible","aws","docker-compose","example","high-availability","infra","terraform","tutorial","webapp"],"latest_commit_sha":null,"homepage":null,"language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Friz-zy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-11T16:00:21.000Z","updated_at":"2018-11-28T14:15:14.000Z","dependencies_parsed_at":"2023-04-01T01:47:57.825Z","dependency_job_id":null,"html_url":"https://github.com/Friz-zy/aws-high-availability-web-worker-setup-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Friz-zy/aws-high-availability-web-worker-setup-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Friz-zy%2Faws-high-availability-web-worker-setup-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Friz-zy%2Faws-high-availability-web-worker-setup-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Friz-zy%2Faws-high-availability-web-worker-setup-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Friz-zy%2Faws-high-availability-web-worker-setup-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Friz-zy","download_url":"https://codeload.github.com/Friz-zy/aws-high-availability-web-worker-setup-example/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Friz-zy%2Faws-high-availability-web-worker-setup-example/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32448399,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T22:27:22.272Z","status":"ssl_error","status_checked_at":"2026-04-29T22:10:49.234Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","aws","docker-compose","example","high-availability","infra","terraform","tutorial","webapp"],"created_at":"2024-12-22T02:33:43.480Z","updated_at":"2026-04-29T23:32:01.607Z","avatar_url":"https://github.com/Friz-zy.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# High Availability Setup Example\n\nInitial estimation: 20h  \nTime spent: 21h 55m during 1 week  \nPlaying with python Vibora: 4h 40m\n\n### Whats going on?!\n\nWe'll implement high availability setup for a web app, so your site or even business always would be available for customers with 99.99% SLA* and zero downtime updates**\n\n*According to [AWS SLA](https://aws.amazon.com/ru/compute/sla/)  \n**At least while you don't wanna do the sql database schema update\n\n\u003cimg src=\"./ascii_schema.png\" width=\"700\" alt=\"ascii aws schema\"\u003e\n\nRequirements:\n  - [Terraform](http://terraform.io/)\n  - [Ansible](https://www.ansible.com/)\n  - [Existed AWS account with admin api credentials](https://docs.aws.amazon.com/en_us/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys)\n  - Understanding that applying this setup will cost you money for used aws resources\n  - Understanding what we are doing here :)\n\nStack:\n  - AWS ec2 load balancer \u0026 instances, efs, rds\n  - Nginx as frontend web server with cache and static content serving\n  - Docker containers and compose for running application\n  - Terraform for configuring AWS infra\n  - Ansible for configuring everything on the hosts\n\nOut of scope:\n  - Security: ssl, access control including VPC and Basition host, waf\n  - Infra: multisite setup, internal LB for app, dns\n  - Terraform: remote backend\n  - Ansible: dynamic inventory\n  - Dev env: docker registry, CD \u0026 CI env and system\n\nHow to do the same without AWS cloud:\n  - Load Balancer:\n      * you still can use some external lb like aws or cloudflare\n      * or you should setup it yourself, maybe with keepalived and haproxy or nginx\n  - Sync between hosts:\n      * some sync script like lsync or syncthing\n      * any cluster fs lile ceph or glusterfs\n      * some ntfs like aws efs\n  - Database:\n      * you still can use some external db like aws rds\n      * Mysql Galera\n      * Mariadb master-master  \n    Anyway you should use keepalived or haproxy or glbd for HA for all except first one\n  - Hosts: can be any :)\n\n### Before we start :)\n\nWe would use some sensitive data that should never be stored in git repo, especially in public git repo  \nData like:\n  - private aws ssh key\n  - private ssh key for app deployment\n  - mysql root password\n  - mysql app password\n\nLucky that I already generated strongly passwords and keys for you and stored it in the encrypted vault :)  \nYou can store encrypted files like this in git repos with minimum risks\n\nVault file is `ansible/group_vars/all/vault.yml` and password is `my_vault_password`  \n(don't use so weaked passwords like this at home)\n\nSo first of all, do this for preventing ansible asking vault password at each execution\n```\ncd ansible\necho 'my_vault_password' \u003e .vault_pass\nansible-vault decrypt group_vars/all/vault.yml\n```\n\nNext step: terraform also require own variables, so we'll convert our ansible yaml file into terraform json\n```\npython -c 'import json, sys, yaml ; \\\ny=yaml.safe_load(open(\"group_vars/all/vault.yml\").read()) ; \\\nopen(\"../terraform/ansible.auto.tfvars\", \"w\").write(json.dumps(y))'\n```\n\nAnd we also need aws ssh private key for applying ansible setup into hosts\n```\npython -c 'import json, sys, yaml ; \\\ny=yaml.safe_load(open(\"group_vars/all/vault.yml\").read()) ; \\\nopen(\"id_rsa_aws\", \"w\").write(y[\"ssh_privkey\"])'\n\nchmod 600 id_rsa_aws\nssh-add id_rsa_aws\n```\n\nFinally close the vault file and change directory to the main\n```\nansible-vault encrypt group_vars/all/vault.yml\ncd ..\n```\n\nNow we are ready to go! :)\n\n### Create infra with Terraform\n\n```\ncd terraform\nterraform init\nexport AWS_ACCESS_KEY_ID=\"XXX\"\nexport AWS_SECRET_ACCESS_KEY=\"YYY\"\nexport AWS_DEFAULT_REGION=\"us-west-2\"\nterraform plan\nterraform apply\n```\n\nAfter dozen of minutes (thanks to rds db creation) you'll finally get something like this:\n```\nApply complete! Resources: 12 added, 0 changed, 0 destroyed.\n\nOutputs:\n\nefs = fs-80a57228.efs.us-west-2.amazonaws.com\nweb-a = 34.219.137.185\nweb-b = 34.221.216.70\nweb-db = web-db.cylt5xazlrm6.us-west-2.rds.amazonaws.com\nweb-lb = web-elb-1230834140.us-west-2.elb.amazonaws.com\n```\n\nTerrafrom also updated ansible variables:\n```\ndiff --git a/ansible/group_vars/all/vars.yml b/ansible/group_vars/all/vars.yml\nindex 853eb00..df57033 100644\n--- a/ansible/group_vars/all/vars.yml\n+++ b/ansible/group_vars/all/vars.yml\n@@ -1,9 +1,9 @@\n ansible_port: 22\n \n # separate aws rds variable as it can be used in multiple apps\n-aws_rds_host: ''\n+aws_rds_host: web-db.cylt5xazlrm6.us-west-2.rds.amazonaws.com\n \n-app_efs: ''\n-app_lb: ''\n+app_efs: fs-80a57228.efs.us-west-2.amazonaws.com\n+app_lb: web-elb\n app_mysql_host: \"{{ aws_rds_host }}\"\n mysql_root_password: \"{{ rds_root_password }}\"\n\\ No newline at end of file\ndiff --git a/ansible/inventory b/ansible/inventory\nindex eee5423..4be85db 100644\n--- a/ansible/inventory\n+++ b/ansible/inventory\n@@ -1,2 +1,2 @@\n-web-a\n-web-b\n+web-a ansible_host=34.219.137.185\n+web-b ansible_host=34.221.216.70\n```\nFor serious setup better use Ansible dynamic inventory but for now it's fine :)\n\n### Setup services and configs with Ansible\n\n`cd ../ansible`\n\nFirst of all we'll install python2 into servers as ansible require it\n```\nansible-playbook playbooks/ansible-bootstrap-ubuntu.yml\n```\n\nAfter that we'll install basic packages, hostname and small fixes\n```\nansible-playbook playbooks/common.yml\n```\n\nAnd finally we'll setup our app environment\n```\nansible-playbook playbooks/setup_application.yml\n```\n\nLatest playbook do almost all the magic:\n  * install necessary soft like nginx, docker \u0026 docker-compose\n  * create directories for shared file system and docker logs\n  * mount aws efs into shared directory\n  * copy nginx and compose configs\n  * copy deployment script with ssh keys\n  * create app user and database\n\n### Finally run the app\n\n`ssh ubuntu@$(cd ../terraform; terraform output web-a)`\n\nWe'll use dummy web app as an our setup payload.\n\nI played a little with python [vibora](https://github.com/vibora-io/vibora) framework  \nand created docker container for you and uploaded it to the [hub](https://hub.docker.com/r/filippfrizzy/pong/).  \n\nMost of the time I struggled with [installing process](https://github.com/vibora-io/vibora/issues/192) or [outdated documentation](https://github.com/vibora-io/vibora/issues/51),  \nso I advise you to use it only for experiments and nothing more =\\  \nHowever this framework looks promisingly.\n\nYou can find sources in the `app` directory in this repo.\n\nThis app would rerutn `hello world` to the `/` request and `pong` to the `/ping` request.  \nWe'll run it in the docker container with our docker-compose config file:\n`cat /shared/configs/docker-compose.yml`.\nAnsible created it from `roles/app/templates/docker-compose.yml.j2` template.\n\nDon't forget to commit this config into git for history!  \n`cd /shared/configs/ \u0026\u0026 git commit -a -m\"add docker-compose.yml\"`\n\nDeployment script will help us with first deployment (and with later updates).  \nYou can check it before execution:  \n`cat /shared/scripts/deploy.sh`  \nIt was also created by Ansible from `roles/app/templates/deploy.sh.j2` template.\n\nSo, just run it:\n```\nsudo bash /shared/scripts/deploy.sh all\n```\n\nIf all was fine and you didn't see something like 'I'm aborting deployment',  \nthen you can log off with `exit` command and finally check your setup:\n```\ncurl -v $(cd ../terraform; terraform output web-lb)\n```\n\nHooray, we did it!\n\n### Don't forget to clean all in the end!\n\n```\ncd ../terraform\nterraform destroy\ncd ../\ngit reset --hard HEAD\n```\n\n### What's next?\n\nTools:\n* https://www.terraform.io\n* https://aws.amazon.com/ru/blogs/apn/terraform-beyond-the-basics-with-aws/\n* https://checklyhq.com/blog/2018/08/an-in-depth-look-at-100-zero-downtime-deployments-with-terraform/\n* https://github.com/leucos/ansible-tuto\n* https://docs.docker.com/compose/\n\nUnderstanding:\n* https://hackernoon.com/the-2018-devops-roadmap-31588d8670cb\n* https://www.digitalocean.com/community/tutorials/what-is-high-availability\n* https://12factor.net/\n* https://landing.google.com/sre/books/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffriz-zy%2Faws-high-availability-web-worker-setup-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffriz-zy%2Faws-high-availability-web-worker-setup-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffriz-zy%2Faws-high-availability-web-worker-setup-example/lists"}