{"id":22933510,"url":"https://github.com/bluebrown/immutable-cluster","last_synced_at":"2025-08-12T16:32:53.986Z","repository":{"id":111481179,"uuid":"359020147","full_name":"bluebrown/immutable-cluster","owner":"bluebrown","description":"AWS nginx instances in ha setup created with packer and terraform","archived":false,"fork":false,"pushed_at":"2021-04-23T07:13:41.000Z","size":97,"stargazers_count":4,"open_issues_count":0,"forks_count":6,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-01T08:38:10.902Z","etag":null,"topics":["ansible","aws","packer","terraform"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bluebrown.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-18T01:40:54.000Z","updated_at":"2022-03-22T05:18:04.000Z","dependencies_parsed_at":"2023-04-29T09:02:19.669Z","dependency_job_id":null,"html_url":"https://github.com/bluebrown/immutable-cluster","commit_stats":null,"previous_names":[],"tags_count":0,"template":true,"template_full_name":null,"purl":"pkg:github/bluebrown/immutable-cluster","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluebrown%2Fimmutable-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluebrown%2Fimmutable-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluebrown%2Fimmutable-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluebrown%2Fimmutable-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bluebrown","download_url":"https://codeload.github.com/bluebrown/immutable-cluster/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluebrown%2Fimmutable-cluster/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270097312,"owners_count":24526645,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-12T02:00:09.011Z","response_time":80,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","aws","packer","terraform"],"created_at":"2024-12-14T11:30:06.748Z","updated_at":"2025-08-12T16:32:53.938Z","avatar_url":"https://github.com/bluebrown.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Immutable Infrastructure in AWS with Packer, Ansible and Terraform\n\n\u003e Immutable infrastructure is an approach to managing services and software deployments on IT resources wherein components are replaced rather than changed. An application or services is effectively redeployed each time any change occurs.\n\nIn this post I am going to show how one can create a workflow based in the idea of immutable infrastructure. You can find the full working code in github \u003chttps://github.com/bluebrown/immutable-cluster\u003e.\n\n## The Recipe\n\nSince the goal is to simply replace the instance on each application release, we want to create a machine image containing the application. This way we can swap out the whole instance quickly without further installation steps after provisioning. This is sometimes called a [golden image](https://opensource.com/article/19/7/what-golden-image). We are going to use [Packer](https://www.packer.io/) \u0026 [Ansible](https://www.ansible.com/) for this.\n\nFurther, we want to manage our infrastructure as code. [Terraform](https://www.terraform.io/) is a good choice for this. It can create, change and destroy infrastructure remotely and keeps track of the current state of our system.\n\n With a golden image and infrastructure as code, we can throw away the complete environment, in this case the vpc with instances, and create a new one within minutes if we wish. Usually we only want to swap the instances though.\n\n## Prerequisites\n\nTo follow along you need:\n\n- an [AWS](https://aws.amazon.com/) account \u0026 access tokens\n- have [Ansible](https://www.ansible.com/) installed\n- have [Packer](https://www.packer.io/) installed\n- have [Terraform](https://www.terraform.io/) installed\n- have the [AWS CLI](https://aws.amazon.com/cli/) version 2 installed\n\n## Creating a Custom Image with Packer\n\n**If you are using a custom vpc, make sure to configure [Packer](https://www.packer.io/) to use a subnet with automatic public ip assignment and a route to the internet gateway.**\n\n### Packer File\n\nFirst we create a [Packer](https://www.packer.io/) file with some information about the image we want to create.\n\nMost importantly We specify the `region`. By default our `AMI` will only be available in this `region`.\n\nWe specify [Ansible](https://www.ansible.com/) as provisioner. It will execute the `playbook` on the temporary instance to apply additional configuration.\n\n```go\nvariable \"aws_access_key\" {\n  sensitive = true\n}\nvariable \"aws_secret_key\" {\n  sensitive = true\n}\n\nsource \"amazon-ebs\" \"example\" {\n  access_key      = var.aws_access_key\n  secret_key      = var.aws_secret_key\n  ssh_timeout     = \"30s\"\n  region          = \"eu-central-1\"\n  // amazon linux 2\n  source_ami      = \"ami-0db9040eb3ab74509\"\n  ssh_username    = \"ec2-user\"\n  ami_name        = \"packer nginx\"\n  instance_type   = \"t2.micro\"\n  skip_create_ami = false\n\n}\n\nbuild {\n  sources = [\n    \"source.amazon-ebs.example\"\n  ]\n  provisioner \"ansible\" {\n    playbook_file = \"playbook.yml\"\n  }\n}\n\n```\n\n### Ansible Playbook\n\nOnce [Packer](https://www.packer.io/) has created the temporary instance, we use [Ansible](https://www.ansible.com/) to apply additional configuration.\n\nThe playbook tells ansible to install and enable the nginx service. The result will be `nginx` serving the default page on port 80 when the `instance` is booted.\n\n```yml\n---\n- name: set up nginx\n\n  hosts: default\n  become: true\n\n  tasks:\n    - name: ensure extra repo is available\n      yum:\n        name: [amazon-linux-extras]\n        state: present\n\n    - name: enable nginx repo\n      shell: amazon-linux-extras enable nginx1\n\n    - name: yum-clean-metadata\n      command: yum clean metadata\n      args:\n        warn: no\n\n    - name: install nginx\n      yum:\n        name: [nginx]\n        state: latest\n\n    - name: enable nginx service\n      service:\n        name: nginx\n        enabled: true\n```\n\n### Build\n\nWith the 2 configuration files we can validate the input and build our custom `AMI` in [AWS](https://aws.amazon.com/).\n\n```console\npacker validate . \npacker build .\n```\n\n### The AMI\n\nOnce the process is completed, we can use the [AWS CLI](https://aws.amazon.com/cli/) to inspect the created `AMI` and find the `ImageId`.\n\n```json\n$ aws ec2 describe-images --owner self --region eu-central-1\n{\n    \"Images\": [\n        {\n            \"Architecture\": \"x86_64\",\n            \"CreationDate\": \"2021-04-18T00:00:05.000Z\",\n            \"ImageId\": \"ami-01cce7ac6df33f08e\",\n            \"ImageLocation\": \"\u003cyour_account_id\u003e/packer nginx\",\n            \"ImageType\": \"machine\",\n            \"Public\": false,\n            \"OwnerId\": \"\u003cyour_account_id\u003e\",\n            \"PlatformDetails\": \"Linux/UNIX\",\n            \"UsageOperation\": \"RunInstances\",\n            \"State\": \"available\",\n            \"BlockDeviceMappings\": [\n                {\n                    \"DeviceName\": \"/dev/xvda\",\n                    \"Ebs\": {\n                        \"DeleteOnTermination\": true,\n                        \"SnapshotId\": \"snap-0692717e44e63cbd1\",\n                        \"VolumeSize\": 8,\n                        \"VolumeType\": \"gp2\",\n                        \"Encrypted\": false\n                    }\n                }\n            ],\n            \"EnaSupport\": true,\n            \"Hypervisor\": \"xen\",\n            \"Name\": \"packer nginx\",\n            \"RootDeviceName\": \"/dev/xvda\",\n            \"RootDeviceType\": \"ebs\",\n            \"SriovNetSupport\": \"simple\",\n            \"VirtualizationType\": \"hvm\"\n        }\n    ]\n}\n```\n\n## Deploying the Infrastructure with Terraform\n\nNow we have our custom `AMI` in the `eu-central-1` region. Next we will use [Terraform](https://www.terraform.io/) to deploy this image together with the required infrastructure.\n\n![image of infrastructure with elb](https://user-images.githubusercontent.com/39703898/115515253-dc43b000-a27c-11eb-8c96-b7fd705b7a9f.png)\n\n```go\nvariable \"aws_access_key\" {\n  sensitive = true\n}\nvariable \"aws_secret_key\" {\n  sensitive = true\n}\n\nvariable \"ami_id\" {\n  default = \"\"\n}\n\nvariable \"domain\" {\n  default = \"\"\n}\n\nterraform {\n  required_providers {\n    aws = {\n      source  = \"hashicorp/aws\"\n      version = \"~\u003e 3.0\"\n    }\n  }\n}\n\nprovider \"aws\" {\n  region     = \"eu-central-1\"\n  access_key = var.aws_access_key\n  secret_key = var.aws_secret_key\n}\n```\n\n### VPC\n\nFirst we create a simple `VPC` with 2 `subnets` in different `availability zones`.\n\n```go\nresource \"aws_vpc\" \"packer\" {\n  cidr_block = \"10.0.0.0/16\"\n  tags = {\n    Name        = \"packer\"\n    Description = \"sample vpc with 2 public subnets in 2 availability zones and a network load balancer for high availability\"\n  }\n\n}\n\nresource \"aws_internet_gateway\" \"inet\" {\n  vpc_id = aws_vpc.packer.id\n  tags = {\n    Name = \"packer internet gateway\"\n  }\n}\n\nresource \"aws_default_route_table\" \"public\" {\n  default_route_table_id = aws_vpc.packer.default_route_table_id\n\n  route {\n    cidr_block = \"0.0.0.0/0\"\n    gateway_id = aws_internet_gateway.inet.id\n  }\n\n  tags = {\n    Name = \"public route table\"\n  }\n}\n\nresource \"aws_subnet\" \"a\" {\n  vpc_id            = aws_vpc.packer.id\n  cidr_block        = \"10.0.0.0/24\"\n  availability_zone = \"eu-central-1a\"\n\n  tags = {\n    Name = \"public subnet a\"\n  }\n\n  map_public_ip_on_launch = true\n}\n\nresource \"aws_subnet\" \"b\" {\n  vpc_id            = aws_vpc.packer.id\n  cidr_block        = \"10.0.1.0/24\"\n  availability_zone = \"eu-central-1b\"\n\n\n  tags = {\n    Name = \"public subnet b\"\n  }\n\n  map_public_ip_on_launch = true\n}\n```\n\n### Security Groups\n\nNext, we create the `security groups`.\n\nThe default security group has only a reference to itself. It is used to allow traffic to flow between the `ALB` and its `targets`.\n\nThe second `security group` is to allow tcp traffic from the public web to the `ALB` on port 80(HTTP) and 443 (HTTPS).\n\n```go\nresource \"aws_default_security_group\" \"internal\" {\n  vpc_id = aws_vpc.packer.id\n\n  tags = {\n    Name = \"default internal sg\"\n  }\n\n  ingress {\n    protocol    = -1\n    self        = true\n    from_port   = 0\n    to_port     = 0\n    description = \"self ref\"\n  }\n\n  egress {\n    protocol    = -1\n    self        = true\n    from_port   = 0\n    to_port     = 0\n    description = \"self ref\"\n  }\n\n}\n\nresource \"aws_security_group\" \"web\" {\n  vpc_id = aws_vpc.packer.id\n\n  tags = {\n    Name = \"web sg for nginx\"\n  }\n\n  ingress {\n    description      = \"http traffic\"\n    from_port        = 80\n    to_port          = 80\n    protocol         = \"tcp\"\n    cidr_blocks      = [\"0.0.0.0/0\"]\n    ipv6_cidr_blocks = [\"::/0\"]\n  }\n\n  ingress {\n    description      = \"http traffic\"\n    from_port        = 443\n    to_port          = 443\n    protocol         = \"tcp\"\n    cidr_blocks      = [\"0.0.0.0/0\"]\n    ipv6_cidr_blocks = [\"::/0\"]\n  }\n\n  egress {\n    from_port        = 0\n    to_port          = 0\n    protocol         = \"-1\"\n    cidr_blocks      = [\"0.0.0.0/0\"]\n    ipv6_cidr_blocks = [\"::/0\"]\n  }\n\n}\n```\n\n### Logging\n\nWe are going to create an `S3 bucket` with the required access `policy` to use it as log destination for the `ALB` in the next section.\n\n```go\nresource \"aws_s3_bucket\" \"logs\" {\n  bucket = \"com.myorg.logs\"\n  acl    = \"private\"\n  force_destroy = true\n  tags = {\n    Name        = \"nginx cluster access logs\"\n    Environment = \"Dev\"\n  }\n}\n\n\ndata \"aws_elb_service_account\" \"main\" {\n    region = \"eu-central-1\"\n}\n\n\nresource \"aws_s3_bucket_policy\" \"logs\" {\n  bucket = aws_s3_bucket.logs.id\n  policy = \u003c\u003cPOLICY\n{\n    \"Version\": \"2012-10-17\",\n    \"Id\": \"allow-elb-logs\",\n    \"Statement\": [\n        {\n            \"Sid\": \"RegionRootArn\",\n            \"Effect\": \"Allow\",\n            \"Principal\": {\n                \"AWS\": \"${data.aws_elb_service_account.main.arn}\"\n            },\n            \"Action\": \"s3:PutObject\",\n            \"Resource\": \"${aws_s3_bucket.logs.arn}/*\"\n        },\n        {\n            \"Sid\": \"AWSLogDeliveryWrite\",\n            \"Effect\": \"Allow\",\n            \"Principal\": {\n                \"Service\": \"delivery.logs.amazonaws.com\"\n            },\n            \"Action\": \"s3:PutObject\",\n            \"Resource\": \"${aws_s3_bucket.logs.arn}/*\",\n            \"Condition\": {\n                \"StringEquals\": {\n                    \"s3:x-amz-acl\": \"bucket-owner-full-control\"\n                }\n            }\n        },\n        {\n            \"Sid\": \"AWSLogDeliveryAclCheck\",\n            \"Effect\": \"Allow\",\n            \"Principal\": {\n                \"Service\": \"delivery.logs.amazonaws.com\"\n            },\n            \"Action\": \"s3:GetBucketAcl\",\n            \"Resource\": \"${aws_s3_bucket.logs.arn}\"\n        }\n    ]\n}\n  POLICY\n}\n```\n\n### Load balancing\n\nNext, an application load balancer (ALB) is created with a 2 `listeners`.\n\nThe first `listener` will listen on port 80 and redirect the traffic to port 443. The\n\nThe second  `listener` will serve a tls certificate, that is imported from `ACM`, on port 443. After the `TLS handshake` it will forward the traffic over http on port 80 to the target group, also known asl `TLS Termination`.\n\nI am assuming that you already have uploaded your own cert or issues one with ACM. There [a branch without tls](https://github.com/bluebrown/immutable-cluster/tree/no-tls) in this repo.\n\nThe `target group` will be populated by the `auto scaling group` in the next section.\n\n```go\nresource \"aws_lb\" \"web\" {\n  name               = \"packer-nginx\"\n  internal           = false\n  load_balancer_type = \"application\"\n  security_groups = [\n    aws_default_security_group.internal.id,\n    aws_security_group.web.id,\n  ]\n  subnets = [\n    aws_subnet.a.id,\n    aws_subnet.b.id\n  ]\n  access_logs {\n    bucket  = aws_s3_bucket.logs.id\n    enabled = true\n  }\n}\n\nresource \"aws_lb_target_group\" \"web\" {\n  name     = \"web-tg\"\n  port     = 80\n  protocol = \"HTTP\"\n  vpc_id   = aws_vpc.packer.id\n}\n\n\nresource \"aws_lb_listener\" \"web\" {\n  load_balancer_arn = aws_lb.web.arn\n  port              = \"80\"\n  protocol          = \"HTTP\"\n  default_action {\n    type = \"redirect\"\n    redirect {\n      port        = \"443\"\n      protocol    = \"HTTPS\"\n      status_code = \"HTTP_301\"\n    }\n  }\n}\n\ndata \"aws_acm_certificate\" \"mycert\" {\n  domain   = var.domain\n  statuses = [\"ISSUED\"]\n  most_recent = true\n}\n\nresource \"aws_lb_listener\" \"websecure\" {\n  load_balancer_arn = aws_lb.web.arn\n  port              = \"443\"\n  protocol          = \"HTTPS\"\n  ssl_policy        = \"ELBSecurityPolicy-2016-08\"\n  certificate_arn   = data.aws_acm_certificate.mycert.arn\n  default_action {\n    type             = \"forward\"\n    target_group_arn = aws_lb_target_group.web.arn\n  }\n}\n```\n\n### Autoscaling\n\nLastly, we create a launch template and auto scaling group to launch new instances of the custom `AMI`.\n\nWe will require a minimum of 2 instances with a desired count of 2 instances. Optionally we allow to scale up to 4 instances if instances reach their resource limit.\n\nThe `strategy` of the `placement group` is set to `partition` which means that the instances should get spread across the racks in physical data center.\n\n```go\nresource \"aws_placement_group\" \"web\" {\n  name     = \"web-pl\"\n  strategy = \"partition\"\n}\n\nresource \"aws_launch_template\" \"web\" {\n  name_prefix   = \"web-lt-\"\n  image_id      = var.ami_id\n  instance_type = \"t2.micro\"\n  vpc_security_group_ids = [\n    aws_default_security_group.internal.id,\n  ]\n}\n\nresource \"aws_autoscaling_group\" \"web\" {\n  name                = \"webscale\"\n  vpc_zone_identifier = [aws_subnet.a.id, aws_subnet.b.id]\n  desired_capacity    = 2\n  max_size            = 4\n  min_size            = 2\n  placement_group     = aws_placement_group.web.id\n  launch_template {\n    id      = aws_launch_template.web.id\n    version = \"$Latest\"\n  }\n  lifecycle {\n    ignore_changes = [load_balancers, target_group_arns]\n  }\n}\n\nresource \"aws_autoscaling_attachment\" \"web\" {\n  autoscaling_group_name = aws_autoscaling_group.web.id\n  alb_target_group_arn   = aws_lb_target_group.web.arn\n}\n```\n\n## Deploy\n\nNow we can deploy the infrastructure. Run `terraform apply` and confirm the prompt. The process will take a couple minutes until all the resources are created and ready.\n\n```console\nterraform apply\n```\n\nYou can use [AWS CLI](https://aws.amazon.com/cli/) to see if the targets of the load balancer are health.\n\nThey may not be ready yet. If that is the case, just wait a couple minutes and check again.\n\n```console\n$ arn=$(aws elbv2 describe-target-groups --name web-tg --query \"TargetGroups[0].TargetGroupArn\" --output text)\n$ aws elbv2 describe-target-health --target-group-arn \"$arn\"\n{\n    \"TargetHealthDescriptions\": [\n        {\n            \"Target\": {\n                \"Id\": \"i-0b8137e9710a695a3\",\n                \"Port\": 80\n            },\n            \"HealthCheckPort\": \"80\",\n            \"TargetHealth\": {\n                \"State\": \"healthy\"\n            }\n        },\n        {\n            \"Target\": {\n                \"Id\": \"i-08db17910a66c9372\",\n                \"Port\": 80\n            },\n            \"HealthCheckPort\": \"80\",\n            \"TargetHealth\": {\n                \"State\": \"healthy\"\n            }\n        }\n    ]\n}\n```\n\nOnce the targets are marked as healthy, we need to point a `CNAME record` from our domain to the `ELB DNS`. I am managing my certificate with `Linode`, so I will give an example of how to do it via [linode-cli](https://www.linode.com/docs/guides/linode-cli/).\n\n```console\ndns=$(aws elbv2 describe-load-balancers --name \"packer-nginx\" --query \"LoadBalancers[0].DNSName\" --output text)\nlinode-cli domains records-create --type CNAME --name elb --target $dns --ttl_sec 300  \u003cmy-domain-id\u003e\n```\n\nYou can now visit the url in your browser under your the configured subdomain.\n\n![nginx default web page](https://user-images.githubusercontent.com/39703898/115831360-64ef5700-a409-11eb-9c8f-5c44fb06be11.png)\n\nThats it!\n\nThe application is deployed from a custom `AMI` across 2 `availability zones` and utilizing `autoscaling`. The traffic is routed via `ELB` which also performs `TLS Termination`.\n\nAdditionally, e have our whole infrastructure as code which we can source control.\n\n## Cleaning Up\n\nIn order to avoid cost, lets remove all created resources.\n\n```console\nterraform destroy\n```\n\nSince the AMI and snapshot was not created with [Terraform](https://www.terraform.io/), it wont be destroyed by the former command. We are going to remove them via CLI.\n\n### Deregister Image\n\n```console\naws ec2 deregister-image --image-id \u003cyour-ami-id\u003e\n```\n\n### Find the snapshot Id\n\n```console\naws ec2 describe-snapshots --owner self\n```\n\n### Delete snapshot\n\n```console\naws ec2 delete-snapshot --snapshot-id \u003cyour-snap-id\u003e\n```\n\n```console\nlinode-cli domains records-delete \u003cmain-id\u003e \u003crecord-id\u003e\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbluebrown%2Fimmutable-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbluebrown%2Fimmutable-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbluebrown%2Fimmutable-cluster/lists"}