https://github.com/bluebrown/immutable-cluster
AWS nginx instances in ha setup created with packer and terraform
https://github.com/bluebrown/immutable-cluster
ansible aws packer terraform
Last synced: 11 months ago
JSON representation
AWS nginx instances in ha setup created with packer and terraform
- Host: GitHub
- URL: https://github.com/bluebrown/immutable-cluster
- Owner: bluebrown
- Created: 2021-04-18T01:40:54.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2021-04-23T07:13:41.000Z (about 5 years ago)
- Last Synced: 2025-04-01T08:38:10.902Z (over 1 year ago)
- Topics: ansible, aws, packer, terraform
- Language: HCL
- Homepage:
- Size: 94.7 KB
- Stars: 4
- Watchers: 3
- Forks: 6
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Immutable Infrastructure in AWS with Packer, Ansible and Terraform
> Immutable infrastructure is an approach to managing services and software deployments on IT resources wherein components are replaced rather than changed. An application or services is effectively redeployed each time any change occurs.
In this post I am going to show how one can create a workflow based in the idea of immutable infrastructure. You can find the full working code in github .
## The Recipe
Since the goal is to simply replace the instance on each application release, we want to create a machine image containing the application. This way we can swap out the whole instance quickly without further installation steps after provisioning. This is sometimes called a [golden image](https://opensource.com/article/19/7/what-golden-image). We are going to use [Packer](https://www.packer.io/) & [Ansible](https://www.ansible.com/) for this.
Further, we want to manage our infrastructure as code. [Terraform](https://www.terraform.io/) is a good choice for this. It can create, change and destroy infrastructure remotely and keeps track of the current state of our system.
With a golden image and infrastructure as code, we can throw away the complete environment, in this case the vpc with instances, and create a new one within minutes if we wish. Usually we only want to swap the instances though.
## Prerequisites
To follow along you need:
- an [AWS](https://aws.amazon.com/) account & access tokens
- have [Ansible](https://www.ansible.com/) installed
- have [Packer](https://www.packer.io/) installed
- have [Terraform](https://www.terraform.io/) installed
- have the [AWS CLI](https://aws.amazon.com/cli/) version 2 installed
## Creating a Custom Image with Packer
**If you are using a custom vpc, make sure to configure [Packer](https://www.packer.io/) to use a subnet with automatic public ip assignment and a route to the internet gateway.**
### Packer File
First we create a [Packer](https://www.packer.io/) file with some information about the image we want to create.
Most importantly We specify the `region`. By default our `AMI` will only be available in this `region`.
We specify [Ansible](https://www.ansible.com/) as provisioner. It will execute the `playbook` on the temporary instance to apply additional configuration.
```go
variable "aws_access_key" {
sensitive = true
}
variable "aws_secret_key" {
sensitive = true
}
source "amazon-ebs" "example" {
access_key = var.aws_access_key
secret_key = var.aws_secret_key
ssh_timeout = "30s"
region = "eu-central-1"
// amazon linux 2
source_ami = "ami-0db9040eb3ab74509"
ssh_username = "ec2-user"
ami_name = "packer nginx"
instance_type = "t2.micro"
skip_create_ami = false
}
build {
sources = [
"source.amazon-ebs.example"
]
provisioner "ansible" {
playbook_file = "playbook.yml"
}
}
```
### Ansible Playbook
Once [Packer](https://www.packer.io/) has created the temporary instance, we use [Ansible](https://www.ansible.com/) to apply additional configuration.
The playbook tells ansible to install and enable the nginx service. The result will be `nginx` serving the default page on port 80 when the `instance` is booted.
```yml
---
- name: set up nginx
hosts: default
become: true
tasks:
- name: ensure extra repo is available
yum:
name: [amazon-linux-extras]
state: present
- name: enable nginx repo
shell: amazon-linux-extras enable nginx1
- name: yum-clean-metadata
command: yum clean metadata
args:
warn: no
- name: install nginx
yum:
name: [nginx]
state: latest
- name: enable nginx service
service:
name: nginx
enabled: true
```
### Build
With the 2 configuration files we can validate the input and build our custom `AMI` in [AWS](https://aws.amazon.com/).
```console
packer validate .
packer build .
```
### The AMI
Once the process is completed, we can use the [AWS CLI](https://aws.amazon.com/cli/) to inspect the created `AMI` and find the `ImageId`.
```json
$ aws ec2 describe-images --owner self --region eu-central-1
{
"Images": [
{
"Architecture": "x86_64",
"CreationDate": "2021-04-18T00:00:05.000Z",
"ImageId": "ami-01cce7ac6df33f08e",
"ImageLocation": "/packer nginx",
"ImageType": "machine",
"Public": false,
"OwnerId": "",
"PlatformDetails": "Linux/UNIX",
"UsageOperation": "RunInstances",
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/xvda",
"Ebs": {
"DeleteOnTermination": true,
"SnapshotId": "snap-0692717e44e63cbd1",
"VolumeSize": 8,
"VolumeType": "gp2",
"Encrypted": false
}
}
],
"EnaSupport": true,
"Hypervisor": "xen",
"Name": "packer nginx",
"RootDeviceName": "/dev/xvda",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
}
]
}
```
## Deploying the Infrastructure with Terraform
Now we have our custom `AMI` in the `eu-central-1` region. Next we will use [Terraform](https://www.terraform.io/) to deploy this image together with the required infrastructure.
```go
variable "aws_access_key" {
sensitive = true
}
variable "aws_secret_key" {
sensitive = true
}
variable "ami_id" {
default = ""
}
variable "domain" {
default = ""
}
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.0"
}
}
}
provider "aws" {
region = "eu-central-1"
access_key = var.aws_access_key
secret_key = var.aws_secret_key
}
```
### VPC
First we create a simple `VPC` with 2 `subnets` in different `availability zones`.
```go
resource "aws_vpc" "packer" {
cidr_block = "10.0.0.0/16"
tags = {
Name = "packer"
Description = "sample vpc with 2 public subnets in 2 availability zones and a network load balancer for high availability"
}
}
resource "aws_internet_gateway" "inet" {
vpc_id = aws_vpc.packer.id
tags = {
Name = "packer internet gateway"
}
}
resource "aws_default_route_table" "public" {
default_route_table_id = aws_vpc.packer.default_route_table_id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.inet.id
}
tags = {
Name = "public route table"
}
}
resource "aws_subnet" "a" {
vpc_id = aws_vpc.packer.id
cidr_block = "10.0.0.0/24"
availability_zone = "eu-central-1a"
tags = {
Name = "public subnet a"
}
map_public_ip_on_launch = true
}
resource "aws_subnet" "b" {
vpc_id = aws_vpc.packer.id
cidr_block = "10.0.1.0/24"
availability_zone = "eu-central-1b"
tags = {
Name = "public subnet b"
}
map_public_ip_on_launch = true
}
```
### Security Groups
Next, we create the `security groups`.
The default security group has only a reference to itself. It is used to allow traffic to flow between the `ALB` and its `targets`.
The second `security group` is to allow tcp traffic from the public web to the `ALB` on port 80(HTTP) and 443 (HTTPS).
```go
resource "aws_default_security_group" "internal" {
vpc_id = aws_vpc.packer.id
tags = {
Name = "default internal sg"
}
ingress {
protocol = -1
self = true
from_port = 0
to_port = 0
description = "self ref"
}
egress {
protocol = -1
self = true
from_port = 0
to_port = 0
description = "self ref"
}
}
resource "aws_security_group" "web" {
vpc_id = aws_vpc.packer.id
tags = {
Name = "web sg for nginx"
}
ingress {
description = "http traffic"
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = ["::/0"]
}
ingress {
description = "http traffic"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = ["::/0"]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = ["::/0"]
}
}
```
### Logging
We are going to create an `S3 bucket` with the required access `policy` to use it as log destination for the `ALB` in the next section.
```go
resource "aws_s3_bucket" "logs" {
bucket = "com.myorg.logs"
acl = "private"
force_destroy = true
tags = {
Name = "nginx cluster access logs"
Environment = "Dev"
}
}
data "aws_elb_service_account" "main" {
region = "eu-central-1"
}
resource "aws_s3_bucket_policy" "logs" {
bucket = aws_s3_bucket.logs.id
policy = <
```
You can now visit the url in your browser under your the configured subdomain.
Thats it!
The application is deployed from a custom `AMI` across 2 `availability zones` and utilizing `autoscaling`. The traffic is routed via `ELB` which also performs `TLS Termination`.
Additionally, e have our whole infrastructure as code which we can source control.
## Cleaning Up
In order to avoid cost, lets remove all created resources.
```console
terraform destroy
```
Since the AMI and snapshot was not created with [Terraform](https://www.terraform.io/), it wont be destroyed by the former command. We are going to remove them via CLI.
### Deregister Image
```console
aws ec2 deregister-image --image-id
```
### Find the snapshot Id
```console
aws ec2 describe-snapshots --owner self
```
### Delete snapshot
```console
aws ec2 delete-snapshot --snapshot-id
```
```console
linode-cli domains records-delete
```