https://github.com/glueops/terraform-module-cloud-aws-kubernetes-cluster
This repository contains a Terraform module for deploying an Amazon EKS cluster on AWS as part of the GlueOps platform. It facilitates setting up VPCs, subnets, EKS clusters, node pools, and the necessary AWS resources for Kubernetes cluster deployment. It includes configurations for addons like CoreDNS and kube-proxy, and supports VPC peering.
https://github.com/glueops/terraform-module-cloud-aws-kubernetes-cluster
aws aws-eks-cluster aws-resources cloudposse eks eks-setup glueops-platform iam kubernetes kubernetes-addons kubernetes-cluster node-pools terraform terraform-module vpc vpc-peering
Last synced: 6 months ago
JSON representation
This repository contains a Terraform module for deploying an Amazon EKS cluster on AWS as part of the GlueOps platform. It facilitates setting up VPCs, subnets, EKS clusters, node pools, and the necessary AWS resources for Kubernetes cluster deployment. It includes configurations for addons like CoreDNS and kube-proxy, and supports VPC peering.
- Host: GitHub
- URL: https://github.com/glueops/terraform-module-cloud-aws-kubernetes-cluster
- Owner: GlueOps
- Created: 2023-01-04T18:03:15.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-12-16T10:03:23.000Z (over 1 year ago)
- Last Synced: 2024-12-16T13:14:28.664Z (over 1 year ago)
- Topics: aws, aws-eks-cluster, aws-resources, cloudposse, eks, eks-setup, glueops-platform, iam, kubernetes, kubernetes-addons, kubernetes-cluster, node-pools, terraform, terraform-module, vpc, vpc-peering
- Language: HCL
- Homepage:
- Size: 191 KB
- Stars: 1
- Watchers: 4
- Forks: 1
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# terraform-module-cloud-aws-kubernetes-cluster
This terraform module is to help you quickly deploy a EKS cluster on Amazon Web Services (AWS). This is part of the opionated GlueOps Platform. If you came here directly then you should probably visit https://github.com/glueops/admiral as that is the start point.
## Prerequisites to use this Terraform module
- A Dedicated AWS Sub account
- Service account with environment variable set
- Service Quotas (Depending on Cluster Size)
For more details see: https://github.com/GlueOps/terraform-module-cloud-aws-kubernetes-cluster/wiki/
### Example usage of module
```hcl
module "captain" {
iam_role_to_assume = "arn:aws:iam::1234567890:role/glueops-captain-role"
source = "git::https://github.com/GlueOps/terraform-module-cloud-aws-kubernetes-cluster.git"
eks_version = "1.33"
# kubernetesVersion and addonName provided
# renovate: eksAddonsFilter={"kubernetesVersion":"1.33","addonName":"aws-ebs-csi-driver"}
csi_driver_version = "v1.54.0-eksbuild.1"
# kubernetesVersion and addonName provided
# renovate: eksAddonsFilter={"kubernetesVersion":"1.33","addonName":"coredns"}
coredns_version = "v1.12.4-eksbuild.1"
# kubernetesVersion and addonName provided
# renovate: eksAddonsFilter={"kubernetesVersion":"1.33","addonName":"kube-proxy"}
kube_proxy_version = "v1.33.5-eksbuild.2"
vpc_cidr_block = "10.65.0.0/26"
region = "us-west-2"
availability_zones = ["us-west-2a", "us-west-2b"]
private_subnets_enabled = false
node_pools = [
# {
# "kubernetes_version" : "1.33",
# "ami_release_version" : "1.33.5-20251217",
# "ami_type" : "AL2023_x86_64_STANDARD",
# "instance_type" : "t3a.large",
# "name" : "glueops-platform-node-pool-1",
# "node_count" : 4,
# "spot" : false,
# "disk_size_gb" : 20,
# "max_pods" : 110,
# "ssh_key_pair_names" : [],
# "kubernetes_labels" : {
# "glueops.dev/role" : "glueops-platform"
# },
# "kubernetes_taints" : [
# {
# key = "glueops.dev/role"
# value = "glueops-platform"
# effect = "NO_SCHEDULE"
# }
# ]
# },
# {
# "kubernetes_version" : "1.33",
# "ami_release_version" : "1.33.5-20251217",
# "ami_type" : "AL2023_x86_64_STANDARD",
# "instance_type" : "t3a.medium",
# "name" : "glueops-platform-node-pool-argocd-app-controller-1",
# "node_count" : 2,
# "spot" : false,
# "disk_size_gb" : 20,
# "max_pods" : 110,
# "ssh_key_pair_names" : [],
# "kubernetes_labels" : {
# "glueops.dev/role" : "glueops-platform-argocd-app-controller"
# },
# "kubernetes_taints" : [
# {
# key = "glueops.dev/role"
# value = "glueops-platform-argocd-app-controller"
# effect = "NO_SCHEDULE"
# }
# ]
# },
# {
# "kubernetes_version" : "1.33",
# "ami_release_version" : "1.33.5-20251217",
# "ami_type" : "AL2023_x86_64_STANDARD",
# "instance_type" : "t3a.medium",
# "name" : "clusterwide-node-pool-1",
# "node_count" : 2,
# "spot" : false,
# "disk_size_gb" : 20,
# "max_pods" : 110,
# "ssh_key_pair_names" : [],
# "kubernetes_labels" : {},
# "kubernetes_taints" : []
# }
]
peering_configs = [
# {
# vpc_peering_connection_id = "pcx-0df92b5241651ba92"
# destination_cidr_block = "10.69.0.0/26"
# }
]
}
```
## VPC Peering
This terraform module expects only to be an accepter VPC. This means a VPC peering request must come from the requesting account. As an accepter VPC you must provide the requester your VPC ID, your AWS Account ID (The subaccount being used for the cluster deployment), and the VPC CIDR you configured for the cluster deployment.
When providing them with the above, please ask them to [enable DNS resolution of hosts within the requester VPC](https://docs.aws.amazon.com/vpc/latest/peering/modify-peering-connections.html#vpc-peering-dns).
### EFS/NFS Example Manifest
```yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv-test
spec:
storageClassName: efs-fun-test
capacity:
storage: 1000Gi # Adjust based on your needs
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
mountOptions:
- timeo=600
- retrans=2
- nfsvers=4.1
- rsize=1048576
- wsize=1048576
- noresvport
- hard
nfs:
path: /
server: nfs.nonprod.antoniostacos.onglueops.com
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-pvc
spec:
accessModes:
- ReadWriteMany
storageClassName: efs-fun-test
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
containers:
- name: my-container
image: nginx
volumeMounts:
- name: my-volume
mountPath: /mnt/data # Mount path within the container
subPath: pod1-fun
volumes:
- name: my-volume
persistentVolumeClaim:
claimName: my-pvc # Name of the PVC to be mounted
```
## Requirements
No requirements.
## Providers
| Name | Version |
|------|---------|
| [aws](#provider\_aws) | n/a |
## Modules
| Name | Source | Version |
|------|--------|---------|
| [kubernetes](#module\_kubernetes) | cloudposse/eks-cluster/aws | 3.0.0 |
| [node\_pool](#module\_node\_pool) | cloudposse/eks-node-group/aws | 3.1.1 |
| [subnets](#module\_subnets) | cloudposse/dynamic-subnets/aws | 2.4.2 |
| [vpc](#module\_vpc) | cloudposse/vpc/aws | 2.2.0 |
| [vpc\_peering\_accepter\_with\_routes](#module\_vpc\_peering\_accepter\_with\_routes) | ./modules/vpc_peering_accepter_with_routes | n/a |
## Resources
| Name | Type |
|------|------|
| [aws_eks_addon.coredns](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) | resource |
| [aws_eks_addon.ebs_csi](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) | resource |
| [aws_eks_addon.kube_proxy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) | resource |
| [aws_iam_role.eks_addon_ebs_csi_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy_attachment.ebs_csi](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_security_group.captain](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group_rule.allow_all_within_group](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_security_group_rule.captain_egress_all_ipv4](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_iam_openid_connect_provider.provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_openid_connect_provider) | data source |
| [aws_iam_policy_document.eks_assume_addon_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
## Inputs
| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| [availability\_zones](#input\_availability\_zones) | The availability zones to deploy into | `list(string)` |
[
"us-west-2a",
"us-west-2b",
"us-west-2c"
]
| no |
| [coredns\_version](#input\_coredns\_version) | You should grab the appropriate version number from: https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html | `string` | `"v1.12.4-eksbuild.1"` | no |
| [csi\_driver\_version](#input\_csi\_driver\_version) | You should grab the appropriate version number from: https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/CHANGELOG.md | `string` | `"v1.54.0-eksbuild.1"` | no |
| [eks\_version](#input\_eks\_version) | The version of EKS to deploy | `string` | `"1.33"` | no |
| [iam\_role\_to\_assume](#input\_iam\_role\_to\_assume) | The full ARN of the IAM role to assume | `string` | n/a | yes |
| [kube\_proxy\_version](#input\_kube\_proxy\_version) | You should grab the appropriate version number from: https://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html | `string` | `"v1.33.5-eksbuild.2"` | no |
| [node\_pools](#input\_node\_pools) | node pool configurations:
- name (string): Name of the node pool. MUST BE UNIQUE! Recommended to use YYYYMMDD in the name
- node\_count (number): number of nodes to create in the node pool.
- instance\_type (string): Instance type to use for the nodes. ref: https://instances.vantage.sh/
- kubernetes\_version (string): Generally this is the same version as the EKS cluster. But if doing a node pool upgrade this may be a different version.
- ami\_release\_version (string): AMI Release version to use for EKS worker nodes. ref: https://github.com/awslabs/amazon-eks-ami/releases
- ami\_type (string): e.g. AMD64 or ARM
- spot (bool): Enable spot instances for the nodes. DO NOT ENABLE IN PROD!
- disk\_size\_gb (number): Disk size in GB for the nodes.
- max\_pods (number): max pods that can be scheduled per node.
- ssh\_key\_pair\_names (list(string)): List of SSH key pair names to associate with the nodes. ref: https://us-west-2.console.aws.amazon.com/ec2/home?region=us-west-2#KeyPairs:
- kubernetes\_labels (map(string)): Map of labels to apply to the nodes. ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
- kubernetes\_taints (list(object)): List of taints to apply to the nodes. ref: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ | list(object({
name = string
node_count = number
instance_type = string
kubernetes_version = string
ami_release_version = string
ami_type = string
spot = bool
disk_size_gb = number
max_pods = number
ssh_key_pair_names = list(string)
kubernetes_labels = map(string)
kubernetes_taints = list(object({
key = string
value = string
effect = string
}))
})) | [
{
"ami_release_version": "1.33.5-20251217",
"ami_type": "AL2023_x86_64_STANDARD",
"disk_size_gb": 20,
"instance_type": "t3a.large",
"kubernetes_labels": {},
"kubernetes_taints": [],
"kubernetes_version": "1.33",
"max_pods": 110,
"name": "default-pool",
"node_count": 1,
"spot": false,
"ssh_key_pair_names": []
}
]
| no |
| [peering\_configs](#input\_peering\_configs) | A list of maps containing VPC peering configuration details | list(object({
vpc_peering_connection_id = string
destination_cidr_block = string
})) | `[]` | no |
| [private\_subnets\_enabled](#input\_private\_subnets\_enabled) | enable private subnets | `bool` | `false` | no |
| [region](#input\_region) | The AWS region to deploy into | `string` | n/a | yes |
| [vpc\_cidr\_block](#input\_vpc\_cidr\_block) | The CIDR block for the VPC | `string` | `"10.65.0.0/26"` | no |
## Outputs
No outputs.