{"id":18445617,"url":"https://github.com/voronenko/devops-tf-eks","last_synced_at":"2025-04-08T00:31:38.604Z","repository":{"id":145345329,"uuid":"140765689","full_name":"Voronenko/devops-tf-eks","owner":"Voronenko","description":"terraform baked AWS kubernetes cluster (EKS) - 1.18 compatible","archived":false,"fork":false,"pushed_at":"2021-06-11T14:48:40.000Z","size":96,"stargazers_count":15,"open_issues_count":1,"forks_count":10,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-23T03:07:09.282Z","etag":null,"topics":["amazon-eks","eks","kubernetes","oops-to-devops","terraform","tf"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Voronenko.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-07-12T21:18:28.000Z","updated_at":"2023-06-26T11:19:48.000Z","dependencies_parsed_at":"2023-07-03T09:17:24.965Z","dependency_job_id":null,"html_url":"https://github.com/Voronenko/devops-tf-eks","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Voronenko%2Fdevops-tf-eks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Voronenko%2Fdevops-tf-eks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Voronenko%2Fdevops-tf-eks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Voronenko%2Fdevops-tf-eks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Voronenko","download_url":"https://codeload.github.com/Voronenko/devops-tf-eks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247755338,"owners_count":20990616,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon-eks","eks","kubernetes","oops-to-devops","terraform","tf"],"created_at":"2024-11-06T07:06:35.241Z","updated_at":"2025-04-08T00:31:38.185Z","avatar_url":"https://github.com/Voronenko.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kubernetes Cluster Setup\n\nImportant: setup follows https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html as of commit date.\n\nIn order to start with cluster setup, you will need:\n\n- Terraform\n- AWS credentials with necessary rights\n- AWS authenticator for EKS, called aws-iam-authenticator:\n  ```sh\n   curl -o ~/dotfiles/bin/aws-iam-authenticator https://amazon-eks.s3.us-west-2.amazonaws.com/1.18.8/2020-09-18/bin/linux/amd64/aws-iam-authenticator\n   chmod +x ~/dotfiles/bin/aws-iam-authenticator\n  ```\n\nNote, that with fresh aws-cli you can use aws-cli subcommands to achieve the same effect.\n\n## Upgrade 1.18-\u003e1.19 via aws UI\n\nUpgrade kube version via aws console.  After that, upgrade all nodegroups to the image recommended for the new cluster version.\nYou will need to update kube-proxy and coredns images manually. Check proper versions at https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html\n\n```sh\nkubectl set image daemonset.apps/kube-proxy -n kube-system kube-proxy=602401143452.dkr.ecr.eu-west-1.amazonaws.com/eks/kube-proxy:v1.19.6-eksbuild.1\n\nkubectl set image --namespace kube-system deployment.apps/coredns coredns=602401143452.dkr.ecr.eu-west-1.amazonaws.com/eks/coredns:v1.8.0-eksbuild.1\n```\n\nLast step would be updating cni, do not forget to update image registry to the registry located in your cluster region\n\n```sh\ncurl -o aws-k8s-cni.yaml https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/v1.7.5/config/v1.7/aws-k8s-cni.yaml\nvim aws-k8s-cni.yaml\n%s/us-west-2/eu-central-1/g\nkubectl apply -f aws-k8s-cni.yaml\n```\n\n\n## Variables\n\nSet the following environment variables:\n\n```\nexport AWS_ACCESS_KEY_ID=\nexport AWS_SECRET_ACCESS_KEY=\nexport TF_VAR_CLUSTER_NAME=\n```\n\n\u003c!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK --\u003e\n## Requirements\n\n| Name | Version |\n|------|---------|\n| terraform | \u003e= 0.14 |\n| aws | ~\u003e 3.29.1 |\n| helm | ~\u003e 2.0.2 |\n| kubernetes | ~\u003e 1.13.3 |\n\n## Providers\n\n| Name | Version |\n|------|---------|\n| aws | ~\u003e 3.29.1 |\n| local | n/a |\n\n## Inputs\n\n| Name | Description | Type | Default | Required |\n|------|-------------|------|---------|:--------:|\n| AWS\\_ACCESS\\_KEY\\_ID | n/a | `any` | n/a | yes |\n| AWS\\_REGION | n/a | `any` | n/a | yes |\n| AWS\\_SECRET\\_ACCESS\\_KEY | n/a | `any` | n/a | yes |\n| CLUSTER\\_NAME | n/a | `string` | `\"istio\"` | no |\n| SCALING\\_DESIRED\\_CAPACITY | n/a | `number` | `2` | no |\n| cluster\\_version | Version of the kubernetes cluster | `string` | `\"1.18\"` | no |\n| domain | n/a | `string` | `\"\"` | no |\n| eks\\_oidc\\_root\\_ca\\_thumbprint | Thumbprint of Root CA for EKS OIDC, Valid until 2037 | `string` | `\"9e99a48a9960b14926bb7f3b02e22da2b0ab7280\"` | no |\n| enable\\_irsa | Whether to create OpenID Connect Provider for EKS to enable IRSA | `bool` | `true` | no |\n| external\\_dns\\_helm\\_chart\\_name | n/a | `string` | `\"external-dns\"` | no |\n| external\\_dns\\_helm\\_chart\\_version | n/a | `string` | `\"3.5.1\"` | no |\n| external\\_dns\\_helm\\_release\\_name | n/a | `string` | `\"external-dns\"` | no |\n| external\\_dns\\_helm\\_repo\\_url | n/a | `string` | `\"https://charts.bitnami.com/bitnami\"` | no |\n| external\\_dns\\_k8s\\_namespace | The k8s namespace in which the alb-ingress service account has been created | `string` | `\"kube-system\"` | no |\n| external\\_dns\\_k8s\\_service\\_account\\_name | The k8s external-dns service account name, ideally should match to helm chart expectations | `string` | `\"external-dns\"` | no |\n| external\\_dns\\_settings | Additional settings for external-dns helm chart check https://github.com/bitnami/charts/tree/master/bitnami/external-dns | `map(any)` | `{}` | no |\n| ingress\\_alb\\_helm\\_chart\\_name | n/a | `string` | `\"aws-load-balancer-controller\"` | no |\n| ingress\\_alb\\_helm\\_chart\\_version | n/a | `string` | `\"1.0.3\"` | no |\n| ingress\\_alb\\_helm\\_release\\_name | n/a | `string` | `\"aws-load-balancer-controller\"` | no |\n| ingress\\_alb\\_helm\\_repo\\_url | n/a | `string` | `\"https://aws.github.io/eks-charts\"` | no |\n| ingress\\_alb\\_k8s\\_dummy\\_dependency | TODO: eliminate allows dirty re-drop | `any` | `null` | no |\n| ingress\\_alb\\_k8s\\_namespace | The k8s namespace in which the alb-ingress service account has been created | `string` | `\"kube-system\"` | no |\n| ingress\\_alb\\_k8s\\_service\\_account\\_name | The k8s alb-ingress service account name, should match to helm chart expectations | `string` | `\"aws-load-balancer-controller\"` | no |\n| ingress\\_alb\\_settings | Additional settings for Helm chart check https://artifacthub.io/packages/helm/helm-incubator/aws-alb-ingress-controller | `map(any)` | `{}` | no |\n| namespaces | List of namespaces to be created in our EKS Cluster. | `list(string)` | `[]` | no |\n| option\\_alb\\_ingress\\_enabled | n/a | `bool` | `false` | no |\n| option\\_external\\_dns\\_enabled | n/a | `bool` | `false` | no |\n| private\\_subnet\\_cidr | CIDR for the Private Subnet | `string` | `\"10.11.1.0/24\"` | no |\n| private\\_subnet\\_cidr2 | CIDR for the Private Subnet | `string` | `\"10.11.3.0/24\"` | no |\n| public\\_subnet\\_cidr | CIDR for the Public Subnet | `string` | `\"10.11.0.0/24\"` | no |\n| public\\_subnet\\_cidr2 | CIDR for the Public Subnet | `string` | `\"10.11.2.0/24\"` | no |\n| use\\_simple\\_vpc | n/a | `bool` | `true` | no |\n| vpc\\_cidr | CIDR for the whole VPC | `string` | `\"10.11.0.0/16\"` | no |\n\n## Outputs\n\n| Name | Description |\n|------|-------------|\n| cluster\\_arn | The Amazon Resource Name (ARN) of the cluster. |\n| cluster\\_certificate\\_authority\\_data | Nested attribute containing certificate-authority-data for your cluster. This is the base64 encoded certificate data required to communicate with your cluster. |\n| cluster\\_endpoint | The endpoint for your EKS Kubernetes API. |\n| cluster\\_id | The id of the EKS cluster. |\n| cluster\\_oidc\\_issuer\\_url | The URL on the EKS cluster OIDC Issuer |\n| cluster\\_version | The Kubernetes server version for the EKS cluster. |\n| kubeconfig | n/a |\n| oidc\\_provider\\_arn | The ARN of the OIDC Provider if `enable_irsa = true`. |\n\n\u003c!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK --\u003e\n\n\n## Deploy\n\n### Plan and apply\n\n```bash\n$ make plan\n$ make apply\n```\n\nPlan will check what changes Terraform needs to apply, then apply deploys the changes.\n\nThe operation takes around 20 minutes.\n\n### Configure\n\nOnce the deployment is done, the kubeconfig will let kubectl know how to connect to cluster:\n```bash\n$ make kubeconfig\nrun the suggested EXPORT\n```\n\nTo make workers join the cluster they need to be have a role associated with it:\n```bash\n$ make config-map-aws-auth\n```\nand wait for nodes to appear.\n\nYour are now ready.\n\n### Login into worker instances\n\nTo login into the instances the private key can be generated from the terraform output:\n\n```bash\n$ make private-key\n```\n\nPrior login you would need to switch the gateway used by the workers private route table from the nat to the internet one\n and open the SSH port on the worker security group.\n\n### Destroy\n\nAs simple as `make destroy`.\n\n## Kubernetes dashboard\n\nFollow the documentation here https://docs.aws.amazon.com/eks/latest/userguide/dashboard-tutorial.html\n\n\n### Working with multiple clusters at a time\n\nBy default, terraform creates one state file, that you will need to replace, if you are working with another cluster.\nTo smoothly work with multiple clusters, you can use terraform workspaces (https://www.terraform.io/docs/state/workspaces.html)\n\nFor example, you can create workspace by cluster name before provisioning cluster, and use `${terraform.workspace}`\nas a cluster name  `cluster_name= \"${terraform.workspace}-cluster\"`\n\n`terraform workspace new dummy`\n\nview workspaces:\n\n```\nterraform workspace list\n  default\n* dummy\n```\n\nselect specific workplace:\n\n```\nterraform workspace select dummy\n```\n\n### Credits\n\nhttps://github.com/TizianoPerrucci  - Formatting, Readme changes, 1.16 upgrade\n\n### Further reading:\n\nPerhaps try  https://github.com/aws-samples/eks-workshop\n\n\n/ end of compact notes\n\n\n===========================================================================================================================\n\nBelow is the original readme as was for 1.11, for historic purposes\n\nLong story:\n\n2BD\n\n# Getting started with kubernetes cluster on AWS\n\nAWS provides comprehensive guide to start with EKS on a link\nhttps://docs.aws.amazon.com/eks/latest/userguide/getting-started.html\n\nNext steps generally follow that guide, difference is that provisioning\nplay is (can be) implemented purely with terraform w/o cloud templates.\n\nIt has advantage in a way how we do deployments, but in your scenario cloud templates might have advantage too.\n\n## First step is creating a VPC with Public and Private Subnets for Your Amazon EKS Cluster\n\nThe simpliest VPC would be all subnets public, like seen on cloud template\n( https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/amazon-eks-vpc-sample.yaml )\n\nBut if your goal is to reuse that in a production, the reasonable would be VPC with private and public parts, like  https://docs.aws.amazon.com/eks/latest/userguide/create-public-private-vpc.html\n\n\nAt the moment there are limitations:  https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html\n\nLet's proceed with recommended network architecture that uses private subnets for your worker nodes and public subnets for Kubernetes to create internet-facing load balancers within\n\n### Step 1.1: Create an Elastic IP Address for Your NAT Gateway(s)\n\nWorker nodes in private subnets require a NAT gateway for outbound internet access. A NAT gateway requires an Elastic IP address in your public subnet, but the VPC wizard does not create one for you. Create the Elastic IP address before running the VPC wizard.\n\nTo create an Elastic IP address\n\nOpen the Amazon VPC console at https://console.aws.amazon.com/vpc/.\n\nIn the left navigation pane, choose Elastic IPs.\nChoose Allocate new address, Allocate, Close.\nNote the Allocation ID for your newly created Elastic IP address; you enter this later in the VPC wizard.\n\n```tf\n\n// reserve elastic ip for nat gateway\n\nresource \"aws_eip\" \"nat_eip\" {\n  vpc = true\n  tags {\n    Environment = \"${local.env}\"\n  }\n}\n\nresource \"aws_eip\" \"nat_eip_2\" {\n  vpc = true\n  tags {\n    Environment = \"${local.env}\"\n  }\n}\n\n```\n\n\n### Step 1.2: Run the VPC Wizard\n\nThe VPC wizard automatically creates and configures most of your VPC resources for you.\n\nTo run the VPC wizard\n\nIn the left navigation pane, choose VPC Dashboard.\nChoose Start VPC Wizard, VPC with Public and Private Subnets, Select.\nFor VPC name, give your VPC a unique name.\nFor Elastic IP Allocation ID, choose the ID of the Elastic IP address that you created earlier.\nChoose Create VPC.\nWhen the wizard is finished, choose OK. Note the Availability Zone in which your VPC subnets were created. Your additional subnets should be created in a different Availability Zone.\nAppropriate terraform part would be\n\n```tf\n\n\n// create vpc\n\n//Base VPC Networking\n//EKS requires the usage of Virtual Private Cloud to provide the base for its networking configuration.\n\n\nresource \"aws_vpc\" \"cluster\" {\n  cidr_block = \"${var.vpc_cidr}\"\n  enable_dns_hostnames = true\n\n  tags = \"${merge(\n    local.common_tags,\n    map(\n      \"Name\", \"${local.env}-eks-vpc\"\n    )\n  )}\"\n\n}\n\n//The below will create a ${var.public_subnet_cidr} VPC,\n//two ${var.public_subnet_cidr} public subnets,\n//two ${var.private_subnet_cidr} private subnets with nat instances,\n//an internet gateway,\n//and setup the subnet routing to route external traffic through the internet gateway:\n\n// public subnets\nresource \"aws_subnet\" \"eks-public\" {\n  vpc_id = \"${aws_vpc.cluster.id}\"\n\n  cidr_block = \"${var.public_subnet_cidr}\"\n  availability_zone = \"${local.availabilityzone}\"\n\n  tags = \"${merge(\n    local.common_tags,\n    map(\n      \"Name\", \"${local.env}-eks-public\"\n    )\n  )}\"\n\n}\n\n\nresource \"aws_subnet\" \"eks-public-2\" {\n  vpc_id = \"${aws_vpc.cluster.id}\"\n\n  cidr_block = \"${var.public_subnet_cidr2}\"\n  availability_zone = \"${local.availabilityzone2}\"\n\n  tags = \"${merge(\n    local.common_tags,\n    map(\n      \"Name\", \"${local.env}-eks-public-2\"\n    )\n  )}\"\n\n}\n\n\n// private subnet\nresource \"aws_subnet\" \"eks-private\" {\n  vpc_id = \"${aws_vpc.cluster.id}\"\n\n  cidr_block = \"${var.private_subnet_cidr}\"\n  availability_zone = \"${local.availabilityzone}\"\n\n  tags = \"${merge(\n    local.common_tags,\n    map(\n      \"Name\", \"${local.env}-eks-private\"\n    )\n  )}\"\n\n}\n\nresource \"aws_subnet\" \"eks-private-2\" {\n  vpc_id = \"${aws_vpc.cluster.id}\"\n\n  cidr_block = \"${var.private_subnet_cidr2}\"\n  availability_zone = \"${local.availabilityzone2}\"\n\n  tags = \"${merge(\n    local.common_tags,\n    map(\n      \"Name\", \"${local.env}-eks-private-2\"\n    )\n  )}\"\n\n}\n\n\n// internet gateway, note: creation takes a while\n\nresource \"aws_internet_gateway\" \"igw\" {\n  vpc_id = \"${aws_vpc.cluster.id}\"\n  tags {\n    Environment = \"${local.env}\"\n  }\n}\n\n\n\n// reserve elastic ip for nat gateway\n\nresource \"aws_eip\" \"nat_eip\" {\n  vpc = true\n  tags {\n    Environment = \"${local.env}\"\n  }\n}\n\nresource \"aws_eip\" \"nat_eip_2\" {\n  vpc = true\n  tags {\n    Environment = \"${local.env}\"\n  }\n}\n\n\n// create nat once internet gateway created\nresource \"aws_nat_gateway\" \"nat_gateway\" {\n  allocation_id = \"${aws_eip.nat_eip.id}\"\n  subnet_id = \"${aws_subnet.eks-public.id}\"\n  depends_on = [\"aws_internet_gateway.igw\"]\n  tags {\n    Environment = \"${local.env}\"\n  }\n}\n\nresource \"aws_nat_gateway\" \"nat_gateway_2\" {\n  allocation_id = \"${aws_eip.nat_eip_2.id}\"\n  subnet_id = \"${aws_subnet.eks-public-2.id}\"\n  depends_on = [\"aws_internet_gateway.igw\"]\n  tags {\n    Environment = \"${local.env}\"\n  }\n\n}\n\n\n//Create private route table and the route to the internet\n//This will allow all traffics from the private subnets to the internet through the NAT Gateway (Network Address Translation)\n\nresource \"aws_route_table\" \"private_route_table\" {\n  vpc_id = \"${aws_vpc.cluster.id}\"\n  tags {\n    Environment = \"${local.env}\"\n    Name = \"${local.env}-private-route-table\"\n  }\n\n}\n\nresource \"aws_route_table\" \"private_route_table_2\" {\n  vpc_id = \"${aws_vpc.cluster.id}\"\n  tags {\n    Environment = \"${local.env}\"\n    Name = \"${local.env}-private-route-table-2\"\n  }\n\n}\n\nresource \"aws_route\" \"private_route\" {\n  route_table_id  = \"${aws_route_table.private_route_table.id}\"\n  destination_cidr_block = \"0.0.0.0/0\"\n  nat_gateway_id = \"${aws_nat_gateway.nat_gateway.id}\"\n}\n\nresource \"aws_route\" \"private_route_2\" {\n  route_table_id  = \"${aws_route_table.private_route_table_2.id}\"\n  destination_cidr_block = \"0.0.0.0/0\"\n  nat_gateway_id = \"${aws_nat_gateway.nat_gateway_2.id}\"\n}\n\n\nresource \"aws_route_table\" \"eks-public\" {\n  vpc_id = \"${aws_vpc.cluster.id}\"\n\n  route {\n    cidr_block = \"0.0.0.0/0\"\n    gateway_id = \"${aws_internet_gateway.igw.id}\"\n  }\n\n  tags {\n    Environment = \"${local.env}\"\n    Name = \"${local.env}-eks-public\"\n  }\n\n}\n\n\nresource \"aws_route_table\" \"eks-public-2\" {\n  vpc_id = \"${aws_vpc.cluster.id}\"\n\n  route {\n    cidr_block = \"0.0.0.0/0\"\n    gateway_id = \"${aws_internet_gateway.igw.id}\"\n  }\n\n  tags {\n    Environment = \"${local.env}\"\n    Name = \"${local.env}-eks-public-2\"\n  }\n}\n\n\n// associate route tables\n\nresource \"aws_route_table_association\" \"eks-public\" {\n  subnet_id = \"${aws_subnet.eks-public.id}\"\n  route_table_id = \"${aws_route_table.eks-public.id}\"\n}\n\nresource \"aws_route_table_association\" \"eks-public-2\" {\n  subnet_id = \"${aws_subnet.eks-public-2.id}\"\n  route_table_id = \"${aws_route_table.eks-public-2.id}\"\n}\n\n\nresource \"aws_route_table_association\" \"eks-private\" {\n  subnet_id = \"${aws_subnet.eks-private.id}\"\n  route_table_id = \"${aws_route_table.private_route_table.id}\"\n}\n\n\nresource \"aws_route_table_association\" \"eks-private-2\" {\n  subnet_id = \"${aws_subnet.eks-private-2.id}\"\n  route_table_id = \"${aws_route_table.private_route_table_2.id}\"\n}\n\n```\n\n\n## Step 2 create your Amazon EKS service role\n\nOpen the IAM console at https://console.aws.amazon.com/iam/.\n\nChoose Roles, then Create role.\n\nChoose EKS from the list of services, then Allows Amazon EKS to manage your clusters on your behalf for your use case, then Next: Permissions.\n\nChoose Next: Review.\n\nFor Role name, enter a unique name for your role, such as eksServiceRole, then choose Create role.\n\n\n\nTerraform definition for the role goes as below.\n\n```tf\n//Kubernetes Masters\n//This is where the EKS service comes into play.\n//It requires a few operator managed resources beforehand so that Kubernetes can properly manage\n//other AWS services as well as allow inbound networking communication from your local workstation\n//(if desired) and worker nodes.\n\n\n//EKS Master Cluster IAM Role\n//\n//IAM role and policy to allow the EKS service to manage or retrieve data from other AWS services.\n//For the latest required policy, see the EKS User Guide.\n\n\nresource \"aws_iam_role\" \"EKSClusterRole\" {\n  name = \"EKSClusterRole-${local.env}\",\n  description = \"Allows EKS to manage clusters on your behalf.\",\n  assume_role_policy = \u003c\u003cPOLICY\n{\n   \"Version\":\"2012-10-17\",\n   \"Statement\":[\n      {\n         \"Effect\":\"Allow\",\n         \"Principal\":{\n            \"Service\":\"eks.amazonaws.com\"\n         },\n         \"Action\":\"sts:AssumeRole\"\n      }\n   ]\n}\nPOLICY\n}\n\n//https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html\n\nresource \"aws_iam_role_policy_attachment\" \"eks-policy-AmazonEKSClusterPolicy\" {\n  policy_arn = \"arn:aws:iam::aws:policy/AmazonEKSClusterPolicy\"\n  role       = \"${aws_iam_role.EKSClusterRole.name}\"\n}\n\nresource \"aws_iam_role_policy_attachment\" \"eks-policy-AmazonEKSServicePolicy\" {\n  policy_arn = \"arn:aws:iam::aws:policy/AmazonEKSServicePolicy\"\n  role       = \"${aws_iam_role.EKSClusterRole.name}\"\n}\n```\n\n\n## Step 3: Create a Control Plane Security Group\n\nWhen you create an Amazon EKS cluster, your cluster control plane creates elastic network interfaces\nin your subnets to enable communication with the worker nodes. You should create a security group that\nis dedicated to your Amazon EKS cluster control plane, so that you can apply inbound and outbound rules\nto govern what traffic is allowed across that connection. When you create the cluster, you specify this security group,\nand that is applied to the elastic network interfaces that are created in your subnets.\n\n\nTo create a control plane security group\n\nIn the left navigation pane, for Filter by VPC, select your VPC and choose Security Groups, Create Security Group.\n\nNote\n\nIf you don't see your new VPC here, refresh the page to pick it up.\nFill in the following fields and choose Yes, Create:\n\nFor Name tag, provide a name for your security group. For example, \u003ccluster-name\u003e-control-plane.\nFor Description, provide a description of your security group to help you identify it later.\nFor VPC, choose the VPC that you are using for your Amazon EKS cluster.\n\n```tf\n//EKS Master Cluster Security Group\n\n//This security group controls networking access to the Kubernetes masters.\n//Needs to be configured also with an ingress rule to allow traffic from the worker nodes.\n\nresource \"aws_security_group\" \"eks-control-plane-sg\" {\n  name        = \"${local.env}-control-plane\"\n  description = \"Cluster communication with worker nodes [${local.env}]\"\n  vpc_id      = \"${aws_vpc.cluster.id}\"\n\n  egress {\n    from_port       = 0\n    to_port         = 0\n    protocol        = \"-1\"\n    cidr_blocks     = [\"0.0.0.0/0\"]\n  }\n}\n\n# OPTIONAL: Allow inbound traffic from your local workstation external IP\n#           to the Kubernetes. You will need to replace A.B.C.D below with\n#           your real IP. Services like icanhazip.com can help you find this.\n//resource \"aws_security_group_rule\" \"eks-ingress-workstation-https\" {\n//  cidr_blocks       = [\"A.B.C.D/32\"]\n//  description       = \"Allow workstation to communicate with the cluster API Server\"\n//  from_port         = 443\n//  protocol          = \"tcp\"\n//  security_group_id = \"${aws_security_group.eks-control-plane-sg.id}\"\n//  to_port           = 443\n//  type              = \"ingress\"\n//}\n```\n\n\nDownload aws authenticator, kind of\n\n```sh\n   curl -o ~/dotfiles/bin/aws-iam-authenticator https://amazon-eks.s3.us-west-2.amazonaws.com/1.18.8/2020-09-18/bin/linux/amd64/aws-iam-authenticator\n   chmod +x ~/dotfiles/bin/aws-iam-authenticator\n```\n\nValidate by running\n\n```sh\naws-iam-authenticator help\n```\n\nCheck if you have latest awscli installed, you should get output similar to one below\n```sh\naws eks describe-cluster --name test-cluster\n\n{\n    \"cluster\": {\n        \"status\": \"ACTIVE\",\n        \"endpoint\": \"https://1F3D31820046A120545D3A2FC1422C04.sk1.us-east-1.eks.amazonaws.com\",\n        \"name\": \"test-cluster\",\n        \"certificateAuthority\": {\n            \"data\": \"CENSORED\"\n        },\n        \"roleArn\": \"arn:aws:iam::CENSORED:role/EKSClusterRole\",\n        \"resourcesVpcConfig\": {\n            \"subnetIds\": [\n                \"subnet-065058abfee2adfca\",\n                \"subnet-0eb04ec6e3f359b5c\",\n                \"subnet-0fd73ea9b4a7dab35\",\n                \"subnet-0ca730350e571154b\"\n            ],\n            \"vpcId\": \"vpc-0daf88b515eac80ab\",\n            \"securityGroupIds\": [\n                \"sg-09de76fce3a132c15\"\n            ]\n        },\n        \"version\": \"1.10\",\n        \"arn\": \"arn:aws:eks:us-east-1:CENSORED:cluster/test-cluster\",\n        \"createdAt\": 1530892117.396\n    }\n}\n\n\n```\n\n\n## Step 4: create  EKS cluster\n\nNow it is time to launch the cluster, either via UI\nor using provisioning tool of your choice\n\n\nMaking cluster to ECS might seem a bit tricky , as by default AWS proposes to hold\ncluster info in the separate files, like  `~/.kube/eksconfig`  and use environment\nvariables for switching context `export KUBECONFIG=~/.kube/eksconfig`\n\nIf you have multiple clusters configured, that's for sure will not be convenient to you.\n\n```\napiVersion: v1\nclusters:\n- cluster:\n    server: ${aws_eks_cluster.demo.endpoint}\n    certificate-authority-data: ${aws_eks_cluster.demo.certificate_authority.0.data}\n  name: kubernetes\ncontexts:\n- context:\n    cluster: kubernetes\n    user: aws\n  name: aws\ncurrent-context: aws\nkind: Config\npreferences: {}\nusers:\n- name: aws\n  user:\n    exec:\n      apiVersion: client.authentication.k8s.io/v1alpha1\n      command: aws-iam-authenticator\n      args:\n        - \"token\"\n        - \"-i\"\n        - \"${var.cluster-name}\"\n        # - \"-r\"\n        # - \"\u003crole-arn\u003e\"\n      # env:\n        # - name: AWS_PROFILE\n        #   value: \"\u003caws-profile\u003e\"\n```\n\nfor example above, you could validate, if you configured your cluster and kubectl right.\n\n```\nkubectx\naws\n\nkubectl get svc\nNAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE\nkubernetes   ClusterIP   172.20.0.1   \u003cnone\u003e        443/TCP   4d\n\n```\n\nTerraform part for the creating cluster action would be\n\n```tf\n\n//\n//EKS Master Cluster\n//This resource is the actual Kubernetes master cluster. It can take a few minutes to provision in AWS.\n\n\n\nresource \"aws_eks_cluster\" \"eks-cluster\" {\n  name     = \"${local.cluster_name}\"\n  role_arn = \"${aws_iam_role.EKSClusterRole.arn}\"\n\n  vpc_config {\n    security_group_ids = [\"${aws_security_group.eks-control-plane-sg.id}\"]\n\n    subnet_ids = [\n      \"${aws_subnet.eks-private.id}\",\n      \"${aws_subnet.eks-private-2.id}\"\n    ]\n  }\n\n  depends_on = [\n    \"aws_iam_role_policy_attachment.eks-policy-AmazonEKSClusterPolicy\",\n    \"aws_iam_role_policy_attachment.eks-policy-AmazonEKSServicePolicy\",\n  ]\n}\n\n\nlocals {\n  kubeconfig = \u003c\u003cKUBECONFIG\n\n\napiVersion: v1\nclusters:\n- cluster:\n    server: ${aws_eks_cluster.eks-cluster.endpoint}\n    certificate-authority-data: ${aws_eks_cluster.eks-cluster.certificate_authority.0.data}\n  name: kubernetes\ncontexts:\n- context:\n    cluster: kubernetes\n    user: aws\n  name: aws-${local.env}\ncurrent-context: aws-${local.env}\nkind: Config\npreferences: {}\nusers:\n- name: aws\n  user:\n    exec:\n      apiVersion: client.authentication.k8s.io/v1alpha1\n      command: aws-iam-authenticator\n      args:\n        - \"token\"\n        - \"-i\"\n        - \"${local.cluster_name}\"\nKUBECONFIG\n}\n\noutput \"kubeconfig\" {\n  value = \"${local.kubeconfig}\"\n}\n```\n\nNote, that instead of patching kubelet config, as per original guide,\nyou can just get ready-to-use file content from `terraform output kubeconfig`.\n\n\n## Step 5,6:  Now it is time to launch worker nodes\n\nAt a time being AWS recommends to launch stack using, for example, following cloud formation template\nhttps://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/amazon-eks-nodegroup.yaml\n\nWhat you would need from outcomes, is record the NodeInstanceRole for the node group that was created.\nYou need this when you configure your Amazon EKS worker nodes.\n\nOn outputs pane you would see output kind of\n\n```\narn:aws:iam::CENSORED:role/test-cluster-worker-nodes-NodeInstanceRole-XF8G1BPXC56B\n```\n\nfinal step would be ensuring that worker nodes can join your cluster.\n\nYou would need template for kubernetes configuration map\n\nhttps://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/aws-auth-cm.yaml\n\n```sh\ncurl -O https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/aws-auth-cm.yaml\n\ncat aws-auth-cm.yaml\n\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: aws-auth\n  namespace: kube-system\ndata:\n  mapRoles: |\n    - rolearn: \u003cARN of instance role (not instance profile)\u003e\n      username: system:node:{{EC2PrivateDNSName}}\n      groups:\n        - system:bootstrappers\n        - system:nodes\n\n```\n\namend role arn and apply changes with\n\n```sh\n\nkubectl apply -f aws-auth-cm.yaml\nconfigmap \"aws-auth\" created\n\n```\n\nIn a few minutes your fleet should be ready\n\n```sh\n\nkubectl get nodes\nNAME                          STATUS     ROLES     AGE       VERSION\nip-10-11-1-210.ec2.internal   NotReady   \u003cnone\u003e    11s       v1.10.3\nip-10-11-3-36.ec2.internal    NotReady   \u003cnone\u003e    14s       v1.10.3\nip-10-11-3-4.ec2.internal     NotReady   \u003cnone\u003e    9s        v1.10.3\n\n~~~~~~~~~~~~\n\nkubectl get nodes\nNAME                          STATUS    ROLES     AGE       VERSION\nip-10-11-1-210.ec2.internal   Ready     \u003cnone\u003e    1m        v1.10.3\nip-10-11-3-36.ec2.internal    Ready     \u003cnone\u003e    1m        v1.10.3\nip-10-11-3-4.ec2.internal     Ready     \u003cnone\u003e    1m        v1.10.3\n```\n\n\nIf you want to omit relying on cloud formation template by URL, and instead stick to fully scripted infrastructure:\n\n### Step 5: on that we prepare worker node and security groups\n\n```tf\n//Kubernetes Worker Nodes\n//The EKS service does not currently provide managed resources for running worker nodes.\n//Here we will create a few operator managed resources so that Kubernetes can properly manage\n//other AWS services, networking access, and finally a configuration that allows\n//automatic scaling of worker nodes.\n\n\n//Worker Node IAM Role and Instance Profile\n//IAM role and policy to allow the worker nodes to manage or retrieve data from other AWS services.\n//It is used by Kubernetes to allow worker nodes to join the cluster.\n//\n//For the latest required policy, see the EKS User Guide.\n\nresource \"aws_iam_role\" \"EKSNodeRole\" {\n  name = \"eks-${local.cluster_name}-node-role\"\n\n  assume_role_policy = \u003c\u003cPOLICY\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Principal\": {\n        \"Service\": \"ec2.amazonaws.com\"\n      },\n      \"Action\": \"sts:AssumeRole\"\n    }\n  ]\n}\nPOLICY\n}\n\nresource \"aws_iam_role_policy_attachment\" \"eks-node-AmazonEKSWorkerNodePolicy\" {\n  policy_arn = \"arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy\"\n  role       = \"${aws_iam_role.EKSNodeRole.name}\"\n}\n\nresource \"aws_iam_role_policy_attachment\" \"eks-node-AmazonEKS_CNI_Policy\" {\n  policy_arn = \"arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy\"\n  role       = \"${aws_iam_role.EKSNodeRole.name}\"\n}\n\nresource \"aws_iam_role_policy_attachment\" \"eks-node-AmazonEC2ContainerRegistryReadOnly\" {\n  policy_arn = \"arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly\"\n  role       = \"${aws_iam_role.EKSNodeRole.name}\"\n}\n\nresource \"aws_iam_instance_profile\" \"eks-node-instance-profile\" {\n  name = \"terraform-eks-demo\"\n  role = \"${aws_iam_role.EKSNodeRole.name}\"\n}\n\n\n//Worker Node Security Group\n//This security group controls networking access to the Kubernetes worker nodes.\n\n\nresource \"aws_security_group\" \"eks-nodes-sg\" {\n  name  =  \"${local.cluster_name}-nodes-sg\"\n  description = \"Security group for all nodes in the cluster [${local.env}] \"\n  vpc_id = \"${aws_vpc.cluster.id}\"\n  //    ingress {\n  //      from_port       = 0\n  //      to_port         = 0\n  //      protocol        = \"-1\"\n  //      description = \"allow nodes to communicate with each other\"\n  //      self = true\n  //    }\n\n  //    ingress {\n  //      from_port       = 1025\n  //      to_port         = 65535\n  //      protocol        = \"tcp\"\n  //      description = \"Allow worker Kubelets and pods to receive communication from the cluster control plane\"\n  //      security_groups = [\"${aws_security_group.eks-control-plane.id}\"]\n  //    }\n\n\n  egress {\n    from_port       = 0\n    to_port         = 0\n    protocol        = \"-1\"\n    cidr_blocks     = [\"0.0.0.0/0\"]\n  }\n\n  tags = \"${\n    map(\n     \"Name\", \"${local.cluster_name}-nodes-sg\",\n     \"kubernetes.io/cluster/${local.cluster_name}\", \"owned\",\n    )\n  }\"\n}\n\n\n//Worker Node Access to EKS Master Cluster\n//Now that we have a way to know where traffic from the worker nodes is coming from,\n//we can allow the worker nodes networking access to the EKS master cluster.\n\nresource \"aws_security_group_rule\" \"https_nodes_to_plane\" {\n  type                     = \"ingress\"\n  from_port                = 443\n  to_port                  = 443\n  protocol                 = \"tcp\"\n  security_group_id        = \"${aws_security_group.eks-control-plane-sg.id}\"\n  source_security_group_id = \"${aws_security_group.eks-nodes-sg.id}\"\n  depends_on = [\"aws_security_group.eks-nodes-sg\", \"aws_security_group.eks-control-plane-sg\" ]\n}\n\n\nresource \"aws_security_group_rule\" \"communication_plane_to_nodes\" {\n  type                     = \"ingress\"\n  from_port                = 1025\n  to_port                  = 65534\n  protocol                 = \"tcp\"\n  security_group_id = \"${aws_security_group.eks-nodes-sg.id}\"\n  source_security_group_id        = \"${aws_security_group.eks-control-plane-sg.id}\"\n  depends_on = [\"aws_security_group.eks-nodes-sg\", \"aws_security_group.eks-control-plane-sg\" ]\n}\n\nresource \"aws_security_group_rule\" \"nodes_internode_communications\" {\n  type = \"ingress\"\n  from_port       = 0\n  to_port         = 0\n  protocol        = \"-1\"\n  description = \"allow nodes to communicate with each other\"\n  security_group_id = \"${aws_security_group.eks-nodes-sg.id}\"\n  self = true\n}\n\n```\n\n### Step 5: on that step we are creating autoscaling group\n\n\n```tf\n//Worker Node AutoScaling Group\n//Now we have everything in place to create and manage EC2 instances that will serve as our worker nodes\n//in the Kubernetes cluster. This setup utilizes an EC2 AutoScaling Group (ASG) rather than manually working with\n//EC2 instances. This offers flexibility to scale up and down the worker nodes on demand when used in conjunction\n//with AutoScaling policies (not implemented here).\n//\n//First, let us create a data source to fetch the latest Amazon Machine Image (AMI) that Amazon provides with an\n//EKS compatible Kubernetes baked in.\n\ndata \"aws_ami\" \"eks-worker\" {\n  filter {\n    name   = \"name\"\n    values = [\"eks-worker-*\"]\n  }\n\n  most_recent = true\n  owners      = [\"602401143452\"] # Amazon Account ID\n}\n\n# EKS currently documents this required userdata for EKS worker nodes to\n# properly configure Kubernetes applications on the EC2 instance.\n# We utilize a Terraform local here to simplify Base64 encoding this\n# information into the AutoScaling Launch Configuration.\n# More information: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/amazon-eks-nodegroup.yaml\nlocals {\n  eks-node-userdata = \u003c\u003cUSERDATA\n#!/bin/bash -xe\n\nCA_CERTIFICATE_DIRECTORY=/etc/kubernetes/pki\nCA_CERTIFICATE_FILE_PATH=$CA_CERTIFICATE_DIRECTORY/ca.crt\nmkdir -p $CA_CERTIFICATE_DIRECTORY\necho \"${aws_eks_cluster.eks-cluster.certificate_authority.0.data}\" | base64 -d \u003e  $CA_CERTIFICATE_FILE_PATH\nINTERNAL_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)\nsed -i s,MASTER_ENDPOINT,${aws_eks_cluster.eks-cluster.endpoint},g /var/lib/kubelet/kubeconfig\nsed -i s,CLUSTER_NAME,${local.cluster_name},g /var/lib/kubelet/kubeconfig\nsed -i s,REGION,${var.AWS_REGION},g /etc/systemd/system/kubelet.service\nsed -i s,MAX_PODS,20,g /etc/systemd/system/kubelet.service\nsed -i s,MASTER_ENDPOINT,${aws_eks_cluster.eks-cluster.endpoint},g /etc/systemd/system/kubelet.service\nsed -i s,INTERNAL_IP,$INTERNAL_IP,g /etc/systemd/system/kubelet.service\nDNS_CLUSTER_IP=10.100.0.10\nif [[ $INTERNAL_IP == 10.* ]] ; then DNS_CLUSTER_IP=172.20.0.10; fi\nsed -i s,DNS_CLUSTER_IP,$DNS_CLUSTER_IP,g /etc/systemd/system/kubelet.service\nsed -i s,CERTIFICATE_AUTHORITY_FILE,$CA_CERTIFICATE_FILE_PATH,g /var/lib/kubelet/kubeconfig\nsed -i s,CLIENT_CA_FILE,$CA_CERTIFICATE_FILE_PATH,g  /etc/systemd/system/kubelet.service\nsystemctl daemon-reload\nsystemctl restart kubelet\nUSERDATA\n}\n\nresource \"aws_launch_configuration\" \"eks-launch-configuration\" {\n  associate_public_ip_address = true\n  iam_instance_profile        = \"${aws_iam_instance_profile.eks-node-instance-profile.name}\"\n  image_id                    = \"${data.aws_ami.eks-worker.id}\"\n  instance_type               = \"t2.small\"\n  name_prefix                 = \"eks-${local.cluster_name}\"\n  security_groups             = [\"${aws_security_group.eks-nodes-sg.id}\"]\n  user_data_base64            = \"${base64encode(local.eks-node-userdata)}\"\n  key_name                    = \"${var.ec2_keyname}\"\n\n  lifecycle {\n    create_before_destroy = true\n  }\n}\n\n\n//Finally, we create an AutoScaling Group that actually launches EC2 instances based on the\n//AutoScaling Launch Configuration.\n\n//NOTE: The usage of the specific kubernetes.io/cluster/* resource tag below is required for EKS\n//and Kubernetes to discover and manage compute resources.\n\nresource \"aws_autoscaling_group\" \"eks-autoscaling-group\" {\n  desired_capacity     = 2\n  launch_configuration = \"${aws_launch_configuration.eks-launch-configuration.id}\"\n  max_size             = 2\n  min_size             = 1\n  name                 = \"eks-${local.cluster_name}\"\n  vpc_zone_identifier  = [\"${aws_subnet.eks-private.id}\", \"${aws_subnet.eks-private-2.id}\"]\n\n\n  tag {\n    key                 = \"Environment\"\n    value               = \"${local.env}\"\n    propagate_at_launch = true\n  }\n\n  tag {\n    key                 = \"Name\"\n    value               = \"eks-${local.cluster_name}\"\n    propagate_at_launch = true\n  }\n\n  tag {\n    key                 = \"kubernetes.io/cluster/${local.cluster_name}\"\n    value               = \"owned\"\n    propagate_at_launch = true\n  }\n}\n\n//NOTE: At this point, your Kubernetes cluster will have running masters and worker nodes, however, the worker nodes will\n//not be able to join the Kubernetes cluster quite yet. The next section has the required Kubernetes configuration to\n//enable the worker nodes to join the cluster.\n\n\n//Required Kubernetes Configuration to Join Worker Nodes\n//The EKS service does not provide a cluster-level API parameter or resource to automatically configure the underlying\n//Kubernetes cluster to allow worker nodes to join the cluster via AWS IAM role authentication.\n\n\n//To output an IAM Role authentication ConfigMap from your Terraform configuration:\n\nlocals {\n  config-map-aws-auth = \u003c\u003cCONFIGMAPAWSAUTH\n\n\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: aws-auth\n  namespace: kube-system\ndata:\n  mapRoles: |\n    - rolearn: ${aws_iam_role.EKSNodeRole.arn}\n      username: system:node:{{EC2PrivateDNSName}}\n      groups:\n        - system:bootstrappers\n        - system:nodes\nCONFIGMAPAWSAUTH\n}\n\noutput \"config-map-aws-auth\" {\n  value = \"${local.config-map-aws-auth}\"\n}\n\n\n//Run\n//\n//terraform output config-map-aws-auth and save the configuration into a file,\n//e.g. config-map-aws-auth.yaml\n//\n//Run kubectl apply -f config-map-aws-auth.yaml\n//\n//You can verify the worker nodes are joining the cluster via: kubectl get nodes --watch\n//At this point, you should be able to utilize Kubernetes as expected!\n//\n\n```\n\nOn that step, you should also have fully working EKS cluster.\nUpon `terraform apply` run you should see smth like\n\n```\nApply complete! Resources: 36 added, 0 changed, 0 destroyed.\n\nOutputs:\n```\n\nTo make cluster fully operational, after provision you should:\n\n1) get cluster kubelet config via\n\n```sh\nmake kubelet-config\n\nterraform output kubeconfig \u003e kubeconfig\nexport KUBECONFIG=/home/slavko/kubernetes/tf/kubeconfig\n```\n\nafter that you should be able to execute commands over your eks cluster\n\n```sh\nkubectl get nodes\nNo resources found.\n```\n\n2) to allow nodes to join, you need to provide cluster with additional config map\n\n```\nmake config-map-aws-auth\nterraform output config-map-aws-auth \u003e config-map-aws-auth.yaml\nkubectl apply -f config-map-aws-auth.yaml\nconfigmap \"aws-auth\" created\nkubectl get nodes --watch\nNAME                         STATUS     ROLES     AGE       VERSION\nip-10-11-1-21.ec2.internal   NotReady   \u003cnone\u003e    1s        v1.10.3\nip-10-11-3-220.ec2.internal   NotReady   \u003cnone\u003e    0s        v1.10.3\n...\nip-10-11-1-21.ec2.internal   Ready     \u003cnone\u003e    31s       v1.10.3\nip-10-11-3-220.ec2.internal   Ready     \u003cnone\u003e    31s       v1.10.3\n```\n\n3) You should be able to run pods and services\n\n```\nkubectl run  --image busybox pod-test --restart=Never\npod \"pod-test\" created\n┌[tf]\u003e\n└──➜ kubectl get pods\nNAME       READY     STATUS      RESTARTS   AGE\npod-test   0/1       Completed   0          8s\n```\n\n## Deploy Kubernetes UI dashboard\n\nas per  https://docs.aws.amazon.com/eks/latest/userguide/dashboard-tutorial.html\n\n###  Step 1: Deploy the Dashboard\nUse the following steps to deploy the Kubernetes dashboard, heapster, and the influxdb backend for CPU and memory metrics to your cluster.\n\nTo deploy the Kubernetes dashboard\n\nDeploy the Kubernetes dashboard to your cluster:\n```\nkubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml\n\nsecret \"kubernetes-dashboard-certs\" created\nserviceaccount \"kubernetes-dashboard\" created\nrole \"kubernetes-dashboard-minimal\" created\nrolebinding \"kubernetes-dashboard-minimal\" created\ndeployment \"kubernetes-dashboard\" created\nservice \"kubernetes-dashboard\" created\n```\n\nDeploy heapster to enable container cluster monitoring and performance analysis on your cluster:\n\n```\nkubectl apply -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/heapster.yaml\n\nserviceaccount \"heapster\" created\ndeployment \"heapster\" created\nservice \"heapster\" created\n```\n\nDeploy the influxdb backend for heapster to your cluster:\n\n```\nkubectl apply -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/influxdb.yaml\n\ndeployment \"monitoring-influxdb\" created\nservice \"monitoring-influxdb\" created\n```\n\nCreate the heapster cluster role binding for the dashboard:\n\n```\nkubectl apply -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/rbac/heapster-rbac.yaml\n\nclusterrolebinding \"heapster\" created\n```\n\n### Step 2: Create an eks-admin Service Account and Cluster Role Binding\n\n\nBy default, the Kubernetes dashboard user has limited permissions. In this section, you create an eks-admin service account and cluster role binding that you can use to securely connect to the dashboard with admin-level permissions. For more information, see Managing Service Accounts in the Kubernetes documentation.\n\nTo create the eks-admin service account and cluster role binding\n\nImportant\n\nThe example service account created with this procedure has full cluster-admin (superuser) privileges on the cluster. For more information, see Using RBAC Authorization in the Kubernetes documentation.\n\nCreate a file called eks-admin-service-account.yaml with the text below. This manifest defines a service account and cluster role binding called eks-admin.\n\n```\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n  name: eks-admin\n  namespace: kube-system\n---\napiVersion: rbac.authorization.k8s.io/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n  name: eks-admin\nroleRef:\n  apiGroup: rbac.authorization.k8s.io\n  kind: ClusterRole\n  name: cluster-admin\nsubjects:\n- kind: ServiceAccount\n  name: eks-admin\n  namespace: kube-system\n```\n\nApply the service account and cluster role binding to your cluster:\n\n```\nkubectl apply -f eks-admin-service-account.yaml\nOutput:\n\nserviceaccount \"eks-admin\" created\nclusterrolebinding.rbac.authorization.k8s.io \"eks-admin\" created\n```\n\n\n### Step 3: Connect to the Dashboard\nNow that the Kubernetes dashboard is deployed to your cluster, and you have an administrator service account that you can use to view and control your cluster, you can connect to the dashboard with that service account.\n\nTo connect to the Kubernetes dashboard\n\nRetrieve an authentication token for the eks-admin service account. Copy the \u003cauthentication_token\u003e value from the output. You use this token to connect to the dashboard.\n\n```\nkubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep eks-admin | awk '{print $1}')\nOutput:\n\nName:         eks-admin-token-b5zv4\nNamespace:    kube-system\nLabels:       \u003cnone\u003e\nAnnotations:  kubernetes.io/service-account.name=eks-admin\n              kubernetes.io/service-account.uid=bcfe66ac-39be-11e8-97e8-026dce96b6e8\n\nType:  kubernetes.io/service-account-token\n\nData\n====\nca.crt:     1025 bytes\nnamespace:  11 bytes\ntoken:      \u003cauthentication_token\u003e\n```\n\nStart the kubectl proxy.\n\n```\nkubectl proxy\n```\n\nOpen the following link with a web browser to access the dashboard endpoint: http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login\n\nChoose Token, paste the \u003cauthentication_token\u003e output from the previous command into the Token field, and choose SIGN IN.\n\n## Step 4 (optional) kube2iam\n\nCheck this nice cluster addition: https://github.com/jtblin/kube2iam\n\nBelow goes extract from the Readme\n\nKube2iam allows you to use IAM roles to give individual pods access to your other AWS resources. Without some way to delegate IAM access to pods, you would instead have to give your worker nodes every IAM permission that your pods need, which is cumbersome to manage and a poor security practice.\n\nFirst, set up RBAC for the kube2iam service account:\n\n```yaml\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n  name: kube2iam\n  namespace: kube-system\n---\napiVersion: v1\nitems:\n  - apiVersion: rbac.authorization.k8s.io/v1\n    kind: ClusterRole\n    metadata:\n      name: kube2iam\n    rules:\n      - apiGroups: [\"\"]\n        resources: [\"namespaces\",\"pods\"]\n        verbs: [\"get\",\"watch\",\"list\"]\n  - apiVersion: rbac.authorization.k8s.io/v1\n    kind: ClusterRoleBinding\n    metadata:\n      name: kube2iam\n    subjects:\n    - kind: ServiceAccount\n      name: kube2iam\n      namespace: kube-system\n    roleRef:\n      kind: ClusterRole\n      name: kube2iam\n      apiGroup: rbac.authorization.k8s.io\nkind: List\n```\n\nThen, install the kube2iam DaemonSet. The kube2iam agent will run on each worker node and intercept calls to the EC2 metadata API.  If your pods are annotated correctly, kube2iam will assume the role specified by your pod to authenticate the request, allowing your pods to access AWS resources using roles, and requiring no change to your application code. The only option we have to pay attention to specifically for EKS is to set the --host-interface option to eni+.\n\nKube2iam has many configuration options that are documented on the GitHub repo, but this manifest will get you started:\n\n```yaml\napiVersion: apps/v1\nkind: DaemonSet\nmetadata:\n  name: kube2iam\n  namespace: kube-system\n  labels:\n    app: kube2iam\nspec:\n  updateStrategy:\n    type: RollingUpdate\n  selector:\n    matchLabels:\n      name: kube2iam\n  template:\n    metadata:\n      labels:\n        name: kube2iam\n    spec:\n      serviceAccountName: kube2iam\n      hostNetwork: true\n      containers:\n        - image: jtblin/kube2iam:0.10.4\n          imagePullPolicy: Always\n          name: kube2iam\n          args:\n            - \"--app-port=8181\"\n            - \"--auto-discover-base-arn\"\n            - \"--iptables=true\"\n            - \"--host-ip=$(HOST_IP)\"\n            - \"--host-interface=eni+\"\n            - \"--auto-discover-default-role\"\n            - \"--log-level=info\"\n          env:\n            - name: HOST_IP\n              valueFrom:\n                fieldRef:\n                  fieldPath: status.podIP\n          ports:\n            - containerPort: 8181\n              hostPort: 8181\n              name: http\n          securityContext:\n            privileged: true\n```\nOnce kube2iam is set up, you can add an annotation in the pod spec of your deployments or other pod controllers to specify which IAM role should be used for the pod like so:\n\n```yaml\n      annotations:\n        iam.amazonaws.com/role: my-role\n```\nMake sure that the roles you are using with kube2iam have been configured so that your worker nodes can assume those roles. This is why we output the worker_role_arn from the Terraform EKS module in the last step. Modify your pod roles so they can be assumed by your worker nodes by adding the following to the Trust Relationship for each role:\n\n```json\n    {\n      \"Sid\": \"\",\n      \"Effect\": \"Allow\",\n      \"Principal\": {\n        \"AWS\": \"WORKER_ROLE_ARN\"\n      },\n      \"Action\": \"sts:AssumeRole\"\n    }\n```\nBe sure to replace WORKER_ROLE_ARN with the ARN of the IAM Role that your EKS worker nodes are configured with, not the ARN of the Instance Profile.\n\n\n## Questions?\n\n\n### Troubleshouting ...\n\nNow lets start smth\n\n```\nkubectl run -i --tty --image busybox dns-test --restart=Never --rm /bin/sh\n```\n\nif you see state pending\n\n```\nNAME       READY     STATUS    RESTARTS   AGE\ndns-test   0/1       Pending   0          1m\n\n```\n\nyou might want immediately check\n\n```sh\nkubectl get events\n```\n\nyou might see smth like\n\n```\nLAST SEEN   FIRST SEEN   COUNT     NAME                        KIND      SUBOBJECT   TYPE      REASON             SOURCE              MESSAGE\n34s         5m           22        dns-test.15404a7104cc67cf   Pod                   Warning   FailedScheduling   default-scheduler   no nodes available to schedule pods\n```\n\nwhere in message you will get hint about the issue\n\nTroubleshouting hints from AWS can be found on\n\nhttps://docs.aws.amazon.com/eks/latest/userguide/troubleshooting.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoronenko%2Fdevops-tf-eks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvoronenko%2Fdevops-tf-eks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvoronenko%2Fdevops-tf-eks/lists"}