{"id":13449021,"url":"https://github.com/kz8s/tack","last_synced_at":"2026-01-17T21:32:32.578Z","repository":{"id":39618213,"uuid":"51404225","full_name":"kz8s/tack","owner":"kz8s","description":"Terraform module for creating Kubernetes cluster running on Container Linux by CoreOS in an AWS VPC","archived":false,"fork":false,"pushed_at":"2018-02-27T16:46:21.000Z","size":2754,"stargazers_count":719,"open_issues_count":40,"forks_count":145,"subscribers_count":40,"default_branch":"master","last_synced_at":"2024-10-28T15:42:27.343Z","etag":null,"topics":["aws-terraform","aws-vpc","coreos","coreos-cluster","docker","etcd-cluster","infrastructure-as-code","kubernetes","kubernetes-coreos-terraform"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kz8s.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-02-09T22:14:00.000Z","updated_at":"2024-07-09T09:44:31.000Z","dependencies_parsed_at":"2022-09-15T13:51:49.586Z","dependency_job_id":null,"html_url":"https://github.com/kz8s/tack","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kz8s%2Ftack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kz8s%2Ftack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kz8s%2Ftack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kz8s%2Ftack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kz8s","download_url":"https://codeload.github.com/kz8s/tack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245002980,"owners_count":20545530,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-terraform","aws-vpc","coreos","coreos-cluster","docker","etcd-cluster","infrastructure-as-code","kubernetes","kubernetes-coreos-terraform"],"created_at":"2024-07-31T06:00:28.731Z","updated_at":"2026-01-17T21:32:32.542Z","avatar_url":"https://github.com/kz8s.png","language":"HCL","readme":"# terraform-aws-coreos-kubernetes\n\n[![Circle CI](https://circleci.com/gh/kz8s/tack.svg?style=svg)](https://circleci.com/gh/kz8s/tack)\n\nOpinionated [Terraform](https://terraform.io) module for creating a Highly Available [Kubernetes](http://kubernetes.io) cluster running on\n[Container Linux by CoreOS](https://coreos.com) (any channel) in an [AWS\nVirtual Private Cloud VPC](https://aws.amazon.com/vpc/). With prerequisites\ninstalled `make all` will simply spin up a default cluster; and, since it is\nbased on Terraform, customization is much easier than\n[CloudFormation](https://aws.amazon.com/cloudformation/).\n\nThe default configuration includes Kubernetes\n[add-ons](https://github.com/kubernetes/kubernetes/tree/master/cluster/addons):\nDNS, Dashboard and UI.\n\n## tl;dr\n```bash\n# prereqs\n$ brew update \u0026\u0026 brew install awscli cfssl jq kubernetes-cli terraform\n\n# build artifacts and deploy cluster\n$ make all\n\n# nodes\n$ kubectl get nodes\n\n# addons\n$ kubectl get pods --namespace=kube-system\n\n# verify dns - run after addons have fully loaded\n$ kubectl exec busybox -- nslookup kubernetes\n\n# open dashboard\n$ make dashboard\n\n# obliterate the cluster and all artifacts\n$ make clean\n```\n\n## Component and Tool Versions\n\ncomponent / tool | version\n---------------- | -------:\nContainer Linux by CoreOS | 1409.7.0, 1465.3.0, 1492.1.0\nkubernetes                | 1.7.4\nflanneld                  | 0.7.1\ndocker                    | 1.12.6\netcd                      | 3.1.6\nrkt                       | 1.25.0\nterraform                 | 0.10.0\ncfssl                     | 1.2.0\naws-cli                   | aws-cli/1.11.129 Python/2.7.10 Darwin/16.7.0 botocore/1.5.92\njq                        | 1.5\n\n\n## Features\n\n* Cluster-internal Certificate Authority infrastructure for TLS certificate generation\n* etcd3\n\n### AWS\n\n* [EC2 Key Pair](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html)\ncreation\n* AWS VPC Public and Private subnets\n* IAM protected S3 bucket for asset distribution\n* Bastion Host\n* Multi-AZ Auto-Scaling Worker Nodes\n* VPC NAT Gateway\n* VPC Endpoint for simplified S3 access from EC2 instances\n\n### Container Linux by CoreOS\n\n* etcd3 DNS Discovery Bootstrap\n* kubelet runs under rkt (using Container Linux by CoreOS recommended [Kubelet Wrapper Script](https://coreos.com/kubernetes/docs/latest/kubelet-wrapper.html))\n\n### Kubernetes\n\n* [Highly Available ApiServer Configuration](https://kubernetes.io/docs/admin/high-availability/)\n* Service accounts enabled\n\n### Terraform\n\n* Container Linux by CoreOS AMI sourcing\n* Terraform Pattern Modules\n\n## Prerequisites\n\n* [AWS Command Line Interface](http://aws.amazon.com/documentation/cli/)\n* [CFSSL: CloudFlare's PKI and TLS toolkit](https://cfssl.org/)\n* [jq](https://stedolan.github.io/jq/)\n* [kubectl](http://kubernetes.io/v1.1/docs/user-guide/kubectl-overview.html)\n* [Terraform](https://www.terraform.io/)\n\nQuick install prerequisites on Mac OS X with [Homebrew](http://brew.sh/):\n\n```bash\n$ brew update \u0026\u0026 brew install awscli cfssl jq kubernetes-cli terraform\n```\n\n## Launch Cluster\n\n`make all` will create:\n- AWS Key Pair (PEM file)\n- AWS VPC with private and public subnets\n- Route 53 internal zone for VPC\n- Bastion host\n- Certificate Authority server\n- etcd3 cluster bootstrapped from Route 53\n- High Availability Kubernetes configuration (masters running on etcd nodes)\n- Autoscaling worker node group across subnets in selected region\n- kube-system namespace and addons: DNS, UI, Dashboard\n\n```bash\nmake all\n```\n\nTo open dashboard:\n\n```bash\nmake dashboard\n```\n\nTo display instance information:\n\n```bash\nmake instances\n```\n\nTo display status:\n\n```bash\nmake status\n```\n\nTo destroy, remove and generally undo everything:\n\n```\nmake clean\n```\n\n`make all` and `make clean` should be idempotent - should an error occur simply try running\nthe command again and things should recover from that point.\n\n## How Tack works\n\n### Tack Phases\n\nTack works in three phases:\n\n1. Pre-Terraform\n2. Terraform\n3. Post-Terraform\n\n#### Pre-Terraform\n\nThe purpose of this phase is to prep the environment for Terraform execution. Some tasks are\nhard or messy to do in Terraform - a little prep work can go a long way here. Determining\nthe Container Linux by CoreOS AMI for a given region, channel and VM Type for instance is easy enough to do\nwith a simple shell script.\n\n#### Terraform\n\nTerraform does the heavy lifting of resource creation and sequencing. Tack uses local\nmodules to partition the work in a logical way. Although it is of course possible to do all\nof the Terraform work in a single `.tf` file or collection of `.tf` files, it becomes\nunwieldy quickly and impossible to debug. Breaking the work into local modules makes the\nflow much easier to follow and provides the basis for composing variable solutions down the track - for example converting the worker Auto Scaling Group to use spot instances.\n\n#### Post-Terraform\n\nOnce the infrastructure has been configured and instantiated it will take some time for it\nto settle. Waiting for the 'master' ELB to become healthy is an example of this.  \n\n### Components\n\nLike many great tools, _tack_ has started out as a collection of scripts, makefiles and other tools. As _tack_ matures and patterns crystalize it will evolve to a Terraform plugin and perhaps a Go-based cli tool for 'init-ing' new cluster configurations. The tooling will compose Terraform modules into a solution based on user preferences - think `npm init` or better yet [yeoman](http://yeoman.io/).\n\n#### TLS Certificates\n\n* [etcd3 coreos cloudint](https://github.com/coreos/coreos-cloudinit/blob/master/config/etcd.go)\n\n```bash\ncurl --cacert /etc/kubernetes/ssl/ca.pem --cert /etc/kubernetes/ssl/k8s-etcd.pem --key /etc/kubernetes/ssl/k8s-etcd-key.pem https://etcd.test.kz8s:2379/health\nopenssl x509 -text -noout -in /etc/kubernetes/ssl/ca.pem\nopenssl x509 -text -noout -in /etc/kubernetes/ssl/k8s-etcd.pem\n```\n\n#### ElasticSearch and Kibana\n\nTo access Elasticseach and Kibana first start `kubectl proxy`.\n\n```bash\n$ kubectl proxy\nStarting to serve on localhost:8001\n```\n\n* [http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging ](http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/elasticsearch-logging)\n* [http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kibana-logging/app/kibana](http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kibana-logging/app/kibana)\n\n## FAQs\n\n* [Create an etcd cluster with more than 3 instances](https://github.com/kz8s/tack/wiki/How-to:-change-etcd-cluster-size)\n\n## Advanced Features and Configuration\n\n### Using an Existing VPC\n\nIf you have an existing VPC you'd like to deploy a cluster into, there is an option for this with _tack_.\n\n#### Constraints\n\n* You will need to allocate 3 static IPs for the etcd servers - Choose 3 unused IPs that fall within the IP range of the first subnet specified in `subnet-ids-private` under `vpc-existing.tfvars`\n* Your VPC has to have private and public subnets (for now)\n* You will need to know the following information:\n  * VPC CIDR Range (e.g. 192.168.0.0/16)\n  * VPC Id (e.g. vpc-abc123)\n  * VPC Internet Gateway Id (e.g. igw-123bbd)\n  * VPC Public Subnet Ids (e.g. subnet-xyz123,subnet-zyx123)\n  * VPC Private Subnet Ids (e.g. subnet-lmn123,subnet-opq123)\n\n#### Enabling Existing VPC Support\n\n* Edit vpc-existing.tfvars\n  * Uncomment the blocks with variables and fill in the missing information\n* Edit modules_override.tf - This uses the [overrides feature from Terraform](https://www.terraform.io/docs/configuration/override.html)\n  * Uncomment the vpc module, this will override the reference to the regular VPC module and instead use the stub vpc-existing module which just pulls in the variables from vpc-existing.tfvars\n* Edit the Makefile as necessary for CIDR_PODS, CIDR_SERVICE_CLUSTER, etc to match what you need (e.g. avoid collisions with existing IP ranges in your VPC or extended infrastructure)\n\n#### Testing Existing VPC Support from Scratch\n\nIn order to test existing VPC support, we need to generate a VPC and then try the overrides with it. After that we can clean it all up.  These instructions are meant for someone wanting to ensure that the _tack_ existing VPC code works properly.\n* Run `make all` to generate a VPC with Terraform\n* Edit terraform.tfstate\n  * Search for the VPC block and cut it out and save it somewhere.  Look for \"path\": [\"root\",\"vpc\"]\n* Run `make clean` to remove everything but the VPC and associated networking (we preserved it in the previous step)\n* Edit as per instructions above\n* Run `make all` to test out using an existing VPC\n* Cleaning up:\n  * Re-insert the VPC block into terraform.tfstate\n  * Run `make clean` to clean up everything\n\n#### Additional Configuration\n\n* You should to [tag your subnets](https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws/aws.go#66) for internal/external load balancers\n\n## Inspiration\n\n* [Code examples to create Container Linux by CoreOS cluster on AWS with Terraform](https://github.com/xuwang/aws-terraform) by [xuwang](https://github.com/xuwang)\n* [kaws: tool for deploying multiple Kubernetes clusters](https://github.com/InQuicker/kaws)\n* [Kubernetes on Container Linux by CoreOS](https://github.com/coreos/coreos-kubernetes)\n* [Terraform Infrastructure Design Patterns](https://www.opencredo.com/2015/09/14/terraform-infrastructure-design-patterns/) by [Bart Spaans](https://www.opencredo.com/author/bart/)\n* [The infrastructure that runs Brandform](https://github.com/brandfolder/infrastructure)\n* [AutoScaling your Kubernetes cluster on AWS](https://renzedevries.wordpress.com/2017/01/10/autoscaling-your-kubernetes-cluster-on-aws/)\n* [Bash template substitution for manifests - from kayrus/elk-kubernetes](https://github.com/kayrus/elk-kubernetes/blob/master/deploy.sh)\n\n## Other Terraform Projects\n\n* [bakins/kubernetes-coreos-terraform](https://github.com/bakins/kubernetes-coreos-terraform)\n* [bobtfish/terraform-aws-coreos-kubernates-cluster](https://github.com/bobtfish/terraform-aws-coreos-kubernates-cluster)\n* [chiefy/tf-aws-kubernetes](https://github.com/chiefy/tf-aws-kubernetes)\n* [cihangir/terraform-aws-kubernetes](https://github.com/cihangir/terraform-aws-kubernetes)\n* [ericandrewlewis/kubernetes-via-terraform](https://github.com/ericandrewlewis/kubernetes-via-terraform)\n* [funkymonkeymonk/terraform-demo](https://github.com/funkymonkeymonk/terraform-demo)\n* [kelseyhightower/kubestack](https://github.com/kelseyhightower/kubestack)\n* [samsung-cnct/kraken](https://github.com/samsung-cnct/kraken)\n* [wearemakery/kubestack-aws](https://github.com/wearemakery/kubestack-aws)\n* [xuwang/aws-terraform](https://github.com/xuwang/aws-terraform)\n\n## References\n\n* [CFSSL: CloudFlare's PKI and TLS toolkit](https://cfssl.org/)\n* [Container Linux by CoreOS - Mounting Storage](https://coreos.com/os/docs/latest/mounting-storage.html)\n* [Deploying Container Linux by CoreOS cluster with etcd secured by TLS/SSL](http://blog.skrobul.com/securing_etcd_with_tls/)\n* [etcd dns discovery bootstrap](https://coreos.com/etcd/docs/latest/clustering.html#dns-discovery)\n* [Generate EC2 Key Pair](https://github.com/xuwang/aws-terraform/blob/master/scripts/aws-keypair.sh)\n* [Generate self-signed certificates](https://coreos.com/os/docs/latest/generate-self-signed-certificates.html)\n* [kubectl Cheat Sheet](https://kubernetes.io/docs/user-guide/kubectl-cheatsheet/)\n* [Makefile `help` target](https://gist.github.com/rcmachado/af3db315e31383502660)\n* [Peeking under the hood of Kubernetes on AWS](https://github.com/kubernetes/kubernetes/blob/release-1.2/docs/design/aws_under_the_hood.md)\n* [Self documenting Makefile](https://gist.github.com/prwhite/8168133)\n* [Setting up etcd to run in production](https://github.com/kelseyhightower/etcd-production-setup)\n* [ssl artifact generation](https://github.com/coreos/coreos-kubernetes/tree/master/lib)\n* [Cluster Autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler)\n* [Persistent Storage - Kubernetes on AWS](http://kubernetes-on-aws.readthedocs.io/en/latest/user-guide/using-volumes.html)\n* [VPC endpoint Terraform example setup](https://gist.github.com/radeksimko/929a41675323eefed023)\n","funding_links":[],"categories":["HCL","Roadmap","kubernetes","Featured On"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkz8s%2Ftack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkz8s%2Ftack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkz8s%2Ftack/lists"}