{"id":23229328,"url":"https://github.com/rootsami/terraform-rancher2","last_synced_at":"2025-04-05T18:30:39.579Z","repository":{"id":51709986,"uuid":"266192604","full_name":"rootsami/terraform-rancher2","owner":"rootsami","description":"Terraform manifests to create e2e production grade k8s cluster","archived":false,"fork":false,"pushed_at":"2021-05-10T15:36:54.000Z","size":238,"stargazers_count":20,"open_issues_count":0,"forks_count":10,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-02-11T15:55:46.803Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rootsami.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-05-22T19:33:49.000Z","updated_at":"2024-03-03T19:07:03.000Z","dependencies_parsed_at":"2022-08-03T09:00:29.951Z","dependency_job_id":null,"html_url":"https://github.com/rootsami/terraform-rancher2","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rootsami%2Fterraform-rancher2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rootsami%2Fterraform-rancher2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rootsami%2Fterraform-rancher2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rootsami%2Fterraform-rancher2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rootsami","download_url":"https://codeload.github.com/rootsami/terraform-rancher2/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247383475,"owners_count":20930288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-19T01:17:54.918Z","updated_at":"2025-04-05T18:30:39.560Z","avatar_url":"https://github.com/rootsami.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# terraform-rancher2\nTerraform manifests to create e2e production-grade Kubernetes cluster on top of cloud providers\n\n## Overview\n\nThis repo is intended to be for creating complete infrastructure pieces on OpenStack that are needed to have a fully provisioned Kubernetes cluster using Terraform and Rancher2. In addition to integration with [cloud-provider-openstack](https://github.com/kubernetes/cloud-provider-openstack)\n\n## Getting started with Infrastructure\n* Clone the repository [terraform-rancher2](https://github.com/rootsami/terraform-rancher2) into a folder.\n* Go into the openstack folder using `cd openstack/`\n* Modify the variables in `terraform.tfvars` to match your current cloud environment. it is important to uncomment the vars `openstack_project` , `openstack_username` and `openstack_password` or export them as env variables with prefix TF_VAR_*  for example: \n```bash\nexport TF_VAR_openstack_username=myusername\nexport TF_VAR_openstack_password=mypassword\nexport TF_VAR_openstack_project=myproject\n```\n* Other variables can be obtained from openstack-cli such as `rancher_node_image_id` , `external_network` by invoking\n```bash\n## image list\nopenstack image list\n## network name\nopenstack network list --external\n## flavors\nopenstack flavor list\n```\n* Run `terraform init` to initialize a working directory containing Terraform configuration files.\n* To apply the creation of the environment, Run `terraform apply --auto-approve` and wait for the output after all resources finish the creation\n```bash\nApply complete! Resources: 25 added, 0 changed, 0 destroyed.\n\nOutputs:\n\nrancher_url = [\n  \"https://xx.xx.xx.xx/\",\n]\n```\n\nUp to this point, use the `rancher_url` from above output and login to rancher instance with username `admin` and password defined in `rancher_admin_password`. Wait for all kubernetes nodes to be discovered, registered, and active.\n\n## Integration with [cloud-provider-openstack](https://github.com/kubernetes/cloud-provider-openstack)\nAs you may notice, that all the nodes have a taint `node.cloudprovider.kubernetes.io/uninitialized`. The usage of `--cloud-provider=external` flag to the kubelet makes it waiting for the clouder-provider to start the initialization.\n* Edit the file `manifests/cloud.conf` with the access information to your openstack environment.\n* Create a secret containing the cloud configuration in the kube-system namespace \n```bash\nkubectl create secret -n kube-system generic cloud-config --from-file=manifests/cloud.conf\n```\n* Create RBAC resources and openstack-cloud-controller-manager deamonset and wait for all the pods in kube-system namespace up and running.\n```bash\nkubectl apply -f manifests/cloud-controller-manager-roles.yaml\nkubectl apply -f manifests/cloud-controller-manager-role-bindings.yaml\nkubectl apply -f manifests/openstack-cloud-controller-manager-ds.yaml\n```\n* Create [cinder-csi-plugin](https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-cinder-csi-plugin.md) which are a set of cluster roles, cluster role bindings, statefulsets, and storageClass to communicate with openstack(cinder).\n```bash\nkubectl apply -f manifests/cinder-csi-plugin.yaml\n```\n\nUp to this point, openstack-cloud-controller-manager and cinder-csi-plugin have been deployed, and they're able to obtain valuable information such as External IP addresses and Zone info.\n\n```bash\n$ kubectl get nodes -o wide\n\nNAME            STATUS   ROLES               AGE     VERSION   INTERNAL-IP     EXTERNAL-IP      OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME\ndemo-master-1   Ready    controlplane,etcd   5h      v1.17.5   192.168.201.6   xx.xx.xx.xx      Ubuntu 18.04.2 LTS   4.15.0-45-generic   docker://19.3.9\ndemo-worker-1   Ready    worker              4h57m   v1.17.5   192.168.201.4   xx.xx.xx.xx      Ubuntu 18.04.2 LTS   4.15.0-45-generic   docker://19.3.9\ndemo-worker-2   Ready    worker              4h56m   v1.17.5   192.168.201.5   xx.xx.xx.xx      Ubuntu 18.04.2 LTS   4.15.0-45-generic   docker://19.3.9\n```\n\n![](docs/cluster-overview.png)\n\n\n\nAlso, as shown in the nodes tab, All nodes are active and labeled by openstack zones.\n\n![](docs/node-details.png)\n\n## Scalability\nWhen it comes to scalability with IaaC (infrastructure-as-a-code), it becomes so easy to obtain any desired state in less consumed efforts and time.\nAll you have to do is to change the number of nodes `count_master` or `count_worker_nodes` and run `terraform apply` again\nFor example, let's increase the number of `count_worker_nodes` by 1\nA few minutes later, after refreshing states and applying updates:\n```bash\n\nApply complete! Resources: 3 added, 0 changed, 0 destroyed.\n\nOutputs:\n\nrancher_url = [\n  \"https://xx.xx.xx.xx\",\n]\n```\nCouple of minutes for the new node to be registered \n```bash\n$ kubectl get nodes -o wide\nNAME            STATUS   ROLES               AGE    VERSION   INTERNAL-IP     EXTERNAL-IP      OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME\ndemo-master-1   Ready    controlplane,etcd   28h    v1.17.5   192.168.201.6   xx.xx.xx.xx      Ubuntu 18.04.2 LTS   4.15.0-45-generic   docker://19.3.9\ndemo-worker-1   Ready    worker              28h    v1.17.5   192.168.201.4   xx.xx.xx.xx      Ubuntu 18.04.2 LTS   4.15.0-45-generic   docker://19.3.9\ndemo-worker-2   Ready    worker              28h    v1.17.5   192.168.201.5   xx.xx.xx.xx      Ubuntu 18.04.2 LTS   4.15.0-45-generic   docker://19.3.9\ndemo-worker-3   Ready    worker              2m2s   v1.17.5   192.168.201.7   xx.xx.xx.xx      Ubuntu 18.04.2 LTS   4.15.0-45-generic   docker://19.3.9\n```\n\n**NOTE: Scaling down the cluster could be made by decreasing the number of nodes in `terrafrom.tfvars`. Node gets deleted, moreover `cloud-provider-openstack` detects that and removes it from the cluster** \n\n## Cleanup\nTo clean up all resources created by this terraform, Just run `terraform destroy`\n\n## References\n* [Rancher2 Provider](https://www.terraform.io/docs/providers/rancher2/)\n* [Openstack Provider](https://www.terraform.io/docs/providers/openstack/)\n* [Cloud-provider-openstack](https://github.com/kubernetes/cloud-provider-openstack)\n* [Running-cloud-controller-manager](https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/#running-cloud-controller-manager)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frootsami%2Fterraform-rancher2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frootsami%2Fterraform-rancher2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frootsami%2Fterraform-rancher2/lists"}