{"id":18458190,"url":"https://github.com/brightbox/kubernetes-cluster","last_synced_at":"2025-04-08T05:34:02.833Z","repository":{"id":33866438,"uuid":"137081073","full_name":"brightbox/kubernetes-cluster","owner":"brightbox","description":"Terraform manifests to create and update a kubernetes cluster on Brightbox Cloud","archived":false,"fork":false,"pushed_at":"2025-02-24T08:33:53.000Z","size":471,"stargazers_count":5,"open_issues_count":0,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-23T06:51:06.546Z","etag":null,"topics":["brightbox-cloud","kubernetes","kubernetes-cluster","terraform"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brightbox.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-12T14:09:40.000Z","updated_at":"2025-02-24T08:33:57.000Z","dependencies_parsed_at":"2023-09-27T13:15:25.854Z","dependency_job_id":"bdc3c8e2-69ee-4268-8900-471e436b8110","html_url":"https://github.com/brightbox/kubernetes-cluster","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brightbox%2Fkubernetes-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brightbox%2Fkubernetes-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brightbox%2Fkubernetes-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brightbox%2Fkubernetes-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brightbox","download_url":"https://codeload.github.com/brightbox/kubernetes-cluster/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247785919,"owners_count":20995641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["brightbox-cloud","kubernetes","kubernetes-cluster","terraform"],"created_at":"2024-11-06T08:17:12.096Z","updated_at":"2025-04-08T05:33:57.804Z","avatar_url":"https://github.com/brightbox.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kubernetes Cluster Builder\n\n## Getting started\nBuild a Kubernetes Cluster on Brightbox Cloud the easy way. [Read our step-by-step guide on deploying a cluster](https://www.brightbox.com/docs/guides/kubernetes/deploy-kubernetes-on-brightbox-cloud/) and start using Kubernetes today.\n\n## Installing kubectl on your workstation.\nThe master node has kubectl set up and ready for operation, but you may want to operate your cluster directly from your workstation\n\n- set the `management_source` variable to the appropriate CIDR that includes your workstation, and apply to the cluster with `terraform apply`. You can do this automaitcally on the command line with\n```\n$ terraform apply -var \"management_source=[\\\"$(curl -s ifconfig.co)/32\\\"]\"\n```\n- [install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) using a method suitable for your workstation.\n- Copy the cluster config from the master node\n```\n$ mkdir ${HOME}/.kube\n$ scp ubuntu@$(terraform output master):.kube/config ~/.kube/config\n$ sed -i \"s/https:.*$/https:\\/\\/$(terraform output master):6443/\" ~/.kube/config\n```\n- Check you can connect by running `kubectl cluster-info`\n\nThe `download-config.sh` script in the `scripts` directory will copy the cluster config into place for you.\n\n## Adding Storage to your cluster\n\nIs described [over here](examples/storage/README.md)\n\n## Running the examples\nIf you are using kubectl on the master node, copy the examples directory to the master node first\n```\nscp -r examples ubuntu@$(terrform output master):.\n```\n### pod-example.yaml\nA simple pod configuration that runs the busybox container\n\n- apply the pod\n```\nkubectl apply -f examples/pod-example.yaml\n```\n- list the pods created\n```\n$ kubectl get pods\nNAME      READY     STATUS    RESTARTS   AGE\nbusybox   1/1       Running   0          1m\n```\n- look at the details of the pod\n```\n$ kubectl describe pod/busybox\nName:               busybox\nNamespace:          default\nPriority:           0\nPriorityClassName:  \u003cnone\u003e\nNode:               srv-rmsqz/10.241.205.182\nStart Time:         Thu, 23 Aug 2018 12:05:07 +0100\nLabels:             \u003cnone\u003e\nAnnotations:        cni.projectcalico.org/podIP=192.168.1.6/32\n                    kubectl.kubernetes.io/last-applied-configuration={\"apiVersion\":\"v1\",\"kind\":\"Pod\",\"metadata\":{\"annotations\":{},\"name\":\"busybox\",\"namespace\":\"default\"},\"spec\":{\"containers\":[{\"command\":[\"sleep\",\"3600\"],...\nStatus:             Running\nIP:                 192.168.1.6\nContainers:\n  busybox:\n    Container ID:  docker://a050e1e6cb92c796fe061b19f0f77524316b0f8fb132fc020d3ffc653aa6e45e\n    Image:         busybox\n    Image ID:      docker-pullable://busybox@sha256:cb63aa0641a885f54de20f61d152187419e8f6b159ed11a251a09d115fdff9bd\n    Port:          \u003cnone\u003e\n    Host Port:     \u003cnone\u003e\n    Command:\n      sleep\n      3600\n    State:          Running\n      Started:      Thu, 23 Aug 2018 12:05:11 +0100\n    Ready:          True\n    Restart Count:  0\n    Environment:    \u003cnone\u003e\n    Mounts:\n      /var/run/secrets/kubernetes.io/serviceaccount from default-token-6vdg8 (ro)\nConditions:\n  Type              Status\n  Initialized       True\n  Ready             True\n  ContainersReady   True\n  PodScheduled      True\nVolumes:\n  default-token-6vdg8:\n    Type:        Secret (a volume populated by a Secret)\n    SecretName:  default-token-6vdg8\n    Optional:    false\nQoS Class:       BestEffort\nNode-Selectors:  \u003cnone\u003e\nTolerations:     node.kubernetes.io/not-ready:NoExecute for 300s\n                 node.kubernetes.io/unreachable:NoExecute for 300s\nEvents:\n  Type    Reason     Age   From                Message\n  ----    ------     ----  ----                -------\n  Normal  Scheduled  2m    default-scheduler   Successfully assigned default/busybox to srv-rmsqz\n  Normal  Pulling    2m    kubelet, srv-rmsqz  pulling image \"busybox\"\n  Normal  Pulled     2m    kubelet, srv-rmsqz  Successfully pulled image \"busybox\"\n  Normal  Created    2m    kubelet, srv-rmsqz  Created container\n  Normal  Started    2m    kubelet, srv-rmsqz  Started container\n```\n- The pod will expire itself after an hour, or you can delete it (delete will wait for the pod to exit)\n```\nkubectl delete -f examples/pod-example.yaml\npod \"busybox\" deleted\n```\n\n### loadbalancer-example.yaml\nThis runs up a simple http service via a Brightbox Loadbalancer and cloud IP.\n\n- apply the service\n```\n$ kubectl apply -f examples/loadbalancer-example.yaml\ndeployment.apps/hello-world created\nservice/example-service created\n```\n- wait until the load balancer service obtains a Cloud IP\n```\n$ kubectl get services\nNAME              TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)        AGE\nexample-service   LoadBalancer   172.30.38.64   109.107.39.75   80:31404/TCP   5m\nkubernetes        ClusterIP      172.30.0.1     \u003cnone\u003e          443/TCP        19m\n```\n- check the service works\n```\n$ curl 109.107.39.75; echo\nHello Kubernetes!\n```\n- and if you have IPv6, check access over IPv6 to your fully dual stacked service\n```\n$ kubectl get service/example-service -o yaml | grep 'hostname:'\n    - hostname: cip-109-107-39-75.gb1s.brightbox.com\n$ curl -v cip-109-107-39-75.gb1s.brightbox.com; echo\n* Rebuilt URL to: cip-109-107-39-75.gb1s.brightbox.com/\n*   Trying 2a02:1348:ffff:ffff::6d6b:274b...\n* TCP_NODELAY set\n* Connected to cip-109-107-39-75.gb1s.brightbox.com (2a02:1348:ffff:ffff::6d6b:274b) port 80 (#0)\n\u003e GET / HTTP/1.1\n\u003e Host: cip-109-107-39-75.gb1s.brightbox.com\n\u003e User-Agent: curl/7.58.0\n\u003e Accept: */*\n\u003e\n\u003c HTTP/1.1 200 OK\n\u003c Date: Thu, 23 Aug 2018 10:47:48 GMT\n\u003c Connection: keep-alive\n\u003c Transfer-Encoding: chunked\n\u003c\n* Connection #0 to host cip-109-107-39-75.gb1s.brightbox.com left intact\nHello Kubernetes!\n```\n- and finally remove the service\n```\nkubectl delete -f examples/loadbalancer-example.yml\n```\n### loadbalancer-annotation-example.yaml\nThis creates a TCP load balancer on Brightbox Cloud with a bespoke\nhttp healthcheck and 'round-robin' balancing policy by adding special\nannotations to the configuration.\n\nCreate, test and delete the example in the same way as the previous example.\n\n## Loadbalancer Source IP support\nBrightbox Cloud load balancers work in either `Cluster` mode or `Local` mode.\n\nIn `Local` mode the source address will always be the address of the\nBrightbox Cloud Load Balancer, with the source address of the client\ncontained in the `X-Forwarded-For` header.\n\nIn `Cluster` mode the source address may be another node in the cluster. The `X-Forwarded-For` header is still set to the source address of the end client.\n\nYou can see the different responses by following the [Source IP test instructions](https://kubernetes.io/docs/tutorials/services/source-ip/#source-ip-for-services-with-type-loadbalancer) on the main k8s documentation site.\n\nTCP loadbalancers obviously don't have the `X-Forwarded-For` header\nset. The source address is set as with HTTP load balancers. See how that\nworks by creating the TCP protocol annotation on the loadbalancer.\n\n```\nkubectl annotate service loadbalancer service.beta.kubernetes.io/brightbox-load-balancer-listener-protocol=tcp\n```\n### loadbalancer-proxy-example.yaml\nBrightbox Cloud load balancers support the [Proxy\nProtocol](https://www.brightbox.com/docs/reference/load-balancers/#proxy-protocol-support)\nallowing the source IP to be obtained easily if your backend software\nsupports it.  This example creates a Percona database which you can\nconnect to with a version 8 client.\n\nApply the example\n```\nkubectl apply -f examples/loadbalancer-proxy-example.yaml\n```\nOnce the load balancer has completed building, get the pod names\n```\nkubectl get pods\n```\nand obtain the root password from the logs\n```\n$ kubectl logs percona-5cb97df57c-db2p5 | grep GENERATED\nGENERATED ROOT PASSWORD: 4q6eNPylWuxykirluM)urIbipoJ\n```\n\nThen connect with a mysql client to the load balancer address and run a show process list\n```\n$ mysql -v -h cip-wyre0.gb1.brightbox.com -e 'show full processlist;' -u root -p\nEnter password:\n--------------\nshow full processlist\n--------------\n\n+----+-----------------+-------------------+------+---------+------+------------------------+-----------------------+-----------+---------------+\n| Id | User            | Host              | db   | Command | Time | State                  | Info                  | Rows_sent | Rows_examined |\n+----+-----------------+-------------------+------+---------+------+------------------------+-----------------------+-----------+---------------+\n|  4 | event_scheduler | localhost         | NULL | Daemon  |  705 | Waiting on empty queue | NULL                  |         0 |             0 |\n| 10 | root            | 192.168.2.1:59186 | NULL | Query   |    0 | starting               | show full processlist |         0 |             0 |\n+----+-----------------+-------------------+------+---------+------+------------------------+-----------------------+-----------+---------------+\n```\nYou'll see that the Host address is a local address from the internal kubernetes network.\n\nNow delete the example\n```\nkubectl delete -f examples/loadbalancer-proxy-example.yaml\n```\nthen edit and uncomment the lines that turn on proxy support in the\nloadbalancer and the database, before applying the example again. Remember\nto get the new root password.\n\nThis time when you connect you'll see that the Host address is the address of the client you are connecting from.\n```\n$ mysql -v -h cip-wyre0.gb1.brightbox.com -e 'show full processlist;' -u root -p\nEnter password:\n--------------\nshow full processlist\n--------------\n\n+----+-----------------+----------------------+------+---------+------+------------------------+-----------------------+-----------+---------------+\n| Id | User            | Host                 | db   | Command | Time | State                  | Info                  | Rows_sent | Rows_examined |\n+----+-----------------+----------------------+------+---------+------+------------------------+-----------------------+-----------+---------------+\n|  4 | event_scheduler | localhost            | NULL | Daemon  |   34 | Waiting on empty queue | NULL                  |         0 |             0 |\n|  8 | root            | 82.132.242.240:47962 | NULL | Query   |    0 | starting               | show full processlist |         0 |             0 |\n+----+-----------------+----------------------+------+---------+------+------------------------+-----------------------+-----------+---------------+\n```\n## Automatic SSL certificate management\nBrightbox Cloud load balancers support [automatic generation of SSL certificates](https://www.brightbox.com/docs/reference/load-balancers/#certificates) via Let's Encrypt.\n\nFirst create a normal HTTP loadbalancer and test that the loadbalancer works as expected. Obtain the address details via kubectl.\n```\n$ kubectl expose deployment source-ip-app --name=loadbalancer --port=80 --target-port=8080 --type=LoadBalancer\nservice/loadbalancer exposed\n$ kubectl get service/loadbalancer\nNAME           TYPE           CLUSTER-IP      EXTERNAL-IP                                                                                                      PORT(S)        AGE\nloadbalancer   LoadBalancer   172.30.73.139   109.107.39.75,2a02:1348:ffff:ffff::6d6b:274b,cip-109-107-39-75.gb1s.brightbox.com,cip-f7uv8.gb1s.brightbox.com   80:31129/TCP   1m\n```\n\nNow map a domain name to the allocated cloudIP via your preferred DNS service - either directly to the addresses of the CloudIP or via a CNAME record to the cip DNS name.\nOnce the domain names resolve correctly, annotate your load balancer with the domain, and change the exposed port to 443.\n```\n$ kubectl patch service/loadbalancer --type='json' -p='[{\"op\": \"replace\", \"path\": \"/spec/ports/0/port\", \"value\":443}]'\nservice/loadbalancer patched\n$ kubectl annotate service loadbalancer service.beta.kubernetes.io/brightbox-load-balancer-ssl-domains=my-domain.co\nservice/loadbalancer annotated\n```\nThe load balancer will automatically obtain the appropriate SSL\ncertificates, install them and turn on the HTTPS redirect service. This\nwill ensure that any access to URLs on the specified domains will go\nvia a secure connection straightaway.\n```\n$ curl https://my-domain.co/\n```\n### Manual Cloud IP allocation example\nIf you allocate a cloud IP manually via the [Brightbox Cloud\nManager](https://www.brightbox.com/docs/guides/manager/getting-started/)\nyou can create an SSL enabled load balancer in one go.\n\n- Select or create a new CloudIP in the Brightbox Manager and map your\nchosen domain to it either via a CNAME record or directly to the addresses\nshown in the Manager. You may want to set the reverse DNS on the CloudIP too.\n- Make a copy of the `load-balancer-ssl-example.yml` manifest and edit it.\n- Enter the name of your domain against the `brightbox-load-balancer-ssl-domains` annotation\n- Alter the value of `LoadBalancerIP` to match the address of the chosen CloudIP\n- Apply the manifest with `kubectl apply -f`\n\nThe load balancer will automatically obtain the appropriate SSL certificates and install them. Once they are in place you can access via an https URL\n\n## Upgrade a Cluster\nThe scripts will upgrade the version of Kubernetes on an existing cluster. Change the `kubernetes_release` version number as required and run `terraform apply`. Both the master and workers will be upgraded to the new version.\nUpgrades will only work if permitted by the `kubeadm upgrade` facility. You can check before hand by logging onto your master and running [`kubeadm upgrade plan`](https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-upgrade/#cmd-upgrade-plan)\n## Adding Workers\nYou can add by changing the `worker_count` variable\nand running `terraform apply`. You can also change the `worker_type` and\neven the `image_desc` and new workers will use those values.\n\n## Reducing workers\nBefore reducing the `worker_count` variable, you will need to drain\nand remove the nodes from Kubernetes.  Reducing workers operates using\nthe last in, first out principle. Run\n\n    $ terraform output\n\nand select the `worker_ids` that are at thoe bottom of their lists.\n\nRun\n\n    kubectl drain srv-abcde --ignore-daemonsets=true\n\nfor each node. Then\n\n    kubectl delete nodes srv-abcde srv-edcba\n\nFinally reduce the `worker_count` variable and run `terraform apply`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrightbox%2Fkubernetes-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrightbox%2Fkubernetes-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrightbox%2Fkubernetes-cluster/lists"}