{"id":13615918,"url":"https://github.com/infraly/k8s-on-openstack","last_synced_at":"2025-04-13T23:34:21.949Z","repository":{"id":54107489,"uuid":"84673053","full_name":"infraly/k8s-on-openstack","owner":"infraly","description":"An opinionated way to deploy a Kubernetes cluster on top of an OpenStack cloud.","archived":false,"fork":false,"pushed_at":"2021-03-09T09:17:14.000Z","size":230,"stargazers_count":112,"open_issues_count":8,"forks_count":48,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-11-08T00:36:10.857Z","etag":null,"topics":["ansible","kubeadm","kubernetes","openstack"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/infraly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-03-11T19:00:17.000Z","updated_at":"2024-01-05T09:42:05.000Z","dependencies_parsed_at":"2022-08-13T06:50:39.189Z","dependency_job_id":null,"html_url":"https://github.com/infraly/k8s-on-openstack","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infraly%2Fk8s-on-openstack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infraly%2Fk8s-on-openstack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infraly%2Fk8s-on-openstack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infraly%2Fk8s-on-openstack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/infraly","download_url":"https://codeload.github.com/infraly/k8s-on-openstack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248797968,"owners_count":21163225,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","kubeadm","kubernetes","openstack"],"created_at":"2024-08-01T20:01:20.503Z","updated_at":"2025-04-13T23:34:16.919Z","avatar_url":"https://github.com/infraly.png","language":"HTML","readme":"# k8s-on-openstack\n\nAn opinionated way to deploy a Kubernetes cluster on top of an OpenStack cloud.\n\nIt is based on the following tools:\n\n  * `kubeadm`\n  * `ansible`\n\n## Getting started\n\nThe following mandatory environment variables need to be set before calling `ansible-playbook`:\n\n  * `OS_*`: standard OpenStack environment variables such as `OS_AUTH_URL`, `OS_USERNAME`, ...\n  * `KEY`: name of an existing SSH keypair\n\nThe following optional environment variables can also be set:\n\n  * `NAME`: name of the Kubernetes cluster, used to derive instance names, `kubectl` configuration and security group name\n  * `IMAGE`: name of an existing Ubuntu 16.04 image\n  * `EXTERNAL_NETWORK`: name of the neutron external network, defaults to 'public'\n  * `FLOATING_IP_POOL`: name of the floating IP pool\n  * `FLOATING_IP_NETWORK_UUID`: uuid of the floating IP network (required for LBaaSv2)\n  * `USE_OCTAVIA`: try to use Octavia instead of Neutron LBaaS, defaults to False\n  * `USE_LOADBALANCER`: assume a loadbalancer is used and allow traffic to nodes (default: false)\n  * `SUBNET_CIDR` the subnet CIDR for OpenStack's network (default: `10.8.10.0/24`)\n  * `POD_SUBNET_CIDR` CIDR of the POD network (default: `10.96.0.0/16`)\n  * `CLUSTER_DNS_IP`: IP address of the cluster DNS service passed to kubelet (default: `10.96.0.10`)\n  * `BLOCK_STORAGE_VERSION`: version of the block storage (Cinder) service, defaults to 'v2'\n  * `IGNORE_VOLUME_AZ`: whether to ignore the AZ field of volumes, needed on some clouds where AZs confuse the driver, defaults to False.\n  * `NODE_MEMORY`: how many MB of memory should nodes have, defaults to 4GB\n  * `NODE_FLAVOR`: allows to configure the exact OpenStack flavor name or ID to use for the nodes. When set, the `NODE_MEMORY` setting is ignored.\n  * `NODE_COUNT`: how many nodes should we provision, defaults to 3\n  * `NODE_AUTO_IP` assign a floating IP to nodes, defaults to False\n  * `NODE_DELETE_FIP`: delete floating IP when node is destroyed, defaults to True\n  * `NODE_BOOT_FROM_VOLUME`: boot node instances using boot from volume. Useful on clouds with only boot from volume\n  * `NODE_TERMINATE_VOLUME`: delete the root volume when each node instance is destroy, defaults to True\n  * `NODE_VOLUME_SIZE`: size of each node volume. defaults to 64GB\n  * `NODE_EXTRA_VOLUME`: create an extra unmounted data volume for each node, defaults to False\n  * `NODE_EXTRA_VOLUME_SIZE`: size of extra data volume for each node, defaults to 80GB\n  * `NODE_DELETE_EXTRA_VOLUME`: delete the extra data volume for each node when node is destroy, defaults to True\n  * `MASTER_BOOT_FROM_VOLUME`: boot the master instance on a volume for data persistence, defaults to True\n  * `MASTER_TERMINATE_VOLUME`: delete the volume when master instance is destroy, defaults to True\n  * `MASTER_VOLUME_SIZE`: size of the master volume. default to 64GB\n  * `MASTER_MEMORY`: how many MB of memory should master have, defaults to 4 GB\n  * `MASTER_FLAVOR`: allows to configure the exact OpenStack flavor name or ID to use for the master. When set, the `MASTER_MEMORY` setting is ignored.\n  * `AVAILABILITY_ZONE`: the availability zone to use for nodes and the default `StorageClass` (defaults to `nova`). This affects `PersistentVolumeClaims` without explicit a storage class.\n  * `HELM_REPOS`: a list of additional helm repos to add, separated by semicolons. Example: `charts* https://github.com/helm/charts;mycharts https://github.com/dev/mycharts`\n  * `HELM_INSTALL`: a list of helm charts and their parameters to install, separated by semicolons. Example: `mycharts/mychart;charts/somechart --name somechart --namespace somenamespace`\n\nSpin up a new cluster:\n\n```console\n$ ansible-playbook site.yaml\n```\n\nDestroy the cluster:\n\n```console\n$ ansible-playbook destroy.yaml\n```\n\nUpgrade the cluster:\n\nThe `upgrade.yaml` playbook implements the upgrade steps described in https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-11/\nAfter editing in `group_vars/all.yaml` the `kubernetes_version` and `kubernetes_ubuntu_version` variables, you can run the following commands.\n\n```console\n$ ansible-playbook upgrade.yaml\n$ ansible-playbook site.yaml\n```\n\n## Open Issues\n\n### Find a better way to configure worker nodes' network plugin\n\nSomehow, the network plugin (kubenet) is not correctly set on the worker node. On the master node `/var/lib/kubelet/kubeadm-flags.env` (created by `kubeadm init`) contains: \n\n```bash\nKUBELET_KUBEADM_ARGS=\"--cgroup-driver=systemd --cloud-provider=external --network-plugin=kubenet --pod-infra-container-image=k8s.gcr.io/pause:3.1 --resolv-conf=/run/systemd/resolve/resolv.conf\"\n```\n\nIt contains the correct `--network-plugin=kubenet` as configured [here](https://github.com/pfisterer/k8s-on-openstack-wip-k8s-1.15/blob/master/files/kubeadm-init.yaml.j2#L9). After joining the k8s cluster, the worker node's copy of `/var/lib/kubelet/kubeadm-flags.env` (created by `kubeadm join`) looks like this: \n\n```bash\nKUBELET_KUBEADM_ARGS=\"--cgroup-driver=systemd --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1 --resolv-conf=/run/systemd/resolve/resolv.conf\"\n```\n\nIt contains `--network-plugin=cni` despite setting `network-plugin: kubenet` [here](https://github.com/pfisterer/k8s-on-openstack-wip-k8s-1.15/blob/master/files/kubeadm-init.yaml.j2#L21). But the JoinConfiguration is ignored by `kubeadm join` when using a join token. \n\nOnce I edit `/var/lib/kubelet/kubeadm-flags.env` to contain --network-plugin=kubenet, the worker node goes online. I've added a hack in [roles/kubeadm-nodes/tasks/main.yaml](https://github.com/pfisterer/k8s-on-openstack-wip-k8s-1.15/blob/master/roles/kubeadm-nodes/tasks/main.yaml#L12) to set the correct value.\n\n\n## Prerequisites\n\n  * Ansible (tested with version 2.9.1)\n  * Shade library required by Ansible OpenStack modules (`python-shade` for Debian)\n\n## CI/CD\n\nThe following environment variables needs to be defined:\n\n  * `OS_AUTH_URL`\n  * `OS_PASSWORD`\n  * `OS_USERNAME`\n  * `OS_DOMAIN_NAME`\n\n# Authors\n\n  * François Deppierraz \u003cfrancois.deppierraz@infraly.ch\u003e\n  * Oli Schacher \u003coli.schacher@switch.ch\u003e\n  * Saverio Proto \u003csaverio.proto@switch.ch\u003e\n  * @HaseHarald \u003chttps://github.com/HaseHarald\u003e\n  * Dennis Pfisterer \u003chttps://github.com/pfisterer\u003e\n\n# References\n\n  * https://kubernetes.io/docs/getting-started-guides/kubeadm/\n  * https://www.weave.works/docs/net/latest/kube-addon/\n  * https://github.com/kubernetes/dashboard#kubernetes-dashboard\n","funding_links":[],"categories":["HTML","openstack"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finfraly%2Fk8s-on-openstack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finfraly%2Fk8s-on-openstack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finfraly%2Fk8s-on-openstack/lists"}