{"id":26778300,"url":"https://github.com/elastisys/compliantkubernetes-kubespray","last_synced_at":"2025-04-16T19:23:41.910Z","repository":{"id":37428688,"uuid":"317570501","full_name":"elastisys/compliantkubernetes-kubespray","owner":"elastisys","description":null,"archived":false,"fork":false,"pushed_at":"2025-04-16T08:33:36.000Z","size":888,"stargazers_count":29,"open_issues_count":33,"forks_count":9,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-04-16T11:13:32.501Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elastisys.png","metadata":{"files":{"readme":"README.md","changelog":"changelog/2.23.0.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-12-01T14:39:43.000Z","updated_at":"2025-04-16T06:10:40.000Z","dependencies_parsed_at":"2023-12-19T17:33:11.716Z","dependency_job_id":"3f2f45fa-12ae-42fe-b56a-46ce75f7d3be","html_url":"https://github.com/elastisys/compliantkubernetes-kubespray","commit_stats":null,"previous_names":[],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastisys%2Fcompliantkubernetes-kubespray","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastisys%2Fcompliantkubernetes-kubespray/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastisys%2Fcompliantkubernetes-kubespray/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastisys%2Fcompliantkubernetes-kubespray/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elastisys","download_url":"https://codeload.github.com/elastisys/compliantkubernetes-kubespray/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249268474,"owners_count":21240935,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-29T05:36:50.625Z","updated_at":"2025-04-16T19:23:41.887Z","avatar_url":"https://github.com/elastisys.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Elastisys Welkin® Kubespray\n\n## On API Stability\n\n⚠️  Please note that the Elastisys Welkin® Kubespray project frequently changes admin-facing API, i.e., configuration, in a backwards-incompatible way. Make sure to read the [change log](CHANGELOG.md) and the [migration steps](/migration). These migration steps are subject to quality assurance and are used in production environments. Hence, if properly executed, they shouldn't cause any downtime.\n\nThe user-facing API changes more rarely, usually as a result of a Kubernetes version upgrade. For details, read the [user-facing release notes](https://elastisys.io/welkin/release-notes/kubespray/).\n\n## Content\n\n- `bin`: wrapper scripts that helps you run kubespray\n- `config`: default config values\n- `conformance-tests`: ck8s conformance tests for bare metal machines\n- `kubespray`: git submodule of the kubespray repository\n\n## Setup\n\n### Requirements\n\n[terraform](https://github.com/hashicorp/terraform/releases) (tested with 1.2.9)\n\nInstalls requirements using the ansible playbook `get-requirements.yaml`\n\n```bash\nansible-playbook -e 'ansible_python_interpreter=/usr/bin/python3' --ask-become-pass --connection local --inventory 127.0.0.1, get-requirements.yaml\n```\n\n## Quick start\n\n1. Init the kubespray config in your config path\n\n    ```bash\n    export CK8S_CONFIG_PATH=~/.ck8s/my-environment\n    ./bin/ck8s-kubespray init \u003cwc|sc\u003e \u003cflavor\u003e [\u003cSOPS fingerprint\u003e]\n    ```\n\n    Arguments:\n    - The init command accepts `wc` (_workload cluster_) or `sc` (_service cluster_) as first argument as to create separate folders for each cluster's configuration files.\n    - `flavor` will determine some default values for a variety of config options.\n      Supported options are `default`, `gcp`, `aws`, `vsphere`, `openstack` and `upcloud`.\n    - `SOPS fingerprint` is the gpg fingerprint that will be used for SOPS encryption.\n      You need to set this or the environment variable `CK8S_PGP_FP` the first time SOPS is used in your specified config path.\n\n1. Edit the `inventory.ini` (found in your config path) to match the VMs (IP addresses and other settings that might be needed for your setup) that should be part of the cluster.\n    Or if you have one created by a terraform script in `kubespray/contrib/terraform` you should use that one.\n\n1. Init and update the [kubespray](https://github.com/kubernetes-sigs/kubespray) gitsubmodule:\n\n    ```bash\n    git submodule init\n    git submodule update\n    ```\n\n1. Run kubespray to set up the kubernetes cluster:\n\n    ```bash\n    ./bin/ck8s-kubespray apply \u003cwc|sc\u003e [\u003coptions\u003e]\n    ```\n\n    Any `options` added will be forwarded to ansible.\n\n1. Done.\n    You should now have a working kubernetes cluster.\n    You should also have an encrypted kubeconfig at `\u003cCK8S_CONFIG_PATH\u003e/.state/kube_config_\u003cwc|sc\u003e.yaml` that you can use to access the cluster.\n\n## Changing authorized SSH keys for a cluster\n\nAuthorized SSH keys can be changed for a cluster using:\n\n```bash\n./bin/ck8s-kubespray apply-ssh \u003cwc|sc\u003e [\u003coptions\u003e]\n```\n\nIt will set the public SSH key(s) found in`\u003cCK8S_CONFIG_PATH\u003e/\u003cwc|sc\u003e-config/group_vars/all/ck8s-ssh-keys.yaml` as authorized keys in your cluster (just add the keys you want to be authorized as elements in `ck8s_ssh_pub_keys_list`).\nNote that the authorized SSH keys for the cluster will be set to these keys _exclusively_, removing any keys that may already be authorized, so make sure the list includes **every SSH key** that should be authorized.\n\nWhen running this command, the SSH keys are applied to each node in the cluster sequentially, in reverse inventory order (first the workers and then the masters).\nA connection test is performed after each node which has to succeed in order for the playbook to continue.\nIf the connection test fails, you may have lost your SSH access to the node; to recover from this, you can set up an SSH connection before running the command and keep it active so that you can change the authorized keys manually.\n\n## Rebooting nodes\n\nYou can reboot all nodes that wants to restart (usually to finish installing new packages) by running:\n\n```bash\n./bin/ck8s-kubespray reboot-nodes \u003cwc|sc\u003e [--extra-vars manual_prompt=true] [\u003coptions\u003e]\n```\n\nIf you set `--extra-vars manual_prompt=true` then you get a manual prompt before each reboot so you can stop the playbook if you want.\n\nNote that this playbook requires you to use ansible version \u003e= 2.10.\n\n## Removing nodes\n\nYou can remove a node from a ck8s cluster by running:\n\n```bash\n./bin/ck8s-kubespray remove-node \u003cwc|sc\u003e \u003cnode-name\u003e[,\u003cnode-name-2\u003e,...] [\u003coptions\u003e]\n```\n\n### Known issues\n\n- The script may fail with the message `error while evaluating conditional (kubelet_heartbeat.rc == 0): 'dict object' has no attribute 'rc'`\n    - In such situations just rerun the script. It will skip the check for that node, so check that it is up and running manually afterwards.\n- The script might fail with a timeout: `Timeout (12s) waiting for privilege escalation prompt`\n    - Try running the script again with a longer ansible timeout: `export ANSIBLE_TIMEOUT=30`\n\n## Running other kubespray playbooks\n\nWith the following command you can run any ansible playbook available in kubespray:\n\n```bash\n./bin/ck8s-kubespray run-playbook \u003cwc|sc\u003e \u003cplaybook\u003e [\u003coptions\u003e]\n```\n\nWhere `playbook` is the filename of the playbook that you want to run, e.g. `cluster.yml` if you want to create a cluster (making the command functionally the same as our `ck8s-kubespray apply` command) or `scale.yml` if you want to just add more nodes. Remember to check the kubespray documentation before running a playbook.\nThis will use the inventory, group-vars, and ssh key in your config path and therefore requires that you first run the init command. Any `options` added will be forwarded to ansible.\n\n## Kubeconfig\n\nWe recommend that you use OIDC kubeconfigs instead of regular cluster-admin kubeconfigs. The default settings will create OIDC kubeconfigs for you when you run `./bin/ck8s-kubespray apply \u003cwc|sc\u003e`, but there are some variables you need to set. See the variables in: `\u003cwc|sc\u003e-config/group_vars/k8s_cluster/ck8s-k8s-cluster.yaml` in your config path.\n\nBut if you need to use a regular cluster-admin kubeconfig in a break-glass situation, then you can ssh to one of the controleplane nodes and use the kubeconfig at `/etc/kubernetes/admin.conf`. We recommend that you do not copy that kubeconfig to your local host, when dealing with production clusters.\n\nFor development you can skip OIDC and instead get a regular cluster admin kubeconfig locally by setting `kubeconfig_localhost: true`. Note that you then must set `create_oidc_kubeconfig: false`.\n\nThe kubeconfig and OIDC cluster admin RBAC are managed with the playbooks `playbooks/kubeconfig.yml` and `playbooks/cluster_admin_rbac.yml`. You can run them manually with:\n\n```bash\n./bin/ck8s-kubespray run-playbook \u003cwc|sc\u003e ../../playbooks/kubeconfig.yml -b\n./bin/ck8s-kubespray run-playbook \u003cwc|sc\u003e ../../playbooks/cluster_admin_rbac.yml -b\n```\n\n## 📜 Licensing Information\n\nAll source files in this repository are licensed under the Apache License, Version 2.0 unless otherwise stated.\nSee the [LICENSE](./LICENSE) file for full details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastisys%2Fcompliantkubernetes-kubespray","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felastisys%2Fcompliantkubernetes-kubespray","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastisys%2Fcompliantkubernetes-kubespray/lists"}