{"id":13928745,"url":"https://github.com/zer0def/kubedee","last_synced_at":"2025-07-19T10:31:48.436Z","repository":{"id":109354853,"uuid":"258323804","full_name":"zer0def/kubedee","owner":"zer0def","description":"Fast multi-node Kubernetes development and test clusters on Incus/LXD.","archived":false,"fork":true,"pushed_at":"2024-11-18T20:49:41.000Z","size":338,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-18T21:52:16.058Z","etag":null,"topics":["force-push-warning","incus","kubernetes","kubernetes-development","lxd"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"schu/kubedee","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zer0def.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-04-23T20:34:24.000Z","updated_at":"2024-11-18T20:49:44.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/zer0def/kubedee","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0def%2Fkubedee","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0def%2Fkubedee/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0def%2Fkubedee/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zer0def%2Fkubedee/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zer0def","download_url":"https://codeload.github.com/zer0def/kubedee/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226588810,"owners_count":17655793,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["force-push-warning","incus","kubernetes","kubernetes-development","lxd"],"created_at":"2024-08-07T18:01:36.342Z","updated_at":"2024-11-26T17:30:54.244Z","avatar_url":"https://github.com/zer0def.png","language":"Shell","funding_links":[],"categories":["kubernetes"],"sub_categories":[],"readme":"![kubedee logo](docs/logo/kubedee.png)\n\n[![builds.sr.ht status](https://builds.sr.ht/~schu/kubedee.svg)](https://builds.sr.ht/~schu/kubedee?)\n\nFast multi-node Kubernetes (\u003e= 1.19) development and test clusters on [Incus](https://github.com/lxc/incus) or [LXD](https://github.com/canonical/lxd).\n\nUnder the hood, [CRI-O](https://github.com/kubernetes-incubator/cri-o) is used\nas container runtime and [Flannel](https://github.com/coreos/flannel) for\nnetworking.\n\nFor questions or feedback, please open an issue.\n\n## Requirements\n\n* [Incus](https://github.com/lxc/incus)/[LXD](https://github.com/canonical/lxd).\n  * Make sure your user is member of the `incus`/`lxd` group (see `incusd --group …` / `lxd --group …`)\n  * btrfs is used a storage driver currently and required\n* [cfssl](https://github.com/cloudflare/cfssl) with cfssljson\n* [jq](https://stedolan.github.io/jq/)\n* kubectl\n\n## Installation\n\nkubedee is meant to and easily installed out of git. Clone the repository\nand link `kubedee` from a directory in your `$PATH`. Example:\n\n```\ncd ~/code\ngit clone https://github.com/schu/kubedee\ncd ~/bin\nln -s ~/code/kubedee/kubedee\n```\n\nThat's it!\n\nkubedee stores all data in `~/.local/share/kubedee/...`. kubedee LXD resources\nhave a `kubedee-` prefix.\n\n`KUBEDEE_DEBUG=1` enables verbose debugging output (`set -x`).\n\n## Usage\n\n### Getting started\n\nkubedee can install clusters based on an upstream version of Kubernetes\nor your own build.\n\nTo install an upstream version, use `--kubernetes-version` to specify\nthe release (Git tag) that you want to install. For example:\n\n```\nkubedee up test --kubernetes-version v1.21.1\n```\n\nTo install a local build, specify the location of the binaries\n(`kube-apiserver` etc.) with `--bin-dir`. For example:\n\n```\nkubedee up test --bin-dir /path/to/my/kubernetes/binaries\n```\n\nThe default for `--bin-dir` is `./_output/bin/` and thus matches the\ndefault location after running `make` in the Kubernetes repository.\nSo in a typical development workflow `--bin-dir` doesn't need to be\nspecified.\n\nNote: after the installation or upgrade of kubedee, kubedee requires some\nextra time to download and update cached packages and images once.\n\nWith a SSD, up-to-date caches and images, setting up a cluster usually takes\nless than 60 seconds for a four node cluster (etcd, controller, 2x worker).\n\n```\n[...]\n\nSwitched to context \"kubedee-test\".\n\n==\u003e Cluster test started\n==\u003e kubectl config current-context set to kubedee-test\n\n==\u003e Cluster nodes can be accessed with 'lxc exec \u003cname\u003e bash'\n==\u003e Cluster files can be found in '/home/schu/.local/share/kubedee/clusters/test'\n\n==\u003e Current node status is (should be ready soon):\nNAME                         STATUS     ROLES    AGE   VERSION\nkubedee-test-controller      NotReady   master   16s   v1.21.1\nkubedee-test-worker-2ma3em   NotReady   node     9s    v1.21.1\nkubedee-test-worker-zm8ikt   NotReady   node     2s    v1.21.1\n```\n\nkubectl's current-context has been changed to the new cluster automatically.\n\n### Incus/LXD VM setup notes\n\nIt might be the case that your particular system prevents any user from using\n`io_uring`, due to a relatively-recent-at-the-time-of-writing outpour of related\nCVEs. This is usually signalled by the `failed to init linux io_uring ring` error.\n\nShould that be the case, in order to spawn VMs through Incus/LXD, the operator\nwould need to either run the respective daemon with `CAP_SYS_ADMIN` priviledges\nor add it's user to a group designated to access `io_uring`. Example follows:\n\n```\nIO_URING_GID=666 IO_URING_GNAME=\"io_uring\"\ngroupadd -r -g \"${IO_URING_GID}\" \"${IO_URING_GNAME}\"  # create io_uring group\nsysctl -w kernel.io_uring_group=\"${IO_URING_GID}\"  # designate it as such by gid\ngpasswd -a incus \"${IO_URING_GNAME}\"  # add respective daemon's user to said group\nsystemctl restart incus  # go nuts\n```\n\n### Cheatsheet\n\nList the available clusters:\n\n```\nkubedee [list]\n```\n\nStart a cluster with less/more worker nodes than the default of 2:\n\n```\nkubedee up --num-workers 4 \u003ccluster-name\u003e\n```\n\nStart a new worker node in an existing cluster:\n\n```\nkubedee start-worker \u003ccluster-name\u003e\n```\n\nDelete a cluster:\n\n```\nkubedee delete \u003ccluster-name\u003e\n```\n\nConfigure the `kubectl` env:\n\n```\neval $(kubedee kubectl-env \u003ccluster-name\u003e)\n```\n\nConfigure the `etcdctl` env:\n\n```\neval $(kubedee etcd-env \u003ccluster-name\u003e)\n```\n\nSee all available commands and options:\n\n```\nkubedee help\n```\n\n## Smoke test\n\nkubedee has a `smoke-test` subcommand:\n\n```\nkubedee smoke-test \u003ccluster-name\u003e\n```\n\n[freenode]: https://freenode.net/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzer0def%2Fkubedee","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzer0def%2Fkubedee","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzer0def%2Fkubedee/lists"}