{"id":13509617,"url":"https://github.com/innobead/kubefire","last_synced_at":"2025-04-06T04:10:00.408Z","repository":{"id":44925813,"uuid":"263789549","full_name":"innobead/kubefire","owner":"innobead","description":"KubeFire 🔥, creates and manages Kubernetes Clusters using Firecracker microVMs","archived":false,"fork":false,"pushed_at":"2025-03-01T02:36:13.000Z","size":8053,"stargazers_count":538,"open_issues_count":12,"forks_count":36,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-03-30T03:05:17.029Z","etag":null,"topics":["firecracker","k0s","k3s","kubeadm","kubernetes","rke2"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/innobead.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-14T01:57:08.000Z","updated_at":"2025-03-28T02:41:46.000Z","dependencies_parsed_at":"2023-12-23T10:30:24.670Z","dependency_job_id":"2e10945b-f33c-483a-8bdc-6bcd1b11ba1e","html_url":"https://github.com/innobead/kubefire","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innobead%2Fkubefire","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innobead%2Fkubefire/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innobead%2Fkubefire/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/innobead%2Fkubefire/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/innobead","download_url":"https://codeload.github.com/innobead/kubefire/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247430869,"owners_count":20937874,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["firecracker","k0s","k3s","kubeadm","kubernetes","rke2"],"created_at":"2024-08-01T02:01:10.418Z","updated_at":"2025-04-06T04:10:00.281Z","avatar_url":"https://github.com/innobead.png","language":"Go","funding_links":[],"categories":["Go","kubernetes","Projects"],"sub_categories":["Security"],"readme":"# What is KubeFire? 🔥\n\nKubeFire is a tool to create and manage Kubernetes clusters running on Firecracker microVMs via **weaveworks/ignite**.\n\n- Uses independent root filesystem (rootfs) and kernel from OCI images instead of traditional VM images like qcow2, vhd, etc.\n- Uses containerd to manage Firecracker processes.\n- Supports different cluster bootstrappers to provision Kubernetes clusters like Kubeadm, K3s, RKE2, and K0s.\n- Supports deploying clusters on different architectures like x86_64/AMD64 and ARM64/AARCH64 (e.g., K3s, RKE2, K0s).\n\n![kubefire in action](./doc/demo.svg)\n\n# Getting Started\n\n## Installing KubeFire\n\nFor official releases, please install the latest version via the command below or use [`huber`](https://github.com/innobead/huber):\n\n```bash\ncurl -sfSL https://raw.githubusercontent.com/innobead/kubefire/master/hack/install-release-kubefire.sh | bash\n```\n\nor\n\n```bash\nhuber install kubefire\n```\n\nFor development purpose, please make sure go 1.14 installed, then build and install `kubefire` in the `GOBIN` path.\n\n```bash\nmake install\n```\n\n## Quickstart\n\nRunning below commands is to quickly have a cluster deployed by kubeadm running in minutes.\n\n```bash\nkubefire install\nkubefire cluster create demo\n```\n\n## Installing or Updating Prerequisites\n\nTo be able to run kubefire commands w/o issues like node/cluster management, there are some prerequisites to have.\nPlease run `kubefire install` command with root permission (or sudo without password) to install or update these prerequisites via the below steps.\n\n- Check virtualization supported\n- Install necessary components including runc, containerd, CNI plugins, and Ignite. See below minimum required versions of components\n  - RuncVersion \u003e= v1.1.3\n  - ContainerdVersion \u003e= v1.6.6\n  - CniVersion \u003e= v1.1.1\n  - IgniteVersion \u003e= v0.10.0\n\n\u003e Notes:\n\u003e - To uninstall the prerequisites, run `kubefire uninstall`.\n\u003e - To check the installation status, run `kubefire info`.\n\n[![asciicast](https://asciinema.org/a/tQKqYjojnsgZOjZqrGbF9Zqh0.svg)](https://asciinema.org/a/tQKqYjojnsgZOjZqrGbF9Zqh0)\n\n## Bootstrapping Cluster\n\n### Bootstrapping with command options, or a declarative config file\n\n`cluster create` provides detailed options to configure the cluster, but it also provides `--config` to accept a cluster configuration file to bootstrap the cluster as below commands.\n\n#### With command options\n```bash\n$ kubefire cluster create -h\nCreates cluster\n\nUsage:\n  kubefire cluster create [name] [flags]\n\nFlags:\n  -b, --bootstrapper string    Bootstrapper type, options: [kubeadm, k3s, rke2, k0s] (default \"kubeadm\")\n  -c, --config string          Cluster configuration file (ex: use 'config-template' command to generate the default cluster config)\n  -o, --extra-options string   Extra options (ex: key=value,...) for bootstrapper\n  -f, --force                  Force to recreate if the cluster exists\n  -h, --help                   help for create\n  -i, --image string           Rootfs container image (default \"ghcr.io/innobead/kubefire-opensuse-leap:15.2\")\n      --kernel-args string     Kernel arguments (default \"console=ttyS0 reboot=k panic=1 pci=off ip=dhcp security=apparmor apparmor=1\")\n      --kernel-image string    Kernel container image (default \"ghcr.io/innobead/kubefire-ignite-kernel:4.19.125-amd64\")\n      --master-count int       Count of master node (default 1)\n      --master-cpu int         CPUs of master node (default 2)\n      --master-memory string   Memory of master node (default \"2GB\")\n      --master-size string     Disk size of master node (default \"10GB\")\n      --no-cache               Forget caches\n      --no-start               Don't start nodes\n  -k, --pubkey string          Public key\n  -v, --version string         Version of Kubernetes supported by bootstrapper (ex: v1.18, v1.18.8, empty)\n      --worker-count int       Count of worker node\n      --worker-cpu int         CPUs of worker node (default 2)\n      --worker-memory string   Memory of worker node (default \"2GB\")\n      --worker-size string     Disk size of worker node (default \"10GB\")\n\nGlobal Flags:\n  -l, --log-level string   log level, options: [panic, fatal, error, warning, info, debug, trace] (default \"info\")\n\n```\n\n#### With declarative config file\n\n```bash\n# Geneate a cluster template configuration, then update the config as per your needs\n$ kubefire cluster config-template \u003e cluster.yaml\n```\n```bash\n$ cat cluster.yaml\nname: \"\"\nbootstrapper: kubeadm\npubkey: \"\"\nprikey: \"\"\nversion: \"\"\nimage: ghcr.io/innobead/kubefire-opensuse-leap:15.2\nkernel_image: ghcr.io/innobead/kubefire-ignite-kernel:4.19.125-amd64\nkernel_args: console=ttyS0 reboot=k panic=1 pci=off ip=dhcp security=apparmor apparmor=1\nextra_options: {}\ndeployed: false\nmaster:\n  count: 1\n  memory: 2GB\n  cpus: 2\n  disk_size: 10GB\nworker:\n  count: 0\n  memory: 2GB\n  cpus: 2\n  disk_size: 10GB\n```\n# Create a cluster with the config file\n\n```bash\nkubefire cluster create demo --config=cluster.yaml\n```\n\n### Bootstrapping with selectable Kubernetes versions\n\n```bash\n# Create a cluster with the latest versions w/o any specified version\n$ kubefire cluster create demo\n\n# Create a cluster with the latest patch version of v1.18\n$ kubefire cluster create demo --version=v1.18\n\n# Create a cluster with a valid specific version v1.18.8\n$ kubefire cluster create demo --version=v1.18.8\n\n# Create a cluster with the latest patch version of supported minor releases\n$ kubefire cluster create demo --version=v1.17\n$ kubefire cluster create demo --version=v1.16\n\n# If the version is outside the supported versions (last 3 minor versions given the latest is v1.18), the cluster creation will be not supported\n$ kubefire cluster create demo --version=v1.15\n```\n\n### Bootstrapping with Kubeadm\n\u003e Supports [the latest supported version](https://dl.k8s.io/release/stable.txt) and last 3 minor versions.\n\n```bash\nkubefire cluster create demo --bootstrapper=kubeadm\n```\n\n#### Add extra Kubeadm deployment options\n\nTo add extra deployment options of the control plane components, use `--extra-options` of `cluster create` command to provide `init_options`, `api_server_options`, `controller_manager_options` or `scheduler_options` key-value pairs as the below example.\n\n\u003e Note: the key-value pairs in `--extra-options` are separated by comma.\n\n- Add extra options of `kubeadm init` into `init_options='\u003coption\u003e,...'`.\n- Add extra options of `API Server` into `api_server_options='\u003coption\u003e,...'`.\n- Add extra options of `Controller Manager` into `controller_manager_options='\u003coption\u003e,...'`.\n- Add extra options of `Scheduler` into `scheduler_options='\u003coption\u003e,...'`.\n\n```bash\nkubefire cluster create demo --bootstrapper=kubeadm --extra-options=\"init_options='--service-dns-domain=yourcluster.local' api_server_options='--audit-log-maxage=10'\"\n```\n\n[![asciicast](https://asciinema.org/a/lQfFfMa1zCXWvz321eUqhNyxB.svg)](https://asciinema.org/a/lQfFfMa1zCXWvz321eUqhNyxB)\n\n### Bootstrapping with K3s\n\u003e Supports [the latest supported version](https://update.k3s.io/v1-release/channels/latest) and last 3 minor versions.\n\nPlease note that K3s only officially supports Ubuntu 16.04 and 18.04, the kernel versions of which are 4.4 and 4.15.\nTherefore, if using the prebuilt kernels, please use `4.19` (which is the default kernel used) instead of `5.4`, otherwise there will be some unexpected errors happening.\nFor rootfs, it's no problem to use other non-Ubuntu images.\n\n```bash\nkubefire cluster create demo --bootstrapper=k3s\n```\n\n### Bootstrapping with K3s on ARM64\n\nFrom 0.3.0, it's able to deploy K3s cluster on ARM64 architecture.\n\n\u003e Notes:\n\u003e K3s supported only, and Kubeadm will be planned to support in the future.\n\n[![asciicast](https://asciinema.org/a/6UVU9PVdcqAAtgN17N9EAaSFq.svg)](https://asciinema.org/a/6UVU9PVdcqAAtgN17N9EAaSFq)\n\n#### Add extra K3s deployment options\n\nTo add extra deployment options of the server or agent nodes, use `--extra-options` of `cluster create` command to provide `server_install_options` or `agent_install_options` key-value pairs as the below example.\n\n\u003e Note: the key-value pairs in `--extra-options` are separated by a comma.\n\n- Add extra options of `k3s server` into `server_install_options='\u003ck3s server option\u003e,...'`.\n- Add extra options of `k3s agent` into `agent_install_options='\u003ck3s agent option\u003e,...'`.\n\n```bash\nkubefire cluster create demo --bootstrapper=k3s --extra-options=\"server_install_options='--disable=traefik,--disable=metrics-server'\"\n```\n\n[![asciicast](https://asciinema.org/a/HqmfS4wZP7pPVS3E7M7gwAzmA.svg)](https://asciinema.org/a/HqmfS4wZP7pPVS3E7M7gwAzmA)\n\n### Bootstrapping with RKE2\n\n```bash\nkubefire cluster create demo --bootstrapper=rke2\n```\n\n#### Add extra RKE2 deployment options\n\nTo add extra deployment options of the RKE2 cluster, use `--extra-options` of `cluster create` command to provide the below options as key-value pairs.\n\n\u003e Note: the key-value pairs in `--extra-options` are separated by a comma.\n\n- Add extra options of `rke2 server` into `server_install_options='\u003crke2 server option\u003e,...'`.\n- Add extra options of `rke2 agent` into `agent_install_options='\u003crke2 agent option\u003e,...'`.\n\n```bash\nkubefire cluster create demo --bootstrapper=rke2 --extra-options=\"server_install_options='--node-label=label1,--node-taint=key=value:NoSchedule'\"\n```\n\n### Bootstrapping with K0s\n\n```bash\nkubefire cluster create demo --bootstrapper=k0s\n```\n\n#### Add extra K0s deployment options\n\nTo add extra deployment options of the K0s cluster, use `--extra-options` of `cluster create` command to provide the below options as key-value pairs.\n\n\u003e Note: the key-value pairs in `--extra-options` are separated by a comma.\n\n- Add extra options of `k0s server` into `server_install_options='\u003ck0s server option\u003e,...'`\n- Add extra options of `k0s worker` into `worker_install_options='\u003ck0s worker option\u003e,...'`\n- Add `cluster_config_file` into `cluster_config_file='\u003ck0s customized cluster.yaml\u003e,...'`\n\n```bash\nkubefire cluster create demo --bootstrapper=k0s --extra-options=\"server_install_options='--debug' cluster_config_file=/tmp/cluster.yaml\"\n```\n\n## Accessing Cluster\n\nDuring bootstrapping, the cluster folder is created at `~/.kubefire/clusters/\u003ccluster name\u003e`. After bootstrapping, there are several files generated in the folder.\n\n- **admin.conf**\n\n  The kubeconfig, downloaded from one of master nodes\n\n- **cluster.yaml**\n\n  The cluster config manifest is for creating the cluster. There is no declarative management based on it for now, but maybe it will be introduced in the future.\n\n- **key, key.pub**\n\n  The private and public keys for SSH authentication to all nodes in the cluster.\n\nThere are two ways below to operate the deployed cluster. After having a valid KUBECONFIG setup, run kubectl commands as usual.\n\n1. run `eval $(kubefire cluster env \u003ccluster name\u003e)` to update KUBECONFIG pointing to `~/.kubefire/clusters/\u003ccluster name\u003e/admin.conf`.\n2. run `kubefire node ssh \u003cmaster node name\u003e` to ssh to one of master nodes, then update KUBECONFIG pointing to `/etc/kubernetes/admin.conf`. For K3s, the kubeconfig is `/etc/rancher/k3s/k3s.yaml` instead.\n\n# Usage\n\n## CLI Commands\n\nMake sure to run kubefire commands with root permission or sudo without password, because ignite needs root permission to manage Firecracker VMs for now, but it is planned to improve in the future release.\n\n```bash\n$ kubefire -h\n\nKubeFire, creates and manages Kubernetes clusters using FireCracker microVMs\n\nUsage:\n  kubefire [flags]\n  kubefire [command]\n\nAvailable Commands:\n  cache       Manages caches\n  cluster     Manages clusters\n  help        Help about any command\n  image       Shows supported RootFS and Kernel images\n  info        Shows info of prerequisites, supported K8s/K3s versions\n  install     Installs or updates prerequisites\n  kubeconfig  Manages kubeconfig of clusters\n  node        Manages nodes\n  uninstall   Uninstalls prerequisites\n  version     Shows version\n\nFlags:\n  -h, --help               help for kubefire\n  -l, --log-level string   log level, options: [panic, fatal, error, warning, info, debug, trace] (default \"info\")\n\n```\n\n```bash\n# Show version\n$ kubefire version\n\n# Show supported RootFS and Kernel images\n$ kubefire image\n\n# Show prerequisites information\n$ kubefire info\n\n# Show supported K8s/K3s versions by builtin bootstrappers\n$ kubefire info -b\n\n# Install or Update prerequisites\n$ kubefire install\n\n# Uninstall prerequisites\n$ kubefire uninstall\n\n# Create a cluster\n$ kubefire cluster create\n\n# Create a cluster w/ a selected version\n$ kubefire cluster create --version=[v\u003cMAJOR\u003e.\u003cMINOR\u003e.\u003cPATCH\u003e | v\u003cMAJOR\u003e.\u003cMINOR\u003e]\n\n# Delete clusters\n$ kubefire cluster delete\n\n# Show a cluster info\n$ kubefire cluster show\n\n# Show a cluster config\n$ kubefire cluster config\n\n# Create the default cluster config template\n$ kubefire cluster config-template\n\n# Stop a cluster\n$ kubefire cluster stop\n\n# Start a cluster\n$ kubefire cluster start\n\n# Restart a cluster\n$ kubefire cluster restart\n\n# List clusters\n$ kubefire cluster list\n\n# Print environment variables of cluster (ex: KUBECONFIG)\n$ kubefire cluster env\n\n# Print cluster kubeconfig\n$ kubefire kubeconfig show\n\n# Download cluster kubeconfig\n$ kubefire kubeconfig download\n\n# SSH to a node\n$ kubefire node ssh\n\n# Show a node info\n$ kubefire node show\n\n# Stop a node\n$ kubefire node stop\n\n# Start a node\n$ kubefire node start\n\n# Restart a node\n$ kubefire node restart\n\n# Show cache info\n$ kubefire cache show\n\n# Delete caches\n$ kubefire cache delete\n```\n\n# Troubleshooting\n\nIf encountering any unexpected behavior like ignite can't allocate valid IPs to the created VMs.\nPlease try to clean up the environment, then verify again. If the issues still cannot be resolved by environment cleanup, please help create issues.\n\n```bash\nkubefire unisntall\nkubefire install\n```\n\n# Supported Container Images for RootFS and Kernel\n\nBesides below prebuilt images, you can also use the images provided by [weaveworks/ignite](https://github.com/weaveworks/ignite/tree/master/images).\n\n## RootFS images\n- ghcr.io/innobead/kubefire-opensuse-leap:15.3\n- ghcr.io/innobead/kubefire-sle15:15.3\n- ghcr.io/innobead/kubefire-rocky:8\n- ghcr.io/innobead/kubefire-centos:8\n- ghcr.io/innobead/kubefire-ubuntu:18.04, 20.04, 20.10\n\n## Kernel images (w/ AppArmor enabled)\n- ghcr.io/innobead/kubefire-ignite-kernel:5.4.43-amd64\n- ghcr.io/innobead/kubefire-ignite-kernel:4.19.125-amd64 (default)\n- ghcr.io/innobead/kubefire-ignite-kernel:5.4.43-arm64\n- ghcr.io/innobead/kubefire-ignite-kernel:4.19.125-arm64 (default)\n\n# References\n\n- [Firecracker](https://github.com/firecracker-microvm/firecracker)\n- [Ignite](https://github.com/weaveworks/ignite)\n- [K3s](https://github.com/k3s-io/k3s)\n- [Kubeadm](https://github.com/kubernetes/kubeadm)\n- [RKE2](https://docs.rke2.io/)\n- [K0s](https://github.com/k0sproject/k0s)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finnobead%2Fkubefire","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finnobead%2Fkubefire","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finnobead%2Fkubefire/lists"}