{"id":16479354,"url":"https://github.com/timebertt/pi-cloud-init","last_synced_at":"2025-03-16T18:31:35.706Z","repository":{"id":40595888,"uuid":"346010159","full_name":"timebertt/pi-cloud-init","owner":"timebertt","description":"Minimal Raspberry Pi OS including cloud-init","archived":false,"fork":false,"pushed_at":"2021-06-15T19:44:12.000Z","size":63,"stargazers_count":95,"open_issues_count":6,"forks_count":13,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-16T04:31:53.801Z","etag":null,"topics":["cloud-init","iot","k3s","kubernetes","raspberry-pi","raspberry-pi-os"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timebertt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-09T13:09:58.000Z","updated_at":"2025-03-13T18:02:17.000Z","dependencies_parsed_at":"2022-08-24T23:40:38.427Z","dependency_job_id":null,"html_url":"https://github.com/timebertt/pi-cloud-init","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timebertt%2Fpi-cloud-init","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timebertt%2Fpi-cloud-init/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timebertt%2Fpi-cloud-init/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timebertt%2Fpi-cloud-init/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timebertt","download_url":"https://codeload.github.com/timebertt/pi-cloud-init/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243910666,"owners_count":20367546,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-init","iot","k3s","kubernetes","raspberry-pi","raspberry-pi-os"],"created_at":"2024-10-11T12:51:42.968Z","updated_at":"2025-03-16T18:31:35.208Z","avatar_url":"https://github.com/timebertt.png","language":"Shell","readme":"# Minimal Raspberry Pi OS including cloud-init\n\nThis repo features a custom [Raspberry Pi OS Lite](https://www.raspberrypi.org/software/operating-systems/) image including [cloud-init](https://cloud-init.io/) built with [pi-gen](https://github.com/RPi-Distro/pi-gen).\n\n## TL;DR: How to use the image :gear:\n\n1. Download the image from GitHub:\n    ```bash\n    ARCH=armhf # either armhf or aarch64 (experimental)\n    curl -sSL -o 2021-05-10-raspios-buster-$ARCH-lite-cloud-init.zip https://github.com/timebertt/pi-cloud-init/releases/download/2021-05-10/2021-05-10-raspios-buster-$ARCH-lite-cloud-init.zip \u0026\u0026 \\\n    unzip -o 2021-05-10-raspios-buster-$ARCH-lite-cloud-init.zip\n    ```\n\n2. Mount the `boot` partition (of the `.img` file) to add `user-data`, `meta-data` and optionally `network-config` files to the root of it. Unmount it again, once added.\n3. Flash image to SD card using [balena etcher](https://www.balena.io/etcher/), [Raspberry Pi Imager](https://www.raspberrypi.org/software/) or similar.\n4. Insert SD card into Pi and power up! :rocket:\n\n## Rationale :bulb:\n\n[Raspberry Pi OS](https://www.raspberrypi.org/software/operating-systems/) is fairly easy to setup. Though, if you want to setup a Raspberry Pi cluster like Alex Ellis and the community [have been doing for five years+](https://alexellisuk.medium.com/five-years-of-raspberry-pi-clusters-77e56e547875), you will have to repeat some manual configuration steps like copying SSH keys, networking setup and so on for each Pi.\n\n[cloud-init](https://cloud-init.io/) is the de-facto standard for solving the same problem on cloud instances. It's baked into most cloud images, which makes cloud instances discover initialization and configuration data from the cloud provider's [metadata service](https://cloudinit.readthedocs.io/en/latest/topics/datasources.html).\n\ncloud-init can be used for bootstrapping Raspberry Pis as well, making it easy to configure networking, packages and remote access consistently across a fleet of Pis. Unfortunately, Raspberry Pi OS doesn't ship with cloud-init out of the box and thus requires [manual installation and configuration](https://gist.github.com/RichardBronosky/fa7d4db13bab3fbb8d9e0fff7ea88aa2).\n\nThis repo features a custom image based on Raspberry Pi OS Lite built with [pi-gen](https://github.com/RPi-Distro/pi-gen) with cloud-init preinstalled, which allows to pass initialization and configuration data (in form of `meta-data`, `user-data` and `network-config` files) to Pis via the boot image flashed to an SD card. This makes it easy to bootstrap multiple Pis in a plug-and-play fashion without attaching a monitor or manually SSHing into each one of them.\n\n**Why not simply use Ubuntu Server?** :question:\n\n[Ubuntu Server](https://ubuntu.com/download/raspberry-pi) comes with cloud-init preinstalled and also features a Raspberry Pi Image. It can be leveraged in a [similar fashion](https://gitlab.com/Bjorn_Samuelsson/raspberry-pi-cloud-init-wifi) to the image built by this project to bootstrap Pis. Though, in my tests Ubuntu Server already consumed more than `300MB` of precious memory on my Pis without anything installed. Therefore I started building a custom image based on Raspberry Pi OS Lite, which consumes only roughly `60MB` of memory out of the box.\n\n**Why not use k3os?** :question:\n\n[k3os](https://github.com/rancher/k3os) is a minimal OS designed to run [k3s](https://github.com/k3s-io/k3s) and be managed by `kubectl`. It also features a `cloud-init`-like configuration file, which can be used to easily configure hosts. Unfortunately, there are currently [no official RPi images](https://github.com/rancher/k3os/issues/309), but you can build one on your own by leveraging [this project](https://github.com/rancher/k3os/issues/309).\nIn my tests, `k3os` performed quite well with some additional overhead in CPU and Memory usage. I chose to build my own images based on Raspberry Pi OS, because of the lower resource usage, the familiarity with Debian-based distros, flexibility in bootstrapping my clusters (not tied to k3s) and the endless resources on Raspberry Pi OS.\n\n## How to build the image :construction:\n\nYou can build the image yourself and customize the build along the way by following these steps.\n\n1. Setup a Debian VM using [vagrant](https://www.vagrantup.com/) which will build the image. This provides a clean build environment and additionally works on a Linux as well as macOS.\n    ```bash\n    # what image to build: either amrhf (default) or aarch64 (experimental)\n    export ARCH=armhf\n    vagrant up --provision\n    ```\n\n2. Start pi-gen build in the VM. This is going to take some time...\n    ```bash\n    vagrant ssh\n    ./pi-cloud-init/build.sh\n    ```\n    When you want to rebuild from scratch (e.g. because you want to build for another arch), you can run\n    ```bash\n    vagrant ssh\n    CLEAN=1 ./pi-cloud-init/build.sh\n    ```\n\n3. Transfer produced image to the host machine and unzip.\n    This requires the `vagrant-scp` plugin, install it first by running:\n    ```bash\n    vagrant plugin install vagrant-scp\n    ```\n    ```bash\n    zip_file=$(date +%Y-%m-%d)-raspios-buster-$ARCH-lite-cloud-init.zip \u0026\u0026 \\\n    vagrant scp raspios-builder:/home/vagrant/pi-cloud-init/$zip_file $zip_file \u0026\u0026 \\\n    unzip -o \"$zip_file\"\n    ```\n\n4. Customize `user-data.yaml`, `meta-data.yaml` and `network-config.yaml` for the instance you're setting up.\n\n5. Mount boot partition to inject `user-data`, `meta-data` and `network-config`.\n    (It's assuming a macOS machine, but you should be able to accomplish the same using `mount` and `umount` on Linux.)\n    ```bash\n    img_file=\"${zip_file%.zip}.img\" \u0026\u0026 \\\n    volume=\"$(hdiutil mount \"$img_file\" | egrep -o '/Volumes/.+')\" \u0026\u0026 \\\n    cp meta-data.yaml \"$volume\"/meta-data \u0026\u0026 \\\n    cp user-data.yaml \"$volume\"/user-data \u0026\u0026 \\\n    cp network-config.yaml \"$volume\"/network-config \u0026\u0026 \\\n    device=\"$(mount | grep \"$volume\" | cut -f1 -d' ' | egrep -o '/dev/disk.')\" \u0026\u0026 \\\n    diskutil umountDisk \"$device\" \u0026\u0026 \\\n    diskutil eject \"$device\"\n    ```\n\n6. Optionally, you can verify the image and cloud-init functionality using [dockerpi](https://github.com/lukechilds/dockerpi) (only works with `armhf` images). It start a Docker container with QEMU in it emulating a Pi. This way you can already verify, that the image and the provided `user-data` is working without flashing a new SD card everytime.\n    ```bash\n    docker run -it -v $PWD/$img_file:/sdcard/filesystem.img lukechilds/dockerpi:vm\n    ...\n    cloud-init[96]: Cloud-init v. 20.2 running 'init-local' at Mon, 08 Mar 2021 19:54:02 +0000. Up 53.20 seconds.\n    ...\n    cloud-init[380]: Cloud-init v. 20.2 running 'init' at Mon, 08 Mar 2021 19:54:42 +0000. Up 93.34 seconds.\n    ...\n    cloud-init[568]: Cloud-init v. 20.2 running 'modules:config' at Mon, 08 Mar 2021 19:55:48 +0000. Up 159.10 seconds.\n    ...\n    cloud-init[620]: Cloud-init v. 20.2 running 'modules:final' at Mon, 08 Mar 2021 19:56:05 +0000. Up 175.50 seconds.\n    cloud-init[620]: Cloud-init v. 20.2 finished at Mon, 08 Mar 2021 19:56:08 +0000. Datasource DataSourceNoCloud [seed=/dev/sda1][dsmode=net].  Up 179.17 seconds\n    ```\n\n7. Now, flash the image including cloud-init data to SD card, using [balena etcher](https://www.balena.io/etcher/), [Raspberry Pi Imager](https://www.raspberrypi.org/software/) or similar.\n\n8. Finally, SSH into your Pi and verify cloud-init functionality. By default, the `pi` user is locked and SSH password authentication is disabled, so make sure to add a custom user with `ssh_authorized_keys` to your `user-data`.\n    ```bash\n    ssh your-user@your-pi\n    less /var/log/cloud-init-output.log\n    ```\n\n## Next steps :running:\n\n### Play around with user-data :video_game:\n\nFind some more user-data examples under [examples](./examples):\n\n- [Simple user-data](./examples/simple)\n- [Static IPv4](./examples/static-ip)\n\n## More fun :tada:\n\n### Spin up a Kubernetes cluster :nerd_face:\n\n[examples/k3s-cluster](./examples/k3s-cluster) contains some user-data files for spinning up a Kubernetes cluster using [k3s](https://github.com/k3s-io/k3s). Use the set of user-data files in `raspberry-0` for bootstrapping your cluster (i.e. for the first Pi). After that, join any number of servers and agents using the user-data files in `raspberry-1`.\n\nThe examples already include configuration for:\n\n- [kube-vip](https://kube-vip.io/) in layer 2 mode (ARP) for API server load balancing\n  - provides an HA control plane endpoint to perform rolling updates on k3s servers (drain, install, reboot one by one)\n  - only used for control plane load balancing\n- [MetalLB](https://metallb.universe.tf/) in layer 2 mode (ARP) for LoadBalancer Services\n  - also includes LoadBalancer IP allocation address ranges specified in configuration ConfigMap (built-in and easier to configure than kube-vip)\n  - not used for control plane load balancing (not supported, ref [metallb/metallb#168](https://github.com/metallb/metallb/issues/168))\n  - [klipper-lb](https://github.com/k3s-io/klipper-lb) is disabled, as it can't provide a dedicated LoadBalancer IP on your local network (just uses host ports to make LoadBalancer services accessible)\n\nGet the kubeconfig from `raspberry-0` and modify it to use the API server LoadBalancer IP:\n\n```bash\nmkdir -p $HOME/.kube/configs/home \u0026\u0026 \\\nssh tim@192.168.0.20 sudo cat /etc/rancher/k3s/k3s.yaml \u003e $HOME/.kube/configs/home/pi-cluster.yaml \u0026\u0026 \\\nexport KUBECONFIG=$HOME/.kube/configs/home/pi-cluster.yaml \u0026\u0026 \\\nkubectl config rename-context default pi-cluster \u0026\u0026 \\\nkubectl config set clusters.default.server https://192.168.0.30:6443\n```\n\nCheck if all Nodes were bootstrapped and got ready:\n\n```bash\n$ kubectl get no -owide\nNAME          STATUS   ROLES                       AGE     VERSION        INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                       KERNEL-VERSION   CONTAINER-RUNTIME\nraspberry-0   Ready    control-plane,etcd,master   18m     v1.20.5+k3s1   192.168.0.20   \u003cnone\u003e        Debian GNU/Linux 10 (buster)   5.10.17-v8+      containerd://1.4.4-k3s1\nraspberry-1   Ready    control-plane,etcd,master   11m     v1.20.5+k3s1   192.168.0.21   \u003cnone\u003e        Debian GNU/Linux 10 (buster)   5.10.17-v8+      containerd://1.4.4-k3s1\nraspberry-2   Ready    control-plane,etcd,master   9m13s   v1.20.5+k3s1   192.168.0.22   \u003cnone\u003e        Debian GNU/Linux 10 (buster)   5.10.17-v8+      containerd://1.4.4-k3s1\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimebertt%2Fpi-cloud-init","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimebertt%2Fpi-cloud-init","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimebertt%2Fpi-cloud-init/lists"}