{"id":17989549,"url":"https://github.com/evilfreelancer/kubernetes-cluster-on-arm","last_synced_at":"2025-11-05T03:02:09.688Z","repository":{"id":47619447,"uuid":"307365441","full_name":"EvilFreelancer/kubernetes-cluster-on-arm","owner":"EvilFreelancer","description":"Project of simple Kubernetes cluster on NanoPi NEO3 ARM64 nodes","archived":false,"fork":false,"pushed_at":"2022-07-21T03:48:50.000Z","size":147,"stargazers_count":21,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-06T14:21:26.786Z","etag":null,"topics":["ansible","arm","arm64","k3s","kubernetes","nanopi","nanopi-neo3","neo3"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EvilFreelancer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-26T12:30:30.000Z","updated_at":"2024-12-06T07:46:43.000Z","dependencies_parsed_at":"2022-09-23T14:19:25.680Z","dependency_job_id":null,"html_url":"https://github.com/EvilFreelancer/kubernetes-cluster-on-arm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/EvilFreelancer/kubernetes-cluster-on-arm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EvilFreelancer%2Fkubernetes-cluster-on-arm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EvilFreelancer%2Fkubernetes-cluster-on-arm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EvilFreelancer%2Fkubernetes-cluster-on-arm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EvilFreelancer%2Fkubernetes-cluster-on-arm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EvilFreelancer","download_url":"https://codeload.github.com/EvilFreelancer/kubernetes-cluster-on-arm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EvilFreelancer%2Fkubernetes-cluster-on-arm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":282750335,"owners_count":26721146,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-05T02:00:05.946Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","arm","arm64","k3s","kubernetes","nanopi","nanopi-neo3","neo3"],"created_at":"2024-10-29T19:14:55.549Z","updated_at":"2025-11-05T03:02:09.645Z","avatar_url":"https://github.com/EvilFreelancer.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kubernetes cluster on ARM nodes (WIP)\n\nProject about simple Kubernetes clusters on ARM nodes.\n\n![Image](assets/arm-cluster.png)\n\n\u003c!-- toc --\u003e\n\n- [Specifications](#specifications)\n  * [Controller](#controller)\n  * [Nodes](#nodes)\n    + [NanoPi NEO3 based solution](#nanopi-neo3-based-solution)\n    + [Raspberry Pi CM3 based solution](#raspberry-pi-cm3-based-solution)\n  * [Software](#software)\n- [Installing Operating Systems](#installing-operating-systems)\n  * [Installing OS to NanoPi NEO3](#installing-os-to-nanopi-neo3)\n  * [Installing OS to Raspberry Pi CM3](#installing-os-to-raspberry-pi-cm3)\n- [Deploying K3S with help of Ansible](#deploying-k3s-with-help-of-ansible)\n  * [Install all required packages on all servers](#install-all-required-packages-on-all-servers)\n  * [Deploy k3s server to controller](#deploy-k3s-server-to-controller)\n  * [Deploy k3s agent to nodes](#deploy-k3s-agent-to-nodes)\n  * [Install and configure kubectl](#install-and-configure-kubectl)\n- [How to enable Kubernetes Dashboard](#how-to-enable-kubernetes-dashboard)\n- [Video blog about this project](#video-blog-about-this-project)\n- [Links](#links)\n\n\u003c!-- tocstop --\u003e\n\n## Specifications\n\nCommunication between nodes and controller:\n\n- 1x [Mercusys MS108G](https://www.mercusys.com/en/product/details/ms108g)\n\n### Controller\n\n- 1x Ethernet cable (with four copper pairs)\n- 1x USB Type-C cable\n- 1x Power supply adapter 5V/2A for USB devices\n- 1x Raspberry Pi 4B (CPU: quad-core, RAM: 8Gb)\n- 1x MicroSD card 32Gb (by Samsung, the EVO series)\n\n### Nodes\n\n#### NanoPi NEO3 based solution\n\n- 4x Ethernet cable (with four copper pairs)\n- 4x USB Type-C cable\n- 4x Power supply adapter 5V/2A for USB devices\n- 4x [NanoPi NEO3](https://www.friendlyelec.com/index.php?route=product/product\u0026product_id=279) (CPU: quad-core, RAM: 2Gb)\n- 4x MicroSD card 32Gb (class 10)\n\n#### Raspberry Pi CM3 based solution\n\n- 2x Ethernet cable (with four copper pairs)\n- 1x Power supply adapter 12V/5A for TuringPi board\n- 1x [TuringPi V1](https://turingpi.com/v1/) cluster board\n- 7x [Raspberry Pi CM3 Lite](https://www.raspberrypi.com/products/compute-module-3/) (CPU: quad-core, RAM: 1Gb)\n- 7x MicroSD card 32Gb (class 10)\n\n### Software\n\n- Operating Systems on all devices in Ubuntu 20.04 LTS (or recent) for ARM64 CPUs;\n- On controller should be installed `k3s server`;\n- On all nodes should be installed `k3s agent`;\n- All agents and controller should work from inside Docker container, for example with help of docker-compose;\n- Deployment of `docker-compose.yml` and other following files should be made via [Ansible](https://www.ansible.com/).\n\n## Installing Operating Systems\n\n### Installing OS to NanoPi NEO3\n\nOn MicroSD cards will be used Ubuntu 20.04 LTS _Focal Fossa_ (or recent).\n\nFirst download `Armbian_21.08.1_Nanopineo3_focal_current_5.10.60.img.xz` archive from:\n\nhttps://armbian.hosthatch.com/archive/nanopineo3/archive/\n\nThen extract an archive:\n\n```shell\nxz -d Armbian_21.08.1_Nanopineo3_focal_current_5.10.60.img.xz\n```\n\nThen connect MicroSD card and check name of device:\n\n```shell\n~$ sudo fdisk -l | grep 'model: Micro' -A 10 -B 2\n\nDisk /dev/sdX: 29,81 GiB, 32010928128 bytes, 62521344 sectors\nDisk model: Micro SD\nUnits: sectors of 1 * 512 = 512 bytes\nSector size (logical/physical): 512 bytes / 512 bytes\nI/O size (minimum/optimal): 512 bytes / 512 bytes\nDisklabel type: dos\nDisk identifier: 0x0000000\n```\n\n`/dev/sdX` is a device name of connected Micro SD card, your probably will be different.\n\nFlash the image to this card:\n\n```shell\ndd if=Armbian_21.08.1_Nanopineo3_focal_current_5.10.60.img of=/dev/sdX bs=1024k status=progress\n```\n\nThen wait some time.\n\n\u003e By default, size of created partition is about 800Mb, try to use `gparted` for increasing space to 32Gb\n\nAfter flashing, you may connect a card to NanoPi NEO3, then turn on the device.\n\nIn a few seconds device will receive and IP address if you have DHCP server in your network,\ntry to [connect via SSH](https://docs.armbian.com/User-Guide_Getting-Started/#how-to-login).\n\n* Username: `root`\n* Password: `1234`\n\nAfter first login NanoPi's prompt will ask you about a new password and some other things.\n\nNanoPi is ready for usage, congrats :)\n\n### Installing OS to Raspberry Pi CM3\n\nOn MicroSD cards will be used Ubuntu 20.04 LTS _Focal Fossa_ (or recent).\n\nNeed to install `rpi-imager` tool:\n\nhttps://github.com/raspberrypi/rpi-imager\n\n```shell\nsudo apt-get install rpi-imager\n```\n\nAfter need to run `rpi-image`, choose Ubuntu Server from **Other general purpose OS** section and select USB flash drive.\n\nYou also may preconfigure names of hosts, default login/pass and SSH key in settings of `rpi-imager`, for this just click to **cog** icon.\n\nAfter you ready click to **Write** button and wait some time.\n\n## Deploying K3S with help of Ansible\n\nHere will be described part about installing additional tools to controller and nodes, plus about deploy\nof docker-compose configs to machines.\n\nFirst need to install ansible tool via package manager:\n\n```shell\nsudo apt-get install ansible\n```\n\nAll commands below will be executed in `ansible` subfolder:\n\n```shell\ncd ansible\n```\n\nAll hosts and groups described in `inventory` file, copy from example, then change to your hosts:\n\n```shell\ncp inventory.dist inventory\nmcedit inventory\n```\n\nAlso need to set username with sudo permissions:\n\n```shell\ncp vars/user.dist.yml vars/user.yml\nmcedit vars/user.yml\n```\n\nDon't forget to change path to `id_rsa.pub` public key file.\n\n### Install all required packages on all servers\n\n\u003e playbook-default.yml\n\nIt's a default playbook, steps described in this file should be executed on all\nmachines of cluster, required sudo privileges.\n\n```shell\nansible-playbook -i inventory playbook-default.yml --ask-become-pass\n```\n\n### Deploy k3s server to controller\n\n\u003e playbook-controller.yml\n\nK3S server is a core of our project, it will execute management operations.\n\nSecond service here is Rancher web-interface.\n\n```shell\nansible-playbook -i inventory playbook-controller.yml\n```\n\nAfter executing this command API of Kubernetes cluster will be available on https://192.168.1.200:6443\n\n### Deploy k3s agent to nodes\n\n\u003e playbook-node.yml\n\nNow we just need add nodes to cluster controller, nodes will join to cluster automatically:\n\n```shell\nansible-playbook -i inventory playbook-node.yml\n```\n\n### Install and configure kubectl\n\nInstruction related to your OS [is here](https://kubernetes.io/docs/tasks/tools/).\n\nNeed to change directory to `~/.kube` and download kubeconfig from k8s-controller:\n\n```shell\n[ -d ~/.kube ] || mkdir ~/.kube\ncd ~/.kube\ncp config config.bak\nscp k8s-controller:/home/pasha/k3s-controller/output/kubeconfig.yaml config\n```\n\nTest installation:\n\n```shell\nkubectl get nodes\n```\n\nIn result will be something like this:\n```text\nNAME             STATUS   ROLES                  AGE   VERSION\nk8s-node4        Ready    \u003cnone\u003e                 31m   v1.22.11+k3s2\nk8s-node2        Ready    \u003cnone\u003e                 31m   v1.22.11+k3s2\nk8s-node3        Ready    \u003cnone\u003e                 31m   v1.22.11+k3s2\nk8s-node1        Ready    \u003cnone\u003e                 31m   v1.22.11+k3s2\nk8s-node6        Ready    \u003cnone\u003e                 31m   v1.22.11+k3s2\nk8s-node7        Ready    \u003cnone\u003e                 30m   v1.22.11+k3s2\nk8s-node5        Ready    \u003cnone\u003e                 31m   v1.22.11+k3s2\nk8s-controller   Ready    control-plane,master   32m   v1.22.11+k3s2\n```\n\n## How to enable Kubernetes Dashboard\n\nGo to folder with dashboard installation scripts: \n\n```shell\ncd k8s-dashboard\n```\n\nRun `install.sh` then wait couple minutes, after script is done run following command:\n\n```shell\nkubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard  9090:80\n```\n\nThen open http://localhost:9090 in browser.\n\n## Video blog about this project\n\nAll videos on Russian language.\n\n1. [Introduction and technical description](https://www.youtube.com/watch?v=jXRgqQrbKAo)\n2. [Installing operating systems](https://www.youtube.com/watch?v=A4kwBo6eRKE)\n3. [Deploy K3S with help of Ansible](https://www.youtube.com/watch?v=u_AuOKlqTNw)\n4. [Moving controller to Raspberry Pi 4B](https://www.youtube.com/watch?v=KBxG_zOksbY)\n5. Kubernetes Dashboard and GoCD server with agents in Kubernetes\n6. TBA...\n\n## Links\n\n* [Note about Proxmox and ARM virtual machine](PROXMOX.md)\n* https://medium.com/@tejaswi.goudru/disable-authentication-https-in-kubernetes-dashboard-2fada478ce91\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevilfreelancer%2Fkubernetes-cluster-on-arm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fevilfreelancer%2Fkubernetes-cluster-on-arm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevilfreelancer%2Fkubernetes-cluster-on-arm/lists"}