{"id":15056953,"url":"https://github.com/jugatsu/cilium-cluster-mesh-workshop","last_synced_at":"2025-04-15T15:12:43.226Z","repository":{"id":240184794,"uuid":"798399413","full_name":"jugatsu/cilium-cluster-mesh-workshop","owner":"jugatsu","description":"🔗 Use Cilium Cluster Mesh for multi-cluster networking.","archived":false,"fork":false,"pushed_at":"2024-05-17T05:21:33.000Z","size":5,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T15:12:33.145Z","etag":null,"topics":["cilium","cilium-cluster-mesh","colima","ebpf","kind","kubernetes"],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jugatsu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-09T17:41:34.000Z","updated_at":"2025-01-13T03:36:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"097243af-39d2-42de-aa4f-f7a8aa571706","html_url":"https://github.com/jugatsu/cilium-cluster-mesh-workshop","commit_stats":null,"previous_names":["jugatsu/cilium-cluster-mesh-workshop"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jugatsu%2Fcilium-cluster-mesh-workshop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jugatsu%2Fcilium-cluster-mesh-workshop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jugatsu%2Fcilium-cluster-mesh-workshop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jugatsu%2Fcilium-cluster-mesh-workshop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jugatsu","download_url":"https://codeload.github.com/jugatsu/cilium-cluster-mesh-workshop/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249094933,"owners_count":21211837,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cilium","cilium-cluster-mesh","colima","ebpf","kind","kubernetes"],"created_at":"2024-09-24T21:59:12.995Z","updated_at":"2025-04-15T15:12:43.208Z","avatar_url":"https://github.com/jugatsu.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cilium Cluster Mesh Workshop\nLearn [Cilium](https://cilium.io/) and [Cilium Cluster Mesh](https://docs.cilium.io/en/stable/network/clustermesh/clustermesh/) by creating a Multi-cluster Kubernetes setup.   \n\n## ✅ System requirements\n\u003e [!WARNING]\n\u003e For MacOS users it is recommended to use [colima](https://github.com/abiosoft/colima) instead of Docker Desktop. See https://github.com/cilium/cilium/issues/30278\n* `colima` (optional for MacOS users) - [Installation instructions](https://github.com/abiosoft/colima?tab=readme-ov-file#installation)\n* `docker` - [Installation instructions](https://docs.docker.com/engine/install)\n* `kind` - [Installation instructions](https://kind.sigs.k8s.io/docs/user/quick-start)\n* `kubectl` - [Installation instructions](https://kubernetes.io/docs/tasks/tools)\n* `cilium-cli` - [Installation instructions](https://github.com/cilium/cilium-cli?tab=readme-ov-file#installation)\n\n### Some notes on `colima` configuration\nEnsure `inotify` setting are [correct](https://github.com/abiosoft/colima/issues/1000):\n```sh\ncp colima/override.yaml ~/.colima/_lima/_config\n```\nStart a virtual machine with 4 cpu and 8 Gb of RAM:\n```sh\ncolima start --cpu 4 --memory 8\n```\n\n## ⚡️ Quick start\nPrepare environment with just one command:\n```bash\nmake all\n```\nThis command can help you with:\n* Create a setup of three Kubernetes clusters using `kind`;\n* Install Cilium CNI in all three clusters using `cilium-cli`;\n* Create Cluster Mesh between all three clusters;\n* Install demo apps in all three clusters.\n\u003e [!TIP]\n\u003e Feel free to check out **Makefile**. It is self-explanatory 😉\n\n## 📖 Play around with Cilium Cluster Mesh\n### Scenario 1: Send http requests to `rebel-base` service from Cluster 1\n\nWe can observe that all requests are load balanced between Cluster 1 and Cluster 2.\n\n```sh\nkubectl --context kind-cluster1 exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'\n```\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eOutput\u003c/b\u003e\u003c/summary\u003e\n\n```json\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n```\n\u003c/details\u003e\n\n### Scenario 2: Send http requests to `rebel-base` service from Cluster 2\n\nSimilarly to scenario above all requests are load balanced between Cluster 2 and Cluster 1.\n```sh\nkubectl --context kind-cluster2 exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'\n```\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eOutput\u003c/b\u003e\u003c/summary\u003e\n\n```json\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n```\n\u003c/details\u003e\n\n### Scenario 3: Use local service affinity for Cluster 1\nSet `service.cilium.io/affinity` annotation for `local`, the Global Service will load-balance across healthy local backends, and only use remote endpoints if all of local backends are not available or unhealthy.\n```sh\nkubectl --context kind-cluster1 annotate service rebel-base io.cilium/service-affinity=local --overwrite\n```\nCheck the destination. As you can see the preferred endpoint destination is Cluster 1 (`local` endpoints).\n```sh\nkubectl --context kind-cluster1 exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'\n```\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eOutput\u003c/b\u003e\u003c/summary\u003e\n\n```json\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n```\n\u003c/details\u003e\n\nRemove `service.cilium.io/affinity` annotation.\n```sh\nkubectl --context kind-cluster1 annotate service rebel-base io.cilium/service-affinity-\n```\nYou will see replies from pods in both clusters as usual.\n```sh\nkubectl --context kind-cluster1 exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'\n```\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eOutput\u003c/b\u003e\u003c/summary\u003e\n\n```json\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n```\n\n\u003c/details\u003e\n\n\u003e [!IMPORTANT]\n\u003e Try to use `service.cilium.io/affinity` with `remote` configuration in Cluster 2 and observe the results.\n\n\u003e [!TIP]\n\u003e Documentation for Global Service Affinity - https://docs.cilium.io/en/latest/network/clustermesh/affinity/\n\n### Scenario 4: Send http requests to `rebel-base` service from Cluster 3\nThat's the fun part. In Cluster 3 we don't have any deployments for `rebel-base`. But we can use global `rebel-base` service to reach pods in Cluster 1 and Cluster 2.\n\nLet's try it out. We got responses from Cluster 1 and Cluster 2.\n\n```sh\nkubectl --context kind-cluster3 exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'\n```\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eOutput\u003c/b\u003e\u003c/summary\u003e\n\n```json\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-1\"}\n```\n\u003c/details\u003e\n\n\u003e [!NOTE]\n\u003e [Feature proposal](https://github.com/cilium/cilium/issues/29200) for global service cluster affinity.\n\n### Scenario 5: High Availability and Fault Tolerance\nLet's try to make service temporarily unavailable in Cluster 1 and observe failover to Cluster 2.\n```sh\nkubectl --context kind-cluster1 scale deploy --replicas=0 rebel-base\n```\nSend some http requests to `rebel-base` in Cluster 1, see failover in action.\n```sh\nkubectl --context kind-cluster1 exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'\n```\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eOutput\u003c/b\u003e\u003c/summary\u003e\n\n```json\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n```\n\u003c/details\u003e\n\nCheck `rebel-base` service from Cluster 2.\n```sh\nkubectl --context kind-cluster2 exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'\n```\n\u003cdetails\u003e\u003csummary\u003e\u003cb\u003eOutput\u003c/b\u003e\u003c/summary\u003e\n\n```json\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n{\"Galaxy\": \"Alderaan\", \"Cluster\": \"Cluster-2\"}\n```\n\u003c/details\u003e\n\nReturn `rebel-base` deployment in Cluster 1 to normal operation.\n```sh\nkubectl --context kind-cluster1 scale deploy --replicas=2 rebel-base\n```\n\u003e [!IMPORTANT]\n\u003e Try scale up/down `rebel-base` deployment in Cluster 2 and observe the result.\n\n## 🗑️ Cleanup\n\n```sh\nmake clean\n```\n\nIt will delete all three clusters.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjugatsu%2Fcilium-cluster-mesh-workshop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjugatsu%2Fcilium-cluster-mesh-workshop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjugatsu%2Fcilium-cluster-mesh-workshop/lists"}