{"id":16927445,"url":"https://github.com/dirien/vcluster-webinar","last_synced_at":"2025-04-13T06:50:22.951Z","repository":{"id":44655532,"uuid":"451995806","full_name":"dirien/vcluster-webinar","owner":"dirien","description":"Demo Repository for vcluster Webinar","archived":false,"fork":false,"pushed_at":"2022-02-01T21:13:27.000Z","size":56,"stargazers_count":3,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-04T23:08:57.512Z","etag":null,"topics":["kubernetes","vcluster"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dirien.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-01-25T18:29:34.000Z","updated_at":"2023-02-07T20:26:02.000Z","dependencies_parsed_at":"2022-09-04T19:01:26.409Z","dependency_job_id":null,"html_url":"https://github.com/dirien/vcluster-webinar","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirien%2Fvcluster-webinar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirien%2Fvcluster-webinar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirien%2Fvcluster-webinar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dirien%2Fvcluster-webinar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dirien","download_url":"https://codeload.github.com/dirien/vcluster-webinar/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248675457,"owners_count":21143766,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kubernetes","vcluster"],"created_at":"2024-10-13T20:34:15.096Z","updated_at":"2025-04-13T06:50:22.930Z","avatar_url":"https://github.com/dirien.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003e This blog entry,was part of my webinar for the Cloud Native Islamabad meetup (27.01.2022)\n\n## TL;DR Code\n\nhttps://github.com/dirien/vcluster-webinar\n\n## Introduction\n\nIn this blog entry, I will show you how to create build a cloud native Kubernetes cluster vending machine with vcluster.\n\nIt's a wrapup of my webinar for the Cloud Native Islamabad meetup (27.01.2022). If you did not see it. You can watch it here:\n\n%[https://www.youtube.com/watch?v=GkaqN8urStg]\n\n## Prerequisites\n\n- You need to have a Scaleway account.\n- `kubectl` and `helm` cli must be installed.\n- `pulumi` cli must be installed, and you need to have a pulumi account to save the state there.\n- `task` cli must be installed (for using Taskfile.yaml as Makefile alternative).\n\nI will use Scaleway as the cloud provider for our managed Kubernetes cluster (called Kosmos)\n\n## Taskfile\n\nTask is a task runner / build tool that aims to be simpler and easier to use than, for example, GNU Make.\n\nOur taskfile looks like this:\n\n````yaml\n# https://taskfile.dev\n\nversion: '3'\n\ntasks:\n  workaround:\n    dir: workaround/kube-prometheus-stack-crds\n    env:\n      KUBECONFIG: ../../infrastructure/controlplane-scaleway/kubeconfig.yaml\n    cmds:\n      - cmd: kubectl delete -k .\n        ignore_error: true\n      - kubectl create -k .\n\n  applications:\n    dir: applications/\n    env:\n      KUBECONFIG: ../infrastructure/controlplane-scaleway/kubeconfig.yaml\n    cmds:\n      - kubectl apply -f applications.yaml\n\n  infrastructure:\n    dir: infrastructure/controlplane-scaleway\n    env:\n      KUBECONFIG: kubeconfig.yaml\n    cmds:\n      - pulumi up -y\n      - pulumi stack output kubeconfig --show-secrets \u003e kubeconfig.yaml\n\n  get-service-cidr:\n    env:\n      KUBECONFIG: infrastructure/controlplane-scaleway/kubeconfig.yaml\n    cmds:\n      - kubectl create service clusterip test --clusterip 1.1.1.1 --tcp=80:80\n````\n\n\n## Scaleway\n\nScaleway is a cloud provider with a variety of services. Scaleway Elements is their Cloud Platform offering, with\neverything you need to run your workload in the Cloud. From virtual machines to serverless functions, you will find a\nhuge range of services.\n\nYou can quickly register here to get an [account](https://console.scaleway.com/register).\n\n### Access And Secret Key\n\nGenerate a new access and secret key (https://console.scaleway.com/project/credentials)\n\nAnd export the keys as ENV variables:\n\n```\nexport ACCESS_KEY=xxx\nexport SECRET_KEY=yyy\nexport ORGANISATION_ID=zzz\n```\n\nSee https://www.scaleway.com/en/docs/generate-api-keys/ for even more details on how Scaleway Credentials work.\n\n## Pulumi\n\nPulumi is an open source infrastructure as code tool for creating, deploying, and managing cloud infrastructure. Pulumi\nworks with traditional infrastructure like VMs, networks, and databases, in addition to modern architectures, including\ncontainers, Kubernetes clusters, and serverless functions. Pulumi supports dozens of public, private, and hybrid cloud\nservice providers.\n\nIn this tutorial, we will use for example golang as our programming language.\n\n### Install Pulumi\n\nInstalling Pulumi is easy, just head over to the [get-stated](https://www.pulumi.com/docs/get-started/install/) website\nand chose the appropriate version and way to download the cli. To store your state files, you can use their\nfree [SaaS](https://app.pulumi.com/signin?reason=401) offering\n\n## Create The Pulumi Infrastructure Program\n\nLet us start with the main Pulumi program. Just create run following commands in your terminal inside your project\nfolder (I use `infrastructure` as the folder):\n\n```bash\npulumi new go\n```\n\nFollow the instructions to create a new project with golang as programing language and fill in the required information.\n\nPulumi offers plenty of templates, if you are unsure of what to use, just use type following command and choose from the\nhuge collection of templates:\n\n```bash\npulumi new\nPlease choose a template:  [Use arrows to move, enter to select, type to filter]\naws-csharp                   A minimal AWS C# Pulumi program\naws-go                       A minimal AWS Go Pulumi program\naws-javascript               A minimal AWS JavaScript Pulumi program\naws-python                   A minimal AWS Python Pulumi program\naws-typescript               A minimal AWS TypeScript Pulumi program\nazure-csharp                 A minimal Azure Native C# Pulumi program\nazure-go                     A minimal Azure Native Go Pulumi program\nazure-javascript             A minimal JavaScript Pulumi program with the native Azure provider\nazure-python                 A minimal Azure Native Python Pulumi program\n....\n```\n\n## Add The Scaleway Provider\n\nThe Scaleway provider binary is a third party binary. It can be installed using the pulumi plugin command.\n\n```bash\npulumi plugin install resource scaleway v0.1.8 --server https://dl.briggs.work/pulumi/releases/plugins\n```\n\nThen you can add the go module via:\n\n```bash\ngo get github.com/jaxxstorm/pulumi-scaleway/sdk/go/scaleway\n```\n\nAnd finally we add the Scaleway Credentials via following commands to the Pulumi config:\n\n```bash\npulumi config set scaleway:access_key YYYY --secret\npulumi config set scaleway:secret_key ZZZZ --secret\n```\n\nMore details -\u003e https://www.pulumi.com/registry/packages/scaleway/\n\n## The Scaleway Infrastructure Deployment\n\nThe deployment of the Scaleway infrastructure is really the bare minimum. We want to focus more on the deployment of the `vcluster`.\n\n```go\ncluster, err := scaleway.NewKubernetesCluster(ctx, \"vcluster-webinar\", \u0026scaleway.KubernetesClusterArgs{\nName:    pulumi.String(\"vcluster-webinar\"),\nVersion: pulumi.String(\"1.23\"),\nRegion:  pulumi.String(\"fr-par\"),\nCni:     pulumi.String(\"cilium\"),\nFeatureGates: pulumi.StringArray{\npulumi.String(\"HPAScaleToZero\"),\n},\nTags: pulumi.StringArray{\npulumi.String(\"pulumi\"),\n},\nAutoUpgrade: \u0026scaleway.KubernetesClusterAutoUpgradeArgs{\nEnable:                     pulumi.Bool(true),\nMaintenanceWindowStartHour: pulumi.Int(3),\nMaintenanceWindowDay:       pulumi.String(\"sunday\"),\n},\nAdmissionPlugins: pulumi.StringArray{\npulumi.String(\"AlwaysPullImages\"),\n},\n})\nif err != nil {\nreturn err\n}\npool, err := scaleway.NewKubernetesNodePool(ctx, \"vcluster-webinar-pool\", \u0026scaleway.KubernetesNodePoolArgs{\nZone:        pulumi.String(\"fr-par-1\"),\nName:        pulumi.String(\"vcluster-webinar-pool\"),\nNodeType:    pulumi.String(\"DEV1-L\"),\nSize:        pulumi.Int(1),\nAutoscaling: pulumi.Bool(true),\nMinSize:     pulumi.Int(1),\nMaxSize:     pulumi.Int(3),\nAutohealing: pulumi.Bool(true),\nClusterId:   cluster.ID(),\n})\nif err != nil {\nreturn err\n}\nctx.Export(\"cluster_id\", cluster.ID())\nctx.Export(\"kubeconfig\", pulumi.ToSecret(cluster.Kubeconfigs.Index(pulumi.Int(0)).ConfigFile()))\n```\n\nNothing special here, we create the cluster in the `fr-par` zone, with the `DEV1-L` node type. To be on the safe side, I activate the Autoscaling feature, and set the minimum and maximum size to 1 and 3. You can adjust this for your needs.\n\nYou can deploy the cluster using the following command:\n\n```bash\ntask infrastructure\n```\n\n## The Minimum ArgoCD Installation\n\nNow comes the funny part, we're going to install ArgoCD on the cluster. But just the bare minimum we need to. We want to deploy everything else via GitOps. Including the ArgoCD.\n\n```go\nprovider, err := kubernetes.NewProvider(ctx, \"kubernetes\", \u0026kubernetes.ProviderArgs{\nKubeconfig: cluster.Kubeconfigs.Index(pulumi.Int(0)).ConfigFile(),\n}, pulumi.Parent(pool))\n\ndep := \u0026ProviderDependency{\nctx:      ctx,\nprovider: provider,\n}\nif err != nil {\nreturn err\n}\nerr = dep.createExternalDns()\nif err != nil {\nreturn err\n}\nerr = dep.createArgoCD()\nif err != nil {\nreturn err\n}\nreturn nil\n```\n\nThe function `createExternalDns` is to install the Scaleway secrets as Kubernetes secret via the Pulumi way. So no need for an extra\nvault solution.\n\nThe next function `createArgoCD` is to deploy the ArgoCD on the cluster. But really the bare minimum we need to.\n\n```go\nargocdNS, err := v1.NewNamespace(p.ctx, \"argocd\", \u0026v1.NamespaceArgs{\nMetadata: \u0026metav1.ObjectMetaArgs{\nName: pulumi.String(\"argocd\"),\n},\n}, pulumi.Provider(p.provider))\nif err != nil {\nreturn err\n}\n_, err = helm.NewRelease(p.ctx, \"argocd-helm\", \u0026helm.ReleaseArgs{\nName:      pulumi.String(\"argocd\"),\nChart:     pulumi.String(\"argo-cd\"),\nVersion:   pulumi.String(\"3.29.5\"),\nNamespace: argocdNS.Metadata.Name(),\nRepositoryOpts: helm.RepositoryOptsArgs{\nRepo: pulumi.String(\"https://argoproj.github.io/argo-helm\"),\n},\nValues: pulumi.Map{\n\"controller\": pulumi.Map{\n\"serviceAccount\": pulumi.Map{\n\"automountServiceAccountToken\": pulumi.Bool(true),\n},\n},\n\"dex\": pulumi.Map{\n\"enabled\": pulumi.Bool(false),\n\"serviceAccount\": pulumi.Map{\n\"automountServiceAccountToken\": pulumi.Bool(true),\n},\n},\n\"redis\": pulumi.Map{\n\"serviceAccount\": pulumi.Map{\n\"automountServiceAccountToken\": pulumi.Bool(true),\n},\n},\n\"server\": pulumi.Map{\n\"config\": pulumi.Map{\n\"url\": pulumi.String(\"https://argocd.ediri.cloud\"),\n},\n\"extraArgs\": pulumi.Array{\npulumi.String(\"--insecure\"),\n},\n\"ingress\": pulumi.Map{\n\"ingressClassName\": pulumi.String(\"nginx\"),\n\"hosts\": pulumi.Array{\npulumi.String(\"argocd.ediri.cloud\"),\n},\n\"enabled\": pulumi.Bool(true),\n\"annotations\": pulumi.Map{\n\"external-dns.alpha.kubernetes.io/hostname\": pulumi.String(\"argocd.ediri.cloud\"),\n\"external-dns.alpha.kubernetes.io/ttl\":      pulumi.String(\"60\"),\n},\n},\n},\n\"repoServer\": pulumi.Map{\n\"serviceAccount\": pulumi.Map{\n\"automountServiceAccountToken\": pulumi.Bool(true),\n},\n},\n},\n}, pulumi.Provider(p.provider), pulumi.Parent(argocdNS))\n```\n\n## Deploy The Rest Of The Application, Using GitOps\n\nNow you can call:\n\n```bash\ntask applications\n```\n\nthis will deploy the missing applications, via GitOps through ArgoCD. We define the ArgoCD application like this:\n\n````yaml\napiVersion: argoproj.io/v1alpha1\nkind: Application\nmetadata:\n  name: applications\n  namespace: argocd\n  annotations:\n    argocd.argoproj.io/sync-wave: \"99\"\nspec:\n  destination:\n    name: in-cluster\n    namespace: argocd\n  project: default\n  source:\n    path: applications\n    repoURL: https://github.com/dirien/vcluster-webinar.git\n    targetRevision: main\n  syncPolicy:\n    automated:\n      prune: true\n      selfHeal: true\n````\n\nPointing to our `kustomization.yaml` file in the `applications` directory.\n\nThere we composite the applications, we want to deploy. You could have a separate GItOps repository for this too.\n\nWe install following applications:\n\n- argocd\n- cert-manager\n- external-dns\n- ingress-nginx\n- kube_prometheus_stack\n- vcluster\n\n\u003e Note: To use Ingress for the `vcluster`, we need to enable passthrough-mode in the `ingress-nginx` Helm chart.\n\n````yaml\n...\nhelm:\n  values: |-\n    ...\n    controller:\n      extraArgs:\n        enable-ssl-passthrough: \"\"\n      ...\nchart: ingress-nginx\n...\n````\n\nWe follow here the `App of Apps` approach. You can find more about this interesting pattern here -\u003e https://argo-cd.readthedocs.io/en/stable/operator-manual/cluster-bootstrapping/#app-of-apps-pattern\n\n![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1643747987777/Lkw3CtCu5j.png)\n\n### ArgoCD waves\n\nIt's important here to mention, that we are using the `argocd.argoproj.io/sync-wave` annotation to define the different waves.\n\nThis is important, because we want to deploy the applications in the right order. This important, if we have some dependencies between the applications.\nFor example Prometheus with the ServiceMonitor.\n\nTo define the waves, we use the `argocd.argoproj.io/sync-wave` annotation and chose the wave number. The wave number is the order in which the applications will be deployed.\n\n```yaml\nmetadata:\n  name: kube-prometheus-stack\n  annotations:\n    argocd.argoproj.io/sync-wave: \"1\"\n```\n\n## vcluster\n\nVirtual clusters are fully working Kubernetes clusters that run on top of other Kubernetes clusters. Compared to fully separate \"real\" clusters, virtual clusters do not have their own node pools. Instead, they are scheduling workloads inside the underlying cluster while having their own separate control plane.\n\nIf you get Inception vibes, welcome to the party!\n![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1643749164726/GUnnylnMH.png)\n\nIf you want to know more, I recomend to watch this latest session of `Containers from the Couch` with Justin, Rich and Lukas\n\n%[https://www.youtube.com/watch?v=a8fIyUd9438]\n\nI created a folder called `vcluster`. And here we are going to use the `App of apps` approach again. You probably spotted the\n`vcluster` application in the `applications` folder. Here we are pointing to the `kustomization.yaml` file, this time in the `vcluster` folder.\n\n````yaml\napiVersion: argoproj.io/v1alpha1\nkind: Application\nmetadata:\n  name: vcluster\n  annotations:\n    argocd.argoproj.io/sync-wave: \"99\"\nspec:\n  destination:\n    name: in-cluster\n    namespace: argocd\n  project: default\n  source:\n    path: vcluster\n    repoURL: https://github.com/dirien/vcluster-webinar.git\n    targetRevision: main\n  syncPolicy:\n    automated:\n      prune: true\n      selfHeal: true\n````\n\n![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1643748035435/YmUHobM2s.png)\n\nInside the `vcluster` folder we define different ArgoCD applications. Depending on the type of `vcluster` we want to deploy.\nWe use `kusomization.yaml` again to glue the ArgoCD applications together. The structure of the folders is completely up to you.\n\n\nCurrently `vcluster` needs, when installed via `helm` the CIDR range provided by the `serviceCIDR` flag.\n\nTo get the CIDR range, we use following target in our `taskfile`:\n\n```bash\ntask get-service-cidr\n\nerror: failed to create ClusterIP service: Service \"test\" is invalid: spec.clusterIPs: Invalid value: []string{\"1.1.1.1\"}: \nfailed to allocate IP 1.1.1.1: the provided IP (1.1.1.1) is not in the valid range. The range of valid IPs is 10.32.0.0/12\n```\n\nThe valid IP range will be displayed in the `taskfile` output. Here it is `10.32.0.0/12`\n\nHere an example of a `vcluster` application, using `k0s`:\n\n````yaml\napiVersion: argoproj.io/v1alpha1\nkind: Application\nmetadata:\n  name: team-4\n  labels:\n    team: team-4\n    department: development\n    project: backend\n    distro: k3s\n    version: v1.21.4-k3s1\n  annotations:\n    argocd.argoproj.io/sync-wave: \"99\"\nspec:\n  destination:\n    name: in-cluster\n    namespace: team-4\n  project: default\n  source:\n    repoURL: 'https://charts.loft.sh'\n    targetRevision: 0.6.0-alpha.7\n    helm:\n      values: |-\n        rbac:\n          clusterRole:\n            create: true\n          role:\n            create: true\n            extended: true\n        syncer:\n          extraArgs:\n            - --tls-san=team-4.ediri.cloud\n            - --out-kube-config-server=https://team-4.ediri.cloud\n        serviceCIDR: 10.32.0.0/12\n        ingress:\n          enabled: true\n          host: team-4.ediri.cloud\n          annotations:\n            external-dns.alpha.kubernetes.io/hostname: team-4.ediri.cloud\n            external-dns.alpha.kubernetes.io/ttl: \"60\"\n    chart: vcluster\n  syncPolicy:\n    syncOptions:\n      - CreateNamespace=true\n````\n\n![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1643748121500/XtoGzSoAv.png)\n\nAs we deployed an ingress controller and external DNS, we can use this to access the `vcluster`.\n\nThis is absolut brilliant, as we can now order via git pull requests new cluster.\n\n### Accessing the vcluster\n\nI am going to use the `vcluster` cli here. But you could also get the `kubeconfig` from your Ops Team\n\n```bash\nvcluster connect team-4 -n team-4 --server=https://team-4.ediri.cloud\n[done] √ Virtual cluster kube config written to: ./kubeconfig.yaml. You can access the cluster via `kubectl --kubeconfig ./kubeconfig.yaml get namespaces`\n```\n\n![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1643749033377/8WqkxazQX.png)\n\nNow we can access the cluster as usual, with the `kubectl` command.\n\n```bash\nkubectl --kubeconfig ./kubeconfig.yaml get namespaces\nNAME              STATUS   AGE\ndefault           Active   5d1h\nkube-system       Active   5d1h\nkube-public       Active   5d1h\nkube-node-lease   Active   5d1h\n```\n\nOr schedule our workload:\n\n```bash\nkubectl run nginx --image=nginx\npod/nginx created\n\nkubectl port-forward pod/nginx 8080:80\nForwarding from 127.0.0.1:8080 -\u003e 80\nForwarding from [::1]:8080 -\u003e 80\nHandling connection for 8080\n\n\ncurl localhost:8080\n\u003c!DOCTYPE html\u003e\n\u003chtml\u003e\n\u003chead\u003e\n\u003ctitle\u003eWelcome to nginx!\u003c/title\u003e\n\u003cstyle\u003e\nhtml { color-scheme: light dark; }\nbody { width: 35em; margin: 0 auto;\nfont-family: Tahoma, Verdana, Arial, sans-serif; }\n\u003c/style\u003e\n\u003c/head\u003e\n\u003cbody\u003e\n\u003ch1\u003eWelcome to nginx!\u003c/h1\u003e\n\u003cp\u003eIf you see this page, the nginx web server is successfully installed and\nworking. Further configuration is required.\u003c/p\u003e\n\n\u003cp\u003eFor online documentation and support please refer to\n\u003ca href=\"http://nginx.org/\"\u003enginx.org\u003c/a\u003e.\u003cbr/\u003e\nCommercial support is available at\n\u003ca href=\"http://nginx.com/\"\u003enginx.com\u003c/a\u003e.\u003c/p\u003e\n\n\u003cp\u003e\u003cem\u003eThank you for using nginx.\u003c/em\u003e\u003c/p\u003e\n\u003c/body\u003e\n\u003c/html\u003e\n```\n\n## Monitoring\n\nWith our monitoring stack, we can monitor our `vcluster`, very comfortably. Just head over to the Grafana and browse the dashboard you need.\n\n![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1643748267309/mCJEsGKTT.png)\n\n## Some links\n\n- https://github.com/dirien/vcluster-webinar\n- https://www.vcluster.com/\n- https://taskfile.dev/\n- https://argoproj.github.io/argo-cd/\n- https://www.scaleway.com/\n\n![image.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1643749091015/wzybDIS81.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdirien%2Fvcluster-webinar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdirien%2Fvcluster-webinar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdirien%2Fvcluster-webinar/lists"}