{"id":17953965,"url":"https://github.com/embano1/kubeconeu22","last_synced_at":"2026-04-16T00:31:49.212Z","repository":{"id":103514046,"uuid":"486529898","full_name":"embano1/kubeconeu22","owner":"embano1","description":"vSphere Event-Driven Tutorial with Knative and VEBA presented at Kubecon EU 2022","archived":false,"fork":false,"pushed_at":"2024-03-06T20:28:09.000Z","size":110,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-25T14:35:42.422Z","etag":null,"topics":["eventing","knative","kubernetes","veba","vmware"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/embano1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-28T09:32:30.000Z","updated_at":"2022-05-13T13:12:57.000Z","dependencies_parsed_at":"2024-06-20T21:41:32.025Z","dependency_job_id":"0f457de5-5398-4925-845b-6ea9e23adabe","html_url":"https://github.com/embano1/kubeconeu22","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/embano1/kubeconeu22","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embano1%2Fkubeconeu22","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embano1%2Fkubeconeu22/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embano1%2Fkubeconeu22/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embano1%2Fkubeconeu22/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/embano1","download_url":"https://codeload.github.com/embano1/kubeconeu22/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embano1%2Fkubeconeu22/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31866262,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"ssl_error","status_checked_at":"2026-04-15T15:24:39.138Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["eventing","knative","kubernetes","veba","vmware"],"created_at":"2024-10-29T10:08:07.335Z","updated_at":"2026-04-16T00:31:49.190Z","avatar_url":"https://github.com/embano1.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tutorial\n\nInstallation instructions and step-by-step examples for the Kubecon 2022 EU\nsession [`Optimize Kubernetes on vSphere with Event-Driven\nAutomation`](https://kccnceu2022.sched.com/event/yttR/optimize-kubernetes-on-vsphere-with-event-driven-automation-steven-wong-michael-gasch-vmware?iframe=no).\n\nThis tutorial uses and deploys the components of the [VMware Event Broker\nAppliance](https://vmweventbroker.io/) (VEBA) directly on a Kubernetes\nenvironment to *create reactive and event-driven solutions for common\noperational use cases*, e.g. notification to 3rd party systems like Slack.\n\nThe solutions described here are meant to serve as an inspiration and starting\npoint. Because with VEBA, you are only limited by your imagination ;)\n\n⚠️ Make sure to follow the [installation](#installing-required-components)\nsection before proceeding with the following scenarios.\n\nThis demo is tested with `vmware-veba/event-router v0.7.3` and Knative `v1.2.0`.\n\n## Scenario: Slack notification on Tags/Label drift\n\nVMware vSphere `categories` and `tags` are often used to identify and group\nrelated resources, e.g. virtual machines running on the same\nhosts/rack/datacenter.\n\nKubernetes can leverage vSphere `tags` and automatically [map\nthem](https://cloud-provider-vsphere.sigs.k8s.io/tutorials/deploying_cpi_with_multi_dc_vc_aka_zones.html)\nto topology `labels`, e.g. `topology.kubernetes.io/zone`.\n\nMapping vSphere `tags` to Kubernetes `labels` ensures that both runtimes\n(schedulers) have the same understanding of the underlying infrastructure and\ntopology.\n\nHowever, due to the dynamic nature of a vSphere environment (vSphere DRS and\nvSphere [proactive] HA), these mappings can get out of sync, often requiring\nmanual operator intervention.\n\n### Solution\n\nDeploy a function which sends a Slack notification when a virtual machine is not\nrunning on the preferred host (i.e. `tag` mismatch).\n\n🎥 [Video walkthrough (Youtube)](https://www.youtube.com/watch?v=2lilm9RTBdM)\n\n![](https://i.ytimg.com/vi_webp/2lilm9RTBdM/maxresdefault.webp)\n\n\n### Setup the Scenario\n\nIf you haven't already done so, set up Slack Incoming Webhooks\n([documentation](https://api.slack.com/messaging/webhooks)).\n\nCreate the appropriate vSphere zone `category` and associated `tags` (if not\nalready done so). This example uses the category `k8s-zone` and two zone `tags`\n(`k8s-zone-eu-1` and `k8s-zone-eu-2`). This can be done with\n[`govc`](https://github.com/vmware/govmomi/tree/master/govc) or through the\nvCenter UI.\n\n```bash\ngovc tags.category.create -d \"Kubernetes zone\" k8s-zone\ngovc tags.create -d \"Kubernetes Zone EU-1\" -c k8s-zone k8s-zone-eu-1\ngovc tags.create -d \"Kubernetes Zone EU-2\" -c k8s-zone k8s-zone-eu-2\n```\n\nAssign the `tags` to your virtual machines and hosts accordingly. For example,\nbelow you can see that `tags` `k8s-zone-eu-1` and `k8s-zone-eu-2` are attached\nto two hosts and one virtual machine each.\n\n```bash\n# EU-1\ngovc tags.attached.ls k8s-zone-eu-1\nHostSystem:host-33\nHostSystem:host-21\nVirtualMachine:vm-56\n\n# EU-2\ngovc tags.attached.ls k8s-zone-eu-2\nHostSystem:host-15\nHostSystem:host-27\nVirtualMachine:vm-57\n```\n\n⚠️ This example function does not support `tags` on objects other than virtual\nmachines and hosts. See the hacking section\n[below](#food-for-further-customization-hacking) for details.\n\nCreate a Kubernetes secret to store the Slack Webhook URL (token).\n\n```bash\nkubectl -n vmware-functions create secret generic slack-credentials --from-literal=token='https://your-token-value...'\n```\n\nCreate the Kubernetes `vsphere-credentials` secret which will be used by the\nfunction to connect to the vCenter Server to retrieve tag assignments.\n\n```bash\nkubectl -n vmware-functions create secret generic vsphere-credentials --from-literal=username='vcenter-username' --from-literal=password='vcenter-password'\n```\n\nConfigure the function. Open the [function.yaml](tagdrift-fn/function.yaml) and\nchange the following values.\n\n```yaml\n- name: VCENTER_URL\n  value: \"https://your-vcenter...\"\n- name: VCENTER_INSECURE\n  value: \"true\" # will not panic on self-signed certificates\n- name: CATEGORY\n  value: \"k8s-zone\" # change this if your zone category has a different name\n```\n\nSave and close the YAML file in the `tagdrift-fn` folder. Deploy the function.\n\n```bash\ncd tagdrift-fn\nkubectl -n vmware-functions create -f function.yaml\nkubectl -n vmware-functions wait --timeout=3m --for=condition=Ready ksvc/tagdrift-fn\n```\n\nInspect the logs that the function is correctly running.\n\n```bash\nkubectl -n vmware-functions logs deploy/tagdrift-fn-00001-deployment  -c user-container\n2022-05-12T12:13:08.475Z        INFO    tagdrift        tagdrift-fn/main.go:55  starting tagdrift function      {\"commit\": \"862f6a31\", \"listenPort\": 8080, \"debug\": true}\n```\n\nIn case the function is not starting (pending), panicking (`CrashLoopBackoff`)\nor throwing warnings/errors make sure that:\n\n- The Kubernetes Slack `secret` was created as described above before creating the function\n- The vSphere configuration values are correct\n- You have sufficient permissions to connect to vCenter and send to Slack\n\nIf the function is working correctly, the following Slack message will be\nposted.\n\n![](static/slack-drift.png)\n\n### Build the Function\n\n⚠️ This is only required if you made changes to the Go code.\n\nThe following descriptions assume a `kind` installation. Change the environment\nvalues accordingly (also if you are not using `kind`).\n\n```bash\nexport KIND_CLUSTER_NAME=kubeconeu22\nexport KO_DOCKER_REPO=kind.local\nexport KO_COMMIT=$(git rev-parse --short=8 HEAD)\n```\n\nThen build and push/load the image with [`ko`](https://github.com/google/ko).\n\n```bash\ncd tagdrift-fn\nko publish -B .\n```\n\n\n#### Food for further customization (hacking):\n\n- Extend the function to retrieve VM `zone` object tags from higher-level\n  objects, e.g. cluster/folder\n- Send the notification to another system, e.g. Teams or PagerDuty\n- Create a trigger for an HA event\n- Extend the integration by having a Slack bot take action to migrate the VM\n  back to a preferred host/site\n\n\n## Scenario: Slack notification on a critical vSphere Alarm\n\nVMware vSphere alarms are an important asset in the DevOps toolbox to\nproactively notify an operator on resource starvation or exhaustion.\n\nBut as of today, these alarms (managed in VMware vCenter Server) are also\nlimited:\n\n- Limited extensibility (send to Slack, anyone?)\n- Security and stability concerns:\n  - requires vCenter access to manage\n  - custom actions only possible with scripts on the vCenter server appliance\n- Resource concerns:\n  - custom solutions often use polling and increase coupling between vCenter and\n    the integration\n- When using out of the box events (push instead of pull), the `AlarmEvent`\n  schema lacks important alarm details\n\n### Solution\n\nDeploy a multi-step event pipeline using the [vSphere Alarm\nServer](https://github.com/embano1/vsphere-alarm-server) to first enrich\nincoming vSphere `AlarmStatusChangedEvents` with critical data and then send\nthem in parallel to the event viewer (Sockeye) and Slack if the severity is\n`yellow` or `red`.\n\n🎥 [Video walkthrough (Youtube)](https://www.youtube.com/watch?v=zm0aAX4suqI)\n\n![](https://i.ytimg.com/vi_webp/zm0aAX4suqI/maxresdefault.webp)\n\n### Setup the Scenario\n\nIf you haven't already done so, set up Slack Incoming Webhooks\n([documentation](https://api.slack.com/messaging/webhooks)).\n\nNext, clone the VMware Event Broker Appliance Github repository which contains\nthe functions and demo code.\n\n```bash\n# from within the current kubeconeu22 git repository\ngit clone https://github.com/vmware-samples/vcenter-event-broker-appliance scenarios\ncd scenarios/examples/knative/powershell/kn-ps-slack-vsphere-alarm\n```\n\n#### Create Secrets\n\nEdit the file `slack_secret.json` and enter your Slack Webhook token URL.\n\nCreate the Kubernetes secret.\n\n```bash\nkubectl -n vmware-functions create secret generic slack-secret --from-file=SLACK_SECRET=slack_secret.json\n```\n\nCreate the Kubernetes `vsphere-credentials` secret which will be used by the\nvSphere Alarm Service to connect to the vCenter Server to retrieve the\nadditional information for a given vSphere Alarm event.\n\n💡 If you already created this secret, skip this step.\n\n```bash\nkubectl -n vmware-functions create secret generic vsphere-credentials --from-literal=username='vcenter-username' --from-literal=password='vcenter-password'\n```\n\n#### Deploy the vSphere Alarm Server\n\nThe vSphere Alarm Server is responsible for enriching (patching) vSphere alarm\nevents with useful alarm details, e.g. thresholds.\n\nDownload the latest deployment manifest for the [vSphere Alarm\nServer](https://github.com/embano1/vsphere-alarm-server).\n\n```bash\ncurl -L -O https://github.com/embano1/vsphere-alarm-server/releases/latest/download/release.yaml\n```\n\nEdit the `release.yaml` file and update the `VCENTER_URL` to match your vCenter\nServer instance. You may also need to update the value of `VCENTER_INSECURE` if\na self-signed TLS certificate is being used by the vCenter Server.\n\nOptionally, enable verbose logging by setting the `DEBUG` variable in the\nmanifest to `\"true\"`.\n\nDeploy the vSphere Alarm Server.\n\n```bash\nkubectl -n vmware-functions apply -f release.yaml\nkubectl -n vmware-functions wait --timeout=3m --for=condition=Available deploy/vsphere-alarm-server\n```\n\n#### Deploy the Alarm Function\n\nDeploy the Slack notification function and associated components.\n\n```bash\nkubectl -n vmware-functions apply -f function.yaml\n```\n\n#### Trigger an Alarm\n\nNow in vCenter trigger an alarm, e.g. by setting a low memory threshold on\nhosts. The alarm should show up as an `com.vmware.event.router/event.AlarmInfo`\nevent in the Sockeye event viewer and also in your configured Slack channel.\n\n![](static/slack-alert.png)\n\n## Installing required components\n\n### Requirements\n\nThe following tools are needed to follow along:\n\n- [Docker](https://www.docker.com/)\n- [`kind`](https://kubernetes.io/docs/tasks/tools/#kind)\n- [`kubectl`](https://kubernetes.io/docs/tasks/tools/#kubectl) \n- [`kn`](https://knative.dev/docs/install/client/#kn)\n- [`helm`](https://helm.sh/docs/intro/install/)\n\n### Knative Serving and Eventing\n\n```bash\nexport KIND_CLUSTER_NAME=kubeconeu22\nexport KO_DOCKER_REPO=kind.local\ncurl -sL install.konk.dev | bash\n```\n\nSource: https://github.com/csantanapr/knative-kind\n\n### `kn` CLI\n\n[kn](https://knative.dev/docs/client/install-kn/) lets you work with Knative\nresources, e.g. `KServices`, `Brokers`, etc. instead of using `kubectl`.\n\n```bash\n# OSX\nbrew install knative/client/kn\n\nkn version\nVersion:      v0.23.0\nBuild Date:   2021-05-19 07:42:12\nGit Revision: 3cc608f9\nSupported APIs:\n* Serving\n  - serving.knative.dev/v1 (knative-serving v0.23.0)\n* Eventing\n  - sources.knative.dev/v1 (knative-eventing v0.23.0)\n  - eventing.knative.dev/v1 (knative-eventing v0.23.0)\n```\n\nSource: https://knative.dev/docs/install/client/#kn\n\n### RabbitMQ\n\nInstalling the RabbitMQ Knative `Broker` ensures reliable and (if needed)\nordered event delivery which the in-memory `Broker` does not. This avoids\ncertain issues for this demo.\n\nSource:\nhttps://github.com/knative-sandbox/eventing-rabbitmq/blob/release-1.4/docs/broker.md\n\n#### Kubernetes Operator\n\n```bash\nkubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml\nkubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml\n\nkubectl -n cert-manager wait --timeout=3m --for=condition=Available deploy --all\nkubectl apply -f https://github.com/rabbitmq/messaging-topology-operator/releases/latest/download/messaging-topology-operator-with-certmanager.yaml\nkubectl -n rabbitmq-system wait --timeout=3m --for=condition=Available deploy --all\n```\n\n#### RabbitMQ Knative `Broker` Resources\n\n```bash\nkubectl apply -f https://github.com/knative-sandbox/eventing-rabbitmq/releases/latest/download/rabbitmq-broker.yaml\nkubectl -n knative-eventing wait --timeout=3m --for=condition=Available deploy --all\n```\n\n#### Deploy a RabbitMQ Cluster\n\nDeploy a RabbitMQ Cluster in the `vmware-functions` namespace, which will be\nused for the Knative `Broker`.\n\n```bash\nkubectl create ns vmware-functions\n\nkubectl apply -f - \u003c\u003c EOF\napiVersion: rabbitmq.com/v1beta1\nkind: RabbitmqCluster\nmetadata:\n  name: rabbitmq\n  namespace: vmware-functions\nspec:\n  replicas: 1\nEOF\n```\n\n#### Deploy a Knative `Broker`\n\nNow we can deploy a Knative `Broker` instance named `default` in the\n`vmware-functions` namespace using the previously created RabbitMQ cluster.\n\n```bash\nkubectl -n vmware-functions wait --timeout=3m --for=condition=Ready pod/rabbitmq-server-0\n\nkubectl apply -f - \u003c\u003c EOF\napiVersion: eventing.knative.dev/v1\nkind: Broker\nmetadata:\n  name: default\n  namespace: vmware-functions\n  annotations:\n    eventing.knative.dev/broker.class: RabbitMQBroker\nspec:\n  config:\n    apiVersion: rabbitmq.com/v1beta1\n    kind: RabbitmqCluster\n    name: rabbitmq\n  delivery:\n    retry: 5\nEOF\n\nkubectl -n vmware-functions wait --timeout=3m --for=condition=Ready broker/default\n```\n\n### Sockeye (Event Viewer)\n\nInstall the event viewer UI.\n\n```bash\nkubectl -n vmware-functions create -f https://github.com/n3wscott/sockeye/releases/download/v0.7.0/release.yaml\n\n# Update Sockeye to disable scale-to-zero\nkn service -n vmware-functions update --scale 1 sockeye\n```\n\n#### Sockeye Event `Trigger`\n\nInstall a `Trigger` which will send **all events** from the `default` `Broker`\nto the Sockeye `KService`.\n\n```bash\nkn trigger -n vmware-functions create sockeye --broker default --sink ksvc:sockeye\nkubectl -n vmware-functions  wait --timeout=3m --for=condition=Ready trigger/sockeye\n```\n\n#### Open UI\n\nRetrieve the Sockeye URL (here\n`http://sockeye.vmware-functions.127.0.0.1.sslip.io`) and open it in a browser.\n\n```bash\nkn service list -n vmware-functions\nNAME      URL                                                  LATEST          AGE   CONDITIONS   READY   REASON\nsockeye   http://sockeye.vmware-functions.127.0.0.1.sslip.io   sockeye-00002   2m    3 OK / 3     True\n```\n\nYou should see the Sockeye interface but no incoming events yet.\n\n⚠️ Sockeye is stateless, i.e. if you refresh the UI all previously displayed\nevents are lost.\n\n### VMware Event Router (VEBA)\n\nInstall the VMware Event Router to retrieve vSphere events, transform them into\nstandardized `CloudEvents` and send them to the `default` `Broker`.\n\nSource: https://vmweventbroker.io/kb/event-router\n\n#### Register VEBA Helm Repository\n\n```bash\nhelm repo add vmware-veba https://projects.registry.vmware.com/chartrepo/veba\n\n# update index in case the repo was already installed\nhelm repo update\n```\n\n#### Install the `router`\n\nInstalls the VMware Event Router in the `vmware-system` namespace.\n\nChange the values in the `vcenter` section accordingly. The `username` must have\nat least read-only access to the vCenter inventory and be able to read from the\nevent stream (database).\n\n```bash\ncat \u003c\u003c EOF | helm install -n vmware-system --create-namespace -f - router vmware-veba/event-router --wait\neventrouter:\n  config:\n    logLevel: debug\n  vcenter:\n    address: https://vcenter.local\n    username: administrator@vsphere.local\n    password: P@ssw0rd\n    insecure: false # fail on TLS certificate warnings\n  eventProcessor: knative\n  knative:\n    destination:\n      ref:\n        apiVersion: eventing.knative.dev/v1\n        kind: Broker\n        name: default\n        namespace: vmware-functions\nEOF\n```\n\nVerify the installation. The `router` deployment should not throw any errors or\nbe in `CrashLoopBackoff` state.\n\n```bash\n# READY must show 1 available pod\nkubectl -n vmware-system get deploy router\nNAME     READY   UP-TO-DATE   AVAILABLE   AGE\nrouter   1/1     1            1           40s\n\n# retrieve logs\nkubectl -n vmware-system logs deploy/router\n\n _    ____  ___                            ______                 __     ____              __\n| |  / /  |/  /      ______ _________     / ____/   _____  ____  / /_   / __ \\____  __  __/ /____  _____\n| | / / /|_/ / | /| / / __  / ___/ _ \\   / __/ | | / / _ \\/ __ \\/ __/  / /_/ / __ \\/ / / / __/ _ \\/ ___/\n| |/ / /  / /| |/ |/ / /_/ / /  /  __/  / /___ | |/ /  __/ / / / /_   / _, _/ /_/ / /_/ / /_/  __/ /\n|___/_/  /_/ |__/|__/\\__,_/_/   \\___/  /_____/ |___/\\___/_/ /_/\\__/  /_/ |_|\\____/\\__,_/\\__/\\___/_/\n\n2022-04-29T12:54:44.900Z        WARN    [VCENTER]       vcenter/vcenter.go:126  using potentially insecure connection to vCenter        {\"address\": \"https://10.192.229.97\", \"insecure\": true}\n2022-04-29T12:54:44.900Z        INFO    [MAIN]          router/main.go:114      connecting to vCenter   {\"commit\": \"92d95709\", \"version\": \"v0.7.2\", \"address\": \"https://10.192.229.97\"}\n2022-04-29T12:54:44.901Z        INFO    [KNATIVE]       injection/injection.go:61       Starting informers...\n2022-04-29T12:54:45.013Z        INFO    [MAIN]          router/main.go:169      created Knative processor       {\"commit\": \"92d95709\", \"version\": \"v0.7.2\", \"sink\": \"http://default-broker-ingress.vmware-functions.svc.cluster.local\"}\n2022-04-29T12:54:45.014Z        WARN    [METRICS]       metrics/server.go:59    disabling basic auth: no authentication data provided\n2022-04-29T12:54:45.014Z        INFO    [METRICS]       metrics/server.go:98    starting metrics server {\"address\": \"http://0.0.0.0:8082/stats\"}\n2022-04-29T12:54:45.214Z        INFO    [VCENTER]       vcenter/vcenter.go:213  checkpointing disabled, setting begin of event stream   {\"beginTimestamp\": \"2022-04-29 12:54:45.129336 +0000 UTC\"}\n2022-04-29T12:54:46.801Z        DEBUG   [VCENTER]       vcenter/vcenter.go:313  no new events, backing off      {\"delaySeconds\": 1}\n2022-04-29T12:54:48.124Z        DEBUG   [VCENTER]       vcenter/vcenter.go:313  no new events, backing off      {\"delaySeconds\": 2}\n```\n\nIf all goes well you might already see `UserLoginSessionEvent` events in the\nSockeye UI.\n\nYou can also manually trigger an event, e.g. by powering on/off a virtual\nmachine.\n\nThis completes the installation. Now you can follow along with the scenarios in\nthe [tutorial](#tutorial) section\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembano1%2Fkubeconeu22","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fembano1%2Fkubeconeu22","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembano1%2Fkubeconeu22/lists"}