{"id":15698925,"url":"https://github.com/developer-guy/kubernetes-response-engine-based-on-event-driven-workflow","last_synced_at":"2025-10-28T19:09:18.588Z","repository":{"id":51838499,"uuid":"364516533","full_name":"developer-guy/kubernetes-response-engine-based-on-event-driven-workflow","owner":"developer-guy","description":"Experimenting to implement Kubernetes Response Engine based on Event-Driven Workflow using Argo Events and Argo Workflows","archived":false,"fork":false,"pushed_at":"2021-05-25T12:50:59.000Z","size":1203,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-09T02:16:59.726Z","etag":null,"topics":["argo","argo-events","argo-workflows","event-driven","falco","falcosidekick","gitops"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/developer-guy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-05T08:55:20.000Z","updated_at":"2024-11-11T13:55:46.000Z","dependencies_parsed_at":"2022-08-23T00:30:52.088Z","dependency_job_id":null,"html_url":"https://github.com/developer-guy/kubernetes-response-engine-based-on-event-driven-workflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/developer-guy%2Fkubernetes-response-engine-based-on-event-driven-workflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/developer-guy%2Fkubernetes-response-engine-based-on-event-driven-workflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/developer-guy%2Fkubernetes-response-engine-based-on-event-driven-workflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/developer-guy%2Fkubernetes-response-engine-based-on-event-driven-workflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/developer-guy","download_url":"https://codeload.github.com/developer-guy/kubernetes-response-engine-based-on-event-driven-workflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253176443,"owners_count":21866143,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["argo","argo-events","argo-workflows","event-driven","falco","falcosidekick","gitops"],"created_at":"2024-10-03T19:36:03.538Z","updated_at":"2025-10-28T19:09:13.543Z","avatar_url":"https://github.com/developer-guy.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kubernetes Response Engine based on Event-Driven Workflow using Argo Events \u0026 Argo Workflows\n\nWe presented in previous blog posts the concept called _Kubernetes Response Engine_, to do so we have used serverless platforms running on top of Kubernetes such as Kubeless, OpenFaaS, and Knative. In a nutshell, this engine aims to provide to users automatic action against threats detected by Falco.\n\nIf you want to get more details about the concept and how we use serverless platforms for this concept, please follow the links below:\n\n\u003e * [Kubernetes Response Engine, Part 1 : Falcosidekick + Kubeless](https://falco.org/blog/falcosidekick-reponse-engine-part-1-kubeless/)\n\u003e * [Kubernetes Response Engine, Part 2 : Falcosidekick + OpenFaas](https://falco.org/blog/falcosidekick-reponse-engine-part-2-openfaas/)\n\u003e * [Kubernetes Response Engine, Part 3 : Falcosidekick + Knative](https://falco.org/blog/falcosidekick-reponse-engine-part-3-knative/)\n\u003e * [Kubernetes Response Engine, Part 4: Falcosidekick + Tekton](https://falco.org/blog/falcosidekick-response-engine-part-4-tekton/)\n\u003e * [Kubernetes Response Engine, Part 5: Falcosidekick + Argo](https://falco.org/blog/falcosidekick-reponse-engine-part-5-argo/)\n\nRecently, a community member, [Edvin](https://github.com/NissesSenap), came with the great idea to use a Cloud Native Workflow system to implement same kind of scenario. Following that, he implemented it by using _Tekton_ and _Tekton Trigger_. To get more details about his work, please follow this [repository](https://github.com/NissesSenap/falcosidekick-tekton).\n\nAfter that, we realized that we can use _Argo Events_ and _Argo Workflows_ to do the same thing. This repository provides an overview of how we can use these tools to implement a _Kubernetes Response Engine_\n\nLet's start with quick a introduction of the tooling.\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n**Table of Contents**\n\n- [Kubernetes Response Engine based on Event-Driven Workflow using Argo Events \u0026 Argo Workflows](#kubernetes-response-engine-based-on-event-driven-workflow-using-argo-events--argo-workflows)\n  - [What is Falco? ¶](#what-is-falco-)\n  - [What is Falcosidekick? ¶](#what-is-falcosidekick-)\n  - [What is Argo Workflows? ¶](#what-is-argo-workflows-)\n  - [What is Argo Events? ¶](#what-is-argo-events-)\n  - [Prerequisites](#prerequisites)\n  - [Demo](#demo)\n    - [Minikube](#minikube)\n    - [Kind](#kind)\n    - [Install Argo Events and Argo Workflows](#install-argo-events-and-argo-workflows)\n    - [Install Falco and Falcosidekick](#install-falco-and-falcosidekick)\n    - [Install Webhook and Sensor](#install-webhook-and-sensor)\n    - [Install argo CLI](#install-argo-cli)\n    - [Argo Worfklows UI](#argo-worfklows-ui)\n    - [Test](#test)\n  - [Furthermore](#furthermore)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n## What is Falco? [¶](https://github.com/falcosecurity/falco)\n\nFalco, the open source cloud native runtime security project, is one of the leading open source Kubernetes threat detection engines. Falco was created by Sysdig in 2016 and is the first runtime security project to join CNCF as an incubation-level project.\n\n## What is Falcosidekick? [¶](https://github.com/falcosecurity/falcosidekick)\n\nA simple daemon for connection Falco to your ecosystem (alerting, logging, metrology, etc).\n\n## What is Argo Workflows? [¶](https://argoproj.github.io/argo-workflows/#what-is-argo-workflows)\n\nArgo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows are declared through a Kubernetes CRD (Custom Resource Definition).\n\n## What is Argo Events? [¶](https://argoproj.github.io/argo-events/#what-is-argo-events)\n\nArgo Events is an event-driven workflow automation framework for Kubernetes which helps you trigger K8s objects, Argo Workflows, Serverless workloads, and others by events from variety of sources like webhook, s3, schedules, messaging queues, gcp pubsub, sns, sqs, etc.\n\n## Prerequisites\n\n* minikube v1.19.0 or kind v0.10.0\n* helm v3.5.4+g1b5edb6\n* kubectl v1.21.0\n* argo v3.0.2\n* ko v0.8.2\n\n## Demo\n\nLet's start with explaining a little bit what we want to achieve in this demo. Basically, Falco, the container runtime security tool, is going to detect an unexpected behaviour at host level, then it will trigger an alert and send it to Falcosidekick. Falcosidekick has _Webhook_ output type we can configure to notify the event source of _Argo Events_. Then, _Argo Events_  will trigger the [argoWorkFlowTrigger](https://github.com/argoproj/argo-events/blob/master/api/sensor.md#argoproj.io/v1alpha1.ArgoWorkflowTrigger) type of trigger of _Argo Workflows_, and this workflow will create a _pod delete_ pod in charge of terminating the compromised pod.\n\nFalco --\u003e Falcosidekick W/webhook --\u003e Argo Events W/webhook --\u003e Argo Workflows W/argoWorkFlowTrigger\n\nNow, let's start with creating our local Kubernetes cluster.\n\n### Minikube\n\n```bash\nminikube start\n```\n\n### Kind\n\nIf you rather use kind.\n\n```shell\n# kind config file\ncat \u003c\u003c'EOF' \u003e\u003e kind-config.yaml.yaml\nkind: Cluster\napiVersion: kind.x-k8s.io/v1alpha4\nnodes:\n- role: control-plane\n  image: kindest/node:v1.20.2\n  extraMounts:\n    # allow Falco to use devices provided by the kernel module\n  - hostPath: /dev\n    containerPath: /dev\n    # allow Falco to use the Docker unix socket\n  - hostPath: /var/run/docker.sock\n    containerPath: /var/run/docker.sock\n- role: worker\n  image: kindest/node:v1.20.2\n  extraMounts:\n    # allow Falco to use devices provided by the kernel module\n  - hostPath: /dev\n    containerPath: /dev\n    # allow Falco to use the Docker unix socket\n  - hostPath: /var/run/docker.sock\n    containerPath: /var/run/docker.sock\n- role: worker\n  image: kindest/node:v1.20.2\n  extraMounts:\n    # allow Falco to use devices provided by the kernel module\n  - hostPath: /dev\n    containerPath: /dev\n    # allow Falco to use the Docker unix socket\n  - hostPath: /var/run/docker.sock\n    containerPath: /var/run/docker.sock\nEOF\n\n# start the kind cluster\n\nkind create cluster --config=./config-kind.yaml\n\n```\n\n\u003e Kind is verified on on linux client only.\n\n### Install Argo Events and Argo Workflows\n\nThen, install _Argo Events_ and _Argo Workflows_ components.\n\n```bash\n# Argo Events Installation\n$ kubectl create namespace argo-events\nnamespace/argo-events created\n\n$ kubectl apply \\\n    --filename https://raw.githubusercontent.com/argoproj/argo-events/stable/manifests/install.yaml\ncustomresourcedefinition.apiextensions.k8s.io/eventbus.argoproj.io created\ncustomresourcedefinition.apiextensions.k8s.io/eventsources.argoproj.io created\ncustomresourcedefinition.apiextensions.k8s.io/sensors.argoproj.io created\nserviceaccount/argo-events-sa created\nclusterrole.rbac.authorization.k8s.io/argo-events-aggregate-to-admin created\nclusterrole.rbac.authorization.k8s.io/argo-events-aggregate-to-edit created\nclusterrole.rbac.authorization.k8s.io/argo-events-aggregate-to-view created\nclusterrole.rbac.authorization.k8s.io/argo-events-role created\nclusterrolebinding.rbac.authorization.k8s.io/argo-events-binding created\ndeployment.apps/eventbus-controller created\ndeployment.apps/eventsource-controller created\ndeployment.apps/sensor-controller created\n\n$ kubectl --namespace argo-events apply \\\n    --filename https://raw.githubusercontent.com/argoproj/argo-events/stable/examples/eventbus/native.yaml\neventbus.argoproj.io/default created\n\n# Argo Workflows Installation\n$ kubectl create namespace argo\nnamespace/argo created\n\n$ kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo-workflows/stable/manifests/quick-start-postgres.yaml\ncustomresourcedefinition.apiextensions.k8s.io/clusterworkflowtemplates.argoproj.io created\ncustomresourcedefinition.apiextensions.k8s.io/cronworkflows.argoproj.io created\ncustomresourcedefinition.apiextensions.k8s.io/workfloweventbindings.argoproj.io created\ncustomresourcedefinition.apiextensions.k8s.io/workflows.argoproj.io created\ncustomresourcedefinition.apiextensions.k8s.io/workflowtemplates.argoproj.io created\nserviceaccount/argo created\nserviceaccount/argo-server created\nserviceaccount/github.com created\nrole.rbac.authorization.k8s.io/argo-role created\nrole.rbac.authorization.k8s.io/argo-server-role created\nrole.rbac.authorization.k8s.io/submit-workflow-template created\nrole.rbac.authorization.k8s.io/workflow-role created\nclusterrole.rbac.authorization.k8s.io/argo-clusterworkflowtemplate-role created\nclusterrole.rbac.authorization.k8s.io/argo-server-clusterworkflowtemplate-role created\nclusterrole.rbac.authorization.k8s.io/kubelet-executor created\nrolebinding.rbac.authorization.k8s.io/argo-binding created\nrolebinding.rbac.authorization.k8s.io/argo-server-binding created\nrolebinding.rbac.authorization.k8s.io/github.com created\nrolebinding.rbac.authorization.k8s.io/workflow-default-binding created\nclusterrolebinding.rbac.authorization.k8s.io/argo-clusterworkflowtemplate-role-binding created\nclusterrolebinding.rbac.authorization.k8s.io/argo-server-clusterworkflowtemplate-role-binding created\nclusterrolebinding.rbac.authorization.k8s.io/kubelet-executor-default created\nconfigmap/artifact-repositories created\nconfigmap/workflow-controller-configmap created\nsecret/argo-postgres-config created\nsecret/argo-server-sso created\nsecret/argo-workflows-webhook-clients created\nsecret/my-minio-cred created\nservice/argo-server created\nservice/minio created\nservice/postgres created\nservice/workflow-controller-metrics created\ndeployment.apps/argo-server created\ndeployment.apps/minio created\ndeployment.apps/postgres created\ndeployment.apps/workflow-controller created\n```\n\nLet's verify if everything is working before we move on to the next step.\n\n```bash\n$ kubectl get pods --namespace argo-events\nNAME                                      READY   STATUS    RESTARTS   AGE\neventbus-controller-7666b44ff7-k8bjf      1/1     Running   0          6m6s\neventbus-default-stan-0                   2/2     Running   0          5m33s\neventbus-default-stan-1                   2/2     Running   0          5m21s\neventbus-default-stan-2                   2/2     Running   0          5m19s\neventsource-controller-7fd7674cb4-jj9sn   1/1     Running   0          6m6s\nsensor-controller-59b64579c9-5fbrv        1/1     Running   0          6m6s\n\n$ kubectl get pods --namespace argo\nNAME                                  READY   STATUS    RESTARTS   AGE\nargo-server-5b86d9f84b-zl5nj          1/1     Running   3          5m32s\nminio-58977b4b48-dnnwx                1/1     Running   0          5m32s\npostgres-6b5c55f477-dp9n2             1/1     Running   0          5m32s\nworkflow-controller-d9cbfcc86-tm2kf   1/1     Running   2          5m32s\n```\n\n### Install Falco and Falcosidekick\n\nLet's install Falco and Falcosidekick.\n\n```bash\n$ helm upgrade --install falco falcosecurity/falco \\\n--namespace falco \\\n--create-namespace \\\n-f hacks/values.yaml\n\nRelease \"falco\" does not exist. Installing it now.\nNAME: falco\nLAST DEPLOYED: Thu May  6 22:43:52 2021\nNAMESPACE: falco\nSTATUS: deployed\nREVISION: 1\nNOTES:\nFalco agents are spinning up on each node in your cluster. After a few\nseconds, they are going to start monitoring your containers looking for\nsecurity issues.\n\n\nNo further action should be required.\n```\n\nIf you are using **kind** the easiest way is to use ebpf.enabled=true.\n\n```shell\n$ helm upgrade --install falco falcosecurity/falco \\\n--namespace falco \\\n--create-namespace \\\n-f values.yaml \\\n--set ebpf.enabled=true\n```\n\nThis way you don't have to install any extra [drivers](https://falco.org/docs/getting-started/installation/#install-driver).\nThis only works on linux.\n\nLet's verify if all components for falco are up and running.\n\n```bash\n$ kubectl get pods --namespace falco\nNAME                                  READY   STATUS    RESTARTS   AGE\nfalco-falcosidekick-9f5dc66f5-nmfdp   1/1     Running   0          68s\nfalco-falcosidekick-9f5dc66f5-wnm2r   1/1     Running   0          68s\nfalco-zwxwz                           1/1     Running   0          68s\n```\n\n### Install Webhook and Sensor\n\nNow, we are ready to set up our workflow by creating the event source and the trigger.\n\n```bash\n# Create event source\n$ kubectl apply -f webhooks/webhook.yaml\neventsource.argoproj.io/webhook created\n\n$ kubectl get eventsources --namespace argo-events\nNAME      AGE\nwebhook   11s\n\n$ kubectl get pods --namespace argo-events\nNAME                                         READY   STATUS    RESTARTS   AGE\neventbus-controller-7666b44ff7-k8bjf         1/1     Running   0          18m\neventbus-default-stan-0                      2/2     Running   0          17m\neventbus-default-stan-1                      2/2     Running   0          17m\neventbus-default-stan-2                      2/2     Running   0          17m\neventsource-controller-7fd7674cb4-jj9sn      1/1     Running   0          18m\nsensor-controller-59b64579c9-5fbrv           1/1     Running   0          18m\nwebhook-eventsource-z9bg6-6769c7bbc8-c6tff   1/1     Running   0          45s # \u003c-- Pod listens webhook event.\n\n# necessary RBAC permissions for trigger and the pod-delete container\n$ kubectl apply -f hacks/workflow-rbac.yaml\nserviceaccount/operate-workflow-sa created\nclusterrole.rbac.authorization.k8s.io/operate-workflow-role created\nclusterrolebinding.rbac.authorization.k8s.io/operate-workflow-role-binding created\n\n$ kubectl apply -f hacks/delete-pod-rbac.yaml\nserviceaccount/falco-pod-delete created\nclusterrole.rbac.authorization.k8s.io/falco-pod-delete-cluster-role created\nclusterrolebinding.rbac.authorization.k8s.io/falco-pod-delete-cluster-role-binding created\n\n# Create trigger\n$ kubectl apply -f sensors/sensors-workflow.yaml\nsensor.argoproj.io/webhook created\n\n$ kubectl get sensors --namespace argo-events\nNAME      AGE\nwebhook   5s\n\n$ kubectl get pods --namespace argo-events\nNAME                                         READY   STATUS    RESTARTS   AGE\neventbus-controller-7666b44ff7-k8bjf         1/1     Running   0          25m\neventbus-default-stan-0                      2/2     Running   0          25m\neventbus-default-stan-1                      2/2     Running   0          25m\neventbus-default-stan-2                      2/2     Running   0          25m\neventsource-controller-7fd7674cb4-jj9sn      1/1     Running   0          25m\nsensor-controller-59b64579c9-5fbrv           1/1     Running   0          25m\nwebhook-eventsource-z9bg6-6769c7bbc8-c6tff   1/1     Running   0          8m11s\nwebhook-sensor-44w7w-7dcb9f886d-bnh8f        1/1     Running   0          74s # \u003c- Pod will create workflow.\n```\n\n\u003e We use google/ko project to build and push container images, but you don't have to do this, we already built an image and pushed it to the registry. If you want to build your own image, install google/ko project and run the command below after having changed the image version inside sensors/sensors-workflow.yaml\n\u003e `\u001dKO_DOCKER_REPO=devopps ko publish . --push=true -B`\n\n### Install argo CLI\n\nThere is one more thing we need to do, this is installation of [argo CLI](https://argoproj.github.io/argo-workflows/cli/) for managing worklows.\n\n```bash\n$ # Download the binary\ncurl -sLO https://github.com/argoproj/argo/releases/download/v3.0.2/argo-darwin-amd64.gz\n\n# Unzip\ngunzip argo-darwin-amd64.gz\n\n# Make binary executable\nchmod +x argo-darwin-amd64\n\n# Move binary to path\nmv ./argo-darwin-amd64 /usr/local/bin/argo\n\n# Test installation\nargo version\n```\n\n### Argo Worfklows UI\n\nArgo Workflows v3.0 comes with a new UI that now also supports Argo Events! The UI is also more robust and reliable.\n\nYou can basically reach out the UI from _localhost_ with doing port-forward the _Kubernetes_ service. There is also needed for using _argo CLI_ properly.\n\n```bash\n$ kubectl -n argo port-forward svc/argo-server 2746:2746\nForwarding from 127.0.0.1:2746 -\u003e 2746\nForwarding from [::1]:2746 -\u003e 2746\n```\n\n### Test\n\nNow, let's test the whole environment. We are going to create an alpine based container, then we'll `exec` into it. At moment we'll exec into the container, Falco will detect it and you should see the status of the Pod as _Terminating_.\n\n```bash\n$ kubectl run alpine --namespace default --image=alpine --restart='Never' -- sh -c \"sleep 600\"\npod/alpine created\n\n$ kubectl exec -i --tty alpine --namespace default -- sh -c \"uptime\" # this will trigger the event\n```\n\nYou should see the similar outputs like the following screen:\n\n![screen_shot](./assets/screenshot.png)\n\n## Furthermore\nThe _Falcosidekick_ and _Argo Events_ are both _CloudEvents_ compliant. [CloudEvents](https://cloudevents.io) is a specification for describing event data in a common way. CloudEvents seeks to dramatically simplify event declaration and delivery across services, platforms, and beyond!\n\nYou can basically achieve the same demo by using _CloudEvents_ instead of _Webhook_ to trigger an action in the _Argo Workflows_. If you are curios about how _CloudEvents_ and _Falco_ can be related with each other, there is a new blog post on [Falco Blog](https://falco.org/blog/) named _Kubernetes Response Engine, Part 3: Falcosidekick + Knative_, you should definitely check that out.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeveloper-guy%2Fkubernetes-response-engine-based-on-event-driven-workflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeveloper-guy%2Fkubernetes-response-engine-based-on-event-driven-workflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeveloper-guy%2Fkubernetes-response-engine-based-on-event-driven-workflow/lists"}