{"id":46233696,"url":"https://github.com/cloud-ark/kubeprovenance","last_synced_at":"2026-03-03T18:14:08.893Z","repository":{"id":43880067,"uuid":"144072985","full_name":"cloud-ark/kubeprovenance","owner":"cloud-ark","description":"Kubernetes Aggregated API Server to track lineage/provenance of Kubernetes Custom resources","archived":false,"fork":false,"pushed_at":"2019-01-01T14:12:28.000Z","size":1223,"stargazers_count":25,"open_issues_count":1,"forks_count":7,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-06-19T17:22:18.031Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloud-ark.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-08-08T22:11:46.000Z","updated_at":"2023-11-30T11:21:34.000Z","dependencies_parsed_at":"2022-07-19T03:32:08.486Z","dependency_job_id":null,"html_url":"https://github.com/cloud-ark/kubeprovenance","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cloud-ark/kubeprovenance","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-ark%2Fkubeprovenance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-ark%2Fkubeprovenance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-ark%2Fkubeprovenance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-ark%2Fkubeprovenance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloud-ark","download_url":"https://codeload.github.com/cloud-ark/kubeprovenance/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud-ark%2Fkubeprovenance/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30054175,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-03T17:46:22.538Z","status":"ssl_error","status_checked_at":"2026-03-03T17:46:22.036Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-03-03T18:14:07.926Z","updated_at":"2026-03-03T18:14:08.884Z","avatar_url":"https://github.com/cloud-ark.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kubeprovenance\n\nA Kubernetes Aggregated API Server to find out Provenance/Lineage information for Kuberentes Custom Resources.\n\n## What is it?\n\n\nKubernetes custom resources extend base API to manage third-party platform elements declaratively.\nIt is important to track chronology of declarative operations performed on custom resources to understand\nhow these operations affect underlying platform elements - e.g. for an instance of Postgres custom resource we may want to know:\nhow many db users were created in a month, when was password changed for a db user, etc.\nFor this, a generic approach is needed to maintain provenance information of custom resources.\n\nkubeprovenance is a tool that helps you find Provenance information about different Kubernetes custom resources in your cluster.\n\nKubeprovenance is a Kubernetes aggregated API server. It uses Kubernetes audit logs for building custom resource provenance.\nProvenance query operators like history, diff, bisect are defined for custom resource instance tracking. Provenance information is accessible via kubectl.\n\n\n## Try it Out:\n\n**1. Setting Up The Environment.**\n\nReference: https://dzone.com/articles/easy-step-by-step-local-kubernetes-source-code-cha\u003cbr/\u003e\nssh to your VM \u003cbr/\u003e\nsudo su - \u003cbr/\u003e\napt-get install -y gcc make socat git wget\u003cbr/\u003e\n\n**2. Install Golang 1.10.3:** \u003cbr/\u003e\nwget https://dl.google.com/go/go1.10.3.linux-amd64.tar.gz \u003cbr/\u003e\nsudo tar -C /usr/local -xzf go1.10.3.linux-amd64.tar.gz \u003cbr/\u003e\nexport PATH=$PATH:/usr/local/go/bin \u003cbr/\u003e\nexport GOROOT=/usr/local/go \u003cbr/\u003e\n\nSet up your Go workspace, set the GOPATH to it. This is where all your Go code should be. \u003cbr/\u003e\nmkdir $HOME/goworkspace \u003cbr/\u003e\nmkdir $HOME/goworkspace/src \u003cbr/\u003e\nmkdir $HOME/goworkspace/bin \u003cbr/\u003e\n\nexport GOPATH=$HOME/goworkspace \u003cbr/\u003e\n\n**3. Install etcd3.2.18:**\ncurl -L https://github.com/coreos/etcd/releases/download/v3.2.18/etcd-v3.2.18-linux-amd64.tar.gz -o etcd-v3.2.18-linux-amd64.tar.gz \u0026\u0026 tar xzvf etcd-v3.2.18-linux-amd64.tar.gz \u0026\u0026 /bin/cp -f etcd-v3.2.18-linux-amd64/{etcd,etcdctl} /usr/bin \u0026\u0026 rm -rf etcd-v3.2.18-linux-amd64* \u003cbr/\u003e\n\n\n**4. Install Docker**\u003cbr/\u003e\nFollow steps here: reference: https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository \u003cbr/\u003e\ndocker version //check if it is installed \u003cbr/\u003e\n\n**5. Get The Kubernetes Source Code:** \u003cbr/\u003e\ngit clone https://github.com/kubernetes/kubernetes $GOPATH/src/k8s.io/kubernetes \u003cbr/\u003e\ncd $GOPATH/src/k8s.io/kubernetes \u003cbr/\u003e\n\n**6. Compile and Run Kubernetes** \u003cbr/\u003e\nexport KUBERNETES_PROVIDER=local \u003cbr/\u003e\nroot@host: $GOPATH/src/k8s.io/kubernetes# hack/local-up-cluster.sh \u003cbr/\u003e\n\nIn a new shell, test that it is working : \u003cbr/\u003e\nroot@host: $GOPATH/src/k8s.io/kubernetes# cluster/kubectl.sh cluster-info \u003cbr/\u003e\nKubernetes master is running at http://127.0.0.1:8080 # =\u003e works! \u003cbr/\u003e\n\nAdd $GOPATH/src/k8s.io/kubernetes/cluster to PATH: \u003cbr/\u003e\n\nexport PATH=$PATH:$GOPATH/src/k8s.io/kubernetes/cluster \u003cbr/\u003e\n\nNow, commands look like kubectl.sh get pods instead of kubectl get pods...\n\n**7. Enabling Auditing:** \u003cbr/\u003e\n\nWe have to enable auditing. reference: https://kubernetes.io/docs/tasks/debug-application-cluster/audit/ \u003cbr/\u003e\nSetting up Log backend (To be added)... \u003cbr/\u003e\n\nIf not in kubernetes directory... \u003cbr/\u003e\ncd $GOPATH/src/k8s.io/kubernetes \u003cbr/\u003e\n\nvi hack/local-up-cluster.sh \u003cbr/\u003e\n\nline 87: Change ENABLE_APISERVER_BASIC_AUDIT to true\n   ENABLE_APISERVER_BASIC_AUDIT=${ENABLE_APISERVER_BASIC_AUDIT:-true}\n\nline 486: add audit-policy file to audit_args:   \u003cbr/\u003e\n   Now you need to add an audit-arg for the audit-policy. add the following line after audit_arg+=\" --audit-log-maxbackup=0\"\n\n   audit_arg += \" --audit-policy-file=/root/audit-policy.yaml\" \u003cbr/\u003e\n\n   The value of --audit-policy-file is where you created your audit-policy.yaml file.  \u003cbr/\u003e\n   There is an example-policy for a Postgres custom resource saved in this repository. \u003cbr/\u003e\n\n   This file defines what actions and resources will generate logs.\n\n   Reference the docs if you are looking to make one: \u003cbr/\u003e\n      https://kubernetes.io/docs/tasks/debug-application-cluster/audit/ \u003cbr/\u003e\n   For running kubeprovenance to track only a Postgres custom resource, audit-policy would look like this:  \u003cbr/\u003e\n   Note: Add more rules to the audit-policy to track different or more than one custom resource:\n\n      root@provenance:~# more audit-policy.yaml\n      apiVersion: audit.k8s.io/v1beta1\n      kind: Policy\n      omitStages:\n        - \"RequestReceived\"\n      rules:\n        - level: Request\n          verbs:\n            - create\n            - delete\n            - patch\n          resources:\n            - group: \"postgrescontroller.kubeplus\"\n              version: \"v1\"\n              resources: [\"postgreses\"]\n\n   Note: The audit log for your custom resource will be saved where this variable is set:\n      APISERVER_BASIC_AUDIT_LOG=/tmp/kube-apiserver-audit.log \u003cbr/\u003e\n\n**8. Running kubeprovenance** \u003cbr/\u003e\n\nInstall dep:  \u003cbr/\u003e\ncurl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh \u003cbr/\u003e\nMove dep executable to somewhere on your $PATH \u003cbr/\u003e\ndep version -- to verify that it is installed correctly \u003cbr/\u003e\n\ngo get github.com/cloud-ark/kubeprovenance \u003cbr/\u003e\ncd $GOPATH/src/github.com/cloud-ark/kubeprovenance \u003cbr/\u003e\ndep ensure -v \u003cbr/\u003e\n\nMake sure Kubernetes is running:\u003cbr/\u003e\n$ kubectl.sh cluster-info\n\nNow to deploy this aggregated api server use these commands:\n1) Build the API Server container image:  \u003cbr/\u003e\n   `$ ./build-provenance-artifacts.sh`\n2) Deploy the API Server in your cluster: \u003cbr/\u003e\n   `$ ./deploy-provenance-artifacts.sh`\n3) Clean-up:  \u003cbr/\u003e\n   `$ ./delete-provenance-artifacts.sh`\n\n\n**9. Deploy Sample Postgres Operator** \u003cbr/\u003e\n\nFollow the steps given [here](https://github.com/cloud-ark/kubeplus/tree/master/postgres-crd-v2)\n\nOnce the kubeprovenance API server is running, you can find provenance information by using the following commands:\n\n1) Get list of version for a Postgres custom resource instance (client25)\n\n```\nkubectl.sh get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/versions\"\n```\n\n2) Get Spec history for Postgres custom resource instance\n\n```\nkubectl.sh get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/spechistory\"\n```\n\n3) Get diff of Postgres custom resource instance between version 1 and version 5\n\n```\nkubectl.sh get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/diff?start=1\u0026end=5\"\n```\n\n4) Get diff of the field databases for a Postgres custom resource instance between version 1 and version 2\n\n```\nkubectl.sh get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/diff?start=1\u0026end=2\u0026field=databases\"\n```\n\n5) Get diff of the field users for a Postgres custom resource instance between version 1 and version 3\n\n```\nkubectl.sh get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/diff?start=1\u0026end=3\u0026field=users\"\n```\n\n6) Find out in which version the user 'pallavi' was given password 'pass123'\n\n```\nkubectl.sh get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/bisect?field1=username\u0026value1=pallavi\u0026field2=password\u0026value2=pass123\"\n```\n\n## Try it on Minikube\n\nNote: Since audit-logging is not supported on minikube yet (https://github.com/kubernetes/minikube/issues/2934), I included a static, pre-generated audit-log to use to see how it works.\n\n**1. Setting up environment.** \u003cbr/\u003e\nsudo su - \u003cbr/\u003e\napt-get install -y gcc make socat git wget\u003cbr/\u003e\n**2. Install Minikube** \u003cbr/\u003e\ncurl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.28.2/minikube-linux-amd64 \u0026\u0026 chmod +x minikube \u0026\u0026 sudo mv minikube /usr/local/bin/ \u003cbr/\u003e\nminikube start \u003cbr/\u003e\nminikube ip -- verify that minikube is up and running \u003cbr/\u003e\n**3. Install Golang 1.10.3:** \u003cbr/\u003e\nwget https://dl.google.com/go/go1.10.3.linux-amd64.tar.gz \u003cbr/\u003e\nsudo tar -C /usr/local -xzf go1.10.3.linux-amd64.tar.gz \u003cbr/\u003e\nexport PATH=$PATH:/usr/local/go/bin \u003cbr/\u003e\nexport GOROOT=/usr/local/go \u003cbr/\u003e\n\nSet up your Go workspace, set the GOPATH to it. This is where all your Go code should be. \u003cbr/\u003e\nmkdir $HOME/goworkspace \u003cbr/\u003e\nmkdir $HOME/goworkspace/src \u003cbr/\u003e\nmkdir $HOME/goworkspace/bin \u003cbr/\u003e\n\nexport GOPATH=$HOME/goworkspace \u003cbr/\u003e\n\n**4. Install etcd3.2.18:**\ncurl -L https://github.com/coreos/etcd/releases/download/v3.2.18/etcd-v3.2.18-linux-amd64.tar.gz -o etcd-v3.2.18-linux-amd64.tar.gz \u0026\u0026 tar xzvf etcd-v3.2.18-linux-amd64.tar.gz \u0026\u0026 /bin/cp -f etcd-v3.2.18-linux-amd64/{etcd,etcdctl} /usr/bin \u0026\u0026 rm -rf etcd-v3.2.18-linux-amd64* \u003cbr/\u003e\n\n\n**5. Install Docker**\u003cbr/\u003e\nFollow steps here: reference: https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository \u003cbr/\u003e\ndocker version //check if it is installed \u003cbr/\u003e\n\n\n**6. Install dep:**\u003cbr/\u003e\ncurl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh \u003cbr/\u003e\nMove dep executable to somewhere on your $PATH \u003cbr/\u003e\ndep version -- to verify that it is installed correctly \u003cbr/\u003e\n\n\n**7. Running kubeprovenance**\u003cbr/\u003e\n\ngo get github.com/cloud-ark/kubeprovenance \u003cbr/\u003e\ncd $GOPATH/src/github.com/cloud-ark/kubeprovenance \u003cbr/\u003e\ndep ensure -v \u003cbr/\u003e\n\n0) Allow Minikube to use local Docker images:   \u003cbr/\u003e\n   `$ eval $(minikube docker-env)`\n1) Build the API Server container image:  \u003cbr/\u003e\n   `$ ./build-provenance-artifacts.sh`\n2) Deploy the API Server in your cluster:  \u003cbr/\u003e\n   `$ ./deploy-provenance-artifacts.sh`\n3) Clean-up:  \u003cbr/\u003e\n   `$ ./delete-provenance-artifacts.sh`\n\nOnce the kubeprovenance API server is running, you can find provenance information by using the following commands:\n\n\n1) Get list of version for a Postgres custom resource instance (client25)\n\n```\nkubectl get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/versions\"\n```\n\n![alt text](https://github.com/cloud-ark/kubeprovenance/raw/master/docs/versions.png)\n\n2) Get Spec history for Postgres custom resource instance\n\n```\nkubectl get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/spechistory\"\n```\n\n![alt text](https://github.com/cloud-ark/kubeprovenance/raw/master/docs/spechistory.png)\n\n\n3) Get diff of Postgres custom resource instance between version 1 and version 5\n\n```\nkubectl get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/diff?start=1\u0026end=5\"\n```\n\n![alt text](https://github.com/cloud-ark/kubeprovenance/raw/master/docs/getfulldiff.png)\n\n\n4) Get diff of the field databases for a Postgres custom resource instance between version 1 and version 2\n\n```\nkubectl get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/diff?start=1\u0026end=2\u0026field=databases\"\n```\n![alt text](https://github.com/cloud-ark/kubeprovenance/raw/master/docs/getfielddiff.png)\n\n\n5) Get diff of the field users for a Postgres custom resource instance between version 1 and version 3\n\n```\nkubectl get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/diff?start=1\u0026end=3\u0026field=users\"\n```\n\n![alt text](https://github.com/cloud-ark/kubeprovenance/raw/master/docs/usersfielddiff.png)\n\n\n6) Find out in which version the user 'pallavi' was given password 'pass123'\n\n```\nkubectl get --raw \"/apis/kubeprovenance.cloudark.io/v1/namespaces/default/postgreses/client25/bisect?field1=username\u0026value1=pallavi\u0026field2=password\u0026value2=pass123\"\n```\n![alt text](https://github.com/cloud-ark/kubeprovenance/raw/master/docs/bisect.png)\n\n\n## Running Unit Tests:\n\n1. go test -v ./...\n\n\n## Troubleshooting tips:\n\n1) Check that the API server Pod is running:\n\n   `$ kubectl get pods -n provenance`\n\n2) Get the Pod name from output of above command and then check logs of the container.\n   For example:\n\n   `$ kubectl logs -n provenance kube-provenance-apiserver-klzpc  -c kube-provenance-apiserver`\n\n\n### Details:\n\nOur experience in building this API server is [here](https://medium.com/@cloudark/our-journey-in-building-a-kubernetes-aggregated-api-server-29a4f9c1de22).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloud-ark%2Fkubeprovenance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloud-ark%2Fkubeprovenance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloud-ark%2Fkubeprovenance/lists"}