{"id":15192053,"url":"https://github.com/dvilaverde/k8s-countermeasures","last_synced_at":"2026-01-15T22:46:17.102Z","repository":{"id":95166610,"uuid":"574295142","full_name":"dvilaverde/k8s-countermeasures","owner":"dvilaverde","description":"Kubernetes operator deploying run-books as code.","archived":false,"fork":false,"pushed_at":"2024-04-27T13:11:27.000Z","size":20739,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-10T02:35:14.440Z","etag":null,"topics":["automation","countermeasure","devops","golang","k8s","kubernetes","operator","operator-sdk","prod-support","runbooks","sre"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dvilaverde.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-05T01:34:54.000Z","updated_at":"2024-04-27T13:11:28.000Z","dependencies_parsed_at":"2024-06-19T16:20:45.655Z","dependency_job_id":"deb0bbb3-32d4-4dc1-b3bc-a92092dd03bf","html_url":"https://github.com/dvilaverde/k8s-countermeasures","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvilaverde%2Fk8s-countermeasures","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvilaverde%2Fk8s-countermeasures/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvilaverde%2Fk8s-countermeasures/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvilaverde%2Fk8s-countermeasures/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dvilaverde","download_url":"https://codeload.github.com/dvilaverde/k8s-countermeasures/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247217220,"owners_count":20903009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","countermeasure","devops","golang","k8s","kubernetes","operator","operator-sdk","prod-support","runbooks","sre"],"created_at":"2024-09-27T21:04:10.950Z","updated_at":"2026-01-15T22:46:17.060Z","avatar_url":"https://github.com/dvilaverde.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kubernetes CounterMeasures\n\n[![Build Status](https://github.com/dvilaverde/k8s-countermeasures/workflows/build/badge.svg)](https://github.com/dvilaverde/k8s-countermeasures/actions)\n\n**Project status: *alpha*** Not all planned features are completed. The API, spec,\nstatus and other user facing objects may change, but in a backward compatible way.\n\nPackaging scripts and instructions for deployment are still in progress\nand looking for contributors.\n\n## TL;DR\n\nKubernetes Operator which allows for automating manual actions, normally documeneted in applcation runbooks and executed \nby Ops or SRE staff, in reaction to an application alert. Simple examples include:\n\n* deleting/restarting a pod on application error that doesn't cause liveness/readiness probes to restart\n* taking a Java thread-dump or enabling profiler such as async-profiler on high CPU usage alert\n\nFor more detailed [examples](config/samples/) and use cases see the [README](docs/README.md) in\nthe [docs](docs) folder.\n\n## Overview\n\nThis project aims to define a API and controller in Kubernetes to codify\nproject runbooks, allowing for automation of actions that are manually\ntaken when on on-call engineer receives an alert.\n\nFor example, imagine a Java application with a runbook that defines when an alert\nfor high CPU is received, the on-call engineer is to take a thread-dump for analysis.\nDoing this manually may prove difficult depending on how long the high CPU event\nlasts and the engineer availability, and whether or not the\ncontainer has the debug tools required.\n\nThis project allows for the automation of the above runbook task by using an operator\nwritten using the [OperatorSDK](https://sdk.operatorframework.io) and a few CRDs\nto define the `event` to monitor and the `actions` to take.\n\nThe operator allows for deployment of an event source, currently only Prometheus\nis supported, and a countermeasure that defines one or more actions. The event source\nwill publish events into an internal event bus to be conssumed by the countermeasures.\n\n## Prerequisites\n\nThe Kubernetes CounterMeasures Operator uses [Ephemeral Containers](https://v1-25.docs.kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/)\nwhich was *alpha* in Kubernetes `1.22.0`, *beta* in `1.23.0`, and stable in `\u003e=1.25.0`.\nTherefore it is recommended to use verion `\u003e=1.25.0`, but development and testing\nwas done with a Kubernetes cluster of version `\u003e=1.23.0`.\n\n## CustomResourceDefinitions\n\nA core feature of the Kubernetes CounterMeasures Operator is to monitor\nthe Kubernetes API server for changes to specific objects and ensure that\nyour application is monitored for any undesirable conditions and when detected\nthe appropriate actions are taken as a counter measure.\nThe Operator acts on the following [custom resource definitions (CRDs)](https://kubernetes.io/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/):\n\n* **`CounterMeasure`**, which defines a condition to watch for and actions to take\nwhen it occurs.\n* **`Prometheus`**, which defines an event source that trigger the counter measures.\n\nThe Kubernetes CounterMeasures operator automatically detects changes\nin the Kubernetes API server to any of the above objects, and ensures\nyour the monitors are updated.\n\nTo learn more about the CRDs introduced by the Kubernetes CounterMeasures Operator\nhave a look at the [documentation](docs/actions.md).\n\n## Dynamic Admission Control\n\nTo provide validation an [admission webhook](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/)\nis provided to validate CRD resources upon initial creation or update\nor during dry run.\n\nFor more information on this feature, see the [user guide](docs/webhook.md).\n\n## Quickstart\n\nTo quickly try out the Kubernetes CounterMeasures Operator inside a [Kind](https://kind.sigs.k8s.io)\ncluster, run the following command:\n\n```bash\n./hack/start-cluster.sh\nmake install\nmake deploy\n```\n\nTo run the Operator outside of a cluster instead of running `make deploy`, use:\n\n```bash\nmake run\n```\n\n## Removal\n\nTo remove the operator, first delete any custom resources you created in each namespace.\n\n```bash\nfor n in $(kubectl get namespaces -o jsonpath={..metadata.name}); do\n  kubectl delete --all --namespace=$n countermeasure\ndone\n```\n\nAfter a couple of minutes you can go ahead and remove the operator itself.\n\n```bash\nmake undeploy\nmake uninstall\n```\n\n## Development\n\n### Prerequisites\n\n* golang environment\n* docker (used for creating container images, etc.)\n* kind (optional)\n\n### Testing\n\n#### Running *unit tests*\n\n`make test`\n\n### Debugging\n\nTo debug the controller locally against a running K8s cluster, add this entry to\nthe `/etc/hosts` file so that the operator can communicate with Prometheus.\n\n```text\n##\n# Host Database\n#\n# localhost is used to configure the loopback interface\n# when the system is booting.  Do not change this entry.\n##\n127.0.0.1 localhost\n# Add for k8s-countermeasures debugging\n127.0.0.1 prometheus-operated.monitoring.svc \n```\n\nthen enable port forwarding from the development host to the promtheus service:\n\n```bash\nkubectl -n monitoring port-forward service/prometheus-operated 9090:9090\n```\n\n## Contributing\n\nMany files (documentation, manifests, ...) in this repository are\nauto-generated. Before proposing a pull request:\n\n1. Commit your changes.\n2. Run `make generate`.\n3. Commit the generated changes.\n\n## Security\n\nIf you find a security vulnerability related to the Kubernetes CounterMeasures\nOperator, please do not report it by opening a GitHub issue, but instead please\nsend an e-mail to the owner of this project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvilaverde%2Fk8s-countermeasures","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdvilaverde%2Fk8s-countermeasures","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvilaverde%2Fk8s-countermeasures/lists"}