{"id":13644585,"url":"https://github.com/xing/kubernetes-deployment-restart-controller","last_synced_at":"2025-09-09T21:48:32.408Z","repository":{"id":49869748,"uuid":"165273704","full_name":"xing/kubernetes-deployment-restart-controller","owner":"xing","description":"Kubernetes controller that restarts Deployments when referenced ConfigMaps or Secrets change.","archived":false,"fork":false,"pushed_at":"2023-10-11T23:25:04.000Z","size":95,"stargazers_count":99,"open_issues_count":2,"forks_count":8,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-05-14T09:21:18.370Z","etag":null,"topics":["k8s","kubernetes","olympus"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xing.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"docs/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-11T16:23:08.000Z","updated_at":"2025-04-04T14:42:20.000Z","dependencies_parsed_at":"2024-01-14T09:32:55.964Z","dependency_job_id":null,"html_url":"https://github.com/xing/kubernetes-deployment-restart-controller","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/xing/kubernetes-deployment-restart-controller","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xing%2Fkubernetes-deployment-restart-controller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xing%2Fkubernetes-deployment-restart-controller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xing%2Fkubernetes-deployment-restart-controller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xing%2Fkubernetes-deployment-restart-controller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xing","download_url":"https://codeload.github.com/xing/kubernetes-deployment-restart-controller/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xing%2Fkubernetes-deployment-restart-controller/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274367727,"owners_count":25272301,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-09T02:00:10.223Z","response_time":80,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["k8s","kubernetes","olympus"],"created_at":"2024-08-02T01:02:08.479Z","updated_at":"2025-09-09T21:48:32.361Z","avatar_url":"https://github.com/xing.png","language":"Go","funding_links":[],"categories":["Operators vs Controllers","Go"],"sub_categories":["Reloaders"],"readme":"# Kubernetes Deployment Restart Controller\n\nThis Kubernetes controller watches ConfigMaps and Secrets referenced by Deployments and\nStatefulSets and triggers restarts as soon as configuration or secret values change.\n\n## Installation\n\nThe [k8s-manifests] folder contains the necessary configuration files. Adjust them to your\ntaste and apply in the given order:\n\n    kubectl apply -f k8s-manifests/rbac.yaml -f k8s-manifests/deployment.yaml\n\nOptionally, create the [metrics Service]:\n\n    kubectl apply -f k8s-manifests/metrics-service.yaml\n\n## Configuration\n\nAutomatic restart functionality is enabled on per-Deployment (StatefulSet) basis.\n\nThe only thing you need to do is set the `com.xing.deployment-restart` annotation on the\ndesired Deployment (or StatefulSet) to `enabled`:\n\n```yml\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  annotations:\n    com.xing.deployment-restart: enabled\n  # the rest of the deployment manifest\n```\n\nController monitors deployment manifests and automatically watches or stops watching\nrelevant ConfigMaps and Secrets. It also stops restarting a deployment as soon as\nannotation is removed or changed to anything else than `enabled`.\n\n## Implementation Details\n\nKubernetes exhibits several constraints that shaped the implementation of the controller a\ngreat deal.\n\nEven though Kubernetes assigns a version to every resource deployed to the cluster, it is\nimpossible to figure out which version of a particular ConfigMap or Secret was used when a\nPod was started. Because of that, the controller maintains its own dataset of\nconfiguration object versions applied to Deployments and StatefulSets. Kubernetes resource\nversions get incremented when any part of the resource definition is changed, not just the\nconfiguration data. To avoid unnecessary restarts, the controller calculates a\nconfiguration data checksum for every ConfigMap and Secret and uses it instead of resource\nversion to detect changes.\n\nAnother constraint is related to resource discovery. Kubernetes Informers are asynchronous\nand do not guarantee that the client gets the most recent cluster changes immediately. The\nthe order in which the client gets the resource definitions is also not fixed. This makes\nit necessary for the controller to always operate on a potentially incomplete view of the\ncluster state. For example, the controller can become aware of a new Deployment resource\nrecently created in the cluster, but yet have no information on a ConfigMap referenced in\nthe Deployment manifest.\n\n### How It Works\n\nEvery deployment [configured for automatic restarts][installation and configuration]\neventually gets updated with a **checksums annotation**. The annotation contains names and\nchecksums of all the configs referenced by the deployment. The controller maintains this\ninformation and uses it to detect config changes.\n\nThe annotation itself is a JSON object:\n\n```yaml\nmetadata:\n  annotations:\n    com.xing.deployment-restart.applied-config-checksums: '{\"configmap/namespace-one/config-one\":\"189832cc316e7594\",\"secret/namespace-one/secret-two\":\"6e79832c18c31594\"}'\n```\n\nThe controller watches all ConfigMap, Secret, Deployment and StatefulSet resources in the\ncluster and builds a **catalog** of deployments and related configs in memory as resources\nbecome known to it.\n\nCatalog entries for config resources can represent either actual resources in the cluster\nand have **checksums** associated with them, or they can be dummies merely stating the\nfact that some of the known deployment resources reference a particular config name.\n\nDeployment entries in the catalog always represent real deployment objects in the cluster.\n\nWhen receiving the information about another config or deployment, controller can\ninstantiate a **change** object. There are several situations when it happens:\n\n * New config is added to the catalog.\n * New deployment is added to the catalog.\n * Known deployment references a config that it was not referencing before.\n * Known deployment does not reference a config that it was referencing before.\n * Known deployment has its checksums annotation changed.\n * Known config has its checksum changed.\n\nEvery change is identified by the name of the resource it was initiated by and has a\ntimestamp and a **counter** associated with it.\n\nInstantiated changes are stored in the queue without any actions to them for the amount of\ntime determined by [RESTART_GRACE_PERIOD setting][command line arguments]. If the resource\nis changed again during the grace period, the change counter gets incremented.\n\nAfter the grace period is exhausted, the change gets processed:\n\n1. Using the catalog, controller discovers deployments that are potentially affected by\nthe change. A deployment change can only affect the deployment itself, while a config\nchange affects all the deployments that reference the config.\n\n2. Every potentially affected deployment has its checksums annotation compared to the\ncurrent state of the catalog. Based on the comparison, controller decides if the\nannotation should be updated and if the deployment needs to be restarted.\n\n3. If an annotation update or a restart is necessary, controller patches the deployment.\nIt always issues a single patch request to update the annotation and restart the\ndeployment at the same time if necessary. A restart is triggered by setting the\n`com.xing.deployment-restart.timestamp` annotation in `spec.template.metadata.annotations`\nof the deployment.\n\nThere are two situations when a deployment restart is triggered by a config update:\n\n1. The deployment **already has a checksum of the updated config** in the checksums\nannotation, and the new checksum of the config is different. This is the \"normal\"\nsituation when a config that deployment has already been referencing for some time gets\nupdated.\n\n2. The deployment does not have a checksum of the updated config in the checksums\nannotation, but **the corresponding change counter is greater than one**. This is the\nsituation when a config got added to the deployment, but before the checksum got saved in\nthe deployment annotation, the config got updated again.\n\n### Caveats\n\n1. Combined with other automation tools, controller can cause deployments to be restarted\nmore often they need to be. For example, if some deployment pipeline takes longer than 5\nseconds ([by default][command line arguments]) to update two ConfigMaps that are\nreferenced by the same deployment, the deployment will be restarted twice. This can be\nmitigated by either changing the pipeline or increasing the restart check period.\n\n2. When forcefully terminated, the controller might miss some restarts. Consider the\nsituation: a new deployment is added to the cluster. Soon after that, a config referenced\nby that deployment is updated, and while that change is being on hold for the grace\nperiod, controller gets forcefully terminated. The fact that there was a config change\nobserved would not be stored anywhere, and another instance of the controller will just\nmark the config as already applied to the deployment. The chances of this happening are\nrather low since controller needs to be killed exactly during the grace period of a\ndeployment change followed by a config change.\n\n## Runtime Metrics\n\nThe controller exposes several metrics at `0.0.0.0:10254/metrics` endpoint in Prometheus\nformat. These metrics can be used to monitor the controller status and observe actions\nthat it takes.\n\nMetric | Type | Description\n------ | ---- | -----------\ndeployment_restart_controller_resource_versions_total | counter | The number of distinct resource versions observed.\ndeployment_restart_controller_configs_total | gauge | The number of tracked configs.\ndeployment_restart_controller_deployments_total | gauge | The number of tracked deployments.\ndeployment_restart_controller_deployment_annotation_updates_total | counter | The number of deployment annotation updates.\ndeployment_restart_controller_deployment_restarts_total | counter | The number of deployment restarts triggered.\ndeployment_restart_controller_changes_processed_total | counter | The number of resource changes processed.\ndeployment_restart_controller_changes_waiting_total | gauge |  The number of changes waiting in the queue.\n\n## Command Line Arguments\n\n```\nUsage:\n  kubernetes-deployment-restart-controller [OPTIONS]\n\nApplication Options:\n  -c, --restart-check-period= Time interval to check for pending restarts in milliseconds\n                              (default: 500) [$RESTART_CHECK_PERIOD]\n  -r, --restart-grace-period= Time interval to compact restarts in seconds (default: 5)\n                              [$RESTART_GRACE_PERIOD]\n      --ignored-errors=       List of error patterns to just warn of, instead of exiting the controller.\n                              Useful if previously legal objects are not valid anymore but have not yet\n                              been updated, e.g. on admission control changes. ENV var splits on ;\n                              (semicolon). [$IGNORED_ERRORS]\n  -v, --verbose=              Be verbose [$VERBOSE]\n      --version               Print version information and exit\n\nHelp Options:\n  -h, --help                  Show this help message\n```\n\n## Development\n\nThis project uses go modules introduced by [go 1.14][go-modules]. Please put the project\nsomewhere outside of your GOPATH to make go automatically recogninze this.\n\nAll build and install steps are managed in the [Makefile](Makefile). `make test` will\nfetch external dependencies, compile the code and run the tests. If all goes well, hack\nalong and submit a pull request. You might need to run the `go mod tidy` after updating\ndependencies.\n\n### Releases\n\nReleases are a two-step process, beginning with a manual step:\n\n* Create a release commit\n  * Increase the version number in [kubernetes-deployment-restart-controller.go/VERSION](kubernetes-deployment-restart-controller.go#25)\n  * Adjust the [CHANGELOG](CHANGELOG.md)\n* Run `make release`, which will create an image, retrieve the version from the\n  binary, create a git tag and push both your commit and the tag\n\nThe Travis CI run will then realize that the current tag refers to the current master commit and\nwill tag the built docker image accordingly.\n\n\n[22368]: https://github.com/kubernetes/kubernetes/issues/22368\n[implementation details]: #implementation-details\n[metrics service]: #runtime-metrics\n[k8s-manifests]: k8s-manifests\n[installation and configuration]: #installation-and-configuration\n[command line arguments]: #command-line-arguments\n[go-modules]: https://github.com/golang/go/wiki/Modules\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxing%2Fkubernetes-deployment-restart-controller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxing%2Fkubernetes-deployment-restart-controller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxing%2Fkubernetes-deployment-restart-controller/lists"}