{"id":13582096,"url":"https://github.com/prymitive/kthxbye","last_synced_at":"2025-05-08T00:33:10.291Z","repository":{"id":37470249,"uuid":"219291513","full_name":"prymitive/kthxbye","owner":"prymitive","description":"Prometheus Alertmanager alert acknowledgement management daemon","archived":false,"fork":false,"pushed_at":"2024-04-15T09:24:58.000Z","size":1149,"stargazers_count":174,"open_issues_count":4,"forks_count":9,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-04-16T06:52:39.871Z","etag":null,"topics":["alerting","alertmanager","prometheus","prometheus-utility"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prymitive.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"docs/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-11-03T11:38:44.000Z","updated_at":"2024-04-21T19:43:39.748Z","dependencies_parsed_at":"2023-10-16T20:12:17.958Z","dependency_job_id":"15960906-7c29-4afe-88e5-07d00e4ce254","html_url":"https://github.com/prymitive/kthxbye","commit_stats":null,"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prymitive%2Fkthxbye","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prymitive%2Fkthxbye/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prymitive%2Fkthxbye/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prymitive%2Fkthxbye/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prymitive","download_url":"https://codeload.github.com/prymitive/kthxbye/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225106472,"owners_count":17421804,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alerting","alertmanager","prometheus","prometheus-utility"],"created_at":"2024-08-01T15:02:25.556Z","updated_at":"2024-11-18T00:06:55.805Z","avatar_url":"https://github.com/prymitive.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# kthxbye\n\nkthxbye is a simple daemon that works as a sidecar for\n[Prometheus Alertmanager](https://github.com/prometheus/alertmanager) and will automatically extend expiring silences.\nThe goal of this project is to provide a simple way of acknowledging alerts,\nwhich is currently not possible with Alertmanager itself\n[see this issue](https://github.com/prometheus/alertmanager/issues/1860).\n\n## Current acknowledgment workflow with Alertmanager\n\nCurrently when a new alert fires in Alertmanager there are 2 options:\n\n- Leave the alert in active state while you work on resolving it\n- Silence this alert for some duration\n\nThis works well in small environments but can cause problems with big teams:\n\n- If you leave alert in active state you need to communicate somehow that\n  you are working on it, otherwise someone else on the team might start\n  working on it too, or (worse) nobody will work on it assuming that someone\n  else already does.\n- If you silence this alert you need to pick the correct duration.\n  If the duration is too short you might need to re-silence it again as it expires, which can be noisy with a lot of Alertmanager users.\n  If the duration is too long you need to remember to expire it after the issue\n  was resolved, otherwise it might mask other problems or issue reoccurring\n  soon.\n\nThere are tools to better manage alert ownership like PagerDuty or Opsgenie,\nwhich can help to avoid this problem, but they require sending all alerts\nto external escalation system.\n\n## How it works\n\nkthxbye will continuously extend silences that are about to expire but are\nmatching firing alerts. Silences will be allowed to expire only if they don't\nmatch any alert.\n\n- A new alert starts to fire in Alertmanager\n- User creates a silence for it with a comment that beings with predefined\n  prefix and short duration\n- kthxbye will continuously poll alerts and silences from Alertmanager:\n  - Get the list of all silences\n  - Get the list of all silenced alerts\n  - Find all silences where comments starts with predefined prefix and are\n    expiring soon\n  - For every such silence:\n    - Check if the silence matches any currently firing alerts\n      - If yes then silence duration will be extended\n      - If no then silence will be allowed to expire\n\nThis allows to silence an alert without worrying about picking correct duration\nfor the silence, so you effectively silence a specific indecent rather than\nthe alert.\n\n## Building binaries\n\nHave the most recent Go version and compile it using the usual `go build`\ncommand:\n\n```shell\ngo build ./...\n```\n\n## Docker images\n\nImages are built automatically for:\n\n- release tags in git - ghcr.io/prymitive/kthxbye:vX.Y.Z\n- main branch commits - ghcr.io/prymitive/kthxbye:latest\n\n## Usage\n\nStart kthxbye and pass the address of Alertmanager you want it to manage.\n\n```shell\nkthxbye -alertmanager.uri http://alertmanager.example.com:9093\n```\n\nBy default kthxbye will only extend silences with comment starting with `ACK!`,\nyou can set a custom prefix with:\n\n```shell\nkthxbye -extend-with-prefix \"MY CUSTOM PREFIX\"\n```\n\nBe sure to set `-extend-if-expiring-in` and `-extend-by` flags that match your\nenvironment.\n`-extend-if-expiring-in` tells kthxbye when to extend a silence, if you set it\nto `6m` then it will extend all silences if they would expire in the next\n6 minutes. Setting it to `90s` would tell kthxbye to extend silences expiring\nin the next 90 seconds.\n`-extend-by` tells kthxbye how much duration should be added to a silence when\nit's extended. Setting it to `10m` would tell kthxbye that exteding a silence\nwould update it to expire 10 minutes from now.\n`-max-duration` can be used to limit maximum duration a silence can reach, if\nset to a non-zero value it will let kthxbye to extend silence duration only if\ntotal duration (from the initial start time to the current expiry time) is less\nthan `-max-duration` value.\n\nNote: duration values must be compatible with Go\n[ParseDuration](https://golang.org/pkg/time/#ParseDuration) and so currently\nvalid time units are \"ns\", \"us\" (or \"µs\"), \"ms\", \"s\", \"m\", \"h\".\n\nBy default kthxbye will wake up and inspect silences every minute, you can\ncustomize it by passing `-interval` flag. Setting it to `30s` would tell kthxbye\nto wake up every 30 seconds.\n\nIt's recommended to run a single kthxbye instance per Alertmanager cluster.\nIf there are multiple instances running there will be a risk of races when\nupdating silences - two (or more) kthxbye instances might try to update the\nsame silence and, because Alertmanager expires the silence that's being updated\nand creates a new silence with new values, it can cause updated silence to be\nforked into two new silences.\nkthxbye can tolerate some downtime, with default settings it only considers\nsilences with less than 5 minutes left, which means that it only needs to\nrun once during those 5 minutes to update such silence. This applies to each\nsilence individually, and they are unlikely to expire in the same time, so in\nreality more than one execution per 5 minutes is required, but it means that\nsmall amount of downtime that might be needed to restart kthxbye on another\nk8s/mesos node (for cloud deployments) if the primary crashes is likely to be\ninvisible to the user.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprymitive%2Fkthxbye","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprymitive%2Fkthxbye","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprymitive%2Fkthxbye/lists"}