{"id":13839380,"url":"https://github.com/pingcap/dead-mans-switch","last_synced_at":"2025-04-07T08:33:03.829Z","repository":{"id":46741822,"uuid":"279315220","full_name":"pingcap/dead-mans-switch","owner":"pingcap","description":"A bypass monitoring prober","archived":false,"fork":false,"pushed_at":"2021-09-28T01:03:41.000Z","size":59,"stargazers_count":33,"open_issues_count":2,"forks_count":16,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-03-22T16:02:11.273Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pingcap.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-13T13:48:06.000Z","updated_at":"2024-10-04T16:59:49.000Z","dependencies_parsed_at":"2022-09-08T10:01:45.111Z","dependency_job_id":null,"html_url":"https://github.com/pingcap/dead-mans-switch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Fdead-mans-switch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Fdead-mans-switch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Fdead-mans-switch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Fdead-mans-switch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pingcap","download_url":"https://codeload.github.com/pingcap/dead-mans-switch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247620852,"owners_count":20968299,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T17:00:21.203Z","updated_at":"2025-04-07T08:32:59.179Z","avatar_url":"https://github.com/pingcap.png","language":"Go","readme":"# dead-mans-switch\ndead mans switch is a simple prometheus alert manager webhook service. it provides a basic mechanisms to ensure alerting pipeline is healthy.\n\n\nprometheus provider a mechanisms to always firing. this is call `WatchDog` in prometheus-operator. If `WatchDog` does not firing, we can think alerting pipeline is unhealthy.\ndead mans switch can used for receive `WatchDog` webhook payload. evaluate alerting payload as expected optional. (sometimes your have a aggregated alert manager services, will have multiply prometheus\ninstance send `WatchDog` alert to alert manager). we can use evaluator to ensure all prometheus is healthy and external label as expected.\n\n```yaml\n- alert: Watchdog\n  annotations:\n    message: |\n      This is an alert meant to ensure that the entire alerting pipeline is functional.\n      This alert is always firing, therefore it should always be firing in Alertmanager\n      and always fire against a receiver. There are integrations with various notification\n      mechanisms that send a notification when this alert is not firing. For example the\n      \"DeadMansSnitch\" integration in PagerDuty.\n  expr: vector(1)\n  labels:\n    severity: none\n```\n      \n## Develop\nbuild binary in local environment\n```sh\nmake build\n```\n\nrunning in local environment\n```sh\ndms -config ./manifest/config.example.yaml\n```\n\nsend alert manager webhook payload\n```sh\ncurl -H \"Content-Type: application/json\" --data @payload.json http://localhost:8080/webhook\n```\n\n## Deploy\n\nThe `manifest/deploy` directory have k8s deploy yaml files, you can copy it and update \u003cpagerduty\u003e in configmap.\nThe `manifest/monitoring` directory have `ServiceMonitor` and `PrometheusRule` crd file, if you use prometheus-operator monitor your k8s clusters, you can trying for it.\n\n### AlertManager config\nlet WatchDog alert send to dead mans switch receivers\n```yaml\nroute:\n  routes:\n    - receiver: dead-mans-switch\n      group_wait: 10s\n      group_interval: 30s\n      repeat_interval: 15s\n      match:\n        alertname: 'Watchdog'\n```\n\nadd dead mans switch service as a new webhook receiver\n```yaml\nreceivers:\n- name: dead-mans-switch\n  webhook_configs:\n  - url: http://dead-mans-switch:8080/webhook\n```","funding_links":[],"categories":["Observability"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpingcap%2Fdead-mans-switch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpingcap%2Fdead-mans-switch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpingcap%2Fdead-mans-switch/lists"}