{"id":19096667,"url":"https://github.com/sapcc/absent-metrics-operator","last_synced_at":"2025-04-30T14:12:12.711Z","repository":{"id":37799913,"uuid":"280971437","full_name":"sapcc/absent-metrics-operator","owner":"sapcc","description":"Absent Metrics Operator creates metric absence alerts atop Kubernetes","archived":false,"fork":false,"pushed_at":"2024-04-12T13:13:59.000Z","size":10587,"stargazers_count":7,"open_issues_count":3,"forks_count":1,"subscribers_count":35,"default_branch":"master","last_synced_at":"2024-04-14T01:06:43.915Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sapcc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-07-19T23:53:39.000Z","updated_at":"2024-04-15T15:23:06.951Z","dependencies_parsed_at":"2023-12-28T03:07:02.643Z","dependency_job_id":"5431eaad-44da-461e-b7e8-6f1f85cdc399","html_url":"https://github.com/sapcc/absent-metrics-operator","commit_stats":null,"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sapcc%2Fabsent-metrics-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sapcc%2Fabsent-metrics-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sapcc%2Fabsent-metrics-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sapcc%2Fabsent-metrics-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sapcc","download_url":"https://codeload.github.com/sapcc/absent-metrics-operator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223783447,"owners_count":17201900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T03:37:26.038Z","updated_at":"2024-11-09T03:37:26.705Z","avatar_url":"https://github.com/sapcc.png","language":"Go","funding_links":[],"categories":["others"],"sub_categories":[],"readme":"# Absent Metrics Operator\n\n[![GitHub Release](https://img.shields.io/github/v/release/sapcc/absent-metrics-operator)](https://github.com/sapcc/absent-metrics-operator/releases/latest)\n[![CI](https://github.com/sapcc/absent-metrics-operator/actions/workflows/ci.yaml/badge.svg)](https://github.com/sapcc/absent-metrics-operator/actions/workflows/ci.yaml)\n[![Coverage Status](https://coveralls.io/repos/github/sapcc/absent-metrics-operator/badge.svg?branch=master)](https://coveralls.io/github/sapcc/absent-metrics-operator?branch=master)\n[![Go Report Card](https://goreportcard.com/badge/github.com/sapcc/absent-metrics-operator)](https://goreportcard.com/report/github.com/sapcc/absent-metrics-operator)\n\nIn this document:\n\n- [Terminology](#terminology)\n- [Overview](#overview)\n- [Motivation](#motivation)\n- [Installation](#installation)\n- [Usage](#usage)\n  - [Metrics](#metrics)\n\nIn other documents:\n\n- [Absence alert rule definition](./docs/absence-alert-rule-definition.md)\n- [Playbook for operators](./docs/playbook.md)\n\n## Terminology\n\nAn **_absence alert rule_** is an alert rule that alerts on the absence of a metric.\n\nA `PrometheusRule` is a custom resource defined by the [Prometheus\noperator](prometheus-operator), it is used to define a set of alerting rules. Within the\nabsent metrics operator documentation and source code, an **_AbsencePrometheusRule_** is a\n`PrometheusRule` resource created (and managed) by the absent metrics operator that\ndefines corresponding **_absence alert rules_** for the metrics that were used in the\nalert rule definitions in a `PrometheusRule`.\n\n## Overview\n\nThe absent metrics operator is a companion operator for the [Prometheus\nOperator][prometheus-operator].\n\nIt monitors all the `PrometheusRule` resources deployed across a\nKubernetes cluster and creates corresponding _absence alert rules_ for\nthe alert rules defined in those resources.\n\n## Motivation\n\nConsider the following alert rule definition:\n\n```yaml\nalert: ImportantAlert\nexpr: foo_bar \u003e 0\nfor: 5m\nlabels:\n  support_group: network\n  service: foo\n  severity: critical\nannotations:\n  summary: Data center is on fire!\n```\n\nThis alert would never trigger if the metric `foo_bar` does not exist in\nPrometheus.\n\nThis can be avoided by using the `absent()` function with the `or` operator so\nthe alert rule expression becomes:\n\n```\nabsent(foo_bar) or foo_bar \u003e 0\n```\n\nHowever, this gets tedious if you have hundreds of alerts deployed across the cluster.\nThere is also the element of human error, e.g. typo or forgetting to include\nthe `absent` function in the alert expression.\n\nThis problem is resolved by the absent metrics operator as it automatically creates the\ncorresponding alert rules that check and alert on metric absence.\n\nFor example, considering the alert rule mentioned above, the operator would generate the following _absence alert rule_:\n\n```yaml\nalert: AbsentNetworkFooBar\nexpr: absent(foo_bar)\nfor: 10m\nlabels:\n  context: absent-metrics\n  severity: info\n  support_group: network\n  service: foo\nannotations:\n  summary: missing foo_bar\n  description: The metric 'foo_bar' is missing. 'ImportantAlert' alert using it may not fire as intended.\n```\n\nRefer to the _absence alert rule_ [definition\ndocumentation](./doc/absence-alert-rule-definition.md) for more information on how these\nalerts are generated and defined.\n\n## Installation\n\nYou can build with `make`, install with `make install`, or `docker build`.\n\nThe `make install` target understands the conventional environment variables for choosing\ninstall locations: `DESTDIR` and `PREFIX`.\n\n## Usage\n\nFor usage instructions:\n\n```\nabsent-metrics-operator --help\n```\n\nIn case of a false positive, the operator can be disabled for a specific alert rule or the\nentire `PrometheusRule` resource. Refer to the [playbook for operators](./docs/playbook.md#disable-the-operator)\nfor instructions.\n\n### Metrics\n\nMetrics are exposed at port `9659`. This port has been\n[allocated](https://github.com/prometheus/prometheus/wiki/Default-port-allocations)\nfor the operator.\n\n| Metric                                              | Labels                                            |\n| --------------------------------------------------- | ------------------------------------------------- |\n| `absent_metrics_operator_successful_reconcile_time` | `prometheusrule_namespace`, `prometheusrule_name` |\n\n[prometheus-operator]: https://github.com/prometheus-operator/prometheus-operator\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsapcc%2Fabsent-metrics-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsapcc%2Fabsent-metrics-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsapcc%2Fabsent-metrics-operator/lists"}