{"id":20593011,"url":"https://github.com/fusakla/promruval","last_synced_at":"2025-04-04T22:02:34.865Z","repository":{"id":38297837,"uuid":"312422819","full_name":"FUSAKLA/promruval","owner":"FUSAKLA","description":"Validate Prometheus/Thanos/Mimir/Loki rules metadata and expression properties to match requirements and constrains of your setup.","archived":false,"fork":false,"pushed_at":"2025-03-17T09:00:16.000Z","size":9808,"stargazers_count":153,"open_issues_count":6,"forks_count":11,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-28T21:02:02.483Z","etag":null,"topics":["alerting","ci","prometheus","prometheus-rules","prometheus-rules-metadata","promruval","promtool","rules","testing","validation"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FUSAKLA.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-12T23:42:12.000Z","updated_at":"2025-03-19T10:23:54.000Z","dependencies_parsed_at":"2023-12-31T04:01:15.574Z","dependency_job_id":"e10e28ff-5e2a-40f0-a75b-317cb0422caa","html_url":"https://github.com/FUSAKLA/promruval","commit_stats":{"total_commits":131,"total_committers":7,"mean_commits":"18.714285714285715","dds":0.0992366412213741,"last_synced_commit":"414a2fc6008248c17fdef794f0d0f3f218a0f1ff"},"previous_names":[],"tags_count":41,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FUSAKLA%2Fpromruval","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FUSAKLA%2Fpromruval/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FUSAKLA%2Fpromruval/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FUSAKLA%2Fpromruval/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FUSAKLA","download_url":"https://codeload.github.com/FUSAKLA/promruval/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247256103,"owners_count":20909240,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alerting","ci","prometheus","prometheus-rules","prometheus-rules-metadata","promruval","promtool","rules","testing","validation"],"created_at":"2024-11-16T07:47:11.623Z","updated_at":"2025-04-04T22:02:34.844Z","avatar_url":"https://github.com/FUSAKLA.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"![image](https://github.com/user-attachments/assets/86877a06-63dd-46d1-bfde-43bdbfafe42e)\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/fusakla/promruval)](https://goreportcard.com/report/github.com/fusakla/promruval)\n[![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/fusakla/promruval/go.yaml)](https://github.com/FUSAKLA/promruval/actions?query=branch%3Amaster)\n[![Docker Pulls](https://img.shields.io/docker/pulls/fusakla/promruval)](https://hub.docker.com/r/fusakla/promruval)\n[![GitHub binaries download](https://img.shields.io/github/downloads/fusakla/promruval/total?label=Prebuilt%20binaries%20downloads)](https://github.com/FUSAKLA/promruval/releases/latest)\n\nPromtool allows users to verify syntactic correctness and test PromQL expressions.\nPromruval aims to validate the rules' metadata and expression properties\nto match requirements and constraints of the particular Prometheus cluster setup.\nUser defines his validation rules in a simple yaml configuration and passes them to\nthe promruval which validates specified files with Prometheus rules same way promtool does.\nUsually it would be used in the CI pipeline.\nYou can read a blog post about the motivation and\nusage [here](https://fusakla.medium.com/promruval-validating-prometheus-rules-9a29f5dc24d2)\nor [watch a lightning talk about it from PromCon](https://www.youtube.com/watch?v=YYSJ--KhlIo\u0026list=PLj6h78yzYM2PZb0QuIkm6ZY-xTuNA5zRO\u0026index=16)\n.\n\n### Example use-cases\n\n- Make sure the playbook, linked by an alert, is a valid URL and really exists.\n- Ensure the range selectors in the `expr` are not lower than three\n  times your Prometheus scrape interval.\n- Avoid querying more data than is retention of used Prometheus by inspecting\n  if the `expr` does not use older data than specified.\n- Make sure `expr` does not use any of the specified labels. Useful when using Thanos, to forbid\n  usage of external labels when alerting on Prometheus to avoid confusion.\n- Ensure alerts has the required labels expected by routing in Alertmanager\n  possibly with allowed values.\n- Make sure Alerts has the expected annotations for rendering the alert template.\n- Forbid usage of some labels or annotations if it got deprecated.\n- and many more...\n\n\u003e As a good starting point you can use the [`docs/default_validation.yaml`](docs/default_validation.yaml) which contains\nsome basic validations that are useful for most of the users.\n\nValidations are quite variable, so you can use them as you fit.\n\n### **👉 Full list of supported validations can be found [here](docs/validations.md).**\n\nIn case you would like to add some, please create a feature request!\n\n### Installation\n\nUsing [prebuilt binaries](https://github.com/FUSAKLA/promruval/releases/latest),\n[Docker image](https://hub.docker.com/r/fusakla/promruval) of build it yourself.\n\n ```bash\ngo install github.com/fusakla/promruval/v3@latest\n```\n\nor\n\n```\nmake build\n```\n\n#### Supported platforms\nPromruval is tested only on the linux amd64. It should work on other platforms as well, but it's not tested.\nEach release contains the binaries for linux, darwin and windows and different architectures (amd64, arm64).\nSo please use them with caution and report any issues.\n\n### Usage\n\n```bash\n$ ./promruval --help-long\nusage: promruval [\u003cflags\u003e] \u003ccommand\u003e [\u003cargs\u003e ...]\n\nPrometheus rules validation tool.\n\nFlags:\n      --[no-]help   Show context-sensitive help (also try --help-long and --help-man).\n  -c, --config-file=CONFIG-FILE ...\n                    Path to validation config file. Can be passed multiple times, only validationRules will be reflected from the additional configs.\n      --[no-]debug  Enable debug logging.\n\nCommands:\nhelp [\u003ccommand\u003e...]\n    Show help.\n\n\nversion\n    Print version and build information.\n\n\nvalidate [\u003cflags\u003e] \u003cpath\u003e...\n    Validate Prometheus rule files in YAML or jsonnet format using validation rules from config file(s).\n\n    -d, --disable-rule=DISABLE-RULE ...\n                                   Allows to disable any validation rules by it's name. Can be passed multiple times.\n    -e, --enable-rule=ENABLE-RULE ...\n                                   Only enable these validation rules. Can be passed multiple times.\n    -o, --output=[text,json,yaml]  Format of the output.\n        --[no-]color               Use color output.\n        --[no-]support-loki        Support Loki rules format.\n        --[no-]support-mimir       Support Mimir rules format.\n        --[no-]support-thanos      Support Thanos rules format.\n\nvalidation-docs [\u003cflags\u003e]\n    Print human readable form of the validation rules from config file.\n\n    -o, --output=[text,markdown,html]\n      Format of the output.\n```\n\n#### Jsonnet support\nPromruval supports the default YAML format (`.yaml` or `.yml`) of rule files but also supports rules written in [Jsonnet](https://jsonnet.org/) (`.jsonnet`).\nIf will be rendered using the [go-jsonnet](https://github.com/google/go-jsonnet) library and then validated as usual, so you don't have to evaluate those by yourself before running the validation.\n\nAdditionaly it supports also the configuration file in the Jsonnet format.\n\n#### Configuration composition\n\nThe `--config-file` flag can be passed multiple times. Promruval will append the additional validation rules from the\nother configs and override the other configurations. The late wins.\nThis allows you to use compose configuration for example if you have specific validations for rules.\n\n**Example**:\n\n```bash\nrules/\n  validations.yaml # Generic validations that apply to all rules\n  prometheus/\n     validations.yaml # Specific validations for Prometheus rules (different Prometheus URL, shorter data retention, no external labels etc)\n     rules.yaml\n  thanos/\n     validations.yaml # Specific validations for Thanos (different URL, longer retention etc)\n     rules.yaml\n```\n\nAnd Promruval would be run as\n\n```bash\npromruval validate --config-file ./rules/validation.yaml --config-file ./rules/prometheus/validation.yaml ./rules/prometheus/*.yaml\n```\n\n### Configuration\n\nPromruval uses a yaml or jsonnet (if config file ends with `.jsonnet`) configuration file to define the validation rules.\n\nBasic structure is:\n\n```yaml\n# OPTIONAL Overrides the annotation used for disabling rules.\ncustomExcludeAnnotation: my_disable_annotation\n\nprometheus:\n  # URL of the running prometheus instance to be used\n  url: https://foo.bar/\n  # OPTIONAL Skip TLS verification\n  insecureSkipTlsVerify: false\n  # OPTIONAL Relative path to a file containing a bearer token to be used for authentication (Bearer token can by set also using the PROMETHEUS_BEARER_TOKEN env variable, which has higher priority)\n  # NOTE: The value will have whitespace trimmed from the beginning and end.\n  bearerTokenFile: bearer_token.txt\n  # OPTIONAL Timeout for any request on the Prometheus instance\n  timeout: 30s\n  # OPTIONAL name of the file to save cache of the Prometheus calls for speedup\n  cacheFile: .promruval_cache.json\n  # OPTIONAL maximum age how old the cache can be to be used\n  maxCacheAge: 1h\n  # OPTIONAL offset(delay) of the query evaluation time (useful for consistency if using remote write for example).\n  queryOffset: 1m\n  # OPTIONAL how long into the past to look in queries supporting time range (just metadata queries for now).\n  queryLookback: 20m\n  # OPTIONAL HTTP headers to be added to the request\n  httpHeaders:\n    foo: bar\n\nvalidationRules:\n  # Name of the validation rule.\n  - name: example-validation\n    # What Prometheus rules to validate, possible values are: 'Group', 'Alert', 'Recording rule', 'All rules'.\n    scope: All rules\n    # List of validations to be used.\n    validations:\n      # Name of the validation type. See the /docs/validations.md.\n      - type: hasLabels\n        # Additional detaild that will be appended to the default error message. Useful to customize the error message.\n        additionalDetails: \"We do this because ...\"\n        # Parameters of the validation. See the /docs/validations.md for details on params of each validation.\n        params:\n          labels: [ \"severity\" ]\n        # OPTIONAL If you want to load the parameters from a separate file, you can use this option.\n        # Its value must be a relative path to the file from the location of the config file.\n        # The content of the file must be in the exact form as the expected params would be.\n        # The option is mutually exclusive with the `params` option.\n        # paramsFromFile: ./params.yaml\n      ...\n    # OPTIONAL If you want the rule validations to apply only to rules/groups which match specified criteria.\n    onlyIf: [] # Same syntax as the `validations` field, all the conditions must be met for the rule to be validated.\n```\n\nFor a complete list of supported validations see the [docs/validations.md](docs/validations.md).\n\nIf you want to see example configuration see the  [`examples/validation.yaml`](examples/validation.yaml).\n\n### How to run it\n\nIf you downloaded the [prebuilt binary](https://github.com/FUSAKLA/promruval/releases/latest) or built it on your own:\n\n```bash\npromruval validate --config-file=examples/validation.yaml examples/rules.yaml\n```\n\nOr using [Docker image](https://hub.docker.com/r/fusakla/promruval)\n\n```bash\ndocker run -it -v $PWD:/rules fusakla/promruval validate --config-file=/rules/examples/validation.yaml /rules/examples/rules.yaml\n```\n\n### Validation using live Prometheus instance\n\nEvent though these validations are useful, they may be flaky and dangerous for the Prometheus instance.\nIf you have large number of rules and run the check often the number of queries can be huge or the instance might go\ndown and your validation\nwould be flaky.\n\nTherefore, it's recommended to use this check as a warning and do not fail if it does not succeed.\nAlso consider running it rather periodically (for example once per day) instead of running it on every commit in CI.\n\n### Disabling validations\nThere are three ways you can disable certain validation:\n - [Using cmd line flag](#using-cmd-line-flag)\n - [Using YAML comments](#using-yaml-comments)\n - [Using PromQL expression comments](#using-promql-expression-comments)\n - [Using alert annotation](#using-alert-annotation)\n\n\u003e The last two are useful if you yse for example jsonnet to generate the rules.\n\u003e Then you can't use the YAML comments, but you can set the comments in the expression or alert annotations.\n\u003e Unfortunately those have limited scope of usage (recording rules cannot have annotations, cannot be disabled on the group or file level).\n\n#### Using cmd line flag\nIf you want to temporarily disable any of the validation rules for all the tested files,\nyou can use the `--disable-rule` flag with value corresponding to the `name`\nof the validation rule you want to disable. Can be passed multiple times.\n\nExample:\n```yaml\n# Promruval validation configuration\nvalidationRules:\n  - name: check-irate\n    scope: Alert\n    validations:\n      - type: expressionDoesNotUseIrate\n```\n\n```bash\npromruval validate --config-file examples/validation.yaml --disable-rule check-irate examples/rules.yaml\n```\n\n#### Using YAML comments\nYou can use comments in YAML to disable certain validations. This can be done on the file, group or rule level.\nThe comment should be in format `# ignore_validations: validationName1, validationName2, ...` where the `validationName`\nis the name of the validation as defined in the [docs/validations.md](./docs/validations.md).\n\n\u003e The `ignore_validations` prefix can be changed using the `customDisableComment` config option in the [config](#configuration).\n\nExample:\n```yaml\n# Disable for the whole file\n# ignore_validations: expressionDoesNotUseIrate\ngroups:\n  # Disable only for the following rule group\n  # ignore_validations: expressionDoesNotUseIrate\n  - name: group1\n    partial_response_strategy: abort\n    interval: 1m\n    limit: 10\n    rules:\n      # Disable only for the following rule\n      # ignore_validations: expressionDoesNotUseIrate\n      - record: recorded_metrics\n        expr: 1\n        labels:\n          foo: bar\n```\n\n#### Using PromQL expression comments\nSame way as in the YAML comments, you can use comments in the PromQL expression to disable certain validations.\nThe comment should be in the same format `# ignore_validations: validationName1, validationName2, ...` where the `validationName`\nis the name of the validation as defined in the [docs/validations.md](./docs/validations.md).\nThe comment can be present multiple times in the expression and can be anywhere in the expression.\n\n\u003e The `ignore_validations` prefix can be changed using the `customDisableComment` config option in the [config](#configuration).\n\nExample:\n```yaml\ngroups:\n  - name: test-group\n    rules:\n      - alert: test-alert\n        expr: |\n          # ignore_validations: expressionDoesNotUseIrate\n          irate(http_requests_total[5m]) # ignore_validations: expressionDoesNotUseIrate\n```\n\n#### Using alert annotation\nIf you can't(or don't want to) use the comments to disable validations, you can use the special annotation\n`disabled_validation_rules`. It represents comma separated list of **validation rule names** to be skipped for the particular alert.\nSince annotations are only available for alerts, **this method can be used only for alerts!**\n\n\u003e The `disabled_validation_rules` annotation name can be changed using the `customExcludeAnnotation` config option in the [config](#configuration).\n\nExample:\n\n```yaml\n# Promruval validation configuration\nvalidationRules:\n  - name: check-irate\n    scope: Alert\n    validations:\n      - type: expressionDoesNotUseIrate\n```\n\n```yaml\n# Prometheus rule file\ngroups:\n  - name: test-group\n    rules:\n      - alert: test-alert\n        expr: 1\n        annotations:\n          disabled_validation_rules: check-irate # Will disable the check-irate validation rule check for this alert\n```\n\n### Other monitoring solutions support\n\n#### Thanos\nIf you want to validate Thanos rules, use the `promruval validate --support-thanos` flag, otherwise you might get errors on unknown fields such as `partial_response_strategy`.\n\nYou can validate it using the [`hasValidPartialResponseStrategy`](./docs/validations.md#hasvalidpartialresponsestrategy) validation.\n\n#### Mimir\nIf you want to validate Mimir rules, use the `promruval validate --support-mimir` flag, otherwise you might get errors on unknown fields such as `source_tenants`.\n\nThe `source_tenants` can be validated using the [`hasSourceTenantsForMetrics`](./docs/validations.md#hassourcetenantsformetrics) or [`hasAllowedSourceTenants`](./docs/validations.md#hasallowedsourcetenants) validations for example.\n\n#### Loki\nIf you want to validate Mimir rules, use the `promruval validate --support-loki` flag, otherwise you might get errors on unknown fields such as `namespace` or `remote_write`.\n\nSince Loki has almost identical rule config as Prometheus, you can use the same validations for Loki rules.\nLoki has special validations for its expressions since it uses different query language [LogQL](https://grafana.com/docs/loki/latest/query/).\nTo see the LogQL specific validations see the [here](./docs/validations.md#logql-expression-validators).\n\n### Human readable validation description\n\nIf you want more human readable validation summary (for a documentation or generating readable git pages)\nyou can use the `validation-docs` command, see the [usage](#usage).\nIt should print out more human readable form than the configuration file is\nand supports multiple output formats such as `text`, `markdown` and `HTML`.\nSee the examples for the output for [Markdown](./examples/human_readable.md) and [HTML](./examples/human_readable.html).\n\n```bash\npromruval validation-docs --config-file examples/validation.yaml --output=html\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffusakla%2Fpromruval","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffusakla%2Fpromruval","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffusakla%2Fpromruval/lists"}