{"id":24961786,"url":"https://github.com/raffis/mongodb-query-exporter","last_synced_at":"2025-04-04T16:17:10.070Z","repository":{"id":41832674,"uuid":"236441425","full_name":"raffis/mongodb-query-exporter","owner":"raffis","description":"Prometheus MongoDB aggregation query exporter","archived":false,"fork":false,"pushed_at":"2025-03-27T19:52:00.000Z","size":703,"stargazers_count":100,"open_issues_count":14,"forks_count":27,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-28T15:08:35.528Z","etag":null,"topics":["metrics","mongodb","mongodb-aggregation","mongodb-query","prometheus","prometheus-exporter"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/raffis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-27T07:46:20.000Z","updated_at":"2025-02-28T03:26:22.000Z","dependencies_parsed_at":"2024-06-05T05:49:45.474Z","dependency_job_id":"83a80b00-6d2e-4dfc-86b9-949e769e756f","html_url":"https://github.com/raffis/mongodb-query-exporter","commit_stats":null,"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raffis%2Fmongodb-query-exporter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raffis%2Fmongodb-query-exporter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raffis%2Fmongodb-query-exporter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/raffis%2Fmongodb-query-exporter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/raffis","download_url":"https://codeload.github.com/raffis/mongodb-query-exporter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247208190,"owners_count":20901570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["metrics","mongodb","mongodb-aggregation","mongodb-query","prometheus","prometheus-exporter"],"created_at":"2025-02-03T08:56:35.907Z","updated_at":"2025-04-04T16:17:10.048Z","avatar_url":"https://github.com/raffis.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Prometheus MongoDB query exporter\n[![release](https://github.com/raffis/mongodb-query-exporter/actions/workflows/release.yaml/badge.svg)](https://github.com/raffis/mongodb-query-exporter/actions/workflows/release.yaml)\n[![Go Report Card](https://goreportcard.com/badge/github.com/raffis/mongodb-query-exporter/v5)](https://goreportcard.com/report/github.com/raffis/mongodb-query-exporter/v5)\n[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/raffis/mongodb-query-exporter/badge)](https://api.securityscorecards.dev/projects/github.com/raffis/mongodb-query-exporter)\n[![Coverage Status](https://coveralls.io/repos/github/raffis/mongodb-query-exporter/badge.svg?branch=master)](https://coveralls.io/github/raffis/mongodb-query-exporter?branch=master)\n[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/mongodb-query-exporter)](https://artifacthub.io/packages/search?repo=mongodb-query-exporter)\n\nMongoDB aggregation query exporter for [Prometheus](https://prometheus.io).\n\n## Features\n\n* Support for gauge metrics\n* Pull and Push (Push is only supported for MongoDB \u003e= 3.6)\n* Supports multiple MongoDB servers\n* Metric caching support\n\nNote that this is not designed to be a replacement for the [MongoDB exporter](https://github.com/percona/mongodb_exporter) to instrument MongoDB internals. This application exports custom MongoDB metrics in the prometheus format based on the queries (aggregations) you want.\n\n## Installation\n\nGet Prometheus MongoDB aggregation query exporter, either as a binaray from the latest release or packaged as a [Docker image](https://github.com/raffis/mongodb-query-exporter/pkgs/container/mongodb-query-exporter).\n\n### Helm Chart\nFor kubernetes users there is an official helm chart for the MongoDB query exporter.\nPlease read the installation instructions [here](https://github.com/raffis/mongodb-query-exporter/blob/master/chart/mongodb-query-exporter/README.md).\n\n### Docker\nYou can run the exporter using docker (This will start it using the example config provided in the example folder):\n```sh\ndocker run -e MDBEXPORTER_CONFIG=/config/configv3.yaml -v $(pwd)/example:/config ghcr.io/raffis/mongodb-query-exporter:latest\n```\n\n## Usage\n\n```\n$ mongodb-query-exporter\n```\n\nUse the `-help` flag to get help information.\n\nIf you use [MongoDB Authorization](https://docs.mongodb.org/manual/core/authorization/), best practices is to create a dedicated readonly user with access to all databases/collections required:\n\n1. Create a user with '*read*' on your database, like the following (*replace username/password/db!*):\n\n```js\ndb.getSiblingDB(\"admin\").createUser({\n    user: \"mongodb_query_exporter\",\n    pwd: \"secret\",\n    roles: [\n        { role: \"read\", db: \"mydb\" }\n    ]\n})\n```\n\n2. Set environment variable `MONGODB_URI` before starting the exporter:\n\n```bash\nexport MDBEXPORTER_MONGODB_URI=mongodb://mongodb_query_exporter:secret@localhost:27017\n```\n\nNote: The URI is substituted using env variables `${MY_ENV}`, given that you may also pass credentials from other env variables. See the example bellow.\n\nIf you use [x.509 Certificates to Authenticate Clients](https://docs.mongodb.com/manual/tutorial/configure-x509-client-authentication/), pass in username and `authMechanism` via [connection options](https://docs.mongodb.com/manual/reference/connection-string/#connections-connection-options) to the MongoDB uri. Eg:\n\n```\nmongodb://CN=myName,OU=myOrgUnit,O=myOrg,L=myLocality,ST=myState,C=myCountry@localhost:27017/?authMechanism=MONGODB-X509\n```\n\n## Credentials from env variables\nYou can pass in credentials from env variables.\n\nGiven the following URI the exporter will look for the ENV variables called `MY_USERNAME` and `MY_PASSWORD` and automatically use them at the referenced position within the URI.\n```bash\nexport MY_USERNAME=mongodb_query_exporter\nexport MY_PASSWORD=secret\nexport MDBEXPORTER_MONGODB_URI=mongodb://${MY_USERNAME}:${MY_PASSWORD}@localhost:27017\n```\n\n## Access metrics\nThe metrics are by default exposed at `/metrics`.\n\n```\ncurl localhost:9412/metrics\n```\n\n## Exporter configuration\n\nThe exporter is looking for a configuration in `~/.mongodb_query_exporter/config.yaml` and `/etc/mongodb_query_exporter/config.yaml` or if set the path from the env `MDBEXPORTER_CONFIG`.\n\nYou may also use env variables to configure the exporter:\n\n| Env variable             | Description                              | Default |\n|--------------------------|------------------------------------------|---------|\n| MDBEXPORTER_CONFIG       | Custom path for the configuration        | `~/.mongodb_query_exporter/config.yaml` or `/etc/mongodb_query_exporter/config.yaml` |\n| MDBEXPORTER_MONGODB_URI  | The MongoDB connection URI               | `mongodb://localhost:27017`\n| MDBEXPORTER_MONGODB_QUERY_TIMEOUT | Timeout until a MongoDB operations gets aborted | `10` |\n| MDBEXPORTER_LOG_LEVEL    | Log level                                | `warning` |\n| MDBEXPORTER_LOG_ENCODING | Log format                               | `json` |\n| MDBEXPORTER_BIND         | Bind address for the HTTP server         | `:9412` |\n| MDBEXPORTER_METRICSPATH  | Metrics endpoint                         | `/metrics` |\n\nNote if you have multiple MongoDB servers you can inject an env variable for each instead using `MDBEXPORTER_MONGODB_URI`:\n\n1. `MDBEXPORTER_SERVER_0_MONGODB_URI=mongodb://srv1:27017`\n2. `MDBEXPORTER_SERVER_1_MONGODB_URI=mongodb://srv2:27017`\n3. ...\n\n## Configure metrics\n\nSince the v1.0.0 release you should use the config version v3.0 to profit from the latest features.\nSee the configuration version matrix bellow.\n\nExample:\n```yaml\nversion: 3.0\nbind: 0.0.0.0:9412\nlog:\n  encoding: json\n  level: info\n  development: false\n  disableCaller: false\nglobal:\n  queryTimeout: 3s\n  maxConnection: 3\n  defaultCache: 0\nservers:\n- name: main\n  uri: mongodb://localhost:27017\naggregations:\n- database: mydb\n  collection: objects\n  servers: [main] #Can also be empty, if empty the metric will be used for every server defined\n  metrics:\n  - name: myapp_example_simplevalue_total\n    type: gauge #Can also be empty, the default is gauge\n    help: 'Simple gauge metric'\n    value: total\n    overrideEmpty: true # if an empty result set is returned..\n    emptyValue: 0       # create a metric with value 0\n    labels: []\n    constLabels:\n      region: eu-central-1\n  cache: 0\n  mode: pull\n  pipeline: |\n    [\n      {\"$count\":\"total\"}\n    ]\n- database: mydb\n  collection: queue\n  metrics:\n  - name: myapp_example_processes_total\n    type: gauge\n    help: 'The total number of processes in a job queue'\n    value: total\n    labels: [type,status]\n    constLabels: {}\n  mode: pull\n  pipeline: |\n    [\n      {\"$group\": {\n        \"_id\":{\"status\":\"$status\",\"name\":\"$class\"},\n        \"total\":{\"$sum\":1}\n      }},\n      {\"$project\":{\n        \"_id\":0,\n        \"type\":\"$_id.name\",\n        \"total\":\"$total\",\n        \"status\": {\n          \"$switch\": {\n              \"branches\": [\n                 { \"case\": { \"$eq\": [\"$_id.status\", 0] }, \"then\": \"waiting\" },\n                 { \"case\": { \"$eq\": [\"$_id.status\", 1] }, \"then\": \"postponed\" },\n                 { \"case\": { \"$eq\": [\"$_id.status\", 2] }, \"then\": \"processing\" },\n                 { \"case\": { \"$eq\": [\"$_id.status\", 3] }, \"then\": \"done\" },\n                 { \"case\": { \"$eq\": [\"$_id.status\", 4] }, \"then\": \"failed\" },\n                 { \"case\": { \"$eq\": [\"$_id.status\", 5] }, \"then\": \"canceled\" },\n                 { \"case\": { \"$eq\": [\"$_id.status\", 6] }, \"then\": \"timeout\" }\n              ],\n              \"default\": \"unknown\"\n          }}\n      }}\n    ]\n- database: mydb\n  collection: events\n  metrics:\n  - name: myapp_events_total\n    type: gauge\n    help: 'The total number of events (created 1h ago or newer)'\n    value: count\n    labels: [type]\n    constLabels: {}\n  mode: pull\n  # Note $$NOW is only supported in MongoDB \u003e= 4.2\n  pipeline: |\n    [\n      { \"$sort\": { \"created\": -1 }},\n      {\"$limit\": 100000},\n      {\"$match\":{\n        \"$expr\": {\n          \"$gte\": [\n            \"$created\",\n            {\n              \"$subtract\": [\"$$NOW\", 3600000]\n            }\n          ]\n        }\n      }},\n      {\"$group\": {\n        \"_id\":{\"type\":\"$type\"},\n        \"count\":{\"$sum\":1}\n      }},\n      {\"$project\":{\n        \"_id\":0,\n        \"type\":\"$_id.type\",\n        \"count\":\"$count\"\n      }}\n    ]\n```\n\nSee more examples in the `/example` folder.\n\n### Info metrics\n\nBy defining no actual value field but set `overrideEmpty` to `true` a metric can sill be exported\nwith labels from the aggregation pipeline but the value is set to a static value taken from `emptyValue`.\nThis is useful for exporting info metrics which can later be used for join queries.\n\n```yaml\nservers:\n- name: main\n  uri: mongodb://localhost:27017\naggregations:\n- database: mydb\n  collection: objects\n  metrics:\n  - name: myapp_info\n    help: 'Info metric'\n    overrideEmpty: true\n    emptyValue: 1\n    labels:\n    - mylabel1\n    - mylabel2\n    constLabels:\n      region: eu-central-1\n  cache: 0\n  mode: pull\n  pipeline: `...`\n```\n\n\n## Supported config versions\n\n| Config version           | Supported since   |\n|--------------------------|-------------------|\n| `v3.0`                   | v1.0.0            |\n| `v2.0`                   | v1.0.0-beta5      |\n| `v1.0`                   | v1.0.0-beta1      |\n\n\n## Cache \u0026 Push\nPrometheus is designed to scrape metrics. During each scrape the mongodb-query-exporter will evaluate all configured metrics.\nIf you have expensive queries there is an option to cache the aggregation result by setting a cache ttl.\nHowever it is more effective to **avoid cache** and design good aggregation pipelines. In some cases a different scrape interval might also be a solution.\nFor individual aggregations and/or MongoDB servers older than 3.6 it might still be a good option though.\n\nA better approach is using push instead a static cache, see bellow.\n\nExample:\n```yaml\naggregations:\n- metrics:\n  - name: myapp_example_simplevalue_total\n    help: 'Simple gauge metric which is cached for 5min'\n    value: total\n  servers: [main]\n  mode: pull\n  cache: 5m\n  database: mydb\n  collection: objects\n  pipeline: |\n    [\n      {\"$count\":\"total\"}\n    ]\n```\n\nTo reduce load on the MongoDB server (and also scrape time) there is a push mode. Push automatically caches the metric at scrape time preferred (If no cache ttl is set). However the cache for a metric with mode push\nwill be invalidated automatically if anything changes within the configured MongoDB collection. Meaning the aggregation will only be executed if there have been changes during scrape intervals.\n\n\u003e**Note**: This requires at least MongoDB 3.6.\n\nExample:\n```yaml\naggregations:\n- metrics:\n  - name: myapp_example_simplevalue_total\n    help: 'Simple gauge metric'\n    value: total\n  servers: [main]\n  # With the mode push the pipeline is only executed if a change occured on the collection called objects\n  mode: push\n  database: mydb\n  collection: objects\n  pipeline: |\n    [\n      {\"$count\":\"total\"}\n    ]\n```\n\n## Debug\nThe mongodb-query-exporters also publishes a counter metric called `mongodb_query_exporter_query_total` which counts query results for each configured aggregation.\nFurthermore you might increase the log level to get more insight.\n\n## Used by\n* The balloon helm chart implements the mongodb-query-exporter to expose general stats from the MongoDB like the number of total nodes or files stored internally or externally.\nSee the [config-map here](https://github.com/gyselroth/balloon-helm/blob/master/unstable/balloon/charts/balloon-mongodb-metrics/templates/config-map.yaml).\n\n\nPlease submit a PR if your project should be listed here!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraffis%2Fmongodb-query-exporter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fraffis%2Fmongodb-query-exporter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraffis%2Fmongodb-query-exporter/lists"}