{"id":13563283,"url":"https://github.com/free/sql_exporter","last_synced_at":"2025-04-04T17:09:29.709Z","repository":{"id":38240321,"uuid":"110008810","full_name":"free/sql_exporter","owner":"free","description":"Database agnostic SQL exporter for Prometheus","archived":false,"fork":false,"pushed_at":"2022-01-19T12:48:53.000Z","size":6250,"stargazers_count":555,"open_issues_count":79,"forks_count":174,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-03-28T16:11:09.172Z","etag":null,"topics":["database","dbms","mysql","postgresql","prometheus","sql-exporter","sql-server"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/free.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-08T17:35:16.000Z","updated_at":"2025-03-18T07:30:11.000Z","dependencies_parsed_at":"2022-07-12T17:13:23.753Z","dependency_job_id":null,"html_url":"https://github.com/free/sql_exporter","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/free%2Fsql_exporter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/free%2Fsql_exporter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/free%2Fsql_exporter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/free%2Fsql_exporter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/free","download_url":"https://codeload.github.com/free/sql_exporter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247217221,"owners_count":20903009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","dbms","mysql","postgresql","prometheus","sql-exporter","sql-server"],"created_at":"2024-08-01T13:01:17.322Z","updated_at":"2025-04-04T17:09:29.693Z","avatar_url":"https://github.com/free.png","language":"Go","readme":"# Prometheus SQL Exporter [![Build Status](https://travis-ci.org/free/sql_exporter.svg)](https://travis-ci.org/free/sql_exporter) [![Go Report Card](https://goreportcard.com/badge/github.com/free/sql_exporter)](https://goreportcard.com/report/github.com/free/sql_exporter) [![GoDoc](https://godoc.org/github.com/free/sql_exporter?status.svg)](https://godoc.org/github.com/free/sql_exporter) [![Docker Pulls](https://img.shields.io/docker/pulls/githubfree/sql_exporter.svg?maxAge=604800)](https://hub.docker.com/r/githubfree/sql_exporter)\n\nDatabase agnostic SQL exporter for [Prometheus](https://prometheus.io).\n\n## Overview\n\nSQL Exporter is a configuration driven exporter that exposes metrics gathered from DBMSs, for use by the Prometheus\nmonitoring system. Out of the box, it provides support for MySQL, PostgreSQL, Microsoft SQL Server and Clickhouse, but\nany DBMS for which a Go driver is available may be monitored after rebuilding the binary with the DBMS driver included.\n\nThe collected metrics and the queries that produce them are entirely configuration defined. SQL queries are grouped into\ncollectors -- logical groups of queries, e.g. *query stats* or *I/O stats*, mapped to the metrics they populate.\nCollectors may be DBMS-specific (e.g. *MySQL InnoDB stats*) or custom, deployment specific (e.g. *pricing data\nfreshness*). This means you can quickly and easily set up custom collectors to measure data quality, whatever that might\nmean in your specific case.\n\nPer the Prometheus philosophy, scrapes are synchronous (metrics are collected on every `/metrics` poll) but, in order to\nkeep load at reasonable levels, minimum collection intervals may optionally be set per collector, producing cached\nmetrics when queried more frequently than the configured interval.\n\n## Usage\n\nGet Prometheus SQL Exporter, either as a [packaged release](https://github.com/free/sql_exporter/releases/latest), as a [Docker image](https://hub.docker.com/r/githubfree/sql_exporter) or\nbuild it yourself:\n\n```\n$ go install github.com/free/sql_exporter/cmd/sql_exporter\n```\n\nthen run it from the command line:\n\n```\n$ sql_exporter\n```\n\nUse the `-help` flag to get help information.\n\n```\n$ ./sql_exporter -help\nUsage of ./sql_exporter:\n  -config.file string\n      SQL Exporter configuration file name. (default \"sql_exporter.yml\")\n  -web.listen-address string\n      Address to listen on for web interface and telemetry. (default \":9399\")\n  -web.metrics-path string\n      Path under which to expose metrics. (default \"/metrics\")\n  [...]\n```\n\n## Configuration\n\nSQL Exporter is deployed alongside the DB server it collects metrics from. If both the exporter and the DB\nserver are on the same host, they will share the same failure domain: they will usually be either both up and running\nor both down. When the database is unreachable, `/metrics` responds with HTTP code 500 Internal Server Error, causing\nPrometheus to record `up=0` for that scrape. Only metrics defined by collectors are exported on the `/metrics` endpoint.\nSQL Exporter process metrics are exported at `/sql_exporter_metrics`.\n\nThe configuration examples listed here only cover the core elements. For a comprehensive and comprehensively documented\nconfiguration file check out \n[`documentation/sql_exporter.yml`](https://github.com/free/sql_exporter/tree/master/documentation/sql_exporter.yml).\nYou will find ready to use \"standard\" DBMS-specific collector definitions in the\n[`examples`](https://github.com/free/sql_exporter/tree/master/examples) directory. You may contribute your own collector\ndefinitions and metric additions if you think they could be more widely useful, even if they are merely different takes\non already covered DBMSs.\n\n**`./sql_exporter.yml`**\n\n```yaml\n# Global settings and defaults.\nglobal:\n  # Subtracted from Prometheus' scrape_timeout to give us some headroom and prevent Prometheus from\n  # timing out first.\n  scrape_timeout_offset: 500ms\n  # Minimum interval between collector runs: by default (0s) collectors are executed on every scrape.\n  min_interval: 0s\n  # Maximum number of open connections to any one target. Metric queries will run concurrently on\n  # multiple connections.\n  max_connections: 3\n  # Maximum number of idle connections to any one target.\n  max_idle_connections: 3\n\n# The target to monitor and the list of collectors to execute on it.\ntarget:\n  # Data source name always has a URI schema that matches the driver name. In some cases (e.g. MySQL)\n  # the schema gets dropped or replaced to match the driver expected DSN format.\n  data_source_name: 'sqlserver://prom_user:prom_password@dbserver1.example.com:1433'\n\n  # Collectors (referenced by name) to execute on the target.\n  collectors: [pricing_data_freshness]\n\n# Collector definition files.\ncollector_files: \n  - \"*.collector.yml\"\n```\n\n### Collectors\n\nCollectors may be defined inline, in the exporter configuration file, under `collectors`, or they may be defined in\nseparate files and referenced in the exporter configuration by name, making them easy to share and reuse.\n\nThe collector definition below generates gauge metrics of the form `pricing_update_time{market=\"US\"}`.\n\n**`./pricing_data_freshness.collector.yml`**\n\n```yaml\n# This collector will be referenced in the exporter configuration as `pricing_data_freshness`.\ncollector_name: pricing_data_freshness\n\n# A Prometheus metric with (optional) additional labels, value and labels populated from one query.\nmetrics:\n  - metric_name: pricing_update_time\n    type: gauge\n    help: 'Time when prices for a market were last updated.'\n    key_labels:\n      # Populated from the `market` column of each row.\n      - Market\n    static_labels:\n      # Arbitrary key/value pair\n      portfolio: income\n    values: [LastUpdateTime]\n    query: |\n      SELECT Market, max(UpdateTime) AS LastUpdateTime\n      FROM MarketPrices\n      GROUP BY Market\n```\n\n### Data Source Names\n\nTo keep things simple and yet allow fully configurable database connections to be set up, SQL Exporter uses DSNs (like\n`sqlserver://prom_user:prom_password@dbserver1.example.com:1433`) to refer to database instances. However, because the\nGo `sql` library does not allow for automatic driver selection based on the DSN (i.e. an explicit driver name must be\nspecified) SQL Exporter uses the schema part of the DSN (the part before the `://`) to determine which driver to use.\n\nUnfortunately, while this works out of the box with the [MS SQL Server](https://github.com/denisenkom/go-mssqldb) and\n[PostgreSQL](github.com/lib/pq) drivers, the [MySQL driver](github.com/go-sql-driver/mysql) DSNs format does not include\na schema and the [Clickhouse](github.com/kshvakov/clickhouse) one uses `tcp://`. So SQL Exporter does a bit of massaging\nof DSNs for the latter two drivers in order for this to work:\n\nDB | SQL Exporter expected DSN | Driver sees\n:---|:---|:---\nMySQL | `mysql://user:passw@protocol(host:port)/dbname` | `user:passw@protocol(host:port)/dbname`\nPostgreSQL | `postgres://user:passw@host:port/dbname` | *unchanged*\nSQL Server | `sqlserver://user:passw@host:port/instance` | *unchanged*\nClickhouse | `clickhouse://host:port?username=user\u0026password=passw\u0026database=dbname` | `tcp://host:port?username=user\u0026password=passw\u0026database=dbname`\n\n## Why It Exists\n\nSQL Exporter started off as an exporter for Microsoft SQL Server, for which no reliable exporters exist. But what is\nthe point of a configuration driven SQL exporter, if you're going to use it along with 2 more exporters with wholly\ndifferent world views and configurations, because you also have MySQL and PostgreSQL instances to monitor?\n\nA couple of alternative database agnostic exporters are available -- https://github.com/justwatchcom/sql_exporter and\nhttps://github.com/chop-dbhi/prometheus-sql -- but they both do the collection at fixed intervals, independent of\nPrometheus scrapes. This is partly a philosophical issue, but practical issues are not all that difficult to imagine:\njitter; duplicate data points; or collected but not scraped data points. The control they provide over which labels get\napplied is limited, and the base label set spammy. And finally, configurations are not easily reused without\ncopy-pasting and editing across jobs and instances.\n","funding_links":[],"categories":["Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffree%2Fsql_exporter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffree%2Fsql_exporter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffree%2Fsql_exporter/lists"}