{"id":18732476,"url":"https://github.com/metrico/distributed-clickhouse-prometheus","last_synced_at":"2026-01-28T06:07:31.647Z","repository":{"id":51059881,"uuid":"519765166","full_name":"metrico/distributed-clickhouse-prometheus","owner":"metrico","description":"Distributed Metrics for ClickHouse + URL Engine (How-To)","archived":false,"fork":false,"pushed_at":"2022-08-04T09:11:36.000Z","size":74,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-06-03T14:12:56.222Z","etag":null,"topics":["clickhouse","clickhouse-server","deta","deta-base","distributed","metrics","prometheus","serverless","timeseries","url-engine"],"latest_commit_sha":null,"homepage":"https://urleng.com","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/metrico.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-31T12:06:36.000Z","updated_at":"2024-11-16T07:46:40.000Z","dependencies_parsed_at":"2022-09-08T02:26:18.107Z","dependency_job_id":null,"html_url":"https://github.com/metrico/distributed-clickhouse-prometheus","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/metrico/distributed-clickhouse-prometheus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metrico%2Fdistributed-clickhouse-prometheus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metrico%2Fdistributed-clickhouse-prometheus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metrico%2Fdistributed-clickhouse-prometheus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metrico%2Fdistributed-clickhouse-prometheus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/metrico","download_url":"https://codeload.github.com/metrico/distributed-clickhouse-prometheus/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metrico%2Fdistributed-clickhouse-prometheus/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28840394,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T05:50:12.573Z","status":"ssl_error","status_checked_at":"2026-01-28T05:49:54.528Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clickhouse","clickhouse-server","deta","deta-base","distributed","metrics","prometheus","serverless","timeseries","url-engine"],"created_at":"2024-11-07T15:06:21.520Z","updated_at":"2026-01-28T06:07:31.629Z","avatar_url":"https://github.com/metrico.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"https://user-images.githubusercontent.com/1423657/182806197-3c20e379-8f1c-4113-bbf4-f3940a7a601a.png\" width=800 /\u003e\n\n\n# _\"Frugal Metrics\"_ \n\n_Metrics can be lots fun, until you have to share them between a bunch of distributed servers... but fear not!_\n\n\u003cbr /\u003e\n\n## Distributed timeseries/metrics endpoint with [UrlEng](https://urleng.com) + [ClickHouse](https://clickhouse.com/docs)\n\nLet's build a serverless prometheus metrics store \u0026 publisher for ClickHouse using [urleng.com](https://urleng.com)\n\n* no coding, no signups, no logins - batteries included! :battery:\t\n* zero config distributed table, simply accessible from anywhere\n* custom HTTP prometheus /metrics scraping endpoint w/ auto-expiration\n* persistent storage courtesy of [deta cloud](https://deta.sh) \n\n\u003cbr/\u003e\n\n## Let's get started!\n\n### Storing metrics in [UrlEng](https://urleng.com)\n\n\n[UrlEng](https://urleng.com) is a _free serverless pastie_ made for ClickHouse tables.\u003cbr\u003e\n\nLet's use it to store a distributed table with a simple schema for our metrics:\n\n| key | help | value | tags |  __expires |\n|---  |---          |---    |---   |---        |\n| metric name | metric help text | metric value | optional json tags | optional expiration unix ts |\n\nTo begin we will keep things basic - performance is not the challenge here but rather flexibiity. A few notes:\n\n* _INSERTs for the same key are considered UPDATEs in URL tables_\n* _JSON Tags must be present or empty stringed_\n* _Metrics can auto-expire using the `__expires` column and a future unix timestamp_\n* **REPLACE `/metrixxx` WITH YOUR OWN UNIQUE STRING OR BE DOOMED**\n\n\u003cbr/\u003e\n\nSimple it is! Let's **INSERT** our first little metric:\n\n```sql\nINSERT INTO FUNCTION url('https://urleng.com/metrixxx', JSONEachRow, 'key String, help String, value Float64, __expire UInt64, tags String') VALUES ('mygauge', 'my little metric', 100, toUnixTimestamp(now()+300), '{\"le\":\"0.05\"}')\n```\n\n\u003cbr/\u003e\n\nThat's easy. Let's take a closer look at our serverless URL table:\n\n```sql\nSELECT * FROM url('https://urleng.com/metrixxx', JSONEachRow)\n\n┌─__expires──┬─help──────────────────┬─key──────┬─tags──────────┬─value─┐\n│ 1659015366 │ my little metric      │ mygauge  │ {\"le\":\"0.05\"} │ 100   │\n└────────────┴───────────────────────┴──────────┴───────────────┴───────┘\n\n```\n\nLooking good! Our distributed metrics table is ready. _Get a materialized view up to insert some fun data._\n\n:tooth: ... but the output is still a bit boring and can't be scraped, so its time to change the output format!\n\n\u003cbr/\u003e\n\n### FORMAT Prometheus\nClickHouse comes with a built in suport for the Prometheus format - its a little strict, but it works, so let's use it!\n```sql\nSELECT ifNull(key, 'undefined') as name, ifNull(toFloat64(value),0) as value, help as help, CAST(JSONExtractKeysAndValues(replaceAll(ifNull(tags,''), '\\'','\\\"'), 'String'), 'Map(String, String)') as labels FROM url('https://urleng.com/metrixxx', JSONEachRow) FORMAT Prometheus\n```\n```\n# HELP mygauge my little metric\nmygauge{le=\"0.05\"} 100\n\n```\n\n\u003c!--\n\n### FORMAT Template\n\nClickHouse [templates](https://clickhouse.com/docs/en/interfaces/formats/#format-template) can be used to handle custom defined formats based on flexible template definitions.\n\nLet's use it to output _\"prometheus looking\"_ metrics from our ClickHouse metric queries:\n\n##### ROW Template\n- Create a row format template for prometheus in `/var/lib/clickhouse/format_schemas/row_out.format`\n```\n# HELP ${0:XML} ${1:XML}\n# TYPE ${0:XML} gauge\n${0:XML}${3:XML} ${2:XML}\n```\n##### PAGE Template\n- Create a page template to display all data in `/var/lib/clickhouse/format_schemas/prom_out.format`\n\n```\n${data}\n```\n\n- Does it work? Let's execute a test SELECT using `system.metrics`\n```sql\nSELECT metric, help, value, '' FROM system.metrics ORDER BY value DESC LIMIT 5 FORMAT Template SETTINGS\nformat_template_resultset = 'prom_out.format', format_template_row = 'row_out.format', format_template_rows_between_delimiter = '\\n'\n```\n\n```\n# HELP MemoryTracking Total amount of memory (bytes) allocated by the server.\n# TYPE MemoryTracking gauge\nMemoryTracking 1416436388\n# HELP MMappedFileBytes Sum size of mmapped file regions.\n# TYPE MMappedFileBytes gauge\nMMappedFileBytes 460284336\n```\n\n##### All together now!\nWe're ready. Let's query our URL table metrics using our Prometheus output template:\n\n```sql\nSELECT key, help, toFloat64(value), tags \nFROM url('https://urleng.com/metrixxx', JSONEachRow)\nORDER BY value DESC LIMIT 5 FORMAT Template SETTINGS\nformat_template_resultset = 'prom_out.format', format_template_row = 'row_out.format', format_template_rows_between_delimiter = '\\n'\n```\n\n```\n# HELP mygauge my little metric\n# TYPE mygauge gauge\nmygauge{type=\"one\"} 100\n```\n\n--\u003e\n\nWe're almost there - all we need is an endpoint we can scrape and luckily, ClickHouse does that too!\n\n\u003cbr/\u003e\n\n### Metrics HTTP Handler\n\nSo let's use all the ClickHouse features we can, shall we? \u003cbr\u003e\nCreate a [custom handler](https://clickhouse.com/docs/en/interfaces/http#predefined_http_interface) for our metrics query, ie: ```/etc/clickhouse-server/config.d/metric_export.xml```\n```xml\n\u003cyandex\u003e\n   \u003cformat_schema_path\u003e/var/lib/clickhouse/format_schemas/\u003c/format_schema_path\u003e\n   \u003ccustom_urleng\u003e'https://urleng.com/metrixxx\u003c/custom_urleng\u003e\n   \u003chttp_handlers\u003e\n      \u003crule\u003e\n        \u003curl\u003e/metrics\u003c/url\u003e\n        \u003cmethods\u003eGET\u003c/methods\u003e\n        \u003chandler\u003e\n            \u003ctype\u003epredefined_query_handler\u003c/type\u003e\n            \u003cquery\u003eSELECT ifNull(key, 'undefined') as name, ifNull(toFloat64(value),0) as value, help, CAST(JSONExtractKeysAndValues(replaceAll(ifNull(tags,''), '\\'','\\\"'), 'String'), 'Map(String, String)') as labels FROM url(getSetting('custom_urleng'), JSONEachRow) FORMAT Prometheus\u003c/query\u003e\n        \u003c/handler\u003e\n      \u003c/rule\u003e\n      \u003cdefaults/\u003e\n   \u003c/http_handlers\u003e\n\u003c/yandex\u003e\n```\n\n\u003cbr/\u003e\n\n## Scrape it and Shake it!\n_Et Voila'!_ Our custom endpoint is ready to be scraped. Let's curl a final test:\n```bash\ncurl 'http://default:password@localhost:8123/metrics'\n```\n```\n# TYPE mygauge gauge\nmygauge{le=\"0.05\"} 100\n\n```\n\n:postbox: _Easy and Fun, isn't it?_ Go ahead, update your metrics from any server/service and scrape away! \n\n\u003cbr /\u003e\n\n### Bonus Steps\n\n:warning: _This guide is intended as a firestarter - go crazy adding timestamping, output formats and anything else!_\n\n#### URL Engine Table\nGoing to use your URL engine store daily? Extend the schema and setup a URL Engine table for quicker access:\n\n```sql\nCREATE TABLE default.prometheus_exporter\n(\n `key` String,\n `value` Float64, \n `help` String,\n `type` String,\n `tags` String,\n `__expires` UInt64 DEFAULT toUnixTimestamp(now()+300),\n)\nENGINE = URL('https://urleng.com/metrixxx', 'JSONEachRow')\n ```\n ```sql\nINSERT INTO default.prometheus_exporter VALUES ('mygauge', 100, 'my little metric', 'gauge', '{\"le\":\"0.05\"}', toUnixTimestamp(now()+300))\n```\n\n----\n\n\n\u003c!--\n#### INSERT w/ Template\n\nLet's parse a fictional query metric\n```\nsome_metric 42 \n```\n\n- Create a row format template in `/opt/ch/row_in.format`\n\n```${name:CSV} ${value:CSV}```\n  \n- Create a page template in `/opt/ch/prom_out.format`\n\n```${data}```\n\n- Execute a query\n```\nINSERT INTO UserActivity FORMAT Template SETTINGS\nformat_template_resultset = '/opt/ch/prom_in.format', format_template_row = '/opt/ch/row_in.format'\n```\n\n--\u003e\n\n\u003cbr/\u003e\n\n## Disclaimers\n\n#### Terms of Use\nNo warranties of any kind, either express or implied. Data can be removed or lost at any moment of time. Use at your own risk.\n#### Security\nThis service does not provide any security or privacy. Traffic is secured by Cloudflare and data stored on Deta cloud. Use at your own risk.\n#### Cookies\nThis service does not collect, store or use cookies.\n#### Limitations\nWorker execution time limits might crop large INSERT queries\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetrico%2Fdistributed-clickhouse-prometheus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmetrico%2Fdistributed-clickhouse-prometheus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetrico%2Fdistributed-clickhouse-prometheus/lists"}