{"id":14972030,"url":"https://github.com/altinity/clickhouse-grafana","last_synced_at":"2025-05-13T20:11:11.925Z","repository":{"id":37338867,"uuid":"77918702","full_name":"Altinity/clickhouse-grafana","owner":"Altinity","description":"Altinity Grafana datasource plugin for ClickHouse®","archived":false,"fork":false,"pushed_at":"2025-05-12T15:23:20.000Z","size":1990045,"stargazers_count":742,"open_issues_count":15,"forks_count":120,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-05-12T16:40:41.476Z","etag":null,"topics":["clickhouse","clickhouse-datasource","grafana"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Altinity.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-01-03T13:18:13.000Z","updated_at":"2025-05-11T17:36:53.000Z","dependencies_parsed_at":"2023-07-12T23:48:57.940Z","dependency_job_id":"9458719b-d09d-40a1-b5d9-9854e82918d7","html_url":"https://github.com/Altinity/clickhouse-grafana","commit_stats":{"total_commits":982,"total_committers":43,"mean_commits":"22.837209302325583","dds":0.675152749490835,"last_synced_commit":"6cb3da17a76055fd4b853c39dc3f6fce18748b95"},"previous_names":["vertamedia/clickhouse-grafana"],"tags_count":69,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Altinity%2Fclickhouse-grafana","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Altinity%2Fclickhouse-grafana/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Altinity%2Fclickhouse-grafana/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Altinity%2Fclickhouse-grafana/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Altinity","download_url":"https://codeload.github.com/Altinity/clickhouse-grafana/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254020611,"owners_count":22000754,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clickhouse","clickhouse-datasource","grafana"],"created_at":"2024-09-24T13:46:14.810Z","updated_at":"2025-05-13T20:11:11.903Z","avatar_url":"https://github.com/Altinity.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Coverage Status](https://coveralls.io/repos/github/Altinity/clickhouse-grafana/badge.svg?branch=master)](https://coveralls.io/github/Altinity/clickhouse-grafana?branch=master)\n\n# Altinity Grafana datasource plugin for ClickHouse® (grafana Grafana 4.6+ supported)\n\nAltinity ClickHouse datasource plugin provides a support for [ClickHouse](https://clickhouse.tech) as a backend database.\n\nInitially plugin developed by Vertamedia, maintaned by Altinity since 2020.\n\n## Quick start\n\n### Grafana 10+ setup notes for plugin version before 3.0.0\n\nOld versions of Altinity ClickHouse datasource plugin for Grafana written in Angular. So you can watch warning like \n```\nAngular plugin\nThis data source plugin uses a deprecated, legacy platform based on AngularJS and will stop working in future releases of Grafana.\n```\n\nDon't worry about warning message, plugin will still working until Grafana 11 will release, after it upgrade to Altinity ClickHouse datasource plugin for Grafana to 3.x version is required.\n\n\n### Grafana 7+ setup notes for plugin version before 2.2.0\n\nWhen 2.0.x and 2.1.x vertamedia-clickhouse-grafana plugin versions released Grafana team didn't provide worked signing method for community plugins.\nCurrent sign process describe on [grafana.com](https://grafana.com/docs/grafana/latest/developers/plugins/sign-a-plugin/)\n\nso, for properly setup 2.0.x and 2.1.x plugins you need change configuration option\n\n```ini\n[plugins]\nallow_loading_unsigned_plugins=vertamedia-clickhouse-datasource\n```\n\nor setup environment variable\n\n```bash\nGF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=vertamedia-clickhouse-datasource\n```\n\nYou can install plugin from [grafana.com](https://grafana.com/plugins/vertamedia-clickhouse-datasource)\n\nOR\n\nCopy files to your [Grafana plugin directory](https://grafana.com/docs/grafana/latest/plugins/installation/#install-plugin-on-local-grafana).\nRestart Grafana, check data sources list at Configuration -\u003e Datasources -\u003e New, choose ClickHouse option.\n\n![Datasources](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/01_data_sources.png)\n![Add new datasource](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/02_add_data_source.png)\n![Datasource types](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/03_filter_click_to_plugin.png)\n\n\n## Features\n\n* Access to CH via HTTP / HTTPS\n* Query setup\n* Raw SQL editor\n* Query formatting\n* Macros support\n* Additional functions\n* Templates\n* Table view\n* SingleStat view\n* Ad-hoc filters\n* Annotations\n* Alerts support\n* Histogram support\n* Logs support\n* Flamegraph support\n* Traces support\n\n## Access to ClickHouse via HTTP / HTTPS\n\nPage configuration is standard\n\n![settings](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/04_datasource_settings.png)\n\nThere is a small feature - ClickHouse treats HTTP Basic Authentication credentials as a database user and will try to run queries using its name.\n\n### [CHProxy](https://github.com/ContentSquare/chproxy) (optional)\n\nUsing of [CHProxy](https://github.com/ContentSquare/chproxy) will bring additional features:\n\n* Easily setup `HTTPS` access to ClickHouse as shown [here](https://github.com/ContentSquare/chproxy#authorize-users-by-passwords-via-https)\nto provide secure access.\n* Limit concurrency and execution time for requests from `Grafana` as shown [here](https://github.com/ContentSquare/chproxy#spread-selects-from-reporting-apps-among-cluster-nodes)\nto prevent `ClickHouse` overloading from `Grafana`.\n* Protection against request bursts for dashboards with numerous graphs. `CHProxy` allows queueing requests and execute them sequentially.\nTo learn more - read about params `max_queue_size` and `max_queue_time` at [CHProxy](https://github.com/ContentSquare/chproxy) page.\n* Response caching for the most frequent queries as shown [here](https://github.com/ContentSquare/chproxy#caching).\n\n`Caching` will protect `ClickHouse` from excessive refreshes and will be optimal option for popular dashboards.\n\u003e Hint - if you need to cache requests like `last 24h` where timestamp changes constantly then try to use `Round` option at `Raw Editor`\n\n## Query setup\n\nQuery setup interface:\n\n![query editor image](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/05_query_settings.png)\n\nFirst row `FROM` contains two options: database and table. Table values depends on a selected database.\nNext rows contains selectors for time filtering:\n\nColumn timestamp time\n* DateTime ([DateTime](https://clickhouse.com/docs/en/sql-reference/data-types/datetime/))\n* DateTime64 ([DateTime64](https://clickhouse.com/docs/en/sql-reference/data-types/datetime64/))\n* TimeStamp ([UInt32](https://clickhouse.com/docs/en/sql-reference/data-types/int-uint/)).\n\nTimestamp column\nDate column\n\n\u003e `Timestmap column` are required for time-based macros and functions because all analytics based on these values.\n\u003e Plugin will try to detect Date, Date32 column automatically\n\nButton `Go to Query` is just a toggler to Raw SQL Editor\n\n## Raw SQL Editor\n\nRaw Editor allows custom SQL queries to be written:\n\n![raw editor image](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/06_raw_sql_editor.png)\n\nRaw Editor allows typing queries, get info about functions and macros, format queries as Clickhouse do.\nTo Execute query on server press \"Run Query\" or just leave focus from SQL editor textarea.\n\nUnder the Editor you can find options which allows setup rounding, time column step \nand `Add metadata` to SQL query which allows know which dashboard and user produce workload to your ClickHouse server.\n\nPress `Show Generated SQL` for see a raw query (all macros and functions have already been replaced) which will be sent directly to ClickHouse.\n![generated sql](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/07_generated_sql.png)\n\n\n## Macros support\n\nPlugin supports the following marcos:\n\n* $table - replaced with selected table name from Query Builder\n* $dateCol - replaced with `Column:Date` value from Query Builder\n* $dateTimeCol - replaced with `Column:DateTime` or `Column:TimeStamp` value from Query Builder\n* $from - replaced with (timestamp with ms)/1000 value of UI selected \"Time Range:From\"\n* $to - replaced with (timestamp with ms)/1000 value of UI selected \"Time Range:To\"\n* $interval - replaced with selected \"Group by a time interval\" value (as a number of seconds)\n* $timeFilter - replaced with currently selected \"Time Range\".\n  Requires Column:Date and Column:DateTime or Column:TimeStamp to be selected.\n* $timeFilterByColumn($column) - replaced with currently selected \"Time Range\" for a column passed as `$column` argument. Use it in queries or query variables as `...WHERE $timeFilterColumn($column)...` or `...WHERE $timeFilterColumn(created_at)...`.\n* $timeSeries - replaced with special ClickHouse construction to convert results as time-series data. Use it as \"SELECT $timeSeries...\".\n* $naturalTimeSeries - replaced with special ClickHouse construction to convert results as time-series with in a logical/natural breakdown. Use it as \"SELECT $naturalTimeSeries...\".\n* $unescape - unescapes variable value by removing single quotes. Used for multiple-value string variables: \"SELECT $unescape($column) FROM requests WHERE $unescape($column) = 5\"\n* $adhoc - replaced with a rendered ad-hoc filter expression, or \"1\" if no ad-hoc filters exist. Since ad-hoc applies automatically only to outer queries the macros can be used for filtering in inner queries.\n\nA description of macros is available by typing their names in Raw Editor\n\n## Functions\n\nFunctions are just templates of SQL queries, and you can check the final query at [Raw SQL Editor mode](https://github.com/Altinity/clickhouse-grafana/blob/master/README.md#raw-sql-editor).\nIf you need some additional complexity - just copy raw sql into Raw Editor and make according changes. Remember that macros are still available to use.\n\nThere are some limits in function use because of poor query analysis:\n\n* Column:Date and Column:DateTime or Column:TimeStamp must be set in Query Builder\n* Query must begin from function name\n* Only one function can be used per query\n\nPlugin supports the following functions:\n\n### $rate(cols...) - converts query results as \"change rate per interval\"\n\nExample usage:\n\n```sql\n$rate(countIf(Type = 200) AS good, countIf(Type != 200) AS bad) FROM requests\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    good / runningDifference(t / 1000) AS goodRate,\n    bad / runningDifference(t / 1000) AS badRate\nFROM\n(\n    SELECT\n        (intDiv(toUInt32(EventTime), 60)) * 1000 AS t,\n        countIf(Type = 200) AS good,\n        countIf(Type != 200) AS bad\n    FROM requests\n    WHERE ((EventDate \u003e= toDate(1482796747)) AND (EventDate \u003c= toDate(1482853383))) AND ((EventTime \u003e= toDateTime(1482796747)) AND (EventTime \u003c= toDateTime(1482853383)))\n    GROUP BY t\n    ORDER BY t\n)\n```\n\n---\n\n### $columns(key, value) - query values as array of [key, value], where key will be used as label\n\nExample usage:\n\n```sql\n$columns(OSName, count(*) c)\nFROM requests\nINNER JOIN oses USING (OS)\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    groupArray((OSName, c)) AS groupArr\nFROM\n(\n    SELECT\n        (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,\n        OSName,\n        count(*) AS c\n    FROM requests\n    INNER JOIN oses USING (OS)\n    WHERE ((EventDate \u003e= toDate(1482796627)) AND (EventDate \u003c= toDate(1482853383))) AND ((EventTime \u003e= toDateTime(1482796627)) AND (EventTime \u003c= toDateTime(1482853383)))\n    GROUP BY\n        t,\n        OSName\n    ORDER BY\n        t,\n        OSName\n)\nGROUP BY t\nORDER BY t\n```\n\nThis will help to build the next graph:\n\n![req_by_os image](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/08_requests_by_os.png)\n\n---\n### $columnsMs(key, value) - same as $columns but for time series with ms\n\nExample usage:\n\n```sql\n$columnsMs(OSName, count(*) c)\nFROM requests\nINNER JOIN oses USING (OS)\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    groupArray((OSName, c)) AS groupArr\nFROM\n(\n    SELECT\n        $timeSeriesMs AS t,\n        OSName,\n        count(*) AS c\n    FROM requests\n    INNER JOIN oses USING (OS)\n    WHERE ((EventDate \u003e= toDate(1482796627)) AND (EventDate \u003c= toDate(1482853383))) AND ((EventTime \u003e= toDateTime64(1482796627,3)) AND (EventTime \u003c= toDateTime64(1482853383,3)))\n    GROUP BY\n        t,\n        OSName\n    ORDER BY\n        t,\n        OSName\n)\nGROUP BY t\nORDER BY t\n```\n\n---\n### $lttb(buckets_number, [field1, ... fieldN], x_field, y_field) - allow show down-sampled time series which will contains more outliers than avg or other kind of aggregation\n\nIf bucket_number is `auto`, then it will calculated as `toUInt64( ($to-$from) / $interval )`\nExample usage:\n\n```sql\n$lttb(auto, category, event_time, count(*) c)\nFROM requests GROUP BY category\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT category, lttb_result.1 AS event_time, lttb_result.2 AS c FROM (\n    SELECT category, untuple(arrayJoin(lttb(toUInt64( ($to - $from) / $interval ))(event_time, cont(*) AS c))) AS lttb_result\n    FROM requests WHERE $timeFilter GROUP BY category\n) ORDER BY event_time\n```\n\n---\n\n---\n### $lttbMs(buckets_number, [field1,... fieldN], x_field, y_field) - same as $lttb but for time series with ms\n\nIf bucket_number is `auto`, then it will calculated as `toUInt64( ($__to-$__from) / $__interval_ms )`\n\nExample usage:\n\n```sql\n$lttbMs(100, event_time, count(*) c)\nFROM requests\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT lttb_result.1 AS event_time, lttb_result.2 AS c FROM (\n    SELECT untuple(arrayJoin(lttb(100)(event_time, count(*) AS c))) AS lttb_result\n    FROM requests WHERE $timeFilterMs    \n) ORDER BY event_time\n```\n\n---\n\n### $rateColumns(key, value) - is a combination of $columns and $rate\n\nExample usage:\n\n```sql\n$rateColumns(OS, count(*) c) FROM requests\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    arrayMap(lambda(tuple(a), (a.1, a.2 / runningDifference(t / 1000))), groupArr)\nFROM\n(\n    SELECT\n        t,\n        groupArray((OS, c)) AS groupArr\n    FROM\n    (\n        SELECT\n            (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,\n            OS,\n            count(*) AS c\n        FROM requests\n        WHERE ((EventDate \u003e= toDate(1482796867)) AND (EventDate \u003c= toDate(1482853383))) AND ((EventTime \u003e= toDateTime(1482796867)) AND (EventTime \u003c= toDateTime(1482853383)))\n        GROUP BY\n            t,\n            OS\n        ORDER BY\n            t,\n            OS\n    )\n    GROUP BY t\n    ORDER BY t\n)\n\n```\n\n---\n\n### $rateColumnsAggregated(key, subkey, aggFunction1, value1, ... aggFunctionN, valueN) - if you need calculate `rate` for higher cardinality dimension and then aggregate by lower cardinality dimension\n\nExample usage:\n\n```sql\n$rateColumnsAggregated(datacenter, concat(datacenter,interface) AS dc_interface, sum, tx_bytes * 1014 AS tx_kbytes, sum, max(rx_bytes) AS rx_bytes) FROM traffic\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    datacenter,\n    sum(tx_kbytesRate) AS tx_bytesRateAgg,\n    sum(rx_bytesRate) AS rx_bytesRateAgg\nFROM\n(\n    SELECT\n        t,\n        datacenter,\n        dc_interface,\n        tx_kbytes / runningDifference(t / 1000) AS tx_kbytesRate,\n        rx_bytes / runningDifference(t / 1000) AS rx_bytesRate\n    FROM\n    (\n        SELECT\n            (intDiv(toUInt32(event_time), 60) * 60) * 1000 AS t,\n            datacenter,\n            concat(datacenter,interface) AS dc_interface,\n            max(tx_bytes * 1024) AS tx_kbytes,\n            max(rx_bytes) AS rx_bytes\n        FROM traffic\n        WHERE ((event_date \u003e= toDate(1482796867)) AND (event_date \u003c= toDate(1482853383))) \n          AND ((event_time \u003e= toDateTime(1482796867)) AND (event_time \u003c= toDateTime(1482853383)))\n        GROUP BY\n            t,\n            datacenter,\n            dc_interface\n        ORDER BY\n            t,\n            datacenter,\n            dc_interface\n    )\n)\nGROUP BY\n  t,\n  datacenter\nORDER BY \n  datacenter,\n  t\n```\n\nlook [issue 386](https://github.com/Altinity/clickhouse-grafana/issues/386) for reasons for implementation  \n\n---\n\n### $perSecond(cols...) - converts query results as \"change rate per interval\" for Counter-like(growing only) metrics\n\nExample usage:\n\n```sql\n$perSecond(Requests) FROM requests\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    if(runningDifference(max_0) \u003c 0, nan, runningDifference(max_0) / runningDifference(t / 1000)) AS max_0_PerSecond\nFROM\n(\n    SELECT\n        (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,\n        max(Requests) AS max_0\n    FROM requests\n    WHERE ((EventDate \u003e= toDate(1535711819)) AND (EventDate \u003c= toDate(1535714715)))\n    AND ((EventTime \u003e= toDateTime(1535711819)) AND (EventTime \u003c= toDateTime(1535714715)))\n    GROUP BY t\n    ORDER BY t\n)\n```\n\n// see [issue 78](https://github.com/Altinity/clickhouse-grafana/issues/78) for the background\n\n---\n\n### $perSecondColumns(key, value) - is a combination of $columns and $perSecond for Counter-like metrics\n\nExample usage:\n\n```sql\n$perSecondColumns(Protocol, Requests) FROM requests WHERE Protocol in ('udp','tcp')\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    groupArray((perSecondColumns, max_0_PerSecond)) AS groupArr\nFROM\n(\n    SELECT\n        t,\n        Protocol,\n        if(runningDifference(max_0) \u003c 0 OR neighbor(perSecondColumns,-1,perSecondColumns) != perSecondColumns, nan, runningDifference(max_0) / runningDifference(t / 1000)) AS max_0_PerSecond\n    FROM\n    (\n        SELECT\n            (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,\n            Protocol AS perSecondColumns,\n            max(Requests) AS max_0\n        FROM requests\n        WHERE ((EventDate \u003e= toDate(1535711819)) AND (EventDate \u003c= toDate(1535714715)))\n        AND ((EventTime \u003e= toDateTime(1535711819)) AND (EventTime \u003c= toDateTime(1535714715)))\n        AND (Protocol IN ('udp', 'tcp'))\n        GROUP BY\n            t,\n            Protocol\n        ORDER BY\n            t,\n            Protocol\n    )\n)\nGROUP BY t\nORDER BY t\n```\n\n// see [issue 80](https://github.com/Altinity/clickhouse-grafana/issues/80) for the background\n\n---\n\n### $perSecondColumnsAggregated(key, subkey, aggFunction1, value1, ... aggFunctionN, valueN) - if you need to calculate `perSecond` for higher cardinality dimension and then aggregate by lower cardinality dimension\n\nExample usage:\n\n```sql\n$perSecondColumnsAggregated(datacenter, concat(datacenter,interface) AS dc_interface, sum, tx_bytes * 1014 AS tx_kbytes, sum, max(rx_bytes) AS rx_bytes) FROM traffic\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    datacenter,\n    sum(tx_kbytesPerSecond) AS tx_bytesPerSecondAgg,\n    sum(rx_bytesPerSecond) AS rx_bytesPerSecondAgg\nFROM\n(\n    SELECT\n        t,\n        datacenter,\n        dc_interface,\n        if(runningDifference(tx_kbytes) \u003c 0 OR neighbor(tx_kbytes,-1,tx_kbytes) != tx_kbytes, nan, runningDifference(tx_kbytes) / runningDifference(t / 1000)) AS tx_kbytesPerSecond,\n        if(runningDifference(rx_bytes) \u003c 0 OR neighbor(rx_bytes,-1,rx_bytes) != rx_bytes, nan, runningDifference(rx_bytes) / runningDifference(t / 1000)) AS rx_bytesPerSecond\n    FROM\n    (\n        SELECT\n            (intDiv(toUInt32(event_time), 60) * 60) * 1000 AS t,\n            datacenter,\n            concat(datacenter,interface) AS dc_interface,\n            max(tx_bytes * 1024) AS tx_kbytes,\n            max(rx_bytes) AS rx_bytes\n        FROM traffic\n        WHERE ((event_date \u003e= toDate(1482796867)) AND (event_date \u003c= toDate(1482853383))) \n          AND ((event_time \u003e= toDateTime(1482796867)) AND (event_time \u003c= toDateTime(1482853383)))\n        GROUP BY\n            t,\n            datacenter,\n            dc_interface\n        ORDER BY\n            t,\n            datacenter,\n            dc_interface\n    )\n)\nGROUP BY\n  t,\n  datacenter\nORDER BY \n  datacenter,\n  t\n```\n\nlook [issue 386](https://github.com/Altinity/clickhouse-grafana/issues/386) for reasons for implementation  \n\n---\n\n### $delta(cols...) - converts query results as \"delta value inside interval\" for Counter-like(growing only) metrics, will negative if counter reset\n\nExample usage:\n\n```sql\n$delta(Requests) FROM requests\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    runningDifference(max_0) AS max_0_Delta\nFROM\n(\n    SELECT\n        (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,\n        max(Requests) AS max_0\n    FROM requests\n    WHERE ((EventDate \u003e= toDate(1535711819)) AND (EventDate \u003c= toDate(1535714715)))\n    AND ((EventTime \u003e= toDateTime(1535711819)) AND (EventTime \u003c= toDateTime(1535714715)))\n    GROUP BY t\n    ORDER BY t\n)\n```\n\n// see [issue 455](https://github.com/Altinity/clickhouse-grafana/issues/455) for the background\n\n---\n\n### $deltaColumns(key, value) - is a combination of $columns and $delta for Counter-like metrics\n\nExample usage:\n\n```sql\n$deltaColumns(Protocol, Requests) FROM requests WHERE Protocol in ('udp','tcp')\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    groupArray((deltaColumns, max_0_Delta)) AS groupArr\nFROM\n(\n    SELECT\n        t,\n        deltaColumns,\n        if (neighbor(deltaColumns,-1,deltaColumns) != deltaColumns, 0, runningDifference(max_0)) AS max_0_Delta\n    FROM\n    (\n        SELECT\n            (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,\n            Protocol AS deltaColumns,\n            max(Requests) AS max_0\n        FROM requests\n        WHERE ((EventDate \u003e= toDate(1535711819)) AND (EventDate \u003c= toDate(1535714715)))\n        AND ((EventTime \u003e= toDateTime(1535711819)) AND (EventTime \u003c= toDateTime(1535714715)))\n        AND (Protocol IN ('udp', 'tcp'))\n        GROUP BY\n            t,\n            Protocol\n        ORDER BY\n            t,\n            Protocol\n    )\n)\nGROUP BY t\nORDER BY t\n```\n\n// see [issue 455](https://github.com/Altinity/clickhouse-grafana/issues/455) for the background\n\n---\n\n### $deltaColumnsAggregated(key, subkey, aggFunction1, value1, ... aggFunctionN, valueN) - if you need to calculate `delta` for higher cardinality dimension and then aggregate by lower cardinality dimension\n\nExample usage:\n\n```sql\n$deltaColumnsAggregated(datacenter, concat(datacenter,interface) AS dc_interface, sum, tx_bytes * 1014 AS tx_kbytes, sum, max(rx_bytes) AS rx_bytes) FROM traffic\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    datacenter,\n    sum(tx_kbytesDelta) AS tx_bytesDeltaAgg,\n    sum(rx_bytesDelta) AS rx_bytesDeltaAgg\nFROM\n(\n    SELECT\n        t,\n        datacenter,\n        dc_interface,\n        if(neighbor(tx_kbytes,-1,tx_kbytes) != tx_kbytes, 0, runningDifference(tx_kbytes) / 1) AS tx_kbytesDelta,\n        if(neighbor(rx_bytes,-1,rx_bytes) != rx_bytes, 0, runningDifference(rx_bytes) / 1) AS rx_bytesDelta\n    FROM\n    (\n        SELECT\n            (intDiv(toUInt32(event_time), 60) * 60) * 1000 AS t,\n            datacenter,\n            concat(datacenter,interface) AS dc_interface,\n            max(tx_bytes * 1024) AS tx_kbytes,\n            max(rx_bytes) AS rx_bytes\n        FROM traffic\n        WHERE ((event_date \u003e= toDate(1482796867)) AND (event_date \u003c= toDate(1482853383))) \n          AND ((event_time \u003e= toDateTime(1482796867)) AND (event_time \u003c= toDateTime(1482853383)))\n        GROUP BY\n            t,\n            datacenter,\n            dc_interface\n        ORDER BY\n            t,\n            datacenter,\n            dc_interface\n    )\n)\nGROUP BY\n  t,\n  datacenter\nORDER BY \n  datacenter,\n  t\n```\n\nlook [issue 386](https://github.com/Altinity/clickhouse-grafana/issues/386) for reasons for implementation\n\n---\n\n### $increase(cols...) - converts query results as \"non-negative delta value inside interval\" for Counter-like(growing only) metrics, will zero if counter reset and delta less zero\n\nExample usage:\n\n```sql\n$increase(Requests) FROM requests\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    if(runningDifference(max_0) \u003c 0, 0, runningDifference(max_0) ) AS max_0_Increase\nFROM\n(\n    SELECT\n        (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,\n        max(Requests) AS max_0\n    FROM requests\n    WHERE ((EventDate \u003e= toDate(1535711819)) AND (EventDate \u003c= toDate(1535714715)))\n    AND ((EventTime \u003e= toDateTime(1535711819)) AND (EventTime \u003c= toDateTime(1535714715)))\n    GROUP BY t\n    ORDER BY t\n)\n```\n\n// see [issue 455](https://github.com/Altinity/clickhouse-grafana/issues/455) for the background\n\n---\n\n### $increaseColumns(key, value) - is a combination of $columns and $increase for Counter-like metrics\n\nExample usage:\n\n```sql\n$increaseColumns(Protocol, Requests) FROM requests WHERE Protocol in ('udp','tcp')\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    groupArray((increaseColumns, max_0_Increase)) AS groupArr\nFROM\n(\n    SELECT\n        t,\n        Protocol,\n        if (runningDifference(max_0) \u003c 0 OR neighbor(increaseColumns,-1,increaseColumns) != increaseColumns, 0, runningDifference(max_0)) AS max_0_Increase\n    FROM\n    (\n        SELECT\n            (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,\n            Protocol AS increaseColumns,\n            max(Requests) AS max_0\n        FROM requests\n        WHERE ((EventDate \u003e= toDate(1535711819)) AND (EventDate \u003c= toDate(1535714715)))\n        AND ((EventTime \u003e= toDateTime(1535711819)) AND (EventTime \u003c= toDateTime(1535714715)))\n        AND (Protocol IN ('udp', 'tcp'))\n        GROUP BY\n            t,\n            Protocol\n        ORDER BY\n            t,\n            Protocol\n    )\n)\nGROUP BY t\nORDER BY t\n```\n\n// see [issue 455](https://github.com/Altinity/clickhouse-grafana/issues/455) for the background\n\n---\n\n### $increaseColumnsAggregated(key, subkey, aggFunction1, value1, ... aggFunctionN, valueN) - if you need to calculate `increase` for higher cardinality dimension and then aggregate by lower cardinality dimension\n\nExample usage:\n\n```sql\n$increaseColumnsAggregated(datacenter, concat(datacenter,interface) AS dc_interface, sum, tx_bytes * 1014 AS tx_kbytes, sum, max(rx_bytes) AS rx_bytes) FROM traffic\n```\n\nQuery will be transformed into:\n\n```sql\nSELECT\n    t,\n    datacenter,\n    sum(tx_kbytesIncrease) AS tx_bytesIncreaseAgg,\n    sum(rx_bytesIncrease) AS rx_bytesIncreaseAgg\nFROM\n(\n    SELECT\n        t,\n        datacenter,\n        dc_interface,\n        if(runningDifference(tx_kbytes) \u003c 0 OR neighbor(tx_kbytes,-1,tx_kbytes) != tx_kbytes, nan, runningDifference(tx_kbytes) / 1) AS tx_kbytesIncrease,\n        if(runningDifference(rx_bytes) \u003c 0 OR neighbor(rx_bytes,-1,rx_bytes) != rx_bytes, nan, runningDifference(rx_bytes) / 1) AS rx_bytesIncrease\n    FROM\n    (\n        SELECT\n            (intDiv(toUInt32(event_time), 60) * 60) * 1000 AS t,\n            datacenter,\n            concat(datacenter,interface) AS dc_interface,\n            max(tx_bytes * 1024) AS tx_kbytes,\n            max(rx_bytes) AS rx_bytes\n        FROM traffic\n        WHERE ((event_date \u003e= toDate(1482796867)) AND (event_date \u003c= toDate(1482853383))) \n          AND ((event_time \u003e= toDateTime(1482796867)) AND (event_time \u003c= toDateTime(1482853383)))\n        GROUP BY\n            t,\n            datacenter,\n            dc_interface\n        ORDER BY\n            t,\n            datacenter,\n            dc_interface\n    )\n)\nGROUP BY\n  t,\n  datacenter\nORDER BY \n  datacenter,\n  t\n```\n\nlook [issue 386](https://github.com/Altinity/clickhouse-grafana/issues/386) for reasons for implementation\n\n---\n\n## Templating\n\n### Query Variable\n\nIf you add a template variable of the type `Query`, you can write a ClickHouse query that can\nreturn things like measurement names, key names or key values that are shown as a dropdown select box.\n\nFor example, you can have a variable that contains all values for the `hostname` column in a table if you specify a query like this in the templating variable *Query* setting.\n\n```sql\nSELECT hostname FROM host\n```\n\nTo use time range dependent macros like `timeFilterByColumn($column)` in your query the refresh mode of the template variable needs to be set to *On Time Range Change*.\n\n```sql\nSELECT event_name FROM event_log WHERE $timeFilterByColumn(time_column)\n```\n\nAnother option is a query that can create a key/value variable. The query should return two columns that are named `__text` and `__value`. The `__text` column value should be unique (if it is not unique then the first value will use). The options in the dropdown will have a text and value that allows you to have a friendly name as text and an id as the value. An example query with `hostname` as the text and `id` as the value:\n\n```sql\nSELECT hostname AS __text, id AS __value FROM host\n```\n\nYou can also create nested variables. For example if you had another variable named `region`. Then you could have the hosts variable only show hosts from the current selected region with a query like this (if `region` is a multi-value variable then use the `IN` comparison operator rather than `=` to match against multiple values):\n\n```sql\nSELECT hostname FROM host WHERE region IN ($region)\n```\n\n### Conditional Predicate\n\nIf you are using templating to feed your predicate, you will face performance degradation when everything will select as the predicate, and it's not necessary. It's also true for textbox when nothing is entered, you have to write specific sql code to handle that.\n\nTo resolve this issue a new macro $conditionalTest(SQL Predicate,$variable) can be used to remove some part of the query.\nIf the variable is type query with all selected or if the variable is a textbox with nothing enter, then the SQL Predicate is not include in the generated query.\n\nTo give an example:\nwith 2 variables\n  $var query with include All option\n  $text textbox\n  $text_with_single_quote textbox with single quote\n\n  The following query\n\n  ```sql\n   SELECT\n     $timeSeries as t,\n     count()\n     FROM $table\n     WHERE $timeFilter\n      $conditionalTest(AND toLowerCase(column) in ($var),$var)\n      $conditionalTest(AND toLowerCase(column2) like '%$text%',$text)\n      $conditionalTest(AND toLowerCase(column3) ilike ${text_with_single_quote:sqlstring},$text_with_single_quote)\n     GROUP BY t\n     ORDER BY t\n  ```\n\n   if the `$var` is selected as \"All\" value, and the `$text` variable is empty, the query will be converted into:\n\n  ```sql\n    SELECT\n      $timeSeries as t,\n      count()\n       FROM $table\n       WHERE $timeFilter\n     GROUP BY t\n     ORDER BY t\n  ```\n\n  If the `$var` template variable have select some elements, and the `$text` template variable has at least one char, the query will be converted into:\n\n  ```sql\n  SELECT\n      $timeSeries as t,\n      count()\n       FROM $table\n       WHERE $timeFilter\n     AND toLowerCase(column) in ($var)\n     AND toLowerCase(column2) like '%$text%'\n     GROUP BY t\n     ORDER BY t\n ```\n### Extended Conditional Test with Else Clause\n\nA new signature of the macro now supports three parameters:\n\n```sql\n$conditionalTest(SQL_if, SQL_else, $variable)\n ```\n\nIf the variable is type query with all selected or if the variable is a textbox with nothing entered, then the SQL_if is included in the generated query. Otherwise, the SQL_else is included.\n\nTo give an example:\nwith 2 variables\n  $var query with include All option\n  $text textbox\n  $text_with_single_quote textbox with single quote\n\n  The following query\n\n  ```sql\n   SELECT\n     $timeSeries as t,\n     count()\n     FROM $table\n     WHERE $timeFilter\n      $conditionalTest(AND toLowerCase(column) in ($var), AND toLowerCase(column) in ($var), $var)\n      $conditionalTest(AND toLowerCase(column2) like '%$text%', AND toLowerCase(column2) like '%$text%', $text)\n      $conditionalTest(AND toLowerCase(column3) ilike ${text_with_single_quote:sqlstring}, AND toLowerCase(column3) ilike ${text_with_single_quote:sqlstring}, $text_with_single_quote)\n     GROUP BY t\n     ORDER BY t\n  ```\n\n   if the `$var` is selected as \"All\" value, and the `$text` variable is empty, the query will be converted into:\n\n  ```sql\n    SELECT\n      $timeSeries as t,\n      count()\n       FROM $table\n       WHERE $timeFilter\n     GROUP BY t\n     ORDER BY t\n  ```\n\n  If the `$var` template variable have select some elements, and the `$text` template variable has at least one char, the query will be converted into:\n\n  ```sql\n  SELECT\n      $timeSeries as t,\n      count()\n       FROM $table\n       WHERE $timeFilter\n     AND toLowerCase(column) in ($var)\n     AND toLowerCase(column2) like '%$text%'\n     GROUP BY t\n     ORDER BY t\n ```\n## Working with panels\n\n### Pie Chart ([https://grafana.com/plugins/grafana-piechart-panel](https://grafana.com/plugins/grafana-piechart-panel))\n\nRemember that pie chart plugin is not welcome for using in grafana - see [Grafana BLog - Friends don't let friends abuse pie charts](https://grafana.com/blog/2015/12/04/friends-dont-let-friends-abuse-pie-charts)\n\n![top users](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/09_requests_by_user_pie_chart.png)\n\nTo create \"Top 5\" diagram we will need two queries: one for 'Top 5' rows and one for 'Other' row.\n\nTop5:\n\n```sql\nSELECT\n    1 AS t, /* fake timestamp value */\n    UserName,\n    sum(Requests) AS Reqs\nFROM requests\nGROUP BY t, UserName\nORDER BY Reqs DESC\nLIMIT 5\n```\n\nOther:\n\n```sql\nSELECT\n    1 AS t, /* fake timestamp value */\n    UserName,\n    sum(Requests) AS Reqs\nFROM requests\nGROUP BY t, UserName\nORDER BY Reqs DESC\nLIMIT 5,10000000000000 /* select some ridiculous number after first 5 */\n```\n\n### Table view ([https://grafana.com/plugins/table](https://grafana.com/plugins/table))\n\nThere are don't contain any tricks in displaying time-series data. To print summary data, omit time column, and format the result as \"Table\" and press \"Run query\".\n\n\n```sql\nSELECT\n    UserName,\n    sum(Requests) as Reqs\nFROM requests\nGROUP BY\n    UserName\nORDER BY\n    Reqs\n```\n\n![table view](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/10_table_view.png)\n\n\n### Vertical histogram ([https://grafana.com/plugins/graph](https://grafana.com/plugins/graph))\n\n![vertical histogram](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/11_vertical_histogram.png)\n\nTo make the vertical histogram from graph panel we will need to edit some settings:\n\n* Display -\u003e Draw Modes -\u003e Bars\n* Axes -\u003e X-Axis -\u003e Mode -\u003e Series\n\nYou can use next query:\n\n```sql\n$columns(\n    Size,\n    sum(Items) Items)\nFROM some_table\n```\n\n// It is also possible to use query without macros\n\n### Worldmap panel ([https://github.com/grafana/worldmap-panel](https://github.com/grafana/worldmap-panel))\n\n![worldmap](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/12_worldmap_example.png)\n\nIf you have a table with country/city codes:\n\n```sql\nSELECT\n    1,\n    Country AS c,\n    sum(Requests) AS Reqs\nFROM requests\nGLOBAL ANY INNER JOIN\n(\n    SELECT Country, CountryCode\n    FROM countries\n) USING (CountryCode)\nWHERE $timeFilter\nGROUP BY\n    c\nORDER BY Reqs DESC\n```\n\nIf you are using [geohash](https://github.com/grafana/worldmap-panel#geohashes-as-the-data-source) set following options:\n\n![Format](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/13_worldmap_format.png)\n\nYou can make following query with `Table` formatting:\n\n![geohash-query](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/14_worldmap_query.png)\n\n## Ad-hoc filters\n\nIf there is an Ad-hoc variable, plugin will fetch all columns of all tables of all databases (except system database) as tags.\nSo in dropdown menu will be options like `database.table.column`. If you specify the default database it will only fetch tables and columns from that database, and the dropdown menu will have an option like `table.column`.\nIf there are ENUM columns, the plugin will fetch their options and use them as tag values.\nAlso, plugin will fetch 300 unique values for fields with other types.\n\nPlugin will apply Ad-hoc filters to all queries on the dashboard if their settings `$database` and `$table` are the same\nas `database.table` specified in Ad-hoc control. If the ad-hoc filter doesn't specify a table, it will apply to all queries regardless of the table.\nThis is useful if the dashboard contains queries to multiple different tables.\n\n![ad-hoc](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/15_adhoc_filter.png)\n\n\u003e There are no option to apply OR operator for multiple Ad-hoc filters - see grafana/grafana#10918\n\u003e There are no option to use IN operator for Ad-hoc filters due to Grafana limitations\n\nThere may be cases when CH contains too many tables and columns so their fetching could take notably amount of time. So, if you need\nto have multiple dashboards with different databases using of `default database` won't help. The best way to solve this will be to have parametrized\nad-hoc variable in dashboard settings. Currently, it's not supported by Grafana interface (see [issue](https://github.com/grafana/grafana/issues/13109)).\nAs a temporary workaround, plugin will try to look for variable with name `adhoc_query_filter` and if it exists will use its value as query to fetch columns.\nFor this purpose we recommend creating some variable `constant` with the name `adhoc_query_filter` and set the value similar to the following one:\n\n```sql\nSELECT database, table, name, type FROM system.columns WHERE table='myTable' ORDER BY database, table\n```\n\nThat should help to control data fetching by ad-hoc queries.\n\n## Template variable values via Query\n\nTo use time range dependent macros like `$from` and `$to` in your query the refresh mode of the template variable needs to be set to On Time Range Change.\n\n```sql\nSELECT ClientID FROM events WHERE EventTime \u003e toDateTime($from) AND EventTime \u003c toDateTime($to)\n```\n\n## Annotations\n\nPlugin support Annotations with regions. To enable this feature open Dashboard `settings` and add new annotation query with `clickhouse` datasource with properly field names.\n\n![Annotation query add](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/16_annotations_query_add.png)\n\n![Annotation query example](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/17_annotations_query_example.png)\n\n![Annotation with regions graph panel](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/18_annotations_graph.png)\n\n## Alerts support\n\nGrafana provide two kind of alerts. Unified alerts and graph panel related alerts (legacy). \nBoth kind of alerts supports by our plugin can't be used together. \nUse `GF_UNIFIED_ALERTING_ENABLED=1` (preferable) or `GF_ALERTING_ENABLED=1` environment variables for switch.\n\n### Panel related alerts (legacy)\nTo enable alerts open \"alerts\" tab in panel, and define alert expression as described on [grafana.com](https://grafana.com/docs/grafana/latest/alerting/)\n\n![Alerts in graph panel](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/19_alerts_tab.png)\n\n\nBe careful with Template variables values, currently grafana doesn't support template variables in alert queries itself.\nAlso, grafana UI doesn't pass template variables values to a backend, after you change it on frontend UI.\n\nSo, the clickhouse grafana plugin can use template variables values, because we have \"Generated SQL\" which pass to backend \"as is\"\nTo ensure template variables values will properly pass to a backend part of the plugin.\nPlease choose the required template variables values for your alerts in UI dropdown,\nensure values properly rendered in \"Generated SQL\" (maybe need change SQL queries in query editor)\nand save a whole dashboard to the Grafana server\n\nWARNING: `Test alert` button doesn't save a current state of alert rules to a backend part of the plugin.\n\nIf the \"Generated SQL\" properly passed into backend part of plugin, you will see something like this:\n![Graph panel with alerts](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/20_alerts_panel.png)\n\n\n### Unified Alerts support\n\nUnified alerts could be provisioned with YAML file, look to https://github.com/Altinity/clickhouse-grafana/tree/master/docker/grafana/provisioning/alerting/\n\n![Unified alerts menu](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/21_unified_alerts_menu.png)\n\n![Unified alerts panel](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/22_unified_alerts_adding.png)\n\nTo export exists unified alerts to YAML use Export alerts\n\n![Unified alerts export](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/24_alerts_export.png)\n\n### Alerts troubleshooting \nTo troubleshoot alerts in clickhouse grafana plugin when enable `level=debug` in `log` section `grafana.ini` or via `GF_LOG_LEVEL=debug` environment variable.\n\n## Histogram support\n![Histogram](https://github.com/Altinity/clickhouse-grafana/raw/master/src/img/histogram.png)\n\nTo show Histogram you need query in format as \"Time Series\"\n\nAccording to https://grafana.com/docs/grafana/latest/panels-visualizations/visualizations/histogram, Histograms support time series and any table results with one or more numerical fields.\n\n## Logs support\n\nTo render your ClickHouse data as Logs, please use special format in \"Format as\" dropdown in Query Editor called \"Logs\". This option helps Grafana recognizes data as logs and shows logs visualization automatically in Explore UI. On dashboards you can use [Logs panel](https://grafana.com/docs/grafana/latest/visualizations/logs-panel/) as well.\n\n![Format as Logs](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/23_logs_support.png)\n  \nTo return suitable for logs data - query should return at least one time field (assumed that it will be first field) and one text field from the ClickHouse.\n\nPlugin is also transforming all text fields, except log line, into the labels using following rules:\n\n* Log line will be taken either from dedicated `content` field or from first in order text field in result\n* All other text fields will be treated as a labels\n\nThere are few dedicated fields that are recognized by Grafana:\n\n* `level` (string) - set the level for each log line\n* `id` (string) - by default, Grafana offers basic support for deduplicating log lines, that can be improved by adding this field to explicitly assign identifiers to each log line\n\nAll other fields returned from data source will be recognized by Grafana as [detected fields](https://grafana.com/docs/grafana/latest/explore/logs-integration/#labels-and-detected-fields)\n\n## Flamegraph support\n![Format as: Flamegraph](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/25_format_as_flamegraph.png)\n\nTo show Flamegraph you need query in format as \"Flame Graph\"\nAccording to https://grafana.com/docs/grafana/latest/panels-visualizations/visualizations/flame-graph/#data-api, you need to have recordset with 4 fields\n- `level` - Numeric - the level of the stack frame. The root frame is level 0.\n- `label` - String - the function name or other symbol which identify\n- `value` - Numeric - the number of samples or bytes that were recorded in this stack trace\n- `self` - Numeric - the number of samples or bytes that were recorded in only this stack frame excluding the children, for clickhouse this is usually zero, but for the last frame in stack requires `self` equals with `value` to properly flamegraph vizualization\n\n**Moreover, rows shall be ordered by stack trace and level**\n\nIf you setup `query_profiler_real_time_period_ns` in profile or query level settings when you can try to visualize it as FlameGraph with the following query  \nLook to [system.trace_log](https://clickhouse.com/docs/en/operations/system-tables/trace_log) table description for how to get data for FlameGraph\nLook to [flamegraph dashboard example](https://github.com/Altinity/clickhouse-grafana/blob/master/docker/grafana/dashboards/flamegraph_and_tracing_support.json) for example of dashboard with FlameGraph\n\n### Flamegraph query example: \n```sql\nSELECT length(trace)  - level_num AS level, label, count() AS value, 0 self\nFROM system.trace_log\n  ARRAY JOIN arrayEnumerate(trace) AS level_num,\n  arrayMap(x -\u003e if(addressToSymbol(x) != '', demangle(addressToSymbol(x)), 'unknown') , trace) AS label\nWHERE trace_type='Real' AND $timeFilter\nGROUP BY level, label, trace\nORDER BY trace, level\n```\n\n## Traces support\nTo show Traces you need query with format as \"Traces\" with following\n![Format as Traces](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/26_format_as_trace.png)\n\n![Trace example](https://github.com/Altinity/clickhouse-grafana/raw/master/.github/images/27_traces_example.png)\n\nFor example, if `\u003copentelemetry_start_trace_probability\u003e1\u003c/opentelemetry_start_trace_probability\u003e` in user profile and `system.opentelemetry_span_log` is not emtpy, then you can show traces about clickhouse query execution\nLook to [system.opentelemetry_span_log](https://clickhouse.com/docs/en/operations/system-tables/opentelemetry_span_log) table description for how to get data for FlameGraph\nLook to [tracing dashboard example](https://github.com/Altinity/clickhouse-grafana/blob/master/docker/grafana/dashboards/flamegraph_and_tracing_support.json) for example of dashboard with FlameGraph\n\nTracing visualization requires following field names (case sensitive):\n- `traceID` - String\n- `spanID` - String\n- `operationName` - String\n- `parentSpanID` - String\n- `serviceName` - String\n- `duration` - UInt64 - duration in milliseconds\n- `startTime` - UInt64 - start time in milliseconds\n- `tags` - map(String, String) - tags for span\n- `serviceTags` - map(String, String) - tags for service (for example 'hostName')\n\n### Traces query example for system.opentelemetry_span_log\n```sql\nSELECT\n  trace_id AS traceID,\n  span_id AS spanID,\n  operation_name AS operationName,\n  parent_span_id AS parentSpanID,\n  'clickhouse' AS serviceName,\n  intDiv(finish_time_us - start_time_us, 1000) AS duration,\n  intDiv(start_time_us,1000) AS startTime,\n  attribute AS tags,\n  map('hostName',hostname) AS serviceTags\nFROM\n  system.opentelemetry_span_log\nWHERE $timeFilter\nORDER BY traceID, startTime\n```\n## Configure the Datasource with Provisioning\n\nIt’s now possible to configure datasources using config files with Grafana’s provisioning system.\nYou can read more about how it works and all the settings you can set for datasources on the [provisioning docs page](http://docs.grafana.org/administration/provisioning/#datasources).\n\nHere are some provisioning example:\n\n```yaml\napiVersion: 1\n\ndatasources:\n - name: Clickhouse\n   type: vertamedia-clickhouse-datasource\n   access: proxy\n   url: http://localhost:8123\n   # \u003cbool\u003e enable/disable basic auth\n   basicAuth: false\n   # \u003cstring\u003e basic auth username\n   basicAuthUser: \"default\"\n   # \u003cbool\u003e enable/disable with credentials headers\n   withCredentials: false\n   # \u003cbool\u003e mark as default datasource. Max one per org\n   isDefault: false\n   # \u003cmap\u003e fields that will be converted to json and stored in json_data\n   jsonData:\n     # \u003cbool\u003e enable/disable sending 'add_http_cors_header=1' parameter\n     addCorsHeader: false\n     # \u003cbool\u003e enable/disable using POST method for sending queries\n     usePOST: false\n     # \u003cbool\u003e enable/disable using Accept-Encoding header in each request\n     useCompression: false\n     # \u003cstring\u003e compression type allowed values: gzip, zstd, br, deflate\n     compressionType: \"\"\n     # \u003cstring\u003e default database name\n     defaultDatabase: \"\"\n     # \u003cbool\u003e enable/disable tls authorization\n     tlsAuth: false\n     # \u003cbool\u003e enable/disable tls authorization with custom ca\n     tlsAuthWithCACert: false\n     # \u003cbool\u003e enable/disable authorization with X-ClickHouse-* headers\n     useYandexCloudAuthorization: false\n     # \u003cstring\u003e X-ClickHouse-Key header value for authorization\n     xHeaderUser: \"\"\n     # \u003cstring\u003e the same value as url when `useYandexCloudAuthorization: true` \n     # @todo remove this workaround when merge https://github.com/grafana/grafana/pull/80858\n     dataSourceUrl: \"http://localhost:8123\"\n   secureJsonData:\n     # \u003cstring\u003e X-ClickHouse-User header value for authorization\n     xHeaderKey: \"\"\n     # \u003cstring\u003e basic auth password\n     basicAuthPassword: \"\"\n     # \u003cstring\u003e custom certificate authority for TLS https connection, base64 encoded \n     tlsCACert: \"\"\n     # \u003cstring\u003e custom client certificate for TLS https connection, base64 encoded \n     tlsClientCert: \"\"\n     # \u003cstring\u003e custom client secret key for TLS https connection, base64 encoded \n     tlsClientKey: \"\"\n```\n\nSome settings and security params are the same for all datasources. You can find them [here](http://docs.grafana.org/administration/provisioning/#example-datasource-config-file).\n\n## FAQ\n\n\u003e Why time series last point is not the real last point?\n\nPlugin extrapolates last datapoint if time range is `last N` to avoid displaying of constantly decreasing graphs\nwhen timestamp in a table is rounded to minute or bigger.\nIf it so then in 99% cases last datapoint will be much less than previous one, because last minute is not finished yet.\nThat's why plugin checks prev datapoints and tries to predict last datapoint value just as it was already written into db.\nThis behavior could be turned off via \"Extrapolation\" checkbox in query editor.\n\n\u003e Which table schema used in SQL query examples?\n\nAll examples in this plugin use following table schema:\n\n```sql\nCREATE TABLE IF NOT EXISTS countries(\n    Country LowCardinality(String),\n    CountryCode LowCardinality(String)\n) ENGINE MergeTree()\nORDER BY (CountryCode, Country);\n\nCREATE TABLE IF NOT EXISTS oses (\n    OSName LowCardinality(String),\n    OS LowCardinality(String)\n) ENGINE MergeTree()\nORDER BY (OS);\n\nCREATE TABLE IF NOT EXISTS requests(\n    EventTime DateTime,\n    EventDate Date,\n    Protocol LowCardinality(String),\n    UserName LowCardinality(String),\n    OS LowCardinality(String),\n    CountryCode LowCardinality(String),\n    Type UInt8,\n    Requests UInt32\n) ENGINE=MergeTree()\nORDER BY (EventDate, EventTime, Type, OS, Protocol, UserName)\nPARTITION BY toYYYYMM(EventDate);\n```\n\n\u003e What about alerts support?\n\nAlerts feature requires changes in `Grafana`'s backend, which can be extended only for Grafana 6.5+. `Grafana`'s maintainers are working on this feature.\nCurrent alerts support for `clickhouse-grafana` datasource plugin in beta.\n\nFor clickhouse grafana plugin 2.2.3+ support only for amd64 architecture for Linux, macOS, Windows and arm64 Linux, macOS (m1).\nOnly amd64 prior 2.2.3 version.\n\n## Contributing\n\nIf you have any idea for an improvement or found a bug do not hesitate to open an issue or submit a pull request.\nWe will appreciate any help from the community which will make working with such amazing products as ClickHouse and Grafana more convenient.\n\n## Development\n\nsee [CONTRIBUTING.md](https://github.com/Altinity/clickhouse-grafana/blob/master/CONTRIBUTING.md) for Development and Pull request Contributing instructions\n\nLicense\n\n---\nMIT License, please see [LICENSE](https://github.com/Altinity/clickhouse-grafana/blob/master/LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faltinity%2Fclickhouse-grafana","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faltinity%2Fclickhouse-grafana","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faltinity%2Fclickhouse-grafana/lists"}