{"id":28633850,"url":"https://github.com/brimdata/grafana-zed-datasource","last_synced_at":"2025-06-12T15:39:32.245Z","repository":{"id":149313227,"uuid":"620531976","full_name":"brimdata/grafana-zed-datasource","owner":"brimdata","description":"Grafana data source plugin for Zed lakes","archived":false,"fork":false,"pushed_at":"2024-06-13T00:15:57.000Z","size":2401,"stargazers_count":1,"open_issues_count":5,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-06-13T05:13:01.502Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brimdata.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-28T21:43:58.000Z","updated_at":"2024-06-13T00:16:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"b6cdc126-7b80-4723-8294-074480d0b6bb","html_url":"https://github.com/brimdata/grafana-zed-datasource","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/brimdata/grafana-zed-datasource","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brimdata%2Fgrafana-zed-datasource","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brimdata%2Fgrafana-zed-datasource/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brimdata%2Fgrafana-zed-datasource/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brimdata%2Fgrafana-zed-datasource/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brimdata","download_url":"https://codeload.github.com/brimdata/grafana-zed-datasource/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brimdata%2Fgrafana-zed-datasource/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259495279,"owners_count":22866654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-12T15:39:18.978Z","updated_at":"2025-06-12T15:39:32.198Z","avatar_url":"https://github.com/brimdata.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Zed Data Source for Grafana\n\nThis [data source plugin](https://grafana.com/grafana/plugins/?type=datasource)\nfor [Grafana](https://grafana.com/) allows the plotting of time-series data\nthat's stored in [Zed lakes](https://zed.brimdata.io/docs/commands/zed/).\n\n---\n  * [Quick Start](#quick-start)\n  * [Install \u0026 Configuration](#install--configuration)\n  * [Best Practices](#best-practices)\n  * [Example Usage in Dashboards](#example-usage-in-dashboards)\n    + [One row per metric](#one-row-per-metric)\n    + [Each measurement (with lot of metrics) in its own row](#each-measurement-with-lot-of-metrics-in-its-own-row)\n    + [Transform between approaches](#transform-between-approaches)\n    + [Variables](#variables)\n    + [Aggregations and the `$__interval` variable](#aggregations-and-the-__interval-variable)\n    + [Annotations](#annotations)\n    + [Logs](#logs)\n  * [Debugging](#debugging)\n  * [Why Unsigned?](#why-unsigned)\n  * [Contributing](#contributing)\n  * [Join the Community](#join-the-community)\n---\n\n## Quick Start\n\nWant to see if this plugin is what you're looking for? Watch\n[this quick video](https://www.youtube.com/watch?v=KcRO6ysOEyA)\nto see how easy it is to install the plugin and make your first chart from Zed\ndata in Grafana. Then keep reading for best practices to plot your\nsophisticated, real-world data.\n\nFor easy cut \u0026 paste, here's the command line used in the video to generate\nyour own simple test data.\n\n```\nNUM=1; while [ $NUM -le 10 ]; do echo $NUM | /opt/Zui/resources/app.asar.unpacked/zdeps/zq -z 'yield {ts: now(), num:this}' -; sleep 1; NUM=`expr $NUM + 1`; done | tee data.zson\n```\n\n## Install \u0026 Configuration\n\nAs an example environment, here the plugin is shown being installed on a fresh\n[Grafana installation on Linux](https://grafana.com/docs/grafana/latest/setup-grafana/installation/debian/).\nFor specifics regarding other platforms or adding a plugin into an existing\nenvironment, refer to the [Grafana documentation](https://grafana.com/docs/grafana/latest/setup-grafana/installation/).\n\n\nTo download and install the latest plugin release, run the following command:\n\n```\nsudo grafana-cli \\\n  --pluginUrl https://github.com/brimdata/grafana-zed-datasource/releases/latest/download/brimdata-zed-datasource.zip \\\n  plugins install brimdata-zed-datasource\n```\n\nAs the plugin is not currently signed ([learn why](#why-unsigned)) an\nadditional [configuration change](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#allow_loading_unsigned_plugins)\nis required to allow the Zed plugin to run. Modify the config file\n`/etc/grafana/grafana.ini`, removing the leading semicolon to\nuncomment this line and then edit it to read:\n\n```\nallow_loading_unsigned_plugins = brimdata-zed-datasource\n```\n\nRestart the Grafana service for the modified config to take effect.\n\n```\nsudo systemctl restart grafana-server\n```\n\nTo confirm the plugin has loaded successfully, look for a line like the\nfollowing in `/var/log/syslog`.\n\n```\nJun  7 09:37:16 ubuntu-22 grafana[22412]: logger=plugin.loader t=2023-06-07T09:37:16.885366925-07:00 level=info msg=\"Plugin registered\" pluginID=brimdata-zed-datasource\n```\n\nOnce inside Grafana (default: http://localhost:3000), the Zed data source\ncan be added by navigating to **Administration \u003e Data Sources \u003e Add data source**,\nthen click the entry for Zed (typically at the bottom of the list).\n\nIf a Zed lake service is listening locally on the default TCP port `9867`\n(as is typical for the lake launched by the [Zui app](https://zui.brimdata.io/)\nor when [`zed serve`](https://zed.brimdata.io/docs/commands/zed#213-serve) is\nrun with default settings) the default URL setting can be used. If your lake\nis listening elsewhere (e.g., with [Zui Insiders](https://github.com/brimdata/zui-insiders)\nit's at http://localhost:9988) change the URL setting as necessary.\n\nWhen **Save \u0026 test** is clicked, the plugin will query the value from\nthe lake service's `/version` endpoint. If successful, the version will be shown\nand the plugin is ready for use in dashboard panel queries.\n\n![Configure and Test Zed Data Source](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/config-zed-data-source.png)\n\n## Best Practices\n\nA Zed lake is not a purpose-built time-series database. However, as a general\ndata platform, it can absolutely be used for storage and query of time-series\ndata at moderate scale.\n\nIn its current state, the plugin relies on the use of Zed queries that\nprepare data for easy transformation to the\n[data frames](https://grafana.com/docs/grafana/latest/developers/plugins/data-frames/)\nthat Grafana uses for rendering plots. Some best practices that help achieve this:\n\n1. **The Time Field that contains timestamps for your data must be of Zed's\n   `time` type.**\n\n2. **The Time Field should ideally be your\n  [pool key](https://zed.brimdata.io/docs/commands/zed#143-pool-key).**\n\n   With this pool configuration, the time range portion of queries initiated\n   via the Grafana dashboard will scan the minimal number of data objects in\n   the Zed lake relevant to the query.\n\n3. **If possible, use `ts` as the name for your Time Field.**\n\n   This field name matches the \"out of the box\" default settings for the plugin.\n   However, the plugin can easily be configured to adapt to a different field\n   name.\n\n4. **Your time-series data should be of a single\n   [shape](https://zed.brimdata.io/docs/language/overview#10-shaping).**\n\n   Grafana's columnar data frames need to be constructed with a specific list\n   of expected fields. Therefore the count of shapes returned by a query is\n   first checked by the plugin and an error is shown if more than one shape is\n   detected. If this occurs, this could be addressed by using the\n   [`cut` operator](https://zed.brimdata.io/docs/language/operators/cut)\n   to trim the set of fields returned by the query or using the\n   [`fuse` operator](https://zed.brimdata.io/docs/language/operators/fuse) to\n   combine the entire query result into a single, wider shape.\n\n5. **Store data in top-level fields of primitive types.**\n\n   Of the fields in a response to a Zed query, the values passed on to Grafana\n   by the plugin will be top-level fields of Zed's\n   [primitive types](https://zed.brimdata.io/docs/formats/zed#1-primitive-types).\n   If you need to use values from\n   [complex Zed types](https://zed.brimdata.io/docs/formats/zed#2-complex-types)\n   in Grafana, modify your Zed query to make them available as top-level fields,\n   e.g., by using the\n   [`put` operator](https://zed.brimdata.io/docs/language/operators/put).\n\nNext we'll walk through some real world examples that leverage these best\npractices.\n\n## Example Usage in Dashboards\n\nAs [often described](https://kb.altinity.com/altinity-kb-schema-design/best-schema-for-storing-many-metrics-registered-from-the-single-source/),\ndifferent schema approaches may be used for storing time-series data. The Zed\nplugin can adapt to multiple approaches, but the Zed query used in the Grafana\npanel will differ. In each of the following sections we'll plot some sample\ntime-series data to illustrate the concepts.\n\n### One row per metric\n\nSome example data that uses this approach is the\n[weekly fuel prices (all data)](https://dgsaie.mise.gov.it/open_data_export.php?export-id=4\u0026export-type=csv)\nlink in the [Italian fuel price data](https://dgsaie.mise.gov.it/open-data), which is freely\navailable under the [IODL 2.0 license](https://it.wikipedia.org/wiki/Italian_Open_Data_License).\nWe'll start by downloading a copy with the English language column headers and\npeek at the data in its original form.\n\n```\n$ curl -o all_prices.csv \\\n  -H 'Accept-Language: en-US' \\\n  'https://dgsaie.mise.gov.it/open_data_export.php?export-id=4\u0026export-type=csv'\n\n$ head -10 all_prices.csv\nSURVEY_DATE,PRODUCT_ID,PRODUCT_NAME,PRICE,VAT,EXCISE,NET,CHANGE\n2005-01-03,1,\"Euro-Super 95\",1115.75,185.96,558.64,371.15,-1.57\n2005-01-03,2,\"Automotive gas oil\",1018.28,169.71,403.21,445.36,-0.33\n2005-01-03,3,\"Heating gas oil\",948.5,158.08,403.21,387.21,-22.55\n2005-01-03,5,LPG,552.5,92.08,156.62,303.8,0.22\n2005-01-03,6,\"Residual fuel oil\",553.25,50.3,166.84,336.11,-12.21\n2005-01-03,8,\"Heavy fuel oil\",229.52,0,31.39,198.13,-5.37\n2005-01-10,1,\"Euro-Super 95\",1088,181.33,558.64,348.03,-27.75\n2005-01-10,2,\"Automotive gas oil\",1004.39,167.4,403.21,433.78,-13.89\n2005-01-10,3,\"Heating gas oil\",947.94,157.99,403.21,386.74,-0.56\n```\n\nPer the approach, we see that the date stamp is repeated for the measurement of\neach of the six different fuel types. Also, being CSV data, this date field\nbegins life as a mere string and therefore must be transformed to the Zed\n`time` type as we store it in the lake.\n\nTaking this into account, we'll perform some preprocessing with\n[`zq`](https://zed.brimdata.io/docs/commands/zq) to prepare the Time\nField and also isolate a subset of other fields, then ultimately load the data\ninto a pool in our Zed lake. For convenience, we'll use `ts` as the name of\nthe transformed time field since this is the plugin's default.\n\n```\n$ zed create prices1\npool created: prices1 2KkHUfmYz7FDdix6WRf7XEjkRfO\n\n$ zq 'cut ts:=time(SURVEY_DATE),PRODUCT_NAME,PRICE' all_prices.csv \\\n  | zed load -use prices1 -\n(2/1) 75.95KB 75.95KB/s\n2KkHZEIfCdlCQJcgO9TAGT8cNpz committed\n```\n\nReading back a sampling of our data, we can see the successful transform.\n\n```\n$ zed query -Z 'from prices1 | sample'\n{\n    ts: 2023-01-16T00:00:00Z,\n    PRODUCT_NAME: \"Automotive gas oil\",\n    PRICE: 1863.68\n}\n```\n\nTo plot the data for all six fuel types in the same panel, we can construct\nsix queries, each of which filters by category. Since the legend would\notherwise show \"PRICE\" for all six, we'll use Zed's\n[`rename` operator](https://zed.brimdata.io/docs/language/operators/rename)  to\nassign a unique field name for each before the data is handed off to Grafana.\nBecause we want to construct field names with spaces, we use\n[field dereferencing with indexing](https://zed.brimdata.io/docs/language/overview#75-field-dereference).\n\nBelow is an example of one of the six queries, followed by the completed panel\nshown in Grafana.\n\n```\nPRODUCT_NAME==\"Automotive gas oil\" | rename this[\"Automotive gas oil\"]:=PRICE\n```\n\n![Example with one row per metric](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/prices1.png)\n\n### Each measurement (with lot of metrics) in its own row\n\nIf our data happens to be in the format with multiple metrics per row, it\nbecomes easy to plot with a single query. We can observe this by working with\nthe data in the\n[weekly fuel prices](https://dgsaie.mise.gov.it/open_data_export.php?export-id=1\u0026export-type=csv)\nlink at the same page we just used.\n\n```\n$ curl -o prices2.csv \\\n  -H 'Accept-Language: en-US' \\\n  'https://dgsaie.mise.gov.it/open_data_export.php?export-id=1\u0026export-type=csv'\n\n$ head -4 prices2.csv\nSURVEY_DATE,EURO-SUPER_95,AUTOMOTIVE_GAS_OIL,LPG,HEATING_GAS_OIL,RESIDUAL_FUEL_OIL,HEAVY_FUEL_OIL\n2005-01-03,1115.75,1018.28,552.5,948.5,553.25,229.52\n2005-01-10,1088,1004.39,552.57,947.94,554.22,238.37\n2005-01-17,1088.14,1004.31,551.88,952.42,562.78,245.89\n```\n\nAs we see, there's now a separate column in the CSV file for each category of fuel\nand each row of measurements appears with a single, shared date stamp. We'll\ncreate another pool and once again transform the date stamp to a field of Zed's\n`time` type as we load it into our lake.\n\n```\n$ zed create prices2\npool created: prices2 2KkJGc9sgl2T7eq5rB0WSff26IV\n\n$ zq 'rename ts:=SURVEY_DATE | ts:=time(ts)' prices2.csv \\\n  | zed load -use prices2 -\n(2/1) 30.29KB 30.29KB/s\n2Kbf3eLGvItvbxLA2TMQyaCab2W committed\n\n$ zed query -Z 'from prices2 | head 1'\n{\n    ts: 2023-01-16T00:00:00Z,\n    \"EURO-SUPER_95\": 1813.58,\n    AUTOMOTIVE_GAS_OIL: 1863.68,\n    LPG: 799.71,\n    HEATING_GAS_OIL: 1651.57,\n    RESIDUAL_FUEL_OIL: 1120.82,\n    HEAVY_FUEL_OIL: 636.78\n}\n```\n\nBecause Grafana defaults to plotting all numeric fields, all six appear on our\nchart if we let the plugin use its default Zed query (`*`) that pulls all points\nfrom the pool that are in the dashboard's current selected time range. The only\nchange from defaults we had to make when configuring our new panel was specifying\nthe pool name \"prices2\".\n\n![Example with many metrics per row](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/prices2.png)\n\nIf we wanted prettier names in the legend, we could add a Zed query to our\npanel config such as:\n\n```\nrename this[\"Euro-Super 95\"] := this[\"EURO-SUPER_95\"],\n       this[\"Automotive Gas Oil\"] := this[\"AUTOMOTIVE_GAS_OIL\"],\n       this[\"Heating Gas Oil\"] := this[\"HEATING_GAS_OIL\"],\n       this[\"Residual Fuel Oil\"] := this[\"RESIDUAL_FUEL_OIL\"],\n       this[\"Heavy Fuel Oil\"] := this[\"HEAVY_FUEL_OIL\"]\n```\n\n### Transform between approaches\n\nNow that we've seen the second approach makes it easier to plot, if you find\nyourself with data that's already stored using the first approach, you could use\na Zed query like what's shown below to transform to the second approach. This\nidiom could be used to preprocess the data before loading it into yet another\npool, or you could use it as part of a Zed query in your Grafana panel config.\n\n```\n$ zed query -Z 'from prices1\n                | map(|{PRODUCT_NAME:PRICE}|) by ts\n                | over map with time=ts =\u003e (\n                  yield {key:[key],value}\n                  | collect(this)\n                  | unflatten(this)\n                  | put ts:=time\n                )'\n\n{\n    LPG: 799.71,\n    \"Euro-Super 95\": 1813.58,\n    \"Heavy fuel oil\": 636.78,\n    \"Heating gas oil\": 1651.57,\n    \"Residual fuel oil\": 1120.82,\n    \"Automotive gas oil\": 1863.68,\n    ts: 2023-01-16T00:00:00Z\n}\n...\n```\n\nIn the future this functionality may be made available in more succinct Zed\nsyntax. Issue [zed/4332](https://github.com/brimdata/zed/issues/4332) tracks\nthis enhancement.\n\n### Variables\n\nThe plugin does not yet support [query variables](https://grafana.com/docs/grafana/latest/developers/plugins/add-support-for-variables/#add-support-for-query-variables-to-your-data-source)\nto populate [dashboard variables](https://grafana.com/docs/grafana/latest/dashboards/variables/)\nwith values pulled from a pool using Zed queries. However, in the meantime,\nqueries in panels can use variables made up of custom values defined in the\ndashboard settings as long as the set of picked values can be expanded into\nsyntactically correct Zed.\n\nBuilding on our prior example, here we've defined a multi-value variable called\n\"fuels\" made up of the six categories of our data.\n\n![Custom variable config](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/custom-variable-config.png)\n\nReturning to our dashboard, we now can enter a Zed query that uses the\n[`cut` operator](https://zed.brimdata.io/docs/language/operators/cut) to\nisolate only the Time Field and a variable reference that expands to a\ncomma-separated field list required by `cut`. Notice that we once again made\nuse of [field dereferencing with indexing](https://zed.brimdata.io/docs/language/overview#75-field-dereference)\nfor the field `EURO-SUPER_95` since it can't be referenced as an identifier due\nto its name containing the character `-`.\n\n![Custom variable in panel](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/custom-variable-in-panel.png)\n\n### Aggregations and the `$__interval` variable\n\nThe examples shown thus far assume that all points in the selected time range\nshould be plotted at their precise values. However, in practice, screen width\nand/or volume of data may make this undesirable or impossible. In these\nsituations it's typical to [summarize](https://zed.brimdata.io/docs/language/operators/summarize)\ntime-bucketed sets of points into single values that can populate a smaller\nnumber of pixels rendered in a chart. This summarization is done by applying\nan [aggregate function](https://zed.brimdata.io/docs/language/aggregates) such\nas [`avg()`](https://zed.brimdata.io/docs/language/aggregates/avg),\n[`min()`](https://zed.brimdata.io/docs/language/aggregates/min),\n[`max()`](https://zed.brimdata.io/docs/language/aggregates/max),\n[`count()`](https://zed.brimdata.io/docs/language/aggregates/count), or\n[`sum()`](https://zed.brimdata.io/docs/language/aggregates/sum) to each set of\nraw points.\n\nTo illustrate this example, we'll use a data source of logged\n[HTTP traffic](https://github.com/brimdata/zed-sample-data/blob/main/zeek-default/http.log.gz)\nfound in the [zed-sample-data repository](https://github.com/brimdata/zed-sample-data).\nIn our example we'll plot the count of [HTTP methods](https://www.rfc-editor.org/rfc/rfc7231#page-21)\nin observed requests over time.\n\nIn the query we'll construct, the use of Grafana's built-in\n[`$__interval`](https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#__interval)\nvariable is essential. The value for this variable is changed automatically by\nGrafana based on the current plot width and slides easily into the `span`\nparameter of Zed's [`bucket()` function](https://zed.brimdata.io/docs/language/functions/bucket).\n\nWe'll once again start by creating a pool and loading our raw test data. Since\nthis data already has a `time`-typed Time Field called `ts`, we don't need to\nperform the same preprocessing of the timestamp we did previously.\n\n```\n$ zed create http\npool created: http 2KkaG0Ms5mNM68kwXYbSj8tRciG\n\n$ zq \"get https://github.com/brimdata/zed-sample-data/blob/main/zeek-default/http.log.gz?raw=true\" \\\n  | zed load -use http -\n(1/1) 8.30MB 8.30MB/s\n2KbzmcybYySkykAkYxTdYsNbL5o committed\n```\n\nBelow is our example aggregation query, followed by the completed panel shown\nin Grafana.\n\n```\ncount() by ts:=bucket(ts,$__interval),method\n| map(|{method:count}|) by ts\n| over map with time=ts =\u003e (\n  yield {key:[key],value}\n  | collect(this)\n  | unflatten(this)\n  | put ts:=time\n)\n| fuse\n```\n\n![HTTP method count panel](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/http-method-count.png)\n\nTo see the effect of the `$__interval` variable, open the Network tab of your\nbrowser's Developer Tools and click the most recent request issued against the Zed\nlake API's `query` endpoint. Here we can see the final query assembled by the\nplugin based on the current panel settings. We can see that the `$__interval`\nvariable was replaced with a duration string `2s` that reflects 2-second time\nbuckets. If you zoom in/out to change the current time range for the plot and\ncheck again, you'll see this value change.\n\n![DevTools Network Tab](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/devtools-network-tab.png)\n\nTo understand what the rest of the Zed is doing, let's look at a sample of\ndata from outside Grafana starting with just the aggregation.\n\n```\n$ zed query -z 'from http\n                | count() by ts:=bucket(ts,2s),method\n                | sort ts'\n\n{ts:2018-03-24T17:15:20Z,method:\"POST\",count:1(uint64)}\n{ts:2018-03-24T17:15:20Z,method:\"OPTIONS\",count:1(uint64)}\n{ts:2018-03-24T17:15:20Z,method:\"HEAD\",count:1(uint64)}\n{ts:2018-03-24T17:15:20Z,method:\"PRI\",count:1(uint64)}\n{ts:2018-03-24T17:15:20Z,method:\"PUT\",count:1(uint64)}\n{ts:2018-03-24T17:15:20Z,method:\"GET\",count:98(uint64)}\n{ts:2018-03-24T17:15:22Z,method:null(string),count:1(uint64)}\n{ts:2018-03-24T17:15:22Z,method:\"POST\",count:3(uint64)}\n{ts:2018-03-24T17:15:22Z,method:\"GET\",count:84(uint64)}\n...\n```\n\nTwo things stand out here:\n\n1. The timestamps are repeated in what's effectively the\n   [one row per metric](#one-row-per-metric) approach discussed above. For\n   this reason in the next several lines of Zed we reuse the\n   [idiom shown previously](#transform-between-approaches) to transform\n   to the approach that consolidates all metrics for a timestamp into the\n   same row.\n\n2. The HTTP methods vary per timestamp, which is different from what we\n   saw with the fuel data where we always saw the same six fuel categories\n   reported for every timestamp. For this reason we apply `fuse` at the\n   end of our Zed to widen each record returned in the query response and\n   add `null` values for methods that did not appear during a time interval.\n   If we'd skipped the `fuse` we'd be attempting to plot multiple shapes and\n   the plugin would kick back an error message (try it!)\n\nApplying our full query and looking at a few lines of output, we can see the\neffect.\n\n```\n$ zed query -Z 'from http\n                | count() by ts:=bucket(ts,1s),method\n                | map(|{method:count}|) by ts\n                | over map with time=ts =\u003e (\n                  yield {key:[key],value}\n                  | collect(this)\n                  | unflatten(this)\n                  | put ts:=time\n                )\n                | fuse\n                | sort ts'\n\n{\n    POST: 1 (uint64),\n    ts: 2018-03-24T17:15:20Z,\n    GET: 63 (uint64),\n    HEAD: null (uint64),\n...\n```\n\nYou may notice lots of \"dots\" in the screenshot above, which are indicative of the\nsparse appearance of rarely-used HTTP methods surrounded by many `null` points.\nTo consider options for representing such data, refer to Grafana's\ndocumentation for the\n[connect null values](https://grafana.com/docs/grafana/latest/panels-visualizations/visualizations/time-series/#connect-null-values)\nsetting.\n\n### Annotations\n\nGrafana's [annotations](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/annotate-visualizations/#querying-other-data-sources)\nfeature can be used to overlay details pulled from a Zed lake onto a\ntime-series plot.\n\nAs an example that builds on top of our plot of counted HTTP methods, the\nfollowing query creates a custom timestamped field called `msg` that populates\nan annotation marking each time a user accessed the Google web site.\n\n```\nhost==\"www.google.com\"\n| yield {ts, msg: \"client \" + string(id.orig_h) + \" accessed \" + host}\n```\n\n![Configure annotation](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/configure-annotation.png)\n\nWhen we refresh our dashboard panel and hover the mouse pointer over the red\nmarker at the bottom of each dotted vertical line, we can see the custom\nmessage.\n\n![Hovering over annotation](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/annotation-hover.png)\n\n### Logs\n\nDue to the previously-described limitations with only handling top-level\nfields of primitive Zed types, the plugin is probably not well-suited for\ngeneral use with Grafana's\n[Logs panel](https://grafana.com/docs/grafana/latest/panels-visualizations/visualizations/logs/).\nHowever, the plugin is configured to permit this and you may find it useful\nfor examining string-based fields.\n\nIn this example we create a simple logs panel that shows the details of the\nHTTP events for accessing the Google web site that we used as the basis for\nour annotations query.\n\n![Logs panel](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/logs-panel.png)\n\n## Debugging\n\nWhen you hit problems, the first thing to check is for an alert shown\nwhen you hover the mouse pointer over a red triangle in the upper-left corner\nof a panel. The errors you may see here are described in this README and\nshould be self-explanatory.\n\n![Hovering over an error message.](https://github.com/brimdata/grafana-zed-datasource/raw/main/src/img/error-hover.png)\n\nIf you can't make sense of the error message, you may find it helpful to look\nin the Network tab of your browser's Developer Tools as shown\n[above](#aggregations-and-the-__interval-variable). The assembled query shown\ncan then be executed outside of Grafana using `zed query` or in the Zui app to\nnarrow down if it's a problem with how the query is constructed or if it's a\nbug/limitation in the plugin or Grafana.\n\nIf you're still stuck, come talk to us on the `#grafana` channel on the\n[Brim Data community Slack](https://www.brimdata.io/join-slack/) or\n[open an issue](https://github.com/brimdata/grafana-zed-datasource/issues).\n\n## Why Unsigned?\n\nIdeally we'd prefer to have the Zed plugin available in the\n[public data source plugin directory](https://grafana.com/grafana/plugins/?type=datasource)\nto make it easier to find and install. This is predicated on the plugin being\n[signed](https://grafana.com/docs/grafana/latest/developers/plugins/sign-a-plugin/).\nAn initial release of the plugin was indeed submitted to the Grafana maintainers\nwith the belief it met the criteria for the \"community\"\n[plugin signature level](https://grafana.com/docs/grafana/latest/developers/plugins/sign-a-plugin/#plugin-signature-levels)\nsince, per that page, the dependent [Zed](https://zed.brimdata.io/) technology \nis offered to the community free of charge as open source and will continue to be\nindefinitely. However, after initial review of our submission we were pointed\nat [more extensive legal terms](https://grafana.com/legal/plugins/)\nwhich explain that to qualify for the community signature level the plugin must\nbe \"not affiliated with any commercial endeavor\". As the corporate sponsor of\nthe Zed project, [Brim Data](https://www.brimdata.io/) is currently a seed stage startup, and the company\ndoes hope to one day operate a viable business such as by offering paid services and\nsupport to users that may want it. Apparently this fact alone is enough to\ndisqualify the plugin from eligibility at the \"community\" level. Furthermore, the\nconstraints of being a seed stage startup mean Brim Data cannot currently\njustify the expense of paying the quoted price to be signed at Grafana's\n\"commercial\" signature level. While this is disappointing, we respect Grafana's\ndecision to run their programs as they wish. We are hopeful that they will\none day change their policies to benefit our combined user base. In the\nmeantime, we hope that users may benefit from using the Zed plugin even in\nits atypical \"unsigned\" state.\n\nIf you have additional questions or concerns about running \"unsigned\", or if\nyou feel strongly about the plugin being \"signed\" and have thoughts on how\nto move past the known barriers, please come talk to us in the `#grafana`\nchannel on the [Brim Data community Slack](https://www.brimdata.io/join-slack/).\n\n## Contributing\n\nContributions are welcomed! Per common practice, please\n[open an issue](https://github.com/brimdata/grafana-zed-datasource/issues)\nbefore sending a pull request.  If you think your ideas might benefit from\nsome refinement via Q\u0026A, come talk to us on the `#grafana` channel on the\n[Brim Data community Slack](https://www.brimdata.io/join-slack/).\n\n## Join the Community\n\nJoin the [Brim Data community Slack](https://www.brimdata.io/join-slack/) workspace for\nannouncements, Q\u0026A, and to trade tips. There's a `#grafana` channel where you\ncan ask questions and get help with this plugin, a `#zed` channel for Q\u0026A about the\nZed language \u0026 lake, and more!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrimdata%2Fgrafana-zed-datasource","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrimdata%2Fgrafana-zed-datasource","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrimdata%2Fgrafana-zed-datasource/lists"}