{"id":34213821,"url":"https://github.com/spacez320/shui","last_synced_at":"2025-12-15T22:00:13.923Z","repository":{"id":207222455,"uuid":"716637909","full_name":"spacez320/shui","owner":"spacez320","description":"Data collection and presentation on the command line.","archived":false,"fork":false,"pushed_at":"2025-05-23T23:29:55.000Z","size":85111,"stargazers_count":7,"open_issues_count":16,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-24T00:28:24.413Z","etag":null,"topics":["cli","data-visualization","logging","monitoring","system-administration","telemetry","terminal","time-series"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spacez320.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-11-09T14:56:07.000Z","updated_at":"2025-04-30T23:13:54.000Z","dependencies_parsed_at":"2023-12-15T00:20:44.990Z","dependency_job_id":"2c896bf0-c9e9-4e2c-a686-960f0232b963","html_url":"https://github.com/spacez320/shui","commit_stats":null,"previous_names":["spacez320/cryptarch","spacez320/shui"],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/spacez320/shui","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacez320%2Fshui","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacez320%2Fshui/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacez320%2Fshui/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacez320%2Fshui/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spacez320","download_url":"https://codeload.github.com/spacez320/shui/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacez320%2Fshui/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27757352,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-15T02:00:09.782Z","response_time":96,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","data-visualization","logging","monitoring","system-administration","telemetry","terminal","time-series"],"created_at":"2025-12-15T22:00:12.414Z","updated_at":"2025-12-15T22:00:13.535Z","avatar_url":"https://github.com/spacez320.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"shui\n====\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/spacez320/shui)](https://goreportcard.com/report/github.com/spacez320/shui)\n[![Go Reference](https://pkg.go.dev/badge/github.com/spacez320/shui/cmd/shui.svg)](https://pkg.go.dev/github.com/spacez320/shui/cmd/shui)\n![GitHub Release](https://img.shields.io/github/v/release/spacez320/shui)\n\nShui is a tool that can be used to extract information with a CLI and observe it over time. It is\nmeant to aid system administrators, platform engineers, or anyone who spends time doing operations\nin a command line.\n\n- It's like a better `watch` that can draw graphs, tables, etc.\n- It's like a simplified metrics database that can run directly in your console.\n- It can act as a bridge between a command line and external monitoring systems.\n\nThis project is in an active, \"alpha\" development phase, but should generally be useable.\n\nSetup\n-----\n\n- Binaries are available from [the releases page](https://github.com/spacez320/shui/releases).\n- Docker images are available on [Docker Hub](https://hub.docker.com/repository/docker/spacez320/shui).\n\nYou can also just:\n\n```sh\ngo install github.com/spacez320/shui/cmd/shui@latest\n```\n\nUsage\n-----\n\nShui expects a **query** to gather data with (e.g. a CLI command, a process ID, etc.). Queries\nproduce **results** that are parsed automatically and stored as a time series.\n\n\u003e NOTE: I'm not good at making gifs, so some of the commands shown may be outdated, even if the\n\u003e functionality isn't.\n\n### Modes\n\nShui has **\"modes\"** that determine what type of query should be provided.\n\n**Query mode** is the default and is for running shell commands.\n\n![Demo of query mode](https://raw.githubusercontent.com/spacez320/shui/master/assets/query-mode.gif)\n\n**Profile mode** is like Query mode except specialized for inspecting systems or processes.\n\n![Demo of profile mode](https://raw.githubusercontent.com/spacez320/shui/master/assets/profile-mode.gif)\n\n### Displays\n\nShui also has **\"displays\"** that determine how data is presented.\n\n**Raw display** and **stream display** simply presents incoming data, the latter being within Shui's\ninteractive window. The examples above use stream displays.\n\n**Table display** will parse results into a table.\n\n![Demo of table display](https://raw.githubusercontent.com/spacez320/shui/master/assets/table-display.gif)\n\n**Graph display** will target a specific field in a result and graph it (this requires the query to\nproduce a number).\n\n![Demo of graph display](https://raw.githubusercontent.com/spacez320/shui/master/assets/graph-display.gif)\n\n### Examples\n\nThese examples show basic usage.\n\n\u003e The examples below have been tested on `GNU bash, version 5.2.15(1)-release`.\n\n```sh\n# See help.\nshui --help\n\n# Execute `whoami` once, printing results to the console and waiting for a user to `^C`.\nshui --query 'whoami'\n\n# Execute `uptime` continuously, printing results to the console, without using persistence.\nshui \\\n    --count -1 \\\n    --history=false \\\n    --query 'uptime'\n\n# Get the size of an NVME disk's used space and output it to a table with the specific label \"NVME\n# Used Space\".\nshui \\\n    --count -1 \\\n    --display 3 \\\n    --labels \"NVME Used Space\" \\\n    --query 'df -h | grep nvme0n1p2 | awk '\\''{print $3}'\\'''\n```\n\nYou can also execute Shui from standard input, which works like query mode.\n\n```sh\nwhile true; do uptime; sleep 1; done | shui\n```\n\n### Configuration\n\nShui can accept instruction from flags or from a configuration file. By default it will look for a\n`shui.toml` in the user's config directory (see: \u003chttps://pkg.go.dev/os#UserConfigDir\u003e), but one may\nalso be provided explicitly.\n\n```sh\nshui --config path/to/config  # Default is \"${HOME}/.config/shui/shui.toml\".\n```\n\nSee [example/shui.toml](example/shui.toml) for an example.\n\n\u003e NOTE: Most options may be mixed, but queries and expressions may only be supplied with either\n\u003e flags or configuration and not both.\n\n### Persistence\n\nShui, by default, will store results and load them when re-executing the same query.\n\nThe only currently supported storage is local disk, located in the user's cache directory. See:\n\u003chttps://pkg.go.dev/os#UserCacheDir\u003e.\n\n### Expressions\n\nShui has the ability to execute \"expressions\" on query results in order to manipulate them\nbefore display (e.g. performing statistics, combining values, producing cumulative series, etc.).\n\nSome key points about expressions:\n\n- Multiple expressions may be provided and execute in the order provided.\n- Filters apply before expressions.\n- It uses [Expr, a Go-centric expression language](https://github.com/expr-lang/expr).\n- The expression language is type sensitive, but results of expressions will always be strings.\n\nExpressions are able to access variables:\n\n1.  `result`, a map of the current result's labels to values.\n2.  `prevResult`, the previous result mapping, for cumulative results. Note that expressions must\n    account for `prevResult` being an empty map for the first result in a series.\n\nSome examples:\n\n```sh\n# Multiply the 5m CPU average by 10. Note that we invoke `get` with a key of `\"9\"` because default\n# labels are string indexes and no labels were provided.\nshui --query 'uptime | tr -d \",\"' -expr 'get(result, \"9\") * 10'\n\n# Cumulatively sum 5m CPU average. Note that we need to account for prevResult being empty and we\n# must convert the prevResult from a string to a float.\nshui --query 'uptime | tr -d \",\"' -filters 9 -expr 'get(result, \"0\") + (\"0\" in prevResult? float(get(prevResult, \"0\")) : 0)'\n```\n\nSee: \u003chttps://expr-lang.org/docs/language-definition\u003e\n\n### Integrations\n\nShui can send its data off to external systems, making it useful as an ad-hoc metrics or log\nexporter. Supported integrations are listed below.\n\n#### Elasticsearch\n\nShui can create Elasticsearch documents from results.\n\n```sh\nshui \\\n    --elasticsearch-addr \u003caddr\u003e \\\n    --elasticsearch-index \u003cindex\u003e \\\n    --elasticsearch-user \u003cuser\u003e \\\n    --elasticsearch-password \u003cpassword\n```\n\n- Documents are structured according to result labels supplied with `--labels`, prefixed with\n  `shui.value.`.\n- Documents will also contain an additional field, `shui.query`.\n- The result `Time` field will be mapped to `timestamp`.\n- Shui must use HTTP Basic Auth (credentials are given with `--elasticsearch-user` and\n  `--elasticsearch-password`).\n- Shui will not attempt to create an index (one must be supplied with `--elasticsearch-index`).\n\nAs an example, given a query `cat file.txt | wc` and `--labels \"newline,words,bytes\"`, the following\nElasticsearch document would be created:\n\n```json\n{\n    \"_index\": \"some-index\",\n    \"_id\": \"some-id\",\n    \"_score\": 1.0,\n    \"_source\": {\n        \"shui.query\": \"cat file.txt | wc\",\n        \"shui.value.bytes\": 3,\n        \"shui.value.newline\": 1,\n        \"shui.value.words\": 2,\n        \"timestamp\": \"2024-06-10T17:40:29.773550719-04:00\"\n    }\n}\n```\n\n#### Prometheus\n\nShui can create Prometheus metrics from numerical results. Both normal Prometheus collection\nand Pushgateway are supported.\n\n```sh\n# Start a Prometheus collection HTTP page.\nshui --prometheus-exporter \u003caddress\u003e\n\n# Specify a Prometheus Pushgateway address to send results to.\nshui --prometheus-pushgateway \u003caddress\u003e\n```\n\n- Metrics namse will have the structure `shui_\u003cquery\u003e` where `\u003cquery\u003e` will be changed to\n  conform to Prometheus naming rules.\n- Shui labels supplied with `--labels` will be saved as a Prometheus label called\n  `shui_label`, creating a unique series for each value in a series of results.\n\nAs an example, given a query `cat file.txt | wc`, and `-labels \"newline,words,bytes\"`, the following\nPrometheus metrics would be created:\n\n```\nshui_cat_file_txt_wc{shui_label=\"newline\"}\nshui_cat_file_txt_wc{shui_label=\"words\"}\nshui_cat_file_txt_wc{shui_label=\"bytes\"}\n```\n\n\u003e **NOTE:** The only currently supported metric is a **Gauge** and queries must provide something\n\u003e numerical to be recorded.\n\nFuture\n------\n\nI've been adding planned work into [project issues](https://github.com/spacez320/shui/issues)\nand [project milestones](https://github.com/spacez320/shui/milestone/1)--take a look there to\nsee what's coming or to make suggestions.\n\nPlanned improvements include things like:\n\n- [ ] Background execution.\n- [x] Persistent results.\n- [x] Ability to perform calculations on streams of data, such as aggregates, rates, or quantile math.\n- [ ] Better text result management, such as diff'ing.\n- [x] Export data to external systems, such as Prometheus.\n- [x] ... and Elasticsearch.\n- [ ] More detailed and varied display modes.\n- [ ] Historical querying.\n\nSimilar Projects\n----------------\n\nThere doesn't seem to be much out there easily visible that matches the same set of functionality,\nbut there are a few projects I've found that do some things similarly.\n\n- [DataDash](https://github.com/keithknott26/datadash), a data visualization tool for the terminal.\n- [Grafterm](https://github.com/slok/grafterm), visualize metrics dashboards on the terminal.\n- [Euoporie](https://github.com/joouha/euporie), a terminal interactive computing environment for\n  Jupyter.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspacez320%2Fshui","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspacez320%2Fshui","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspacez320%2Fshui/lists"}