https://github.com/spacez320/shui
Data collection and presentation on the command line.
https://github.com/spacez320/shui
cli data-visualization logging monitoring system-administration telemetry terminal time-series
Last synced: about 2 months ago
JSON representation
Data collection and presentation on the command line.
- Host: GitHub
- URL: https://github.com/spacez320/shui
- Owner: spacez320
- License: mit
- Created: 2023-11-09T14:56:07.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2025-05-23T23:29:55.000Z (8 months ago)
- Last Synced: 2025-05-24T00:28:24.413Z (8 months ago)
- Topics: cli, data-visualization, logging, monitoring, system-administration, telemetry, terminal, time-series
- Language: Go
- Homepage:
- Size: 81.2 MB
- Stars: 7
- Watchers: 2
- Forks: 0
- Open Issues: 16
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
shui
====
[](https://goreportcard.com/report/github.com/spacez320/shui)
[](https://pkg.go.dev/github.com/spacez320/shui/cmd/shui)
Shui is a tool that can be used to extract information with a CLI and observe it over time. It is
meant to aid system administrators, platform engineers, or anyone who spends time doing operations
in a command line.
- It's like a better `watch` that can draw graphs, tables, etc.
- It's like a simplified metrics database that can run directly in your console.
- It can act as a bridge between a command line and external monitoring systems.
This project is in an active, "alpha" development phase, but should generally be useable.
Setup
-----
- Binaries are available from [the releases page](https://github.com/spacez320/shui/releases).
- Docker images are available on [Docker Hub](https://hub.docker.com/repository/docker/spacez320/shui).
You can also just:
```sh
go install github.com/spacez320/shui/cmd/shui@latest
```
Usage
-----
Shui expects a **query** to gather data with (e.g. a CLI command, a process ID, etc.). Queries
produce **results** that are parsed automatically and stored as a time series.
> NOTE: I'm not good at making gifs, so some of the commands shown may be outdated, even if the
> functionality isn't.
### Modes
Shui has **"modes"** that determine what type of query should be provided.
**Query mode** is the default and is for running shell commands.
**Profile mode** is like Query mode except specialized for inspecting systems or processes.
### Displays
Shui also has **"displays"** that determine how data is presented.
**Raw display** and **stream display** simply presents incoming data, the latter being within Shui's
interactive window. The examples above use stream displays.
**Table display** will parse results into a table.
**Graph display** will target a specific field in a result and graph it (this requires the query to
produce a number).
### Examples
These examples show basic usage.
> The examples below have been tested on `GNU bash, version 5.2.15(1)-release`.
```sh
# See help.
shui --help
# Execute `whoami` once, printing results to the console and waiting for a user to `^C`.
shui --query 'whoami'
# Execute `uptime` continuously, printing results to the console, without using persistence.
shui \
--count -1 \
--history=false \
--query 'uptime'
# Get the size of an NVME disk's used space and output it to a table with the specific label "NVME
# Used Space".
shui \
--count -1 \
--display 3 \
--labels "NVME Used Space" \
--query 'df -h | grep nvme0n1p2 | awk '\''{print $3}'\'''
```
You can also execute Shui from standard input, which works like query mode.
```sh
while true; do uptime; sleep 1; done | shui
```
### Configuration
Shui can accept instruction from flags or from a configuration file. By default it will look for a
`shui.toml` in the user's config directory (see: ), but one may
also be provided explicitly.
```sh
shui --config path/to/config # Default is "${HOME}/.config/shui/shui.toml".
```
See [example/shui.toml](example/shui.toml) for an example.
> NOTE: Most options may be mixed, but queries and expressions may only be supplied with either
> flags or configuration and not both.
### Persistence
Shui, by default, will store results and load them when re-executing the same query.
The only currently supported storage is local disk, located in the user's cache directory. See:
.
### Expressions
Shui has the ability to execute "expressions" on query results in order to manipulate them
before display (e.g. performing statistics, combining values, producing cumulative series, etc.).
Some key points about expressions:
- Multiple expressions may be provided and execute in the order provided.
- Filters apply before expressions.
- It uses [Expr, a Go-centric expression language](https://github.com/expr-lang/expr).
- The expression language is type sensitive, but results of expressions will always be strings.
Expressions are able to access variables:
1. `result`, a map of the current result's labels to values.
2. `prevResult`, the previous result mapping, for cumulative results. Note that expressions must
account for `prevResult` being an empty map for the first result in a series.
Some examples:
```sh
# Multiply the 5m CPU average by 10. Note that we invoke `get` with a key of `"9"` because default
# labels are string indexes and no labels were provided.
shui --query 'uptime | tr -d ","' -expr 'get(result, "9") * 10'
# Cumulatively sum 5m CPU average. Note that we need to account for prevResult being empty and we
# must convert the prevResult from a string to a float.
shui --query 'uptime | tr -d ","' -filters 9 -expr 'get(result, "0") + ("0" in prevResult? float(get(prevResult, "0")) : 0)'
```
See:
### Integrations
Shui can send its data off to external systems, making it useful as an ad-hoc metrics or log
exporter. Supported integrations are listed below.
#### Elasticsearch
Shui can create Elasticsearch documents from results.
```sh
shui \
--elasticsearch-addr \
--elasticsearch-index \
--elasticsearch-user \
--elasticsearch-password
# Specify a Prometheus Pushgateway address to send results to.
shui --prometheus-pushgateway
```
- Metrics namse will have the structure `shui_` where `` will be changed to
conform to Prometheus naming rules.
- Shui labels supplied with `--labels` will be saved as a Prometheus label called
`shui_label`, creating a unique series for each value in a series of results.
As an example, given a query `cat file.txt | wc`, and `-labels "newline,words,bytes"`, the following
Prometheus metrics would be created:
```
shui_cat_file_txt_wc{shui_label="newline"}
shui_cat_file_txt_wc{shui_label="words"}
shui_cat_file_txt_wc{shui_label="bytes"}
```
> **NOTE:** The only currently supported metric is a **Gauge** and queries must provide something
> numerical to be recorded.
Future
------
I've been adding planned work into [project issues](https://github.com/spacez320/shui/issues)
and [project milestones](https://github.com/spacez320/shui/milestone/1)--take a look there to
see what's coming or to make suggestions.
Planned improvements include things like:
- [ ] Background execution.
- [x] Persistent results.
- [x] Ability to perform calculations on streams of data, such as aggregates, rates, or quantile math.
- [ ] Better text result management, such as diff'ing.
- [x] Export data to external systems, such as Prometheus.
- [x] ... and Elasticsearch.
- [ ] More detailed and varied display modes.
- [ ] Historical querying.
Similar Projects
----------------
There doesn't seem to be much out there easily visible that matches the same set of functionality,
but there are a few projects I've found that do some things similarly.
- [DataDash](https://github.com/keithknott26/datadash), a data visualization tool for the terminal.
- [Grafterm](https://github.com/slok/grafterm), visualize metrics dashboards on the terminal.
- [Euoporie](https://github.com/joouha/euporie), a terminal interactive computing environment for
Jupyter.