Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/okuvshynov/fewlines
Create histograms, time series charts and dashboards in command-line output and log files.
https://github.com/okuvshynov/fewlines
cli dashboard distribution histogram logging python
Last synced: 17 days ago
JSON representation
Create histograms, time series charts and dashboards in command-line output and log files.
- Host: GitHub
- URL: https://github.com/okuvshynov/fewlines
- Owner: okuvshynov
- License: mit
- Created: 2023-06-29T00:11:52.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-06-03T16:08:54.000Z (8 months ago)
- Last Synced: 2025-01-06T17:55:31.924Z (about 1 month ago)
- Topics: cli, dashboard, distribution, histogram, logging, python
- Language: Python
- Homepage:
- Size: 213 KB
- Stars: 10
- Watchers: 4
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fewlines - log your dashboard
Whether we like it or not, we debug things by putting print statements around.
`fewlines` is a supplement for this, allowing to plot bar charts which only take few lines in a terminal output, but can be a very useful piece of information.
In a more extended form fewlines can be used for logging collections of charts in a dashboard-like form.
Currently python only.
c++ version in progress and experimental.
Requires Unicode block characters.
Horizon-style color output requires terminal with 256 ANSI colors.
Monochrome version can be used for logging distributions to text files.
Example: plotting distribution of weights, gradients and activations for layers in LoRA neural net modules:
```
=== WEIGHTS ===
~~~~~~~~~ -0.000912|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~0~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|0.000912
Q0.B|▃▇▅▄▄▃▃▃▃▂▂▂▃▂▂▂▂▂▂▂▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▃▂▃▃▂▂▂▂▃▃▃▃▄▄▆▇▃|
V0.B|▁▆▇▇▆▅▄▃▃▄▃▃▃▃▃▃▃▂▂▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▃▂▃▃▃▃▃▃▃▃▃▄▄▅▆▇▇▆▁|
Q1.B|▃▇▇▇▅▅▄▄▃▄▃▃▃▃▃▃▃▃▃▃▃▃▃▃▃▃▃▃▃▃▂▃▃▃▃▃▃▃▃▂▃▃▃▃▃▃▃▃▃▃▃▃▄▄▅▅▆▇▇▃|
V1.B|▂▇▇▆▅▄▄▃▃▂▂▂▂▂▂▂▂▁▂▂▁▂▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁▂▁▂▂▁▁▂▂▂▂▂▂▂▂▂▃▄▅▅▆▇▇▂|
Q2.B|▂▆▇▆▅▄▃▃▃▃▃▃▃▃▃▃▂▃▃▃▃▂▃▃▂▂▂▂▂▃▂▂▃▂▃▃▂▃▃▂▃▃▃▃▃▃▃▃▃▃▃▃▃▄▄▅▆▇▇▂|
V2.B|▂▇▇▆▅▄▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▂▂▂▁▁▂▁▁▂▁▁▁▂▂▂▂▂▂▂▂▂▂▂▂▂▂▃▃▃▄▅▆▇▇▂|
Q3.B|▃▇▇▇▅▅▄▄▃▃▃▃▃▃▃▃▂▃▂▂▂▂▂▂▂▂▂▂▂▃▂▂▂▂▂▂▂▂▂▂▃▂▂▃▃▂▃▃▃▃▃▃▄▄▅▅▆▇▇▃|
V3.B|▃▇▇▆▅▄▃▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▂▂▂▂▂▃▄▄▆▇▇▃|
Q4.B|▂▆▇▇▅▅▄▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▂▁▁▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▃▃▄▅▅▆▇▆▂|
V4.B|▃▇▆▅▄▃▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▂▃▄▅▆▇▃|
Q5.B|▃▇▇▆▆▅▄▄▃▃▃▃▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▃▂▃▃▂▂▃▂▃▃▃▃▃▄▄▄▅▆▇▇▆▃|
V5.B|▃▇▆▅▄▃▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▂▂▃▄▅▆▇▄|
Q6.B|▃▇▇▆▅▄▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▂▂▂▂▂▁▁▂▁▂▂▂▂▂▂▂▂▂▂▂▂▂▂▃▃▄▅▆▇▇▃|
V6.B|▄▇▆▄▃▃▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▃▄▅▆▇▃|
Q7.B|▃▇▇▆▅▄▄▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▃▃▃▄▅▆▇▇▃|
V7.B|▄▇▆▄▃▂▂▁▁▁▁▁▁▁▁▁▁▁▁ ▁▁ ▁▁ ▁ ▁ ▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▃▄▆▇▅|
Q8.B|▄▇▆▅▄▃▃▂▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▂▁▂▂▂▂▂▂▂▃▃▃▄▅▆▇▄|
V8.B|▅▇▅▄▃▂▂▁▁▁▁▁▁▁▁▁▁▁ ▁▁ ▁▁ ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▂▃▄▅▇▅|
```
As we can see, it's very compact, can be put to log files with any other relevant info and gives a good insight into what do weights look like: bimodal, close to 1e-4, earlier layers move closer to center.
While created originally to plot distributions of weights/gradients or ML models it is not restricted to ML use cases, obviously. Can be used to show latency distribution, for example.
python installation:
```
pip install fewlines
```
## Usage example
### Basic charts
```
from fewlines import charts as fc
import numpy as np
A = np.random.normal(size=100)
print('Just the list of numbers')
for l in fc.histogram_chart(A):
print(l)
print('the list of numbers with extra options')
for l in fc.histogram_chart((A, {'n_lines': 3})):
print(l)
print('the dict title -> numbers')
for l in fc.histogram_chart({'series_A': A}):
print(l)
print('the dict title -> numbers, options')
for l in fc.histogram_chart({'series_A': (A, {'n_lines': 3})}):
print(l)
```
Output:
Note that multi-line charts rendering might depend on terminal spacing settings. Here in markdown it is messed up, so I'm attaching it as image.
### Dashboards
```
from fewlines import metrics as fm
from fewlines import dashboard as fd
import numpy as np
import time
# logging data somewhere
for i, v in enumerate(np.random.lognormal(mean=1.0, sigma=0.7, size=1000)):
fm.add('ssd_read_latency', v, time.time() - i)
for i, v in enumerate(np.random.lognormal(mean=3.0, sigma=0.7, size=1500)):
fm.add('nw_recv_latency', v, time.time() - i)
print("\n## Default one-line dashboards with wildcard")
for s in fd.histograms('*latency'):
print(s)
for s in fd.timeseries('*latency'):
print(s)
print()
print('\n## two histograms with separate scales with larger height and horizon colors')
for s in fd.dashboard({"charts": [('*latency', 'histogram')], "n_lines": 3, "color": 'green'}):
print(s)
print()
print('\n## two histograms sharing the scale as they are part of the same group with larger height and horizon colors')
for s in fd.dashboard({"charts": [[('*latency', 'histogram')]], "n_lines": 3, "color": 'green'}):
print(s)
print()
conf = {
"title": "Custom Dashboard",
"charts": [
('ssd_read_latency', 'timeseries', {'n_lines': 3, 'color': None}),
[
('ssd_read_latency', 'timeseries'),
('ssd_read_latency', 'timeseries', {'agg': 'max'}),
('ssd_read_latency', 'timeseries', {'agg': 'min'}),
],
('ssd_read_latency', 'histogram', {'n_lines': 4, 'color': 'green'}),
('ssd_read_latency', 'histogram', {'n_lines': 4, 'color': 'gray'}),
('ssd_read_latency', 'histogram', {'n_lines': 6, 'color': None}),
],
"time": -600, # default -3600
"bins": 60, # default 60
"title_width": 40, # default 30
"n_lines": 3,
"color": None,
}
print('\n## detailed complicated config with different aggregations')
for s in fd.dashboard(conf):
print(s)
```
Output:
