Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://gitlab.com/arvidnl/vizpl2
Visualizes GitLab CI pipeline execution times.
https://gitlab.com/arvidnl/vizpl2
gitlab gitlab-ci typescript
Last synced: about 2 months ago
JSON representation
Visualizes GitLab CI pipeline execution times.
- Host: gitlab.com
- URL: https://gitlab.com/arvidnl/vizpl2
- Owner: arvidnl
- Created: 2021-11-12T14:04:14.796Z (about 3 years ago)
- Default Branch: master
- Last Synced: 2024-10-06T08:07:14.101Z (3 months ago)
- Topics: gitlab, gitlab-ci, typescript
- Stars: 2
- Forks: 1
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# vizpl2
Visualizes GitLab CI pipeline execution times.
![](https://gitlab.com/arvidnl/vizpl2/-/raw/master/.images/demo.gif)
## Installation
`vizpl2` can either be installed locally using `npm` or run through Docker.
### Installation with `npm`
```sh
npm install --global vizpl2
```This will put `vizpl2` in your path:
```sh
$ vizpl2 --graph 'arvidnl/vizpl2#725494153'
...
$ vizpl2 --help
```### Usage with Docker
Docker images for the vizpl2 `master` branch are pushed to
`registry.gitlab.com/arvidnl/vizpl2:master`.For ease of use, consider the following alias:
```sh
alias vizpl2="docker run --rm --tty registry.gitlab.com/arvidnl/vizpl2:master"
```The `--tty` flag ensures that vizpl2's `--graph` mode displays ANSI
colors correctly.## Usage
Pipelines are referred to either as:
- A pipeline reference `namespace/project#pipeline_id`, `project#pipeline_id`,
`#pipeline_id` or `pipeline_id`. At the time of writing `namespace`
defaults to `arvidnl` and `project` default to `vizpl2`. So
`arvidnl/vizpl2#725494153`, `vizpl2#725494153`, `'#725494153'` and
`725494153` all refer to the same pipeline. Be sure to quote
references given without namespace and project
(i.e. `'#725494153'`), to avoid the hash being interpreted as a
shell comment.
- A pipeline URL,
i.e. `https://gitlab.com/namespace/project/-/pipelines/pipeline_id`,
which is equivalent to the pipeline reference
`namespace/project#pipeline_id`.
- `local_file.json`, where `local_file.json` contain JSON data in the
output format of GitLab's [List pipeline jobs API
endpoint](https://docs.gitlab.com/ee/api/jobs.html#list-pipeline-jobs).## Graphs
Single pipelines can be visualized as a graph by passing `--graph`:
```sh
$ vizpl2 --graph '#725494153'
Dataset Job Time (█ = 5s, █ = time queued)
---------- ------------------------ ------------------------------
#725494153 [pipeline] ▏█████████████████████████████ (NaN, 2m25s)
#725494153 > [install] ▏ █████▋ (0s, 28s)
#725494153 - install ▏ █████▋ (0s, 28s)
#725494153 > [build] ▏ ████████▋ (0s, 43s)
#725494153 - docker:build_publish ▏ ███████████████▌ (0s, 1m17s)
#725494153 - build ▏ ██████▎ (0s, 31s)
#725494153 - format ▏ ████▎ (0s, 21s)
#725494153 > [test] ▏ ███████▋ (0s, 38s)
#725494153 - test ▏ ███████▋ (0s, 38s)
```## Tables
The duration of one or multiple pipelines can be presented as a table by passing `--table`:
```sh
$ npx vizpl2 --table '#725494153'
+--------------------------------------------------------------+
| Job durations |
+--------------------------+-----------------------------------+
| Job | #725494153 (n=1): Duration (mean) |
+--------------------------+-----------------------------------+
| [pipeline] | 2m25.17s |
| > [install] | 28.15s |
| - install | 28.15s |
| > [build] | 43.43s |
| - docker:build_publish | 1m17.76s |
| - build | 31.06s |
| - format | 21.47s |
| > [test] | 38.26s |
| - test | 38.26s |
| | |
| Sequential time (mean): | 3m16.69s |
+--------------------------+-----------------------------------+
```The final row, labeled `Sequential time (mean)`, is the sum of the
duration of each job in the pipeline.### Tables with multiple data sets
You can provide multiple pipeline references on the command line:
```sh
$ npx vizpl2 --table '#725494153' '#725491719'
+--------------------------------------------------------------------------------------------------+
| Job durations |
+--------------------------+-----------------------------------+-----------------------------------+
| Job | #725494153 (n=1): Duration (mean) | #725491719 (n=1): Duration (mean) |
+--------------------------+-----------------------------------+-----------------------------------+
| [pipeline] | 2m25.17s | 2m21.19s (- %2.82) |
| > [install] | 28.15s | 29.62s (+ %5.23) |
| - install | 28.15s | 29.62s (+ %5.23) |
| > [build] | 43.43s | 41.9s (- %3.64) |
| - docker:build_publish | 1m17.76s | 1m11.64s (- %8.54) |
| - build | 31.06s | 32.26s (+ %3.88) |
| - format | 21.47s | 21.81s (+ %1.57) |
| > [test] | 38.26s | 38.78s (+ %1.37) |
| - test | 38.26s | 38.78s (+ %1.37) |
| | | |
| Sequential time (mean): | 3m16.69s | 3m14.11s (- %1.33) |
+--------------------------+-----------------------------------+-----------------------------------+
```### Combining pipelines into data sets
You can average sets of pipelines by combining them into
data sets. Pipeline references are combined into a labeled data set by
terminating them with a `--label ` option:```sh
$ vizpl2 --table \
'#725494153' '#725491719' --label 'Dataset1' \
'#725488623' '#725487593' --label 'Dataset2'
+----------------------------------------------------------------------------------------------+
| Job durations |
+--------------------------+---------------------------------+---------------------------------+
| Job | Dataset1 (n=2): Duration (mean) | Dataset2 (n=2): Duration (mean) |
+--------------------------+---------------------------------+---------------------------------+
| [pipeline] | 2m23.18s | 2m20.09s (- %2.21) |
| > [install] | 28.88s | 28.46s (- %1.5) |
| - install | 28.88s | 28.46s (- %1.5) |
| > [build] | 42.67s | 43.64s (+ %2.27) |
| - docker:build_publish | 1m14.7s | 1m17.69s (+ %4.01) |
| - build | 31.66s | 30.51s (- %3.77) |
| - format | 21.64s | 22.71s (+ %4.95) |
| > [test] | 38.52s | 32.88s (- %17.15) |
| - test | 38.52s | 32.88s (- %17.15) |
| | | |
| Sequential time (mean): | 3m15.4s | 3m12.25s (- %1.64) |
+--------------------------+---------------------------------+---------------------------------+
```In this example, pipelines `#725494153` and `#725491719` are combined
into the data set labeled `Dataset1`, and pipelines `#725488623` and
`#725487593` are combined into the data set labeled `Dataset2`. Within
each data set, the durations of jobs is averaged.Similarly, the `Sequential time` of each dataset is the mean of the
sequential time of each pipeline in that dataset.## JSON output
You can get the output of vizpl2 in JSON format using `--json`.
vizpl2 --json tezos/tezos#874232916
This can be combined with `jq` to easily filter the output it like this:
vizpl2 --json tezos/tezos#874232916 | \
jq 'map(select(.Job.type == "stage" or .Job.type == "pipeline"))'## Options
For the full list of options, pass `--help` to `vizpl2`.
## Interpreting results
How to read the results:
- the mean duration of the pipeline (job name written as
`[pipeline]`) is the mean end time of the job finishing last.- the mean duration time of stages (job name written as `> [ stage
name ]`) is the average duration of slowest job of the stage in
each pipeline of the data set: `average(forall ds in data sets:
max(forall job in ds.stage: job.duration))`. In other words, it
is not the `mean end time of the stage - mean start time` of the
stage.This means that the mean duration of pipelines depends on how runners
are scheduled to jobs. On the other hand, this is not true for the
mean duration of stages, and therefore comparisons made there allow to
disregard variations in runner availability.Ideally, we should calculate the critical path for a pipeline, and
them sum the mean duration of each step on the critical path, but the
tools does not yet handle this.# Development
Use the package scripts defined in `package.json`:
- `npm run build`: to compile TypeScript to JavaScript.
- `npm run start`: to compile TypeScript to JavaScript continuously in the background.
- `npm run test`: to run the test suite.