Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/pwwang/pipen-board

Visualizing configuration and running of pipen pipelines on the web
https://github.com/pwwang/pipen-board

pipeline pipelines

Last synced: 14 days ago
JSON representation

Visualizing configuration and running of pipen pipelines on the web

Host: GitHub
URL: https://github.com/pwwang/pipen-board
Owner: pwwang
Created: 2023-04-05T06:52:01.000Z (over 1 year ago)
Default Branch: master
Last Pushed: 2024-08-17T04:38:37.000Z (5 months ago)
Last Synced: 2024-12-10T04:27:02.448Z (24 days ago)
Topics: pipeline, pipelines
Language: Svelte
Homepage:
Size: 4.55 MB
Stars: 1
Watchers: 2
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

# pipen-board

Visualize configuration and running of [pipen][1] pipelines on the web.

![pipen-board](https://pwwang.github.io/immunopipe/pipen-board.gif)

## Installation

```bash
pip install pipen-board
```

## Usage

```bash
$ pipen board --help
Usage: pipen board [options] -- [pipeline options]

Configure and run pipen pipelines from the web

Required Arguments:
pipeline The pipeline and the CLI arguments to run the pipeline. For the
pipeline either `/path/to/pipeline.py:` or
`:` `` must be an instance of
`Pipen` and running the pipeline should be called under `__name__ ==
'__main__'.

Optional Arguments:
-h, --help show help message and exit
-p PORT, --port PORT Port to serve the UI wizard [default: 18521]
-a FILE, --additional FILE
Additional arguments for the pipeline, in YAML, INI, JSON or TOML
format. Can have sections `ADDITIONAL_OPTIONS` and `RUNNING_OPTIONS`.
It can also have other sections and items to override the
configurations generated from the pipeline. If the pipeline is
provided as a python script, such as
`/path/to/pipeline.py:`, and `` runs under
`__name__ == '__main__'`, the additional file can also be specified
as `auto` to generate a `RUNNING OPTIONS/Local` section to run the
pipeline locally.
--loglevel {auto,debug,info,warning,error,critical}
The logging level. If `auto`, it will be set to `debug` if `--dev` is
set, otherwise `info`. [default: auto]
--dev Run the pipeline in development/debug mode. This will reload the
server when changes are made to this package and reload the pipeline
when page reloads for new configurations. Page cache is also disabled
in this mode.
-w WORKDIR, --workdir WORKDIR
The working directory of the pipeline. [default: .pipen]
-s SCHEMA_DIR, --schema-dir SCHEMA_DIR
The directory to store the configuration schemas. [default: ~/.pipen-
board]
```

## Describing arguments in docstring

### Docstring schema

```python
class ProcessOrProcessGroup:
"""Short summary

Long description
Long description

Args:
arg1 (): description
- subarg1 (): description
- subarg2 (): description
arg2 (): description

:

"""
```

The metadata can have multiple attributes, separated by semicolon (`;`). For example:

```
arg1 (action=ns;required): description
```

### Marks

You can mark a process using `pipen.utils.mark(=)` as a decorator to decorate a process. For example:

```python
from pipen import Proc
from pipen.utils import mark

@mark(board_config_no_input=True)
class MyProc(Proc):
pass
```

Available marks:

- `board_config_no_input`: Whether to show the input section for the process in configuation page. Only affects the start processes. Default to `False`.
- `board_config_hidden`: Whether to hide the process options in the configuration page. Note that the process is still visible in the process list. Default to `False`.

### Metadata for arguments

| Name | Description | Allowed values |
| -------- | ----------- | -------------- |
| `action` | Like the `action` argument in [`argx`][2]*. | `store_true`, `store_false`, `ns`, `namespace`, `append`, `extend`, `clear_append`, `clear_extend` (other values are allowed but ignore, they may be effective for CLI use) |
| `btype` | Board type (option type specified directly). If specified, `action` will be ignored | `ns`, `choice`, `mchoice`, `array`, `list`, `json`, `int`, `float`, `bool`, `str`, `text`, `auto`* |
| `type` | Fallback for `action` and `btype` | Same as `btype` |
| `flag` | Fallback for `action=store_true` | No values needed |
| `text`/`mline`/`mlines` | Shortcut for `btype=text` | No values needed |
| `ns`/`namespace` | Shortcut for `btype=ns` | No values needed |
| `choices`/`choice` | Shortcut for `btype=choice` | No values needed |
| `mchoices`/`mchoice` | Shortcut for `btype=mchoice` | No values needed |
| `array`/`list` | Shortcut for `btype=array`/`btype=list` | No values needed |
| `choices`/`choice` | Shortcut for `btype=choice` | No values needed |
| `mchoices`/`mchoice` | Shortcut for `btype=mchoice` | No values needed |
| `order` | The order of the argument in the UI. | Any integer |
| `readonly` | Whether the argument is readonly. | No values needed (True if specified, otherwise False) |
| `required` | Whether the argument is required. | No values needed (True if specified, otherwise False) |
| `placeholder` | The placeholder in the UI for the argument. | Any string |
| `bitype` | The type of the elements in an array or list. | `int`, `float`, `bool`, `str`, `json`, `auto`* |
| `itype` | Fallback for `bitype` | Same as `bitype` |

- `argx*`: An argument parser for Python, compatible with `argparse`.
- `auto*`: Automatically infer the type from a string value.
- Any of `True`, `TRUE`, `true`, `False`, `FALSE`, `false` will be inferred as a `bool` value.
- Any of `None`, `NONE`, `none`, `null`, `NULL` will be inferred as `None`.
- Any integers will be inferred as `int`.
- Any floats will be inferred as `float`.
- Try to parse the value as JSON. If succeed, the value will be inferred as `json`.
- Otherwise, the value will be inferred as `str`.

### Types of options in the UI

The type of an option in the UI is determined by the `btype`, `action` or `type` metadata. If neither is specified, a `PlainText` will be used.

- `BoolOption`: Shown as a switch
- `TextOption`: Shown as a textarea (allow multiple lines)
- `ChoiceOption`: Shown as a dropdown list (`subarg1` and `subarg2` in the example above are used as the choices)
- `MChoiceOption`: Shown as a multiple choice list (`subarg1` and `subarg2` in the example above are used as the choices)
- `JsonOption`: Shown as a textarea, but the value will be validated and parsed as JSON
- `ArrayOption`: Shown as a tag input. Items can be added or removed.
- `AutoOption`: Shown as a 1-row textarea, and the value will be parsed automatically
- `PlainText`: Shown as a plain text. No validation or parsing will be performed.
- `MoreLikeOption`: Show as a box with buttons to add or remove sub-options. It's usally used together with `ns` type. If there is a sub-option under the option in the docstring wrapped by `<...>`, it indicates that we may have more sub-options.

[1]: https://github.com/pwwang/pipen
[2]: https://github.com/pwwang/argx