Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/pwwang/pipen-runinfo
Generate running information for jobs in pipen pipelines.
https://github.com/pwwang/pipen-runinfo
pipeline runinfo sessioninfo
Last synced: 14 days ago
JSON representation
Generate running information for jobs in pipen pipelines.
- Host: GitHub
- URL: https://github.com/pwwang/pipen-runinfo
- Owner: pwwang
- Created: 2023-06-09T21:52:11.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2024-08-14T19:26:38.000Z (5 months ago)
- Last Synced: 2024-12-10T04:27:02.258Z (24 days ago)
- Topics: pipeline, runinfo, sessioninfo
- Language: Python
- Homepage:
- Size: 256 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# pipen-runinfo
Generate running information for jobs in [pipen][1] pipelines.
Including session info (packages and versions), time, and device info.
## Install
```bash
pip install -U pipen-runinfo
```
## Enable/Disable the plugin
The plugin is registered via entrypoints. It's by default enabled. To disable it: plugins=[..., "no:runinfo"], or uninstall this plugin.
## Plugin options
- `runinfo_lang`: The name of the language to run the job script for session info.
Default is `None`, which means it will be inferred from the `proc.lang`
This should be a process-level option, unless you only have one single
process in your pipeline.
- `runinfo_path`: Whether to include paths for the modules in the session information.
Default is `True`.
This option could be either specified in the process-level or the pipeline-level.
Only works for `python`.
- `runinfo_submod`: Whether to include submodules in the session information.
Default is `False`.
This option could be either specified in the process-level or the pipeline-level.
Only works for `python`.
## Supported languages for session info
`python`, `R`, `bash`, and `fish`.
## Usage
The plugin will generate 3 files in the job directory of the pipeline.
### `job.runinfo.session`
The session information of the job, including the interpreter, packages and their versions.
#### Python
Generates a TSV file with the following columns:
- `Name`: The name of the module, or python itself
- `__version__`: The version fetched by `module.__version__` or `module.version`
- `importlib.metadata`: The version fetched by `importlib.metadata.version(package)`
- `Path`: The path of the module (only if `runinfo_path` is `True`)
#### R
Generates a text file `sessionInfo()` output.
#### Bash
Generates a TSV file with the following columns:
- `SHELL`: The value of `$SHELL`
- `BASH_VERSION`: The value of `$BASH_VERSION`
- `BASH_ARGV0`: The value of `$BASH_ARGV0`
- `BASH_SOURCE`: The value of `$BASH_SOURCE`
- `proc-exe`: The real path of the executable from `/proc//exe`
- `proc-exe-version`: The version of the executable from `/proc//exe --version`
#### Fish
Generates a TSV file with the following columns:
- `SHELL`: The value of `$SHELL`
- `FISH_VERSION`: The value of `$FISH_VERSION`
- `proc-exe`: The real path of the executable from `/proc//exe`
- `proc-exe-version`: The version of the executable from `/proc//exe --version`
### `job.runinfo.time`
The time spent on the job, and more, generated by `time -v` command.
### `job.runinfo.device`
The device (cpu and memory) information of the job, generated by `lscpu`/`lsmem` command.
[1]: https://github.com/pwwang/pipen