Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mskcc/pluto
Utilities for pluto-cwl and helix_filters repos
https://github.com/mskcc/pluto
Last synced: about 1 month ago
JSON representation
Utilities for pluto-cwl and helix_filters repos
- Host: GitHub
- URL: https://github.com/mskcc/pluto
- Owner: mskcc
- License: gpl-3.0
- Created: 2021-02-04T14:59:13.000Z (almost 4 years ago)
- Default Branch: master
- Last Pushed: 2023-01-20T16:25:56.000Z (almost 2 years ago)
- Last Synced: 2024-05-12T00:46:17.869Z (7 months ago)
- Language: Python
- Size: 251 KB
- Stars: 0
- Watchers: 4
- Forks: 1
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pluto
[](https://pluto-util.readthedocs.io/en/latest/?badge=latest)
**P**ost-processing & **L**ightweight **U**pdates **T**o pipeline **O**utput
Utilities module for [pluto-cwl](https://github.com/mskcc/pluto-cwl) and [helix_filters_01](https://github.com/mskcc/helix_filters_01) repos
This repository contains object classes to make it easier to run CWL files, and to run test cases against CWL files.
Repo contents include:
#### Python Libraries
`tools.py`
- `CWLFile`: convenience class to map a CWL file by basename back to its full object path in relation to the `cwl` dir
- `CWLRunner`: helper class for running a CWL file from the command line via `cwltool` or Toil
- `TableReader`, `MafReader`: class for reading data from tabular files
- `MafWriter`: class for writing records to a tabular .maf format file
- `PlutoTestCase`: the most important class in the `pluto` module, this class bundles a large variety of helper methods and functionality into a standard Python `unittest.TestCase` object to reduce the amount of code needed when writing CWL test cases, and to provide an easy standard interface to control CWL test case execution parameters. Uses environment variables inherited from `settings.py` to control how tests are executed. Also includes a large number of custom test case 'assertion' methods in order to easily validate CWL workflow outputs. A noteworthy method is `assertCWLDictEqual` which performs a `cwltool`/Toil-agnostic comparison of CWL JSON dict's to validate workflow output.
`settings.py`
- contains a large amount of environment variables that can be supplied on the command line to control how CWL and test cases are executed
`serializer.py`
- `OFile`, `ODir`: special subclasses of the base Python `dict` object which are used to simulate the standard CWL JSON dict record format of workflow output item descriptions. Use these classes to create objects that expand out to a full CWL JSON, in order to use with methods such as `assertDictEquals`. The class signature is designed for succint expression of a large dynamically generated dictionary structure, and can cleanly reduce a standard CWL JSON representation in-code from ~8 lines down to 1-2 lines.
- `serializer.py` can also be run as a script on the saved output.json from a CWL workflow in order to convert a workflow output JSON into its Python representation using `OFile` and `ODir`, to allow for quickly creating new fixtures from existing JSON outputs.
#### Scripts
- `Makefile`: contains the install recipe for the `pluto` repo, and testing recipe for `pluto` itself
- `run-cwltool.sh`: convenience wrapper script for running a CWL with `cwltool`
- `run-toil.sh`: convenience wrapper script for running a CWL with Toil (this is the one you probably should be using)
- `env.juno.sh`: environment initialization script for use on MSKCC's Juno HPC cluster
- `scripts/`: more handy scripts for tedious tasks