Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/common-workflow-language/cwl-utils

Python utilities for CWL
https://github.com/common-workflow-language/cwl-utils

common-workflow-language commonwl cwl

Last synced: about 16 hours ago
JSON representation

Python utilities for CWL

Awesome Lists containing this project

README 

        
|Linux Build Status| |Code coverage| |Documentation Status|



.. |Linux Build Status| image:: https://github.com/common-workflow-language/cwl-utils/actions/workflows/ci-tests.yml/badge.svg?branch=main

   :target: https://github.com/common-workflow-language/cwl-utils/actions/workflows/ci-tests.yml

.. |Code coverage| image:: https://codecov.io/gh/common-workflow-language/cwl-utils/branch/main/graph/badge.svg

   :target: https://codecov.io/gh/common-workflow-language/cwl-utils

.. |Documentation Status| image:: https://readthedocs.org/projects/cwl-utils/badge/?version=latest

   :target: https://cwl-utils.readthedocs.io/en/latest/?badge=latest

   :alt: Documentation Status



cwl-utils

---------



Python Utilities and Autogenerated Classes for loading and parsing `CWL

v1.0 `__,

`CWL

v1.1 `__,

and `CWL

v1.2 `__

documents.



Requires Python 3.8+



Installation

------------



::



   pip3 install cwl-utils



To install from source::



   git clone https://github.com/common-workflow-language/cwl-utils.git

   cd cwl-utils

   pip3 install .



Usage

-----



Pull the all referenced software container images

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



``cwl-docker-extract`` is useful to cache or pre-pull all software

container images referenced in a CWL CommandLineTool or CWL Workflow

(including all referenced CommandLineTools and sub-Workflows and so on).



The default behaviour is to use the Docker engine to download and save

the software container images in Docker format.



.. code:: bash



   cwl-docker-extract path_to_my_workflow.cwl



.. code:: bash



   cwl-docker-extract --dir DIRECTORY path_to_my_workflow.cwl



Or you can use the Singularity software container engine to download and

save the software container images and convert them to the Singularity

format at the same time.



.. code:: bash



   cwl-docker-extract --singularity --dir DIRECTORY path_to_my_workflow.cwl



Print all referenced software packages

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



``cwl-cite-extract`` prints all software packages found (recursively) in the

specified CWL document.



Currently the package name and any listed specs and version field are printed

for all ``SoftwareRequirement`` s found.



.. code:: bash



   cwl-cite-extract path_to_my_workflow.cwl



Replace CWL Expressions with concrete steps

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



``cwl-expression-refactor`` refactors CWL documents so that any CWL Expression

evaluations are separate steps (either CWL ExpressionTools or CWL CommandLineTools.)

This allows execution by CWL engines that do not want to support inline expression

evaluation outside of concrete steps, or do not want to directly support CWL's

optional ``InlineJavascriptRequirement`` at all.



.. code:: bash



   cwl-expression-refactor directory/path/to/save/outputs path_to_my_workflow.cwl [more_workflows.cwl]



Split a packed CWL document

~~~~~~~~~~~~~~~~~~~~~~~~~~~



``cwl-graph-split`` splits a packed CWL document file into multiple files.



Packed CWL documents use the $graph construct to contain multiple CWL Process

objects (Workflow, CommandLineTool, ExpressionTool, Operation). Typically

packed CWL documents contain a CWL Workflow under the name "main" and the

workflow steps (including any sub-workflows).



.. code:: bash



   cwl-graph-split --outdir optional/directory/path/to/save/outputs path_to_my_workflow.cwl



Normalize a CWL document

~~~~~~~~~~~~~~~~~~~~~~~~



``cwl-normalizer`` normalizes one or more CWL document so that for each document,

a JSON format CWL document is produces with it and all of its dependencies packed

together, upgrading to CWL v1.2, as needed. Can optionally refactor CWL

Expressions into separate steps in the manner of cwl-expression-refactor.



.. code:: bash



   cwl-normalizer directory/path/to/save/outputs path_to_my_workflow.cwl [more_workflows.cwl]



Generate for Workflow Parameters from a CWL document

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



``cwl-inputs-schema-gen`` generates a JSON Schema for workflow input parameters from a CWL document.



.. code:: bash



   cwl-inputs-schema-gen path_to_my_workflow.cwl



Using the CWL Parsers

~~~~~~~~~~~~~~~~~~~~~



.. code:: python



   from pathlib import Path

   from ruamel import yaml

   import sys



   from cwl_utils.parser import load_document_by_uri, save



   # File Input - This is the only thing you will need to adjust or take in as an input to your function:

   cwl_file = Path("testdata/md5sum.cwl")  # or a plain string works as well



   # Import CWL Object

   cwl_obj = load_document_by_uri(cwl_file)



   # View CWL Object

   print("List of object attributes:\n{}".format("\n".join(map(str, dir(cwl_obj)))))



   # Export CWL Object into a built-in typed object

   saved_obj = save(cwl_obj)

   print(f"Export of the loaded CWL object: {saved_obj}.")



Development

-----------



Regenerate parsers

~~~~~~~~~~~~~~~~~~



To regenerate install the ``schema_salad`` package and run:



``cwl_utils/parser/cwl_v1_0.py`` was created via

``schema-salad-tool --codegen python https://github.com/common-workflow-language/common-workflow-language/raw/codegen/v1.0/extensions.yml --codegen-parser-info "org.w3id.cwl.v1_0" > cwl_utils/parser/cwl_v1_0.py``



``cwl_utils/parser/cwl_v1_1.py`` was created via

``schema-salad-tool --codegen python https://github.com/common-workflow-language/cwl-v1.1/raw/codegen/extensions.yml --codegen-parser-info "org.w3id.cwl.v1_1" > cwl_utils/parser/cwl_v1_1.py``



``cwl_utils/parser/cwl_v1_2.py`` was created via

``schema-salad-tool --codegen python https://github.com/common-workflow-language/cwl-v1.2/raw/codegen/extensions.yml --codegen-parser-info "org.w3id.cwl.v1_2" > cwl_utils/parser/cwl_v1_2.py``



Release

~~~~~~~



To release CWLUtils, bump the version in ``cwl_utils/__meta__.py``, and

tag that commit with the new version. The

`gh-action-pypi-publish `__

should release that tag.