https://github.com/giannisdoukas/ipython2cwl-demo
IPython2CWL Demo
https://github.com/giannisdoukas/ipython2cwl-demo
common-workflow-language cwl jupyter-notebook
Last synced: about 2 months ago
JSON representation
IPython2CWL Demo
- Host: GitHub
- URL: https://github.com/giannisdoukas/ipython2cwl-demo
- Owner: giannisdoukas
- Created: 2020-06-30T23:46:57.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2020-07-09T20:04:23.000Z (over 5 years ago)
- Last Synced: 2025-06-08T05:44:45.924Z (9 months ago)
- Topics: common-workflow-language, cwl, jupyter-notebook
- Language: Jupyter Notebook
- Homepage:
- Size: 116 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# IPython2CWL DEMO
This demo requires ipython2cwl
```sh
$ pip install ipython2cwl
```
This example is based on demo1.ipynb.
* You can clone the repo or,
* You can execute:
* jupyter-repo2cwl https://github.com/giannisdoukas/ipython2cwl-demo -o .
* That command will generate a docker image & a file named demo1.cwl in your current working directory
* You can still execute the Jupyter Notebook by executing: `$ docker run -p 127.0.0.1:8888:8888 THE_NAME_OF_THE_GENERATED_IMAGE`
* You can find the generated image id or in the logs produced by the jupyter-ipython2cwl or in the generated cwl file.
---
Users can use typing hints in string format or by importing the library.
```python
!echo "Hello World 1" > message1.txt
file_input1: 'CWLFilePathInput' = 'message1.txt'
with open(file_input1, 'r') as f:
data1 = f.read()
```
```python
!echo "Hello World 2" > message2.txt
from ipython2cwl.iotypes import CWLFilePathInput
file_input2: CWLFilePathInput = 'message2.txt'
with open(file_input2, 'r') as f:
data2 = f.read()
```
For each variable which has the typing hints the tool will remove the presented assigment and it will generify it and expose it as a command line argument.
Currently the supported type hints are:
* CWLFilePathInput
* CWLBooleanInput
* CWLStringInput
* CWLIntInput
* CWLFilePathOutput
These types can be also combined with List & Optional.
```python
import os
from typing import List
from ipython2cwl.iotypes import CWLStringInput
messages1: List[CWLStringInput] = ["message1:", "hello", "world", '!']
messages2: List['CWLStringInput'] = ["message2:", "hello", "world", "!!"]
messages3: 'List[CWLStringInput]' = ["message3:", "hello", "world", "!!!"]
messages4: 'Optional[CWLStringInput]' = "messages4"
print(' '.join(messages1))
print(' '.join(messages2))
print(' '.join(messages3))
if messages4 is None:
print('='*20)
print("Message 4 is None")
print('='*20)
else:
print(messages4)
```
message1: hello world !
message2: hello world !!
message3: hello world !!!
messages4
To store the results to a file we have to define a variable and give the hint CWLFilePathOutput.
```python
output: 'CWLFilePathOutput' = 'output.txt'
with open(output, 'w') as f:
f.write(' '.join(messages1))
f.write(os.linesep)
f.write('-'*10)
f.write(os.linesep)
f.write(' '.join(messages2))
f.write(os.linesep)
f.write('-'*10)
f.write(os.linesep)
f.write(' '.join(messages3))
```
```python
!cat output.txt
```
message1: hello world !
----------
message2: hello world !!
----------
message3: hello world !!!
The auto-generated CWL file will look like:
```python
# !cat ../out/demo1.cwl
```
arguments:
- --
baseCommand: /app/cwl/bin/demo1
class: CommandLineTool
cwlVersion: v1.1
hints:
DockerRequirement:
dockerImageId: r2d-2fvar-2ffolders-2fk8-2f800hfw-5fn2md-5f2zb44lhhtqqr0000gn-2ft-2frepo2cwl-5fixi144l4-2frepo1593555577
inputs:
file_input1:
inputBinding:
prefix: --file_input1
type: File
file_input2:
inputBinding:
prefix: --file_input2
type: File
messages1:
inputBinding:
prefix: --messages1
type: string[]
messages2:
inputBinding:
prefix: --messages2
type: string[]
messages3:
inputBinding:
prefix: --messages3
type: string[]
messages4:
inputBinding:
prefix: --messages4
type: string?
outputs:
output:
outputBinding:
glob: output.txt
type: File
and the auto generated script looks like:
```sh
$ docker run -it r2d-2fvar-2ffolders-2fk8-2f800hfw-5fn2md-5f2zb44lhhtqqr0000gn-2ft-2frepo2cwl-5fixi144l4-2frepo1593555577 cat /app/cwl/bin/demo1 # this command will display the auto-generated script to the terminal
```
```python
#!/usr/bin/env ipython
"""
DO NOT EDIT THIS FILE
THIS FILE IS AUTO-GENERATED BY THE ipython2cwl.
FOR MORE INFORMATION CHECK https://github.com/giannisdoukas/ipython2cwl
"""
def main(file_input1, file_input2, messages1, messages2, messages3, messages4):
get_ipython().system('echo "Hello World 1" > message1.txt')
with open(file_input1, 'r') as f:
data1 = f.read()
get_ipython().system('echo "Hello World 2" > message2.txt')
with open(file_input2, 'r') as f:
data2 = f.read()
import os
from typing import List
print(' '.join(messages1))
print(' '.join(messages2))
print(' '.join(messages3))
if messages4 is None:
print('=' * 20)
print('Message 4 is None')
print('=' * 20)
else:
print(messages4)
output = 'output.txt'
with open(output, 'w') as f:
f.write(' '.join(messages1))
f.write(os.linesep)
f.write('-' * 10)
f.write(os.linesep)
f.write(' '.join(messages2))
f.write(os.linesep)
f.write('-' * 10)
f.write(os.linesep)
f.write(' '.join(messages3))
get_ipython().system('cat output.txt')
if __name__ == '__main__':
import argparse
import pathlib
parser = argparse.ArgumentParser()
parser.add_argument('--file_input1', type=pathlib.Path, required=True)
parser.add_argument('--file_input2', type=pathlib.Path, required=True)
parser.add_argument('--messages1', type=str, required=True, nargs='+')
parser.add_argument('--messages2', type=str, required=True, nargs='+')
parser.add_argument('--messages3', type=str, required=True, nargs='+')
parser.add_argument('--messages4', type=str, required=False, default=None)
args = parser.parse_args()
main(file_input1=args.file_input1, file_input2=args.file_input2,
messages1=args.messages1, messages2=args.messages2, messages3=args.
messages3, messages4=args.messages4)
```
You can use try execute the generated workflow with the files in.yaml & in2.yaml
```sh
$ cwltool demo1.cwl in.yaml
$ cwltool demo1.cwl in1.yaml
```