https://github.com/pwwang/pyppl_export

https://github.com/pwwang/pyppl_export

Last synced: 14 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/pwwang/pyppl_export
• Owner: pwwang
• License: mit
• Created: 2019-12-24T17:43:54.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2020-06-06T05:10:41.000Z (over 4 years ago)
• Last Synced: 2024-12-10T04:27:12.642Z (24 days ago)
• Language: Python
• Size: 39.1 KB
• Stars: 1
• Watchers: 3
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# pyppl_export

Export outputs generated by PyPPL pipelines:

1. export whole output to a directory

2. export partial output to a directory

3. resume process from export directory

```python

# export all output

PyPPL(config_export_dir = './export')

# export output with key outfile1

# pXXX.output = 'outfile1:file:..., outfile2:file:...'

PyPPL(config_export_dir = './export', config_export_part = 'outfile1')

# export output with .txt files only

PyPPL(config_export_dir = './export', config_export_part = '*.txt')

# export output with output key "outfile"

PyPPL(config_export_dir = './export', config_export_part = 'outfile')

# Use proc and job data to select

PyPPL(config_export_dir = './export', config_export_part = '*.{{args.ext}}')

# use export directory to make cache job cached

PyPPL(config_export_dir = './export', cache = 'export')

# copy the output to export directory

PyPPL(config_export_dir = './export', config_export_how = 'copy')

# move, but keep symblic links in the output directory

PyPPL(config_export_dir = './export', config_export_how = 'move')

# gzip single file and tar-gzip the directory

PyPPL(config_export_dir = './export', config_export_how = 'gz')

# Just create symblic links in export directory

PyPPL(config_export_dir = './export', config_export_how = 'link')

# Don't overwrite existing files

PyPPL(config_export_dir = './export', config_export_ow = False)

```