https://github.com/gluonfield/nf-pipeline-test

https://github.com/gluonfield/nf-pipeline-test

Last synced: 4 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/gluonfield/nf-pipeline-test
• Owner: gluonfield
• Created: 2024-06-25T15:33:34.000Z (7 months ago)
• Default Branch: main
• Last Pushed: 2024-06-25T22:51:12.000Z (6 months ago)
• Last Synced: 2025-01-01T10:38:37.126Z (6 days ago)
• Language: Shell
• Size: 32.2 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Testing nf-float staging speed

## Goal

Figure out if there's a bottleneck staging many large files.

## Instructions

1. Specify list of files to stage in `nextflow_command`.

2. Adjust `outdir` and `env BUCKET`

3. Run the workflow

```

./float submit --hostInit transient_JFS_AWS.sh \

-i docker.io/memverge/juiceflow \

--vmPolicy '[onDemand=true]' \

--migratePolicy '[disable=true]' \

--dataVolume '[size=60]:/mnt/jfs_cache' \

--dirMap /mnt/jfs:/mnt/jfs \

-c 2 -m 4 \

-n staging-testing-speed-1 \

--securityGroup sg-0e3a2750bdf58794c \

--env BUCKET=https://cfdx-juicefs.s3.us-east-1.amazonaws.com \

-j job_submit_AWS.sh

```

## Results