Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/flow-r/flowr
Robust and efficient workflows using a simple language agnostic approach
https://github.com/flow-r/flowr
bioinformatics cloud cluster cran flowr pipeline tsv workflow
Last synced: 2 months ago
JSON representation
Robust and efficient workflows using a simple language agnostic approach
- Host: GitHub
- URL: https://github.com/flow-r/flowr
- Owner: flow-r
- License: other
- Created: 2014-05-01T19:20:29.000Z (over 10 years ago)
- Default Branch: master
- Last Pushed: 2021-03-10T15:43:53.000Z (over 3 years ago)
- Last Synced: 2024-05-13T23:20:27.225Z (5 months ago)
- Topics: bioinformatics, cloud, cluster, cran, flowr, pipeline, tsv, workflow
- Language: R
- Homepage: http://flow-r.github.io/flowr
- Size: 31.2 MB
- Stars: 83
- Watchers: 10
- Forks: 10
- Open Issues: 6
-
Metadata Files:
- Readme: README.Rmd
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- jimsghstars - flow-r/flowr - Robust and efficient workflows using a simple language agnostic approach (R)
README
---
output:
html_document:
keep_md: yes
packagedocs:
toc: false
navpills: |
---
[](https://travis-ci.org/sahilseth/flowr)
[](https://cran.r-project.org/package=flowr)
## [ Streamlining Computing workflows](http://flow-r.github.io/flowr/)
**Latest documentation: [flow-r.github.io/flowr](http://flow-r.github.io/flowr/)**
Flowr framework allows you to design and implement complex pipelines, and
deploy them on your institution's computing cluster. This has been built
keeping in mind the needs of bioinformatics workflows. However, it is
easily extendable to any field where a series of steps (shell commands)
are to be executed in a (work)flow.
### Highlights
- No new **syntax or language**. Put all shell commands as a tsv file called [flow mat](http://flow-r.github.io/flowr/overview.html#flow_matrix).
- Define the [flow of steps](http://flow-r.github.io/flowr/overview.html#relationships) using a simple tsv file (serial, scatter, gather, burst...) called [flow def](http://flow-r.github.io/flowr/overview.html#flow_definition).
- Works on your laptop/server or cluster (/cloud).
- Supports **multiple cluster computing platforms** (torque, lsf, sge, slurm ...), cloud (star cluster) OR a local machine.
- One line installation (`install.packages("flowr")`)
- **Reproducible** and **transparent**, with cleanly structured execution logs
- **Track** and **re-run** flows
- **Lean** and **Portable**, with easy installation
- **Fine grain** control over resources (CPU, memory, walltime, etc.) of each step.
### Example
[](https://github.com/flow-r/fastq_bam)
### A few lines, to get started
``` {r, eval=FALSE}
## Official stable release from CRAN (updated every other month)
## visit flow-r.github.io/flowr/install for more details
install.packages("flowr", repos = "http://cran.rstudio.com")
# or a latest version from DRAT, provide cran for dependencies
install.packages("flowr", repos = c(CRAN="http://cran.rstudio.com", DRAT="http://sahilseth.github.io/drat"))
library(flowr) ## load the library
setup() ## copy flowr bash script; and create a folder flowr under home.
# Run an example pipeline
# style 1: sleep_pipe() function creates system cmds
flowr run x=sleep_pipe platform=local execute=TRUE
# style 2: we start with a tsv of system cmds
# get example files
wget --no-check-certificate http://raw.githubusercontent.com/sahilseth/flowr/master/inst/pipelines/sleep_pipe.tsv
wget --no-check-certificate http://raw.githubusercontent.com/sahilseth/flowr/master/inst/pipelines/sleep_pipe.def
# submit to local machine
flowr to_flow x=sleep_pipe.tsv def=sleep_pipe.def platform=local execute=TRUE
# submit to local LSF cluster
flowr to_flow x=sleep_pipe.tsv def=sleep_pipe.def platform=lsf execute=TRUE
```
**Example pipelines** [inst/pipelines](https://github.com/flow-r/flowr/tree/master/inst/pipelines)
### Resources
- For a quick overview, you may browse through,
these [introductory slides](http://sahilseth.com/slides/flowrintro/).
- The [overview](http://flow-r.github.io/flowr/overview.html) provides additional details regarding
the ideas and concepts used in flowr
- We have a [tutorial](http://flow-r.github.io/flowr/tutorial.html) which can walk you through creating a
new pipeline
- Additionally, a subset of important functions are described in the [package reference](http://flow-r.github.io/flowr/rd.html)
page
- You may follow detailed instructions on [installing and configuring](http://flow-r.github.io/flowr/install.html)
- You can use flow creator: https://sseth.shinyapps.io/flow_creator), a shiny app to aid in
designing a *shiny* new flow. This provides a good example of the concepts
### Updates
This package is under active-development,
you may watch for changes using
the [watch link above](https://help.github.com/articles/watching-repositories/).
### Feedback
Please feel free to raise a [github issue](https://github.com/flow-r/flowr/issues) with questions and comments.
### Acknowledgements
- Jianhua Zhang
- Samir Amin
- Roger Moye
- Kadir Akdemir
- Ethan Mao
- Henry Song
- An excellent resource for writing your own R packages:
[r-pkgs.org](https://r-pkgs.org/)