{"id":50499965,"url":"https://github.com/dienerlab/pipelines","last_synced_at":"2026-06-02T11:00:32.545Z","repository":{"id":225657580,"uuid":"766502048","full_name":"dienerlab/pipelines","owner":"dienerlab","description":"A monorepo containing all of our Nextflow pipelines.","archived":false,"fork":false,"pushed_at":"2026-06-01T08:37:00.000Z","size":1260,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-06-01T10:21:54.262Z","etag":null,"topics":["metagenomics","metatranscriptomics","microbiome","nextflow"],"latest_commit_sha":null,"homepage":"https://dienerlab.github.io/pipelines","language":"Nextflow","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dienerlab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-03-03T12:55:25.000Z","updated_at":"2026-06-01T08:36:46.000Z","dependencies_parsed_at":"2024-07-22T14:44:59.019Z","dependency_job_id":"7644ff67-9f34-4721-82f9-fa4a8c6a9d39","html_url":"https://github.com/dienerlab/pipelines","commit_stats":null,"previous_names":["dienerlab/pipelines"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dienerlab/pipelines","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dienerlab%2Fpipelines","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dienerlab%2Fpipelines/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dienerlab%2Fpipelines/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dienerlab%2Fpipelines/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dienerlab","download_url":"https://codeload.github.com/dienerlab/pipelines/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dienerlab%2Fpipelines/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33818568,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-02T02:00:07.132Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["metagenomics","metatranscriptomics","microbiome","nextflow"],"created_at":"2026-06-02T11:00:19.712Z","updated_at":"2026-06-02T11:00:32.419Z","avatar_url":"https://github.com/dienerlab.png","language":"Nextflow","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![checks](https://github.com/dienerlab/pipelines/actions/workflows/lint.yml/badge.svg)](https://github.com/dienerlab/pipelines/actions/workflows/lint.yml)\n\n[![docs](https://github.com/dienerlab/pipelines/actions/workflows/docs.yml/badge.svg)](https://dienerlab.github.io/pipelines)\n\n# :hammer: :triangular_ruler: Pipelines\n\nThis repo contains various analysis pipelines for the lab. Here are the basic\nrules:\n\n- each folder includes pipelines for a particular analysis - data type combination\n- pipelines are [nextflow](https://www.nextflow.io/) workflows\n- each pipeline comes with a list of conda environment files that manage the required software\n\n## Data layout\n\nPipelines will usually operate from a top level project\ndirectory structured in the following way:\n\n```\n[project root]\n├─ [pipeline].nf\n├─ data\n│  ├─ raw\n│  │  ├─ sample1_R1.fastq.gz\n│  │  ├─ sample1_R2.fastq.gz\n│  │  └─ ...\n│  ├─ figures\n│  ├─ fig1.png\n│  │  └─ ...\n│  └─ ...\n└─ refs\n   ├─ eggnog\n   └─ kraken2\n```\n\nThe initial raw data lives in `data/raw` and all analysis artifacts should\nbe written into `data/` as well. Figures go into `figures/`.\n\n## Setup\n\nThe first step is to copy or symlink the pipeline files into the top project\ndirectory. After that you can set up a conda environment that includes all software\nfor the pipeline (please see individual pipelines for variations on that).\n\n```bash\nconda env create -f conda.yml\n```\n\nEither activate the environment (usualy named after the pipeline):\n\n```bash\nconda activate metagenomics\n```\n\nor run the pipeline with the `-with-conda /my/envs/metagenomics` option (required for HPC).\n\n## Nextflow Configuration\n\nYou may also create a [nextflow config](https://www.nextflow.io/docs/latest/config.html) either in the project\ndirectory as `nextflow.config` or in your user HOME as `~/.nextflow/config`. A template config is\n[included in this repo](nextflow.config). If you are a lab member please use the [optimized\nversion from the wiki](https://github.com/dienerlab/internal/wiki/configs).\n\nTo install it as a global configuration:\n\nOn the server run the following to create the config directory\n\n```bash\nmkdir ~/.nextflow\n```\n\nAfter that edit and copy the config:\n\n```bash\ncp /path/to/pipelines/nextlow.config ~/.nextflow/config\n```\n\nAdd in your token if you want to use [Nextflow Tower](https://tower.nf) to track your pipeline.\n\nFor slurm substitute the partition name `default` with the SLURM partition.\n\n## Run the pipeline\n\nAfter setup you can test the pipeline with\n\n```bash\nnextflow run [WORKFLOW].nf -profile local -resume\n```\n\nBy default this will use all available 12 CPUs and 128 GB RAM unless specified otherwise in the personal [netxflow config](https://www.nextflow.io/docs/latest/config.html#scope-executor).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdienerlab%2Fpipelines","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdienerlab%2Fpipelines","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdienerlab%2Fpipelines/lists"}