{"id":24479550,"url":"https://github.com/jetbrains-research/washu","last_synced_at":"2026-04-16T00:32:14.791Z","repository":{"id":91051408,"uuid":"58488696","full_name":"JetBrains-Research/washu","owner":"JetBrains-Research","description":"Reproducible and scalable technical pipelines for ChIP-Seq and RNA-Seq processing","archived":false,"fork":false,"pushed_at":"2023-03-13T10:13:22.000Z","size":166903,"stargazers_count":0,"open_issues_count":12,"forks_count":0,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-09-10T23:22:05.038Z","etag":null,"topics":["analysis","bash","chip-seq","parallel-computing","peak-calling","pipeline","qc","qsub","rna-seq","visualization"],"latest_commit_sha":null,"homepage":"https://artyomovlab.wustl.edu/aging","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JetBrains-Research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-05-10T19:39:14.000Z","updated_at":"2021-11-18T20:12:21.000Z","dependencies_parsed_at":"2025-01-21T10:39:09.309Z","dependency_job_id":null,"html_url":"https://github.com/JetBrains-Research/washu","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/JetBrains-Research/washu","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JetBrains-Research%2Fwashu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JetBrains-Research%2Fwashu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JetBrains-Research%2Fwashu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JetBrains-Research%2Fwashu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JetBrains-Research","download_url":"https://codeload.github.com/JetBrains-Research/washu/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JetBrains-Research%2Fwashu/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31866309,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"ssl_error","status_checked_at":"2026-04-15T15:24:39.138Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analysis","bash","chip-seq","parallel-computing","peak-calling","pipeline","qc","qsub","rna-seq","visualization"],"created_at":"2025-01-21T10:24:16.899Z","updated_at":"2026-04-16T00:32:14.767Z","avatar_url":"https://github.com/JetBrains-Research.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![JetBrains Research](https://jb.gg/badges/research.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)\nLicense [![license](https://img.shields.io/github/license/mashape/apistatus.svg)](https://opensource.org/licenses/MIT)\nTests [![tests](http://teamcity.jetbrains.com/app/rest/builds/buildType:(id:Biolabs_WashU_Tests)/statusIcon.svg)](http://teamcity.jetbrains.com/viewType.html?buildTypeId=Biolabs_WashU_Tests\u0026guest=1)\nChIPSeq Pipeline Tests [![tests](http://teamcity.jetbrains.com/app/rest/builds/buildType:(id:Biolabs_WashU_Pipeline_Tests)/statusIcon.svg)](http://teamcity.jetbrains.com/viewType.html?buildTypeId=Biolabs_WashU_Pipeline_Tests\u0026guest=1)\n\nDisclaimer\n==========\nThese pipelines use `qsub` and pure `bash` cpu level parallelism.\\\nPlease have a look at the updated [snakemake](https://snakemake.readthedocs.io/en/stable/#) pipeline [chipseq-smk-pipeline](https://github.com/JetBrains-Research/chipseq-smk-pipeline).\n\n\nPipelines\n=========\nScalable and reproducible technical pipelines for ChIP-Seq and RNA-Seq processing.\\\nParallel execution is supported with zero configuration on Portable Batch System (`qsub`) and local machines.\\\nReproducibility is guaranteed by automated testing of all the steps in Docker using Continuous Integration.\n\nChIP-Seq pipeline was used for [Epigenetic changes in aging human monocytes](http://artyomovlab.wustl.edu/aging/index.html) ChIP-Seq data analysis.\n\n* `pipeline_chipseq.py` - Pipeline for batch ChIP-Seq processing, including QC, alignment, peak calling\n* `pipeline_tf.py`      - Pipeline for batch Transcription Factor ChIP-Seq processing\n* `pipeline_rnaseq.py`  - Pipeline for batch RNA-Seq processing, including QC, alignment, quantification\n\nHow do I launch the ChIP-Seq pipeline?\n--------------------------------------\nFollow these instructions to launch ChIP-Seq pipeline:\n* Configure environment, see **Requirements** section\n* Place all the `.fastq` files to a single `\u003cFASTQ_FOLDER\u003e`\n* Create `\u003cINDEXES\u003e` folder to store all the indexes required\n* Launch the pipeline with desired `\u003cgenome\u003e`, e.g. `mm9` or `hg19` \n```bash\npython3 pipeline_chipseq.py \u003cFASTQ_FOLDER\u003e \u003cINDEXES\u003e \u003cgenome\u003e\n```\n\nHow do I launch the RNA-Seq pipeline?\n-------------------------------------\nFollow these instructions to launch RNA-Seq pipeline:\n* Configure environment, see **Requirements** section\n* Place all the `.fastq` files to a single `\u003cFASTQ_FOLDER\u003e`\n* Create `\u003cINDEXES\u003e` folder to store all the indexes required\n* Launch the pipeline with desired `\u003cgenome\u003e`, e.g. `mm9` or `hg19` \n```bash\npython3 pipeline_rnaseq.py \u003cFASTQ_FOLDER\u003e \u003cINDEXES\u003e \u003cgenome\u003e\n```\n\nRequirements\n------------\n* Ensure you have Python 3 installed as default interpreter\n* Add the following to `~/.bashrc` (Linux) or `~/.bash_profile` (MacOS):\n```bash\n# Configure project path\nexport WASHU_ROOT=\"\u003cPATH_TO_REPOSITORY\u003e\"\n\n# Configure correct python code execution\nexport PYTHONPATH=\"$WASHU_ROOT:$PYTHONPATH\"\n\n# Configure local machine parallelism\nexport WASHU_PARALLELISM=8\n```\n\n* Install required tools using [Conda](https://conda.io/docs/)\n```bash\nconda install --channel bioconda samtools bedtools bowtie bowtie2 fastqc multiqc sra-tools macs2 sicer \\\n    ucsc-bedgraphtobigwig ucsc-bedclip ucsc-bigwigaverageoverbed \\\n    star rseg \n```\nFor more details see `docker/biolabs/washu/Dockerfile`.\n * Download [Picard tools](https://github.com/broadinstitute/picard):\n```bash \ncurl --location https://github.com/broadinstitute/picard/releases/download/2.10.7/picard.jar \\\n    --output ~/picard.jar\n```\n* Download and extract [Phantompeakqualtools](https://github.com/kundajelab/phantompeakqualtools):\n```bash\ncurl --location https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/phantompeakqualtools/ccQualityControl.v.1.1.tar.gz \\\n    --output ~/phantompeakqualtools.tar.gz \ntar xvf ~/phantompeakqualtools.tar.gz\n```\n* Download [SPAN](https://github.com/JetBrains-Research/span):\n```bash\ncurl --location https://download.jetbrains.com/biolabs/span/span-1.1.5628.jar \\\n    --output ~/span.jar \n```\n\nProject structure\n-----------------\n* `/bed`            - BED files manipulations - intersection, ChromHMM enrichment, closes gene, etc.\n* `/docker`         - Docker configuration files with tools and test data. See Tests section.\n* `/parallel`       - Scripts for parallel execution of Portable Batch System (`qsub`) or on local machine. \\\nParallelism level on local machine can be configured via **WASHU_PARALLELISM** environment variable. \n* `/scripts`        - QC, Visualization, BAM conversions, Reads In Peaks, etc.\n* `/test`           - Tests for pipelines.\n\nTests\n-----\nExplore preconfigured Continuous Integration configurations on [TeamCity](https://www.jetbrains.com/teamcity/?fromMenu):\n* [ChIP-Seq Pipeline tests](http://teamcity.jetbrains.com/viewType.html?buildTypeId=Epigenome_Tools_WashuPipelineTests\u0026guest=1)   \n* [Other tests](http://teamcity.jetbrains.com/viewType.html?buildTypeId=Epigenome_Tools_Washu\u0026guest=1)\n\nFetch Docker image `biolabs/washu` with all the necessary tools for pipeline and test data.\n```bash\ndocker pull biolabs/washu\n```\nLaunch tests.\n```bash\n# Change working directory\ncd \u003cproject_path\u003e\n\n# General tests\ndocker run -v $(pwd):/washu -t -i biolabs/washu /bin/bash -c \\\n    \"source activate py3.5 \u0026\u0026 cd /washu \u0026\u0026 bash test.sh\"\n\n# ChIP-Seq Pipeline tests\ndocker run -v $(pwd):/washu -t -m 2G -e JAVA_OPTIONS=\"-Xmx1G\" -i biolabs/washu /bin/bash -c \\\n    \"source activate py3.5 \u0026\u0026 cd /washu \u0026\u0026 bash test_pipeline_chipseq.sh\"\n```\nExplore the results of ChIP-Seq pipeline in `out` folder after executing these tests. \n\nTools used\n---------- \n[Bedtools](https://bedtools.readthedocs.io/en/latest/), \n[Bowtie](http://bowtie-bio.sourceforge.net/index.shtml), \n[Bowtie2](http://bowtie-bio.sourceforge.net/bowtie2/index.shtml), \n[FastQC](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/),\n[MACS2](https://github.com/taoliu/MACS),\n[MANorm](https://www.ncbi.nlm.nih.gov/pubmed/22424423),\n[MultiQC](http://multiqc.info/),\n[Phantompeakqualtools](https://github.com/kundajelab/phantompeakqualtools),\n[Picardtools](https://github.com/broadinstitute/picard),\n[RSeg](https://academic.oup.com/bioinformatics/article/27/6/870/236489/Identifying-dispersed-epigenomic-domains-from-ChIP),\n[Samtools](http://samtools.sourceforge.net/),\n[SICER](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2732366/),\n[SPAN](https://github.com/JetBrains-Research/span)\n\n[STAR](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3530905/), \n[RSEM](https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-12-323)\n\nData standards and pipelines\n--------------\n* ENCODE data [standards](https://www.encodeproject.org/data-standards/)\n* ENCODE ChIP-Seq [pipeline](https://github.com/ENCODE-DCC/chip-seq-pipeline)\n* Blueprint ChIP-Seq [pipeline](http://dcc.blueprint-epigenome.eu/#/md/chip_seq_grch38)\n\nUseful links\n------------\n* JetBrains Research BioLabs [homepage](https://research.jetbrains.org/groups/biolabs)\n* Washington University in Saint Louis Maxim Artyomov LAB [homepage](https://artyomovlab.wustl.edu/site/)\n* Review on ChIP-Seq, ATAC-Seq and DNAse-Seq processing in latex [format](https://github.com/olegs/bioinformatics)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjetbrains-research%2Fwashu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjetbrains-research%2Fwashu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjetbrains-research%2Fwashu/lists"}