{"id":15009043,"url":"https://github.com/khuang28jhu/bs3","last_synced_at":"2025-04-09T16:34:28.091Z","repository":{"id":201272532,"uuid":"65881430","full_name":"khuang28jhu/bs3","owner":"khuang28jhu","description":"BS-Seeker3: An Ultra-fast, Versatile Pipeline for Mapping Bisulfite-treated Reads.","archived":false,"fork":false,"pushed_at":"2019-07-05T17:38:00.000Z","size":40191,"stargazers_count":27,"open_issues_count":33,"forks_count":13,"subscribers_count":3,"default_branch":"dev1","last_synced_at":"2025-03-23T18:51:45.427Z","etag":null,"topics":["alignment","bisulfite","methylation-rate-calculation","ngstools","python-2"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/khuang28jhu.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-08-17T06:22:06.000Z","updated_at":"2024-12-05T10:38:10.000Z","dependencies_parsed_at":"2024-05-10T20:45:50.808Z","dependency_job_id":null,"html_url":"https://github.com/khuang28jhu/bs3","commit_stats":null,"previous_names":["khuang28jhu/bs3"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khuang28jhu%2Fbs3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khuang28jhu%2Fbs3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khuang28jhu%2Fbs3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/khuang28jhu%2Fbs3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/khuang28jhu","download_url":"https://codeload.github.com/khuang28jhu/bs3/tar.gz/refs/heads/dev1","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248068140,"owners_count":21042432,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alignment","bisulfite","methylation-rate-calculation","ngstools","python-2"],"created_at":"2024-09-24T19:22:35.454Z","updated_at":"2025-04-09T16:34:28.074Z","avatar_url":"https://github.com/khuang28jhu.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BS-Seeker3 \nBS-Seeker3 maps bisulfite-treated reads (bs-seq) with high accuracy and ultra-fast speed. While being 1.5 time faster than BSMAP and 10 times faster than Bismark, BS-Seeker3 can map twice as many reads than both aligners. In addition to its high-throughput performance, BS-Seeker3 offers additional downstream analysis to further investigate and visualize methylation pattern post-alignment.\n\n# \u003ca name=\"NewFeatures\"\u003e\u003c/a\u003eNew Features\n* BS-seeker3 now employs an improved index, conducts fast alignment with SNAP, and incorporates a highly optimized pipeline to process SNAP outputs. \n* BS-seeker3 now executes local alignment through the Unnoken Algorithm, which allows high mappability and accuracy without sacrificing too much runtime. \n* BS-seeke3 also outputs a preliminary quality control graph, a meta-gene plot, and a bisulfite unconversion rate histogram. Additional downstream methylation analysis is supported by MethGo. \n\n# BS-Seeker3 Pipeline\n\n![overview](https://github.com/khuang28jhu/bs3/blob/master/Overview.png)\n\n### Annoucement\n\nI am aware and fixing a file-directory problem that's preventing the code to be run in a more flexible manner. Also improving the RRBS segment and problem with matching short reads (length \u003c 50). MS1 obgligations are making this go slowly but it's getting there.\n\n# Table of Contents\n\n* ## [New Features](#NewFeatures)\n* ## [System Requirements](#SystemRequirements)\n* ## [BS-Seeker3 Usage](#RunningBS-Seeker3)\n   * ### [Download BS-Seeker3](#DownloadBS-Seeker3)\n   * ### [Index Buidling](#IndexBuidling)\n   * ### [Alignment](#Alignment)\n   * ### [Methylation Rate Calculation](#MethylationRateCalculation)\n   * ### [Methylation Rate Statistics Display](#MethylationRateStatisticsDisplay)\n* ## [Example Use Case](#ExampleUseCase)\n* ## [Linking with MethGo](#Linkingw/MethGo)\n\n# \u003ca name=\"SystemRequirements\"\u003e\u003c/a\u003eSystem Requirements\n* Linux or Mac OS Environment\n* Python2 (version 2.5.2 or above; it should be pre-installed in both Linux and Mac). Type 'Python' to see the installed version. Python2 could be downloaded from http://www.python.org/download/ )\n* GCC 5.4.0 +\n* SNAP version 1.0beta.23, which could be downloaded from https://github.com/amplab/snap\n* Python Modules 'Pysam' and 'Metplotlib'. To install the packages, use the following commands on an UNIX terminal:\n```\npip install pysam\n``` \n```\npip install Matplolib\n```\n\n# \u003ca name=\"RunningBS-Seeker3\"\u003e\u003c/a\u003eRunning BS-Seeker3\nBS-Seeker3 is a 3-steps pipeline: index-building, bs-seq alignment, and methylation rate calculation. Prior to alignment, BS3 first builds a custom-index for the reference genome (the user should adjust specific index-building parameters based on the reference genome size, see below for details). During alignment, BS3 uses SNAP to map bisulfite reads, and then sorts through the non-unique and incorrectly converted mappings. After alignment, methylation rate is then calcualted at the single-base resolution.\n\n### \u003ca name=\"DownloadBS-Seeker3\"\u003e\u003c/a\u003eDownload BS-Seeker3\nType the following commands in an Unix Terminal:\n* To download the Mac verion:\n```\nStay tuned.\n```\n* To download the Linux version:\n```\ngit clone https://github.com/khuang28jhu/bs3\n\n```\n\n### \u003ca name=\"IndexBuilding\"\u003e\u003c/a\u003eIndex Buidling\nUse the script **bs3-build.py** to build an index for a reference genome. \n\n**Usage:**\n```\n$ ./bs3-build -h \nUsage: ./bs3-build -h [options]\n\n-f                   Path to the reference genome; the reference genome should be in fasta format\n\n-s                   Seed size (default: 20), a SNAP option; SNAP uses a hashtable strucutre. \n                     It builds the index by breaking the reference genome into multiple seqeunces\n                     (seed) of a set length. This option determines the length of each \n                     seqeunce (seed size), and SNAP can process seed sizes to 23. A seed size of\n                     20 is recommended for bisulfite reads of 100 bp long; a longer size should be\n                     used for raw reads of longer length. \n                     \n-L                   (default: 4), a SNAP option specific to the Linux implementation; This option \n                     determines the byte size to store the location of each seed along the \n                     reference genome. It ranges from 4 to 8 bytes. For larger genomes, a larger \n                     location size should be used; for example, to build an index based on the human \n                     genome, a location size of 5 bytes is recommended. \n```\n### \u003ca name=\"Alignment\"\u003e\u003c/a\u003eAlignment\nUse the script **bs3-align.py** to map the raw bisulfite reads. \n\n**Input:**\n* BS reads file in fastq\n```\n@SRR019072.2842 HWI-EAS365_1060:4:1:51:313 length=87\nTAATTAGATTTGTGTTATAGATTATTTGTAAAGAAAGTAATTATTAAAGGAAATGTTAGTTTTTATTTGATATATGATAAGAGAACG\n+SRR019072.2842 HWI-EAS365_1060:4:1:51:313 length=87\nBBBCC@)8ABA/\u003c2\u003eCB:=.:?BBABB1-:@74@B@?=@@ABB@B7@@5/98\u003c;)\u003c\u003e56:?\u003e:;A?A?A@\u003e=AABB@A\u003c3(@@=086\n```\n* BS reads file in fasta\n```\n\u003eread1\nTCCATTATACCGTAACCCAATACAAAAATTATTTAT\n\u003eread2\nTCTGTAGACGGGTCGAATGGGGAGTTCATAGGGGGG\n```\n**Usage:**\n```\n$ ./bs3-align -h \nUsage: ./bs3-align -h [options] \n\nFor single end reads:\n\n-i INFILE,           Input read file (FORMAT:  fasta, fastq). Ex: read.fa or read.fa.gz\n\nFor pair end reads:\n\n-1 FILE,             Input read file, mate 1 (FORMAT: fasta, fastq)\n\n-2 FILE,             Input read file, mate 2 (FORMAT: fasta, fastq)\n\nImportant General options:\n\n-K ALIGNMENT LENGTH, Neglect alignments with length below this value\n\n-g GENOME,           Name of the reference genome (should be the same as \"-f\" in bs3-build.py ) [ex.\n                     chr21_hg18.fa]\n\n-m NO_MISMATCHES,    Set the number(\u003e=1)/percentage([0, 1)) of mismatche in a read. Ex: 8 (allow 8 \n                     mismatches) or 0.08 (allow 8% mismatches) [Default: 12]\n                     \n-l INT,              Split the input file into smaller files based on this number. Each smaller file \n                     is processed in paralell. The result is then merged. [Default: 12800000]\n                     \n-o OUTFILE           The name of output file \n\nRelevant Aligner Options:\n\n--snap-h             MaxHits, (default: 250 on the Mac version, 300 on Linux) a SNAP option; There \n                     are often seeds matching to multiple locations in the genomes. Sorting throught all\n                     the putative hits is a time-consuming process. This option sets a threshold on the \n                     number of locations that a seed can match to. Seeds matching to locations more than \n                     this number are ignored during the alignment.\n                     \nMethylation Rate Statistics Display Option:\n\n--qcf=QC_F           Supply the length of the raw bisulfite reads to plot a quality control plot. A \n                     quality control plot tabulates the average rate of mismatche at each position \n                     on a raw read.\n```\n**Output:** \n* Alignment Summary in .stat file\n```\n\tBS-seeker3 Result\n\n\tFinal Alignment Report\n================================================\n\nNumber of reads in total: 20000\nNumber of unique-hits reads (before post-filtering): 9990.0\nNumber of reads mapped after post-filtering 9934.0\n\nAlignment Time: 1.14915108681secs\n\n\tFinal Cytosine Report\n================================================\n\nTotal Number of Cytosines: 348392.0\nTotal Number of Cs in CpG context: 45242.0\nTotal Number of Cs in CHG context: 49610.0\nTotal Number of Cs in CHH context: 253540.0\n\nRate of Methylation\n mCG  0.999%\n mCHG  0.999%\n mCHH  0.999%\n```\n* List of Aligned Reads in SAM Format ([SAM Fields Description](https://samtools.github.io/hts-specs/SAMv1.pdf))\n\n```\nSRR2058107.412129\t0\t10_w_c\t42386003\t1\t90M\t*\t0\t0\tTGGATTGGAAGGTAATTATTATTGAATGGAATTGAATGGAATTATTGAATGGATTTGAATGGAATAATTATTGAATGGAATTGAATGGAA\tIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII\tPG:Z:SNAP\tNM:i:3\tRG:Z:FASTQ\tPL:Z:Illumina\tPU:Z:pu\tLB:Z:lb\tSM:Z:sm\n```\n\n### \u003ca name= \"MethylationRateCalculation\" \u003e\u003c/a\u003e Methylation Rate Calculation\nUse the script **bs3-align.py** to map raw bisulfite reads.\n\n**Input:**\n* SAM file from the previous step\n\n**Usage:**\n```\n$ ./bs3-call_methylation -h \nUsage: ./bs3-call_methylation -h [options]\n\nOptions:\n\n-i INFILE,          Input alinged read file in SAM format; output from bs3-align.py\n\n-d DBPATH,          Path to the reference genome index (generated during index-buidling) (optional)\n\n-o OUTFILE,         The output prefix to create the CGmap, ATCGmap and wiggle files\n\n--sorted,           Specify when the input bam file is already sorted, the sorting step will be \n                    skipped [Default: False]\n```\n\n\u003ca name=\"Outputaa\"\u003e\u003c/a\u003e**Output:**\n\n- wig file\n\n    Sample:\n\n        variableStep chrom=chr1\n        3000419\t0.000000\n        3000423\t-0.2\n        3000440\t0.000000\n        3000588\t0.5\n        3000593\t-0.000000\n\n\n        Format descriptions:\n        WIG file format. Negative value for 2nd column indicate a Cytosine on minus strand.\n\n\n- CGmap file\n\n    Sample:\n\n        chr1\tG\t3000851\tCHH\tCC\t0.1\t1\t10\n        chr1\tC\t3001624\tCHG\tCA\t0.0\t0\t9\n        chr1\tC\t3001631\tCG\tCG\t1.0\t5\t5\n\n    Format descriptions:\n\n        (1) chromosome\n        (2) nucleotide on Watson (+) strand\n        (3) position\n        (4) context (CG/CHG/CHH)\n        (5) dinucleotide-context (CA/CC/CG/CT)\n        (6) methylation-level = #_of_C / (#_of_C + #_of_T).\n        (7) #_of_C (methylated C, the count of reads showing C here)\n        (8) = #_of_C + #_of_T (all Cytosines, the count of reads showing C or T here)\n\n\n- ATCGmap file\n\n    Sample:\n\n        chr1\tT\t3009410\t--\t--\t0\t10\t0\t0\t0\t0\t0\t0\t0\t0\tna\n        chr1\tC\t3009411\tCHH\tCC\t0\t10\t0\t0\t0\t0\t0\t0\t0\t0\t0.0\n        chr1\tC\t3009412\tCHG\tCC\t0\t10\t0\t0\t0\t0\t0\t0\t0\t0\t0.0\n        chr1\tC\t3009413\tCG\tCG\t0\t10\t50\t0\t0\t0\t0\t0\t0\t0\t0.83\n\n\n    Format descriptions:\n\n        (1) chromosome\n        (2) nucleotide on Watson (+) strand\n        (3) position\n        (4) context (CG/CHG/CHH)\n        (5) dinucleotide-context (CA/CC/CG/CT)\n\n        (6) - (10) plus strand\n        (6) # of reads from Watson strand mapped here, support A on Watson strand\n        (7) # of reads from Watson strand mapped here, support T on Watson strand\n        (8) # of reads from Watson strand mapped here, support C on Watson strand\n        (9) # of reads from Watson strand mapped here, support G on Watson strand\n        (10) # of reads from Watson strand mapped here, support N\n\n        (11) - (15) minus strand\n        (11) # of reads from Crick strand mapped here, support A on Watson strand and T on Crick strand\n        (12) # of reads from Crick strand mapped here, support T on Watson strand and A on Crick strand\n        (13) # of reads from Crick strand mapped here, support C on Watson strand and G on Crick strand\n        (14) # of reads from Crick strand mapped here, support G on Watson strand and C on Crick strand\n        (15) # of reads from Crick strand mapped here, support N\n\n        (16) methylation_level = #C/(#C+#T) = C8/(C7+C8) for Watson strand, =C14/(C11+C14) for Crick strand;\n        \"nan\" means none reads support C/T at this position.\n\n**Convert Output to Bismark BedGraph Format**\n```\nsh toBismarkBedgraph.sh {output}.CGmap.gz\n```\n\n### \u003ca name=\"MethylationRateStatisticsDisplay\"\u003e\u003c/a\u003eMethylation Rate Statistics Display\nUse the script **bs3-methyl_display.py** to plot a meta-gene plot or a quality control plot.\u003cbr / \u003e\u003cbr / \u003e \n\n**Input:**\n* 'CGmap' file from the 'Methylation Rate Calculation' step\n* For a Metagene Plot based on a paritcular genomic structure (gene or transposon), the gene annotation file (in gff3); [Description of the fields in a gff3 file](http://gmod.org/wiki/GFF3#GFF3_Format)\n* For a QC Plot, the '.qc' file from the 'Alignment' step; \n\n**Usage:**\n```\n$ ./bs3-methyl_display -h \nUsage: ./bs3-methyl_display -h [options] \n\n-m MET             Supply the single-base resolution methylation level report from the \n                   methylation rate calculation step (in CG format)\n                   \n-a ANNOTATION      Suppply the gene annotation file to build a meta-plot (in gff3 format)\n\n-r GENOME_REGION   Select the genomeic region to plot for the meta-plot of gene. \n                   Select each with the option ```-r gene```; (default: gene)\n                   \n-q QC_F            Plot Quality Control Graph, supply the .qc file generated during the alignment \n                   step\n\n--meta=META        Plot metagene plot\n```\n**Output**\n* Example Meta-gene Plot\n![meta](https://github.com/khuang28jhu/bs3/blob/master/metaplot1.png)\n* Example Meta-gene Plot based on an Average Chromosomal View\n![meta](https://github.com/khuang28jhu/bs3/blob/master/metaplot2.png)\n* Example Quality Control Plot\n![qclot](https://github.com/khuang28jhu/bs3/blob/master/QC_Plot.png)\n\n\nUse the script **bs3-unconversion.py** to calculate the unconversion rate of the bisulfite reads if your data contains control reads from the lambda phage library. The lambda phage DNA is free of DNA methylation, so all cytosine of the genome should have been converted to uracil. The unconverted cytosines thus reveal the unconversionr rate.\n\u003cbr / \u003e\u003cbr / \u003e**Usage:**\n```\n$ ./bs3-unconversion -h\nUsage: ./bs3-unconversion -h [options]\n\n-f INPUT          The path to the raw bisulfite read file.\n\n-g GENOME         The path to the genome file.\n```\n**Output**\n* \u003ca name=\"Example\"\u003e\u003c/a\u003eExample Unconversion Rate Plot\n![unconversion](https://github.com/khuang28jhu/bs3/blob/master/Unconversion_Rate.png)\n\n# \u003ca name=\"ExampleUseCase\"\u003e\u003c/a\u003eExample Use Case\nPlease download SNAP version 1.0beta.23 from  https://github.com/amplab/snap, move ```snap-aligner``` to BSseeker3 home directory, and rename ```snap-aligner```  ```snap```.\n```\nmv snap-aligner snap\n```\n#### [Download BS-Seeker3](#DownloadBS-Seeker3)\n#### Build Indexes for the Reference Genome\n```\n./bs3-build -f reference_genome/genome.fa --aligner=snap\n```\n   This will build SNAP indexes in the directory bs_align/bs_utils/reference_genomes/genome.fa_snap\n#### Map the Sample Reads \n```\n./bs3-align -i test_data/WGBS.fa -o WGBS -f sam -g reference_genome/genome.fa -d reference_genome/\n```\nPaired-end reads:\n```\n./bs3-align -1 test_data/pair1.fq -2 test_data/pair2.fq -o WGBC -f sam -g reference_genome/genome.fa -d reference_genome/\n```\n   This will produce the output file ``` WGBS.sam ```, which contains the aligned reads in SAM format ([SAM Fields Description](https://samtools.github.io/hts-specs/SAMv1.pdf))\n#### Return Genome-wide Methylation Report for the Sample Reads \n```\n./bs3-call_methylation -i WGBS -o output  --db reference_genome/genome.fa_snap/\n```\n   This will produce a genome-wide methylation report of the data, ```output.wig.gz```,```output.ATCGmap.gz``` and ```output.CGmap.gz```; Description of the file formats is [here](#Outputaa).\n####  Plot QC Plot and Metagene Graph for the Sample Reads\n```\n./bs3-methyl_display --meta y -m output.CGmap.gz\n```\nThis returns an average chromosomal distribution of the methylation level for the reads (the annotation file is not supplied ).\n```\n./bs3-align -i test_data/WGBS.fa -o WGBS -f sam -g reference_genome/genome.fa --qcf 100\n```\n```\n./bs3-methyl_display -q WGBS.qc\n```\nThis returns a quality contol plot of the reads based on the number of mismatches per read position.\n#### Calculate the Unconversion Rate of the Data \n```\n./bs3-unconversion -f test_data/WGBS.fa -g reference_genome/lamdba.fa\n```\n   This will map the sample reads against the lamda phage library and output the [graph](#Example)```Unconversion_Rate.png``` summarizing the unconversion rate of the data.\n# \u003ca name=\"Linkingw/MethGo\"\u003e\u003c/a\u003eMethGo\n\nMethGo is a simple and effective tool designed to analyze data from whole genome bisulfite sequencing (WGBS) and reduced representation bisulfite sequencing (RRBS). MethGo provides 5 major modules:\n\nCOV: Coverage distribution of each cytosine\n\nMET: Both global and gene-centric cytosince methylation levels\n\nTXN: Cytosine methylation levels at transcription factor binding sites (TFBSs)\n\nSNP: Single nucleotide polymorphism (SNP) calling\n\nCNV: Copy number variation calling\n\nFor a complete introduction of Methgo and to download of its dependcies, please go to here: [MethGo Tutorial] (https://methgo.readthedocs.io/en/latest/)\n\nPlease use toMethgo.py to transition to MethGo. toMethgo.py takes in and delivers to the MethGo pipeline the .sam ouput from the [Alignment](#Alignment) stage and the .CGmap file from the [Methylation Rate Calculation](#MethylationRateCalculation) stage.\n\nSee the instructions below to perform the relevant MethGo modules: \n\n```\n$ python toMethGo.py -h \nUsage: python toMethGo.py [module tag] y --[relevant input tags] [input filenames]\n\nex: python toMethGo.py -CNV y --cnv ex.refindex -i ex.sam\n\n  -i INFILE,      --input=INFILE   SAM output from bs_seeker3-align\n  \n  -m MET                            Single-based-resolution methylation level file (CG format)\n  \n  -g FILE,        --genome=FILE     Genome File Name\n  \n  --gtf=FILE,     --gtf=FILE        Gene Annotation File Name\n  \n  --txn=FILE,     --txn=FILE        Txn Labels File Name\n  \n  --bind=FILE,    --bind=FILE       Motif Binding Site File Name\n  \n  --cnv=FILE,     --cnv=FILE        Input rference genome index file\n  \n  \n  --MET=METM            To perform MET module of MethGo\n  \n  --SNP=SNPM            To perform SNP module of MethGo\n  \n  --TXN=TXNM            To perform TXN module of MethGo\n  \n  --COV=COVM            To perform COV module of MethGo\n  \n  --CNV=CNVM            To perform CNV module of MethGo\n \n```\n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhuang28jhu%2Fbs3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkhuang28jhu%2Fbs3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhuang28jhu%2Fbs3/lists"}