{"id":29188109,"url":"https://github.com/pengni/deepsignal-plant","last_synced_at":"2025-07-01T22:09:07.094Z","repository":{"id":40664103,"uuid":"222630712","full_name":"PengNi/deepsignal-plant","owner":"PengNi","description":"Detecting methylation using signal-level features from Nanopore sequencing reads of plants","archived":false,"fork":false,"pushed_at":"2024-07-01T09:58:19.000Z","size":346,"stargazers_count":61,"open_issues_count":19,"forks_count":12,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-06-27T11:53:10.473Z","etag":null,"topics":["bioinformatics","methylation","nanopore-sequencing","plant","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PengNi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-11-19T07:04:12.000Z","updated_at":"2025-06-09T08:34:05.000Z","dependencies_parsed_at":"2023-01-21T05:39:42.741Z","dependency_job_id":"9d6fb6d1-810d-4fdf-9956-dd91860be34f","html_url":"https://github.com/PengNi/deepsignal-plant","commit_stats":{"total_commits":141,"total_committers":3,"mean_commits":47.0,"dds":"0.014184397163120588","last_synced_commit":"37430f368359ca6e9d9b0aa645d0cde2dc81ae69"},"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/PengNi/deepsignal-plant","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PengNi%2Fdeepsignal-plant","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PengNi%2Fdeepsignal-plant/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PengNi%2Fdeepsignal-plant/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PengNi%2Fdeepsignal-plant/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PengNi","download_url":"https://codeload.github.com/PengNi/deepsignal-plant/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PengNi%2Fdeepsignal-plant/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263042347,"owners_count":23404459,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","methylation","nanopore-sequencing","plant","pytorch"],"created_at":"2025-07-01T22:09:06.067Z","updated_at":"2025-07-01T22:09:07.051Z","avatar_url":"https://github.com/PengNi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DeepSignal-plant\n\n[![Python](https://img.shields.io/pypi/pyversions/deepsignal-plant)](https://www.python.org/)\n[![GitHub-License](https://img.shields.io/github/license/PengNi/deepsignal-plant)](https://github.com/PengNi/deepsignal-plant/blob/master/LICENSE)\n\n[![PyPI-version](https://img.shields.io/pypi/v/deepsignal-plant)](https://pypi.org/project/deepsignal-plant/)\n[![PyPI-Downloads](https://static.pepy.tech/badge/deepsignal-plant)](https://pepy.tech/project/deepsignal-plant)\n\u0026emsp;[![Conda-version](https://img.shields.io/conda/vn/bioconda/deepsignal-plant)](https://anaconda.org/bioconda/deepsignal-plant)\n[![Conda-Downloads](https://img.shields.io/conda/dn/bioconda/deepsignal-plant)](https://anaconda.org/bioconda/deepsignal-plant)\n\u003c!--[![PyPI-Downloads/m](https://pepy.tech/badge/deepsignal-plant/month)](https://pepy.tech/project/deepsignal-plant)--\u003e\n\n## A deep-learning method for detecting methylation state from Oxford Nanopore sequencing reads of plants.\ndeepsignal-plant applies BiLSTM to detect methylation from Nanopore reads. It is built on **Python3** and **PyTorch**.\n\n\n#### Known issues\n- [The VBZ compression issue] Please try adding ont-vbz-hdf-plugin to your environment as follows when all fast5s failed in `tombo resquiggle` and/or `deepsignal_plant call_mods`. Normally it will work after setting `HDF5_PLUGIN_PATH`:\n```shell\n# 1. install hdf5/hdf5-tools (maybe not necessary)\n# ubuntu\nsudo apt-get install libhdf5-serial-dev hdf5-tools\n# centos\nsudo yum install hdf5-devel\n\n# 2. download ont-vbz-hdf-plugin-1.0.1-Linux-x86_64.tar.gz (or newer version) and set HDF5_PLUGIN_PATH\n# https://github.com/nanoporetech/vbz_compression/releases\nwget https://github.com/nanoporetech/vbz_compression/releases/download/v1.0.1/ont-vbz-hdf-plugin-1.0.1-Linux-x86_64.tar.gz\ntar zxvf ont-vbz-hdf-plugin-1.0.1-Linux-x86_64.tar.gz\nexport HDF5_PLUGIN_PATH=/abslolute/path/to/ont-vbz-hdf-plugin-1.0.1-Linux/usr/local/hdf5/lib/plugin\n```\nReferences: [issue #8](https://github.com/PengNi/deepsignal-plant/issues/8), [tombo issue #254](https://github.com/nanoporetech/tombo/issues/254), and [vbz_compression issue #5](https://github.com/nanoporetech/vbz_compression/issues/5).\n\n\n## Contents\n- [Installation](#Installation)\n- [Trained models](#Trained-models)\n- [Example data](#Example-data)\n- [Quick start](#Quick-start)\n- [Usage](#Usage)\n\n## Installation\ndeepsignal-plant is built on [Python3](https://www.python.org/) and [PyTorch](https://pytorch.org/). [Guppy](https://nanoporetech.com/community) and [tombo](https://github.com/nanoporetech/tombo) are required to basecall and re-squiggle the raw signals from nanopore reads before running deepsignal-plant.\n   - Prerequisites: \\\n       [Python3.*](https://www.python.org/) (version\u003e=3.8)\\\n       [Guppy](https://nanoporetech.com/community) (version\u003e=3.6.1)\\\n       [tombo](https://github.com/nanoporetech/tombo) (version 1.5.1)\n   - Direct dependencies: \\\n       [numpy](http://www.numpy.org/) \\\n       [h5py](https://github.com/h5py/h5py) \\\n       [statsmodels](https://github.com/statsmodels/statsmodels/) \\\n       [scikit-learn](https://scikit-learn.org/stable/) \\\n       [PyTorch](https://pytorch.org/) (version \u003e=1.2.0, \u003c=1.11.0)\n   - Non-direct dependencies: \\\n       [scipy](https://scipy.org/) \\\n       [pandas](https://pandas.pydata.org/)\n\n#### Option 1. One-step installation\nInstall deepsignal-plant, its dependencies, and other required packages in one step using [conda](https://conda.io/docs/) and [environment.yml](environment.yml):\n\n```shell\n# download deepsignal-plant\ngit clone https://github.com/PengNi/deepsignal-plant.git\n\n# install tools in environment.yml\nconda env create --name deepsignalpenv -f /path/to/deepsignal-plant/environment.yml\n\n# then the environment can be activated to use\nconda activate deepsignalpenv\n```\n\n#### Option 2. Step-by-step installation\n##### (1) create an environment\nWe highly recommend using a virtual environment for the installation of deepsignal-plant and its dependencies. A virtual environment can be created and (de)activated as follows using [conda](https://conda.io/docs/):\n```bash\n# create\nconda create -n deepsignalpenv python=3.8\n\n# activate\nconda activate deepsignalpenv\n\n# deactivate\nconda deactivate\n```\nThe virtual environment can also be created using [virtualenv](https://github.com/pypa/virtualenv/).\n\n##### (2) Install deepsignal-plant\nAfter the environment being created and activated, deepsignal-plant can be installed using [conda](https://anaconda.org/bioconda/deepsignal-plant)/[pip](https://pypi.org/project/deepsignal-plant/), or from github directly:\n```bash\n# install using conda\nconda install -c bioconda deepsignal-plant\n\n# or install using pip\npip install deepsignal-plant\n\n# or install from github (latest version)\ngit clone https://github.com/PengNi/deepsignal-plant.git\ncd deepsignal-plant\npython setup.py install\n```\n\n##### (3) Re-install pytorch if needed\n[PyTorch](https://pytorch.org/) can be automatically installed during the installation of deepsignal-plant. However, if the version of [PyTorch](https://pytorch.org/) installed is not appropriate for your OS, an appropriate version should be re-installed in the same environment as the [instructions](https://pytorch.org/get-started/locally/):\n```bash\n# install using conda\nconda install pytorch==1.11.0 cudatoolkit=10.2 -c pytorch\n\n# or install using pip\npip install torch==1.11.0\n```\n\n##### (4) Install tombo\n[tombo (version 1.5.1)](https://github.com/nanoporetech/tombo) is required to be installed:\n```bash\n# install using pip\npip install ont-tombo\n\n# or install using conda\nconda install -c bioconda ont-tombo\n```\n\n**Note:**\n\nGuppy (version\u003e=3.6.1) is also required, which can be downloaded from [Nanopore Community (login required)](https://nanoporetech.com/community).\n\n\n## Trained models\n\nCurrently, we have trained the following models:\n   * _[model.dp2.CNN.arabnrice2-1_120m_R9.4plus_tem.bn13_sn16.both_bilstm.epoch6.ckpt](https://drive.google.com/file/d/1HnDKPEfCAXgo7vPN-zaD44Kqz1SDw160/view?usp=sharing)_: A 5mC model trained using _A. thaliana_ and _O. sativa_ R9.4 1D reads.\n\n\n## Example data\n\n   * _[fast5s.sample.tar.gz](https://drive.google.com/file/d/1PauSQH-3Wpi6FNjNycH9n3GSxkW8C3s0/view?usp=sharing)_: 4000 _A. thaliana_ R9.4 raw reads, with a genome reference.\n\n\n## Quick start\nTo call modifications, the raw fast5 files should be basecalled by [Guppy (version\u003e=3.6.1)](https://nanoporetech.com/community) and then be re-squiggled by [tombo (version 1.5.1)](https://github.com/nanoporetech/tombo). At last, modifications of specified motifs can be called by deepsignal. Belows are commands to call 5mC in CG, CHG, and CHH contexts:\n```bash\n# Download and unzip the example data and pre-trained models.\n# 1. guppy basecall using GPU\nguppy_basecaller -i fast5s/ -r -s fast5s_guppy \\\n  --config dna_r9.4.1_450bps_hac_prom.cfg \\\n  --device CUDA:0\n\n# 2. tombo resquiggle\ncat fast5s_guppy/*.fastq \u003e fast5s_guppy.fastq\ntombo preprocess annotate_raw_with_fastqs --fast5-basedir fast5s/ \\\n  --fastq-filenames fast5s_guppy.fastq \\\n  --sequencing-summary-filenames fast5s_guppy/sequencing_summary.txt \\\n  --basecall-group Basecall_1D_000 --basecall-subgroup BaseCalled_template \\\n  --overwrite --processes 10\ntombo resquiggle fast5s/ GCF_000001735.4_TAIR10.1_genomic.fna \\\n  --processes 10 --corrected-group RawGenomeCorrected_000 \\\n  --basecall-group Basecall_1D_000 --overwrite\n\n# 3. deepsignal-plant call_mods\n# 5mCs in all contexts (CG, CHG, and CHH) can be called at one time\nCUDA_VISIBLE_DEVICES=0 deepsignal_plant call_mods --input_path fast5s/ \\\n  --model_path model.dp2.CNN.arabnrice2-1_120m_R9.4plus_tem.bn13_sn16.both_bilstm.epoch6.ckpt \\\n  --result_file fast5s.C.call_mods.tsv \\\n  --corrected_group RawGenomeCorrected_000 \\\n  --motifs C --nproc 30 --nproc_gpu 6\ndeepsignal_plant call_freq --input_path fast5s.C.call_mods.tsv \\\n  --result_file fast5s.C.call_mods.frequency.tsv\n# split 5mC call_freq file into CG/CHG/CHH call_freq files\npython /path/to/deepsignal_plant/scripts/split_freq_file_by_5mC_motif.py \\\n  --freqfile fast5s.C.call_mods.frequency.tsv\n```\n\n\n## Usage\n#### 1. Basecall and re-squiggle\nBefore running deepsignal, the raw reads should be basecalled by [Guppy (version\u003e=3.6.1)](https://nanoporetech.com/community) and then be processed by the *re-squiggle* module of [tombo (version 1.5.1)](https://github.com/nanoporetech/tombo).\n\nNote:\n- If the fast5 files are in multi-read FAST5 format, please use _multi_to_single_fast5_ command from the [ont_fast5_api package](https://github.com/nanoporetech/ont_fast5_api) to convert the fast5 files before using [Guppy](https://nanoporetech.com/community) and [tombo](https://nanoporetech.com/community) (Ref to [issue #173](https://github.com/nanoporetech/tombo/issues/173) in [tombo](https://github.com/nanoporetech/tombo)).\n```bash\nmulti_to_single_fast5 -i $multi_read_fast5_dir -s $single_read_fast5_dir -t 30 --recursive\n```\n- If the basecall results are saved as fastq, run the [*tombo proprecess annotate_raw_with_fastqs*](https://nanoporetech.github.io/tombo/resquiggle.html) command before *re-squiggle*.\n\nFor the example data:\n```bash\n# 1. run multi_to_single_fast5 if needed\nmulti_to_single_fast5 -i $multi_read_fast5_dir -s $single_read_fast5_dir -t 30 --recursive\n\n# 2. basecall using GPU, fast5s/ is the $single_read_fast5_dir\nguppy_basecaller -i fast5s/ -r -s fast5s_guppy \\\n  --config dna_r9.4.1_450bps_hac_prom.cfg \\\n  --device CUDA:0\n# or using CPU\nguppy_basecaller -i fast5s/ -r -s fast5s_guppy \\\n  --config dna_r9.4.1_450bps_hac_prom.cfg\n\n# 3. proprecess fast5 if basecall results are saved in fastq format\ncat fast5s_guppy/*.fastq \u003e fast5s_guppy.fastq\ntombo preprocess annotate_raw_with_fastqs --fast5-basedir fast5s/ \\\n  --fastq-filenames fast5s_guppy.fastq \\\n  --sequencing-summary-filenames fast5s_guppy/sequencing_summary.txt \\\n  --basecall-group Basecall_1D_000 --basecall-subgroup BaseCalled_template \\\n  --overwrite --processes 10\n\n# 4. resquiggle, cmd: tombo resquiggle $fast5_dir $reference_fa\ntombo resquiggle fast5s/ GCF_000001735.4_TAIR10.1_genomic.fna \\\n  --processes 10 --corrected-group RawGenomeCorrected_000 \\\n  --basecall-group Basecall_1D_000 --overwrite\n```\n\n#### 2. extract features\nFeatures of targeted sites can be extracted for training or testing.\n\nFor the example data (By default, deepsignal-plant extracts 13-mer-seq and 13*16-signal features of each CpG motif in reads. Note that the value of *--corrected_group* must be the same as that of *--corrected-group* in [tombo](https://github.com/nanoporetech/tombo).):\n```bash\n# extract features of all Cs\ndeepsignal_plant extract -i fast5s \\\n  -o fast5s.C.features.tsv --corrected_group RawGenomeCorrected_000 \\\n  --nproc 30 --motifs C\n```\n\nThe extracted_features file is a tab-delimited text file in the following format:\n   - **chrom**: the chromosome name\n   - **pos**:   0-based position of the targeted base in the chromosome\n   - **strand**:    +/-, the aligned strand of the read to the reference\n   - **pos_in_strand**: 0-based position of the targeted base in the aligned strand of the chromosome (_legacy column, not necessary for downstream analysis_)\n   - **readname**:  the read name\n   - **read_strand**:   t/c, template or complement\n   - **k_mer**: the sequence around the targeted base\n   - **signal_means**:  signal means of each base in the kmer\n   - **signal_stds**:   signal stds of each base in the kmer\n   - **signal_lens**:   lens of each base in the kmer\n   - **raw_signals**:  signal values for each base of the kmer, splited by ';'\n   - **methy_label**:   0/1, the label of the targeted base, for training\n\n#### 3. call modifications\n\nTo call modifications, either the extracted-feature file or **the raw fast5 files (recommended)** can be used as input. \n\n**GPU/Multi-GPU support**: Use `CUDA_VISIBLE_DEVICES=${cuda_number} ccsmeth call_mods [options]` to call modifications with specified GPUs (_e.g._, `CUDA_VISIBLE_DEVICES=0` or `CUDA_VISIBLE_DEVICES=0,1`).\n\nFor the example data:\n```bash\n# call 5mCs for instance\n\n# extracted-feature file as input, use CPU\nCUDA_VISIBLE_DEVICES=-1 deepsignal_plant call_mods --input_path fast5s.C.features.tsv \\\n  --model_path model.dp2.CNN.arabnrice2-1_120m_R9.4plus_tem.bn13_sn16.both_bilstm.epoch6.ckpt \\\n  --result_file fast5s.C.call_mods.tsv \\\n  --nproc 30\n# extracted-feature file as input, use GPU\nCUDA_VISIBLE_DEVICES=0 deepsignal_plant call_mods --input_path fast5s.C.features.tsv \\\n  --model_path model.dp2.CNN.arabnrice2-1_120m_R9.4plus_tem.bn13_sn16.both_bilstm.epoch6.ckpt \\\n  --result_file fast5s.C.call_mods.tsv \\\n  --nproc 30 --nproc_gpu 6\n\n# fast5 files as input, use CPU\nCUDA_VISIBLE_DEVICES=-1 deepsignal_plant call_mods --input_path fast5s/ \\\n  --model_path model.dp2.CNN.arabnrice2-1_120m_R9.4plus_tem.bn13_sn16.both_bilstm.epoch6.ckpt \\\n  --result_file fast5s.C.call_mods.tsv \\\n  --corrected_group RawGenomeCorrected_000 \\\n  --motifs C --nproc 30\n# fast5 files as input, use GPU\nCUDA_VISIBLE_DEVICES=0 deepsignal_plant call_mods --input_path fast5s/ \\\n  --model_path model.dp2.CNN.arabnrice2-1_120m_R9.4plus_tem.bn13_sn16.both_bilstm.epoch6.ckpt \\\n  --result_file fast5s.C.call_mods.tsv \\\n  --corrected_group RawGenomeCorrected_000 \\\n  --motifs C --nproc 30 --nproc_gpu 6\n```\n\nThe modification_call file is a tab-delimited text file in the following format:\n   - **chrom**: the chromosome name\n   - **pos**:   0-based position of the targeted base in the chromosome\n   - **strand**:    +/-, the aligned strand of the read to the reference\n   - **pos_in_strand**: 0-based position of the targeted base in the aligned strand of the chromosome (_legacy column, not necessary for downstream analysis_)\n   - **readname**:  the read name\n   - **read_strand**:   t/c, template or complement\n   - **prob_0**:    [0, 1], the probability of the targeted base predicted as 0 (unmethylated)\n   - **prob_1**:    [0, 1], the probability of the targeted base predicted as 1 (methylated)\n   - **called_label**:  0/1, unmethylated/methylated\n   - **k_mer**:   the kmer around the targeted base\n\n#### 4. call frequency of modifications\nA modification-frequency file can be generated by `call_freq` function with the call_mods file as input:\n```bash\n# call 5mCs for instance\n\n# output in tsv format\ndeepsignal_plant call_freq --input_path fast5s.C.call_mods.tsv \\\n  --result_file fast5s.C.call_mods.frequency.tsv\n# output in bedMethyl format\ndeepsignal_plant call_freq --input_path fast5s.C.call_mods.tsv \\\n  --result_file fast5s.C.call_mods.frequency.bed --bed\n# use --sort to sort the results\ndeepsignal_plant call_freq --input_path fast5s.C.call_mods.tsv \\\n  --result_file fast5s.C.call_mods.frequency.bed --bed --sort\n```\n\nThe modification_frequency file can be either saved in [bedMethyl](https://www.encodeproject.org/data-standards/wgbs/) format (by setting `--bed` as above), or saved as a tab-delimited text file in the following format by default:\n   - **chrom**: the chromosome name\n   - **pos**:   0-based position of the targeted base in the chromosome\n   - **strand**:    +/-, the aligned strand of the read to the reference\n   - **pos_in_strand**: 0-based position of the targeted base in the aligned strand of the chromosome (_legacy column, not necessary for downstream analysis_)\n   - **prob_0_sum**:    sum of the probabilities of the targeted base predicted as 0 (unmethylated)\n   - **prob_1_sum**:    sum of the probabilities of the targeted base predicted as 1 (methylated)\n   - **count_modified**:    number of reads in which the targeted base counted as modified\n   - **count_unmodified**:  number of reads in which the targeted base counted as unmodified\n   - **coverage**:  number of reads aligned to the targeted base\n   - **modification_frequency**:    modification frequency\n   - **k_mer**:   the kmer around the targeted base\n\n#### 5. denoise training samples\n```bash\n# please use deepsignal_plant denoise -h/--help for instructions\ndeepsignal_plant denoise --train_file /path/to/train/file\n```\n\n#### 6. train new models\nA new model can be trained as follows:\n```bash\n# need to split training samples to two independent datasets for training and validating\n# please use deepsignal_plant train -h/--help for instructions\ndeepsignal_plant train --train_file /path/to/train/file \\\n  --valid_file /path/to/valid/file \\\n  --model_dir /dir/to/save/the/new/model\n```\n\nExtra\n=====\nWe are testing deepsignal-plant on a zebrafish sample...\n\nLicense\n=========\nCopyright (C) 2020 [Jianxin Wang](mailto:jxwang@mail.csu.edu.cn), [Feng Luo](mailto:luofeng@clemson.edu), [Peng Ni](mailto:nipeng@csu.edu.cn)\n\nThis program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.\n\nThis program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License along with this program. If not, see \u003chttps://www.gnu.org/licenses/\u003e.\n\n[Jianxin Wang](mailto:jxwang@mail.csu.edu.cn), [Peng Ni](mailto:nipeng@csu.edu.cn),\nSchool of Computer Science and Engineering, Central South University, Changsha 410083, China\n\n[Feng Luo](mailto:luofeng@clemson.edu), School of Computing, Clemson University, Clemson, SC 29634, USA\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpengni%2Fdeepsignal-plant","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpengni%2Fdeepsignal-plant","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpengni%2Fdeepsignal-plant/lists"}