Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/nanoporetech/katuali

Analysis pipelines from Oxford Nanopore Technologies' Research Division
https://github.com/nanoporetech/katuali

Last synced: about 1 year ago
JSON representation

Analysis pipelines from Oxford Nanopore Technologies' Research Division

Awesome Lists containing this project

README 

          
![Oxford Nanopore Technologies logo](images/ONT_logo_590x106.png)



Katuali

=======



`Katuali` is a flexible consensus pipeline implemented in

[Snakemake](https://snakemake.readthedocs.io) to basecall, assemble, and polish

Oxford Nanopore Technologies' sequencing data.



Features

--------



  * fast5 to high quality assembly in a single command.

  * Recommended fixed "standard" and "fast" pipelines.

  * Interchange basecaller, assembler, and consensus components of the

    pipelines simply by changing the target filepath.

  * [Medaka](https://github.com/nanoporetech/medaka) training pipeline

    including generation of training data, model training, and model evaluation.

  * Seemless distribution of tasks over local or distributed compute.

  * Open source (Mozilla Public License 2.0).



Documentation can be found at https://nanoporetech.github.io/katuali/.



Installation

------------



A Makefile is provided to create a

[virtual environment](https://docs.python.org/3/tutorial/venv.html) into which

all necessary dependencies will be installed. Katuali has been tested on Linux

(specifically Ubuntu 16) and MacOS.



To setup the environment run:



    git clone https://github.com/nanoporetech/katuali.git

    cd katuali

    make install

    source ./venv/bin/activate



A conda package is forthcoming.



Dependencies

------------



Katuali makes use of a number of tools to do basecalling, assembly, and

polishing that need to be installed to perform such steps of a pipeline.

Katuali does not enforce that all possible tools are present; only those

required to run a user's pipeline need be installed. If Katuali cannot

find a tool, it will alert the user.



The list below indicates possible options for pipeline steps and the default

filesystem location where Katuali expects to find them. These locations can be

changed by the user, see the

[documentation](https://nanoporetech.github.io/katuali/) for more information.

Please refer to the documentation of each of these tools for installation

instructions.



**General Tools**



* [pomoxis](https://github.com/nanoporetech/pomoxis): "~/git/pomoxis/venv/bin/activate"



Pomoxis is used as a general utility throughout much of katuali, its

installation is required for most functionality.



**Basecalling Tools**



* [guppy](https://community.nanoporetech.com/downloads): "/usr/bin/guppy_basecaller"



Guppy is recommended, both scrappie and flappie are research prototypes.



**Assembly Tools**



* [canu](https://github.com/marbl/canu): "~/git/canu-1.8/Linux-amd64/bin/canu"

* [flye](https://github.com/fenderglass/Flye): "~/git/Flye/bin/flye"

* [shasta](https://github.com/chanzuckerberg/shasta): "~/git/shasta-Linux-0.3.0"

* [pomoxis](https://github.com/nanoporetech/pomoxis): "~/git/pomoxis/venv/bin/activate"



In the context of Katuali, pomoxis functions as a wrapper to

[miniasm](https://github.com/lh3/miniasm). Canu is utilised in the standard

workflow, while pomoxis is used in the fast workflow.



**Consensus Tools**



* [medaka](https://github.com/nanoporetech/medaka): "~/git/medaka/venv/bin/activate"

* [nanopolish](https://github.com/jts/nanopolish): "~/git/nanopolish"



Medaka is recommended in combination with the latest "flip-flop" algorithm

in the guppy basecaller, and for rapid consensus. Nanopolish may still be

preferred if using older basecallers.



Usage

-----



The `Katuali` tests contain examples of how to basecall, assemble, and polish

a small dataset that comes bundled with `Katuali`. These tests can be run with:



    make test



from the `Katuali` source directory. Running the tests requires installation of

`scrappie`, `pomoxis`, `medaka`, and `nanopolish`. See the

[documentation](https://nanoporetech.github.io/katuali/installation.html#tests)

for more information on the outputs of running the above.



To run with other data, start by creating a directory of reads (which could

contain subdirectories of reads) within a run directory:



    mkdir -p run1

    cd run1

    ln -s /path/to/fast5 reads  # create a softlink to the fast5 data

    cd ..

    

Then make a copy of the katuali config into your working directory,



    katuali_config my_config.yaml



and update the katuali config to reflect your data:

    

    DATA:

        'run1':

            'GENOME_SIZE': '4.0M'  # for canu we need to specify genome size



There are two predefined pipelines.



1. To perform basecalling, a quick assembly with miniasm, and consensus with racon

   and medaka it is sufficient to run:



       katuali --configfile my_config.yaml fast_assm_polish



2. Alternatively to assemble with canu run:



       katuali --configfile my_config.yaml standard_assm_polish



The [documentation](https://nanoporetech.github.io/katuali/examples.html)

describes how to create a pipeline with any combination of basecallers,

assemblers and polishers. 

    



Medaka training pipeline

------------------------



It is possible to train medaka models starting from folders of fast5s in a

single command once the config has been modified to reflect your input data

(fast5s and genomes for each run as well as training and evaluation region

definitions).



Running:



    katuali all_medaka_train_features --keep-going



will:



* basecall all the runs

* align each run to its reference

* create subsampled sets of basecalls over the desired regions and depths

* assemble those sets of basecalls

* create medaka training features for all those sets



Running:



    katuali medaka_train_replicates --keep-going



will do all the tasks of `all_medaka_train_features` and additionally launch

multiple medaka model-training replicates.



If some of your input runs have insufficient coverage-depth for some of the

training regions, some of the training feature files will not be made. In this

case the config flag `USE_ONLY_EXISTING_MEDAKA_FEAT` can be set to true to allow

katuali to train using only those features which exist already:



    USE_ONLY_EXISTING_MEDAKA_FEAT: true 



Refer to comments in the config (katuali/config.yaml) to see how this process

can be controlled. 



Help

----



**Licence and Copyright**



© 2018 Oxford Nanopore Technologies Ltd.



`katuali` is distributed under the terms of the Mozilla Public License 2.0.



**Research Release**



Research releases are provided as technology demonstrators to provide early

access to features or stimulate Community development of tools. Support for

this software will be minimal and is only provided directly by the developers.

Feature requests, improvements, and discussions are welcome and can be

implemented by forking and pull requests. However much as we would

like to rectify every issue and piece of feedback users may have, the 

developers may have limited resource for support of this software. Research

releases may be unstable and subject to rapid iteration by Oxford Nanopore

Technologies.