{"id":16849200,"url":"https://github.com/mbhall88/psdm","last_synced_at":"2025-03-17T05:32:08.243Z","repository":{"id":57657808,"uuid":"422810671","full_name":"mbhall88/psdm","owner":"mbhall88","description":"Compute a pairwise SNP distance matrix from one or two alignment(s)","archived":false,"fork":false,"pushed_at":"2024-09-12T01:34:04.000Z","size":389,"stargazers_count":24,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-11T03:40:52.099Z","etag":null,"topics":["alignment","bioinformatics","distance","fasta","matrix","pairwise","rust","snp"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mbhall88.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-30T07:10:30.000Z","updated_at":"2024-11-29T02:32:52.000Z","dependencies_parsed_at":"2024-09-12T08:44:31.148Z","dependency_job_id":"ffdbc2a2-4012-4a46-bc47-a7fa7834288d","html_url":"https://github.com/mbhall88/psdm","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbhall88%2Fpsdm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbhall88%2Fpsdm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbhall88%2Fpsdm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbhall88%2Fpsdm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mbhall88","download_url":"https://codeload.github.com/mbhall88/psdm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243846976,"owners_count":20357294,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alignment","bioinformatics","distance","fasta","matrix","pairwise","rust","snp"],"created_at":"2024-10-13T13:14:35.724Z","updated_at":"2025-03-17T05:32:07.751Z","avatar_url":"https://github.com/mbhall88.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!--todo--\u003e\n[![codecov](https://codecov.io/gh/mbhall88/psdm/branch/main/graph/badge.svg?token=WVIARYNHYZ)](https://codecov.io/gh/mbhall88/psdm)\n[![Rust CI](https://github.com/mbhall88/psdm/actions/workflows/rust-ci.yaml/badge.svg)](https://github.com/mbhall88/psdm/actions/workflows/rust-ci.yaml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![github release version](https://img.shields.io/github/v/release/mbhall88/psdm)](https://github.com/mbhall88/psdm/releases)\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5706784.svg)](https://doi.org/10.5281/zenodo.5706784)\n\nCompute a pairwise SNP distance matrix from one or two alignment(s)\n\n[TOC]: #\n\n## Table of Contents\n- [Motivation](#motivation)\n- [Install](#install)\n  - [`cargo`](#cargo)\n  - [`conda`](#conda)\n  - [Precompiled binaries](#precompiled-binaries)\n  - [`homebrew`](#homebrew)\n  - [Container](#container)\n  - [Local](#local)\n- [Usage](#usage)\n  - [Quick](#quick)\n  - [Full](#full)\n- [Benchmark](#benchmark)\n- [Contributing](#contributing)\n- [Cite](#cite)\n\n## Motivation\n\nA key point of difference for `psdm` is the pairwise SNP distance between **two**\nalignment files. This is particularly beneficial if you have computed a SNP distance\nmatrix for many samples already and want to update the distances with some new samples -\nwithout rerunning the analysis for all samples in the original file.\n\nAnother potential use is having two alignment files for the same samples but with\nsequences generated by different techniques. For example, if you have produced consensus\nsequences from SNP calls from Illumina and Nanopore and want to see how similar the\nIllumina-to-Nanopore (inter-technology) distances are - compared to the intra-technology\ndistances.\n\nDespite these motivations, `psdm` can still be used to compute a \"traditional\" pairwise\nSNP distance matrix for a single FASTA alignment file.\n\n## Install\n\n\n### `cargo`\n\n[![Crates.io](https://img.shields.io/crates/v/psdm.svg)](https://crates.io/crates/psdm)\n\nPrerequisite: [`rust` toolchain][rust] (min. v1.55.0)\n\n```shell\n$ cargo install psdm\n```\n\n### `conda`\n\n[![Conda (channel only)](https://img.shields.io/conda/vn/bioconda/psdm)](https://anaconda.org/bioconda/psdm)\n[![bioconda version](https://anaconda.org/bioconda/psdm/badges/platforms.svg)](https://anaconda.org/bioconda/psdm)\n![Conda](https://img.shields.io/conda/dn/bioconda/psdm)\n\nPrerequisite: [`conda`][conda] (and bioconda channel [correctly set up][channels])\n\n```shell\n$ conda install psdm\n```\n\n### Precompiled binaries\n\n**tl;dr**: Run the following snippet to download the latest binary for your system to\nthe current directory and show the help menu.\n\n```shell\nversion=\"0.3.0\"\nOS=$(uname -s)                                                                                                       \nif [ \"$OS\" = \"Linux\" ]; then                                                                                         \n    triple=\"x86_64-unknown-linux-musl\"                                                                              \nelif [ \"$OS\" = \"Darwin\" ]; then                                                                                        \n    triple=\"x86_64-apple-darwin\"                                                         \nelse                                                      \n    echo \"ERROR: $OS not a recognised operating system\"\nfi              \nif [ -n \"$triple\" ]; then   \n    URL=\"https://github.com/mbhall88/psdm/releases/download/${version}/psdm-${version}-${triple}.tar.gz\"\n    wget \"$URL\" -O - | tar -xzf -\n    ./psdm-${version}-${triple}/psdm --help             \nfi\n```\n\nThese binaries _do not_ require that you have the `rust` toolchain installed.\n\nCurrently, there are two pre-compiled binaries available:\n- Linux kernel `x86_64-unknown-linux-musl` (works on most Linux distributions I tested)\n- OSX kernel `x86_64-apple-darwin` (works for any post-2007 Mac)\n\nAn example of downloading one of these binaries using `wget`\n\n```shell\n$ version=\"0.3.0\"\n$ URL=\"https://github.com/mbhall88/psdm/releases/download/${version}/psdm-${version}-x86_64-unknown-linux-musl.tar.gz\"\n$ wget \"$URL\" -O - | tar -xzf -\n$ ./psdm*/psdm --help\n```\n\nIf these binaries do not work on your system please raise an issue and I will\npotentially add some additional [target triples][triples].\n\n### `homebrew`\n\nPrerequisite: [`homebrew`][homebrew]\n\nThe `homebrew` installation is done via the [homebrew-bio tap][brew-tap].\n\n```shell\n$ brew install brewsci/bio/psdm\n```\n\n### Container\n\nDocker images are hosted at [quay.io].\n\n#### `singularity`\n\nPrerequisite: [`singularity`][singularity]\n\n```shell\n$ URI=\"docker://quay.io/mbhall88/psdm\"\n$ singularity exec \"$URI\" psdm --help\n```\n\nThe above will use the latest version. If you want to specify a version then use a\n[tag][quay.io] (or commit) like so.\n\n```shell\n$ VERSION=\"0.3.0\"\n$ URI=\"docker://quay.io/mbhall88/psdm:${VERSION}\"\n```\n\n#### `docker`\n\n[![Docker Repository on Quay](https://quay.io/repository/mbhall88/psdm/status \"Docker Repository on Quay\")](https://quay.io/repository/mbhall88/psdm)\n\nPrerequisite: [`docker`][docker]\n\n```shhell\n$ docker pull quay.io/mbhall88/psdm\n$ docker run quay.io/mbhall88/psdm psdm --help\n```\n\nYou can find all the available tags on the [quay.io repository][quay.io].\n\n### Local\n\nPrerequisite: [`rust` toolchain][rust]\n\n```sh\n$ git clone https://github.com/mbhall88/psdm.git\n$ cd psdm\n$ cargo build --release\n$ target/release/psdm --help\n# if you want to check everything is working ok\n$ cargo test\n```\n\n## Usage\n\n### Quick\n\n#### Single alignment file\n\n**`aln1.fa`**\n\n```\n\u003es1\nABCDEFGH\n\u003es2\naBN-XFnH\n\u003es0\nAbCdEfG-\n```\n\n```shell\n$ psdm aln1.fa\n,s1,s2,s0\ns1,0,1,0\ns2,1,0,1\ns0,0,1,0\n```\n\n#### Two alignment files\n\n**`aln2.fa.gz`**\n\n```\n\u003es2\nxXNNfoo=\n\u003es5 description\nAB-DEFGG\n```\n\n```shell\n$ psdm aln1.fa aln2.fa.gz\n,s1,s2,s0\ns2,6,5,5\ns5,1,2,0\n```\n\nThe column names represent the **first** alignment file provided.\n\n### Full\n\nI'd like the sequences to be sorted by identifier in the output\n\n```shell\n$ psdm -s aln1.fa\n,s0,s1,s2\ns0,0,0,1\ns1,0,0,1\ns2,1,1,0\n```\n\nI want a tab-delimited (TSV) matrix instead of a comma-delimited (CSV) one\n\n```shell\n$ psdm -d \"\\t\" aln1.fa\n        s1      s2      s0\ns1      0       1       0\ns2      1       0       1\ns0      0       1       0\n```\n\nThe case of nucleotides matters to me - i.e., `acgt` is *not* the same as `ACGT`\n\n```shell\n$ psdm -c aln1.fa\n,s1,s2,s0\ns1,0,3,3\ns2,3,0,5\ns0,3,5,0\n```\n\nBy default, `psdm` ignores `N`'s and gaps (`-`). However, maybe you also want to ignore\n`X`'s\n\n```shell\n$ psdm -e NX- aln1.fa\n,s1,s2,s0\ns1,0,0,0\ns2,0,0,0\ns0,0,0,0\n```\n\nOr maybe you don't want to ignore anything\n\n```shell\n$ psdm -e \"\" aln1.fa\n,s1,s2,s0\ns1,0,5,4\ns2,5,0,8\ns0,4,8,0\n```\n\nI'm impatient, use all the threads I have!\n\n```shell\n$ psdm -t 0 aln1.fa\n```\n\nGive me long-form output instead of a matrix\n\n```shell\n$ psdm -l aln1.fa\ns1,s1,0\ns1,s2,1\ns1,s0,0\ns2,s1,1\ns2,s2,0\ns2,s0,1\ns0,s1,0\ns0,s2,1\ns0,s0,0\n```\n\nI'd like to know the progress of the pairwise comparisons\n\n```\n$ psdm -P -t 8 aln.fa\n[2024-06-20T02:50:32Z INFO  psdm] Using 8 thread(s)\n[2024-06-20T02:50:32Z INFO  psdm] Loading first alignment file...\n[2024-06-20T02:50:38Z INFO  psdm] Loaded 200 sequences with length 4411532bp\n[2024-06-20T02:50:38Z INFO  psdm] Calculating 20100 pairwise distances...\nProgress: 50.00% (10050 / 20100)\n```\n\nWrite the matrix to a file please\n\n```shell\n$ psdm -o dists.csv aln1.fa\n```\n\n```\n$ psdm --help\npsdm 0.3.0\nMichael Hall \u003cmichael@mbh.sh\u003e\nCompute a pairwise SNP distance matrix from one or two alignment(s)\n\nUSAGE:\n    psdm [OPTIONS] [ALIGNMENTS]...\n\nARGS:\n    \u003cALIGNMENTS\u003e...\n            FASTA alignment file(s) to compute the pairwise distance for.\n\n            Providing two files will compute the distances for all sequences in one file against all\n            sequences from the other file - i.e., not between sequences in the same file. The first\n            file will be the column names, while the second is the row names. The alignment file(s)\n            can be compressed.\n\nOPTIONS:\n    -c, --case-sensitive\n            Case matters - i.e., dist(a, A) = 1\n\n    -d, --delim \u003cDELIMITER\u003e\n            Delimiting character for the output table\n\n            [default: ,]\n\n    -e, --ignored-chars \u003cIGNORED_CHARS\u003e\n            String of characters to ignore - e.g., `-e N-` -\u003e dist(A, N) = 0 and dist(A, -) = 0\n\n            Note, if using `--case-sensitive` the upper- and lower-case form of a character is\n            needed. To not ignore any characters, use `-e ''` or `-e \"\"`\n\n            [default: N-]\n\n    -h, --help\n            Print help information\n\n    -l, --long\n            Output as long-form (\"melted\") table\n\n            By default the output is a N x N or N x M table where N is the number of sequences in\n            the first alignment and M is the number of sequences in the (optional) second alignment.\n\n    -o, --output \u003cOUTPUT\u003e\n            Output file name [default: stdout]\n\n    -P, --progress\n            Show a progress bar\n\n    -q, --quiet\n            No logging (except progress info if `-P` is given)\n\n    -s, --sort\n            Sort the alignment(s) by ID\n\n    -t, --threads \u003cTHREADS\u003e\n            Number of threads to use. Setting to 0 will use all available\n\n            [default: 1]\n\n    -V, --version\n            Print version information\n```\n\n## Benchmark\n\nWe benchmark against [`snp-dists`][snp-dists] (v0.8.2). `snp-dists` is a brilliant tool\nfor computing SNP distance matrices and is the inspiration for `psdm`.\n\nAs `snp-dists` doesn't allow inter-alignment comparisons, we will benchmark the\nintra-alignment mode. We use an alignment file of 151 sequences with a length of\n4411532bp.\n\nThe benchmark times were recorded with [`hyperfine`][hyperfine] with 10 runs for each\ntool/threads combination.\n\n![Benchmark plot](benchmark/benchmark.png)\n\n| tool      |   threads |   mean |   SD |   min |   median |   max |\n|:----------|----------:|-------:|-----:|------:|---------:|------:|\n| psdm      |         1 |   95.4 |  1.0 |  94.2 |     95.3 |  97.9 |\n| snp-dists |         1 |  231.0 |  4.8 | 224.2 |    231.8 | 240.4 |\n| psdm      |         2 |   54.3 |  1.5 |  53.2 |     53.6 |  57.5 |\n| snp-dists |         2 |  120.3 |  0.8 | 118.9 |    120.3 | 121.3 |\n| psdm      |         4 |   33.0 |  0.8 |  32.5 |     32.7 |  35.0 |\n| snp-dists |         4 |   61.8 |  0.2 |  61.5 |     61.8 |  62.2 |\n| psdm      |         8 |   22.3 |  0.1 |  22.1 |     22.3 |  22.4 |\n| snp-dists |         8 |   33.1 |  0.3 |  32.8 |     33.1 |  33.7 |\n| psdm      |        16 |   17.6 |  0.2 |  17.4 |     17.5 |  18.0 |\n| snp-dists |        16 |   20.7 |  0.2 |  20.4 |     20.7 |  21.1 |\n\n## Contributing\n\nContributions are always welcome. For changes to be accepted, they must pass the CI and\ncoverage checks. These include:\n\n- Code is formatted (`cargo fmt`).\n- There are no compiler errors/warnings. `cargo clippy --all-features --all-targets --\n  -D warnings`\n- Test code coverage has not reduced.\n\nPlease also add a succinct description of the contribution in the\n[CHANGELOG](CHANGELOG.md).\n\n# Cite\n\n`psdm` is archived [at Zenodo](https://zenodo.org/record/5706785).\n\n```bibtex\n@software{Hall2021psdm,\n  author       = {Hall, Michael B.},\n  title        = {{psdm: Compute a pairwise SNP distance matrix from \n                   one or two alignment(s)}},\n  month        = nov,\n  year         = 2021,\n  publisher    = {Zenodo},\n  version      = {0.3.0},\n  doi          = {10.5281/zenodo.5706784},\n  url          = {https://doi.org/10.5281/zenodo.5706785}\n}\n```\n\n[rust]: https://www.rust-lang.org/tools/install\n[channels]: https://bioconda.github.io/user/install.html#set-up-channels\n[conda]: https://docs.conda.io/projects/conda/en/latest/user-guide/install/\n[quay.io]: https://quay.io/repository/mbhall88/psdm\n[singularity]: https://sylabs.io/guides/3.4/user-guide/quick_start.html#quick-installation-steps\n[docker]: https://docs.docker.com/v17.12/install/\n[homebrew]: https://docs.brew.sh/Installation\n[brew-tap]: https://github.com/brewsci/homebrew-bio\n[triples]: https://clang.llvm.org/docs/CrossCompilation.html#target-triple\n[snp-dists]: https://github.com/tseemann/snp-dists\n[hyperfine]: https://github.com/sharkdp/hyperfine\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmbhall88%2Fpsdm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmbhall88%2Fpsdm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmbhall88%2Fpsdm/lists"}