{"id":13704082,"url":"https://github.com/brentp/somalier","last_synced_at":"2026-04-17T18:02:44.062Z","repository":{"id":45141677,"uuid":"143888326","full_name":"brentp/somalier","owner":"brentp","description":"fast sample-swap and relatedness checks on BAMs/CRAMs/VCFs/GVCFs... \"like damn that is one smart wine guy\"","archived":false,"fork":false,"pushed_at":"2026-04-16T17:42:25.000Z","size":1552,"stargazers_count":308,"open_issues_count":62,"forks_count":44,"subscribers_count":7,"default_branch":"master","last_synced_at":"2026-04-16T18:31:29.595Z","etag":null,"topics":["bioinformatics","cancer-genomics","genomics"],"latest_commit_sha":null,"homepage":"","language":"Nim","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brentp.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2018-08-07T14:52:04.000Z","updated_at":"2026-04-16T17:40:40.000Z","dependencies_parsed_at":"2023-01-20T05:33:59.329Z","dependency_job_id":"964719de-700c-4c57-ad4c-7d2d3b897907","html_url":"https://github.com/brentp/somalier","commit_stats":null,"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"purl":"pkg:github/brentp/somalier","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brentp%2Fsomalier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brentp%2Fsomalier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brentp%2Fsomalier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brentp%2Fsomalier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brentp","download_url":"https://codeload.github.com/brentp/somalier/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brentp%2Fsomalier/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31939788,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T17:29:20.459Z","status":"ssl_error","status_checked_at":"2026-04-17T17:28:47.801Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","cancer-genomics","genomics"],"created_at":"2024-08-02T21:01:03.987Z","updated_at":"2026-04-17T18:02:44.051Z","avatar_url":"https://github.com/brentp.png","language":"Nim","funding_links":[],"categories":["Next Generation Sequencing"],"sub_categories":["BAM File Utilities"],"readme":"# somalier: extract informative sites, evaluate relatedness, and perform quality-control on BAM/CRAM/BCF/VCF/GVCF\n\n[![Actions Status](https://github.com/brentp/somalier/workflows/Docker%20Image%20CI/badge.svg)](https://github.com/brentp/somalier/actions)\n[![Cite](https://img.shields.io/badge/cite-genome%20medicine-blue)](https://genomemedicine.biomedcentral.com/articles/10.1186/s13073-020-00761-2)\n[![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg?style=flat)](http://bioconda.github.io/recipes/somalier/README.html)\n\n## Quick Start \n`somalier` makes checking any number of samples for identity easy **directly from the alignments** or from jointly-called VCFs:\n\nThe first step is to extract sites. For **VCF** just use:\n```\nsomalier extract -d extracted/ --sites sites.vcf.gz -f /data/human/g1k_v37_decoy.fa $cohort.vcf.gz\n```\n\nwith a sites file from [releases](https://github.com/brentp/somalier/releases/latest)\n\nFor **BAM** or **CRAM**, use:\nThis is parallelizable by sample via cluster or cloud, but here, using a for loop:\n```\nfor f in *.cram; do\n    somalier extract -d extracted/ --sites sites.vcf.gz -f /data/human/g1k_v37_decoy.fa $f\ndone\n```\n\n`--sites` is a VCF of known polymorphic sites in VCF format. A good set is provided in\nthe [releases](https://github.com/brentp/somalier/releases) but any set of common variants will work.\n\n\n\u003e :warning: `somalier` can work on GVCF and individual VCFs, but it is\n\u003e recommended to extract from bam/cram when possible. It is also good to\n\u003e extract from a jointly-called VCF/BCF when only looking within that cohort.\n\u003e While extracting from a single-sample VCF is possible (with --unknown) and\n\u003e GVCF is also supported, these options are less accurate and more prone to\n\u003e problems.\n\n\nThe next step is to calculate relatedness on the extracted data:\n\n```\nsomalier relate --ped $pedigree extracted/*.somalier\n```\nThis will create text and interactive HTML output\nthat makes it fast and easy to detect mismatched samples and sample-swaps.\nThe files created are:\n+ `{output_prefix}.samples.tsv # creates a .ped like file with extra QC columns\n+ `{output_prefix}.pairs.tsv # shows IBS for all possible sample pairs\n+ `{output_prefix}.groups.tsv # shows pairs of samples above a certain relatedness\n+ `{output_prefix}.html # interactive html\n\nExample html output is [here](https://brentp.github.io/somalier/ex.html)\n\nNote that the `somalier relate` command runs extremely quickly (\u003c 2 seconds for 600 samples and ~1 minute for 4,500 samples) so it's possible\nto add/remove samples or adjust a pedigree file and re-run iteratively.\n\nFor example to add the **n + 1th** samples, just run `somalier extract` on the new sample and then re-use\nthe already extracted data from the `n` original samples.\n\nFor *huge* sample-sets, if you run into a bash error for *argument list too long*, you can pass the somalier files as quoted\nglob strings like:  `\"/path/to/set-a/*.somalier\" \"/path/to/set-b/*.somalier\"`.\n\n## Example Output\n\n+ Interactive output from `somalier relate` is [here](https://brentp.github.io/somalier/ex.html)\n+ Interactive output from `somalier ancestry` is [here](https://brentp.github.io/somalier/ex.somalier-ancestry.html)\n\n## Infer\n\n`somalier` can also infer first-degree relationships (parent-child) when both-parents\nare present and can often build entire pedigrees on high-qualty data. To do this, use\n `somalier relate --infer ...` and the **samples.tsv** output will be a pedigree file\nindicating the inferred relationships.\n\nSee [wiki](https://github.com/brentp/somalier/wiki/pedigree-inference) for more detail.\n\n\n## Usage\n\nThe usage is also described above. Briefly, after downloading the somalier binary and a sites vcf from the [releases](https://github.com/brentp/somalier/releases/latest) run:\n```\nsomalier extract -d cohort/ --sites sites.hg38.vcf.gz -f $reference $sample.bam\n```\nfor each sample to create a small binary file of the ref and alt counts for the variants listed\nin sites.hg38.vcf.gz.\n\nfor a vcf, run:\n```\nsomalier extract -d cohort/ --sites sites.hg38.vcf.gz -f $reference $cohort.bcf\n```\n\n`somalier` can `extract` from a multi or single-sample VCF or a GVCF. This will be much faster, in cases where it's available,\nthis would look like:\n\n```\nsomalier extract -d extracted/ --sites sites.vcf.gz -f /data/human/g1k_v37_decoy.fa joint.vcf.gz\n```\n\nfollowing this, there will be a `$sample.somalier` file for each sample in the `joint.vcf.gz`\n\nNote that `somalier` uses the `AD` field to extract depth information. If that FORMAT field is not present in the\nheader, then it will use the genotypes only and use a total depth of 20 (10,10 for heterozygote), etc.\n\n\nThen run:\n```\nsomalier relate --ped $pedigree_file cohort/*.somalier\n```\nThis will create an html file for QC in a few seconds. \n\nNote that if a new sample is added to the cohort, it's only necessary to perform\nthe `extract` step on that sample and then run the (fast) `relate` step again with all\nof the extracted files.\n\nFor cancer cohorts, tumor-normal QC, CNV-robust concordance, and the\n`somalier contamination` workflow, see\n[Cancer, Concordance, Contamination](cancer-concordance-contamination.md).\n\n## Extended Usage\nFor each command of somalier, extended parameters are listed in `--help` of each subcommand.\n```\n$./somalier --help\nCommands:\n  extract      :   extract genotype-like information for a single sample from VCF/BAM/CRAM.\n  relate       :   aggregate `extract`ed information and calculate relatedness among samples.\n  ancestry     :   perform ancestry prediction on a set of samples, given a set of labeled samples\n  AB_plot      :   make an html plot of autosomal allele balance and depth across sites.\n  contamination :   estimate contamination from extracted somalier files.\n  find-sites   :   create a new sites.vcf.gz file from a population VCF (this is rarely needed).\n```\n### somalier extract\n\nextract genotype-like information for a single-sample at selected sites\n\n```\nUsage:\n  somalier extract [options] sample_file\n\nArguments:\n  sample_file      single-sample CRAM/BAM/GVCF file or multi/single-sample VCF from which to extract\n\nOptions:\n  -s, --sites=SITES          sites vcf file of variants to extract\n  -f, --fasta=FASTA          path to reference fasta file\n  -d, --out-dir=OUT_DIR      path to output directory (default: .)\n  --sample-prefix=SAMPLE_PREFIX\n                             prefix for the sample name stored inside the digest\n```\n### somalier relate\n\ncalculate relatedness among samples from extracted, genotype-like information\n\n```\nUsage:\n  somalier relate [options] [extracted ...]\n\nArguments:\n  [extracted ...]  $sample.somalier files for each sample. the first 10 are tested as a glob patterns\n\nOptions:\n  -g, --groups=GROUPS        optional path  to expected groups of samples (e.g. tumor normal pairs).\nspecified as comma-separated groups per line e.g.:\n    normal1,tumor1a,tumor1b\n    normal2,tumor2a\n  --sample-prefix=SAMPLE_PREFIX\n                             optional sample prefixes that can be removed to find identical samples. e.g. batch1-sampleA batch2-sampleA\n  -p, --ped=PED              optional path to a ped/fam file indicating the expected relationships among samples.\n  -d, --min-depth=MIN_DEPTH  only genotype sites with at least this depth. (default: 7)\n  --min-ab=MIN_AB            hets sites must be between min-ab and 1 - min_ab. set this to 0.2 for RNA-Seq data (default: 0.3)\n  -u, --unknown              set unknown genotypes to hom-ref. it is often preferable to use this with VCF samples that were not jointly called\n  -i, --infer                infer relationships (https://github.com/brentp/somalier/wiki/pedigree-inference)\n  -o, --output-prefix=OUTPUT_PREFIX\n                             output prefix for results. (default: somalier)\n```\n\nNote that for large cohorts, by default, `somalier relate` will subset to interesting sample-pairs so as not to\nballoon the size of the output. By default, pairs that are expected to be unrelated and have a relatedness \u003c= 0.05.\nIf you wish to force all samples to be reported, then set the environment variable `SOMALIER_REPORT_ALL_PAIRS` to any non-empty\nvalue, e.g. `export SOMALIER_REPORT_ALL_PAIRS=1`\n\n### somalier contamination\n\nestimate contamination from extracted `.somalier` files.\n\nFor tumor-normal use, CNV-robust concordance, CHARR, and the paired\n`somalier contamination` workflow, see\n[Cancer, Concordance, Contamination](cancer-concordance-contamination.md).\n\n## find-sites\n\nTo create a set of new sites, use `somalier find-sites` on a population VCF. More info on this tool is available in [the wiki](https://github.com/brentp/somalier/wiki/find-sites)\n\n## AB_plot\n\nTo inspect allele balance and depth along the autosomal sites in the sites file, run:\n\n```\nsomalier AB_plot --sites sites.hg38.vcf.gz extracted/*.somalier\n```\n\nThis creates an HTML page with a Plotly plot where the x-axis is the autosomal site index from the sites file, the left y-axis is allele balance, and the right y-axis is depth.\n\n## Install\n\nSomalier is available via bioconda, see [here](https://bioconda.github.io/recipes/somalier/README.html).\n\nAlternatively, you can get a static binary from [here](https://github.com/brentp/somalier/releases).\n\nUsers can also get a docker image [here](https://hub.docker.com/r/brentp/somalier/tags)\nwhich contains htslib and a somalier binary ready-for-use.\n\n## How it works\n\n`somalier` takes a list of known polymorphic sites. Even a few hundred (or dozen) sites\ncan be a very good indicator of relatedness. The best sites are those with a population\nallele frequency close to 0.5 as that maximizes the probability that any 2 samples will differ.\nA list of such sites is provided in the [releases](https://github.com/brentp/somalier/releases)\nfor GRCh37 and hg38.\n\nIn order to quickly calculate genotypes at these sites, `somalier` assays the exact base.\nThe extraction step is done directly from the bam/cram files 1 sample at a time.\n\nThe `relate` step is run on the output of the `extract` commands. It runs extremely quickly\nso that new samples can be added and compared. It uses 3 bit-vectors per sample for hom-ref,\nhet, hom-alt. Each bitvector is a sequence of 64 bit integers where each bit is set if\nthe variant at that index in the sample is for example, heterozygous. With this setup,\nwe can use fast bitwise operations and [popcount](https://en.wikichip.org/wiki/population_count)\nhardware instructions to calculate relatedness extremely quickly.\n\nFor each sample-pair, it reports:\n1. IBS0 -- the number of sites where one sample is hom-ref and another is hom-alt\n2. IBS2 -- the number of sites where the samples have the same genotype\n3. shared-hets -- the number of sites where both samples are heterozygotes\n4. shared-hom-alts -- the number of sites where both samples are homozygous alternate\n\nThese are used to calculate [relatedness](https://en.wikipedia.org/wiki/Coefficient_of_relationship)\nand a measure of relatedness that is unaffected by loss-of-heterozygosity that is often seen in some \ncancers. The interactive output allows toggling between any of these measures.\n\nIt also reports depth information and the count of `HET`, `HOM_REF`, `HOM_ALT`, and `unknown` genotypes for each sample\nalong with a number of metrics that are useful for general QC.\n\n## Example\n\n![example](https://user-images.githubusercontent.com/1739/43783575-4863f13c-9a1f-11e8-9cf8-622f784edc69.png)\n\nHere, each point is a pair of samples. We can see that the expected identical sample-pairs (e.g. tumor-normal pairs) specified by the user\nand drawn in red mostly cluster together on the right. Unrelateds cluster on the lower left. The sample-swaps are the blue points that cluster with\nthe red. In the somalier output, the user can **hover to see which sample-pairs are involved each point**\n\n\n## Ancestry Estimate\n\n`somalier` can predict ancestry on a set of query samples given a set of labelled samples. You can read more about this in [the wiki](https://github.com/brentp/somalier/wiki/ancestry)\n\n## Usage\n\nUsage is intentionally very simple and running `somalier extract` or `somalier relate` will give sufficient help for nearly\nall cases.\n\nBy default `somalier` will only consider variants that have a \"PASS\" or \"RefCall\" FILTER. To extend this list, set\nthe environment variable `SOMALIER_ALLOWED_FILTERS` to a comma-delimited list of additional filters to allow.\n\nby default sites with an allele balance \u003c 0.01 will be considered homozygous reference. To adjust this, use e.g. : \n`SOMALIER_AB_HOM_CUTOFF=0.04 somalier relate ...`\n \n## Other Work\n\nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC5499645/\n\nhttps://academic.oup.com/bioinformatics/article/33/4/596/2624551\n\n\n## Acknowledgement\n\nThis work was motivated by interaction and discussions with Preeti Aahir and several\nearly users who provided valuable feedback.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrentp%2Fsomalier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrentp%2Fsomalier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrentp%2Fsomalier/lists"}