{"id":16619207,"url":"https://github.com/tanghaibao/quota-alignment","last_synced_at":"2026-03-12T00:02:01.211Z","repository":{"id":804372,"uuid":"508176","full_name":"tanghaibao/quota-alignment","owner":"tanghaibao","description":"Guided synteny alignment between duplicated genomes (within specified quota constraint)","archived":false,"fork":false,"pushed_at":"2017-09-05T18:12:06.000Z","size":3104,"stargazers_count":57,"open_issues_count":3,"forks_count":20,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-18T02:12:25.169Z","etag":null,"topics":["comparative-genomics","evolution","genomics","synteny"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tanghaibao.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2010-02-08T18:00:21.000Z","updated_at":"2025-03-06T09:36:32.000Z","dependencies_parsed_at":"2022-08-16T10:55:24.157Z","dependency_job_id":null,"html_url":"https://github.com/tanghaibao/quota-alignment","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tanghaibao%2Fquota-alignment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tanghaibao%2Fquota-alignment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tanghaibao%2Fquota-alignment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tanghaibao%2Fquota-alignment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tanghaibao","download_url":"https://codeload.github.com/tanghaibao/quota-alignment/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244815225,"owners_count":20514917,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["comparative-genomics","evolution","genomics","synteny"],"created_at":"2024-10-12T02:24:33.803Z","updated_at":"2026-03-12T00:02:01.205Z","avatar_url":"https://github.com/tanghaibao.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Quota synteny alignment\n=========================\n\n:Author: Haibao Tang (`tanghaibao \u003chttp://github.com/tanghaibao\u003e`_),\n         Brent Pedersen (`brentp \u003chttp://github.com/brentp\u003e`_)\n:Email: tanghaibao@gmail.com\n:License: `BSD \u003chttp://creativecommons.org/licenses/BSD/\u003e`_\n\n.. contents ::\n\nIntroduction\n------------\n\nTypically in comparative genomics, we can identify anchors, chain them into\nsyntenic blocks and interpret these blocks as derived from a common descent.\nHowever, when comparing two genomes undergone ancient genome duplications\n(plant genomes in particular), we have large number of blocks that are not\northologous, but are paralogous. This has forced us sometimes to use *ad-hoc*\nrules to screen these blocks. So the question is: **given the expected depth\n(quota) along both x- and y-axis, select a subset of the anchors with maximized\ntotal score**.\n\n.. image:: http://lh4.ggpht.com/_srvRoIok9Xs/TT9wAmt6fQI/AAAAAAAABMQ/EWtrxS5rvww/s800/quota_chart1.png\n    :alt: before quota-align\n.. image:: http://lh6.ggpht.com/_srvRoIok9Xs/TT9wAj9gYYI/AAAAAAAABMU/bbupmSrE5G8/s800/quota_chart2.png\n    :alt: after quota-align\n\nThis program tries to screen the clusters based on the depth constraints\nenforced by the user. For example, between rice-sorghum comparison, we can\nenforce ``1:1`` ratio to get all the orthologous blocks; or maybe ``4:2`` to\ngrab orthologous blocks between athaliana-poplar. But the quota has to be given\nby the user. The program than tries to optimize the scores of these blocks\nglobally.\n\nTo see the algorithm in action without installation, please go to `CoGe SynMap\ntool \u003chttp://genomevolution.org/CoGe/SynMap.pl\u003e`_. Select \"Analysis Options\",\nselect algorithm options for \"Merge Syntenic Blocks\" (``quota_align.py--merge``)\nand/or \"Syntenic Depth\" (``quota_align.py --quota``).\n\nInstallation\n------------\n\n- Download the most recent codes at::\n\n    git clone http://github.com/tanghaibao/quota-alignment.git\n\n**Required dependencies**\n\n- Python version \u003e=2.7\n\n- GNU linear programming kit `GLPK \u003chttp://www.gnu.org/software/glpk/\u003e`_.\n\nPlease put the executable ``glpsol`` on the ``PATH``.\n\n**Optional dependencies**\n\n- `SCIP \u003chttp://scip.zib.de/download.shtml\u003e`_.\n\nFaster integer programming solver, choose the binary (32-bit, 64-bit) that fits\nyour machine. SCIP might have dependency on `LAPACK \u003chttp://www.netlib.org/lapack/\u003e`_,\nwhich needs to be installed too. Please rename the executable ``scip`` and put it on the\n``PATH``, for example::\n\n    sudo cp scip-x.x.0.linux.x86_64 /usr/local/bin/scip\n    sudo chmod +x !$\n\n- `bx-python \u003chttp://bitbucket.org/james_taylor/bx-python/wiki/Home\u003e`_ package.\n\nThis is only required when user wants to analyze ``.maf`` formatted data (use\n``maf_utils.py``)::\n\n    easy_install bx-python\n\n- `BCBio \u003chttp://github.com/chapmanb/bcbb/tree/master/gff/BCBio/\u003e`_ package.\n\nThis is only required when user wants to convert ``.gff`` file to ``.bed``\nformat, see section `Pre- and post-processing`_.\n\n\nCookbook\n-------------------------\nDefault package comes with the test data for case 1 and 2 in ``run.sh``. More\ntest data set can be downloaded `here\n\u003chttp://chibba.agtec.uga.edu/duplication/index/downloads/\u003e`_.\nUnpack into the folder, and execute ``run.sh``.\n\nBLAST anchors chaining and quota-based screening\n::::::::::::::::::::::::::::::::::::::::::::::::::::\nFirst you need to figure out a way to convert the BLAST result into the\nfollowing format (called ``.raw`` format), see section `Pre- and\npost-processing`_, in particular on filtering BLAST output::\n\n    1       6       1       4848    12\n    1       7       1       4847    10\n    1       8       1       4847    50\n    1       9       1       4846    14\n\nWhere the five columns correspond to ``chr1``, ``pos1``, ``chr2``, ``pos2``,\nand ``-log10(E-value)``. Then we can do something like::\n\n    quota_align.py --format=raw --merge --Dm=20 --min_size=5 --quota=2:1 maize_sorghum.qa\n\n``--merge`` asks for chaining, distance cutoff ``--Dm=20`` for extending the\nchain, ``--min_size=5`` for keeping the chains that are long enought;\n``--quota=2:1`` turns on the quota-based screening (and asks for two-to-one\nmatch, in this case, lineage specific WGD in maize genome, make every **2**\nmaize region matching **1** sorghum region). Note that if you set the quota\nwrong, e.g. suppose you don't know the quota ratio between maize and sorghum,\nand you typed ``1:1``, you will see the ``coverage`` reports to be too low::\n\n    write (134) clusters to 'data/maize_sorghum.qa.filtered'\n    genome X coverage: 62.6%\n    genome Y coverage: 97.5%\n\nIn this case, genome X (maize) has only slightly over half of the genome\naligned, missing the duplicated counterpart.\n\nBLASTZ anchors chaining and quota-based screening\n:::::::::::::::::::::::::::::::::::::::::::::::::::::\nMost often you will have the ``.maf`` file. First convert it to ``.qa`` format::\n\n    cluster_utils.py --format=maf athaliana_lyrata.maf athaliana_lyrata.qa\n\nThen you want to do the chaining and the screening in one step::\n\n    quota_align.py --merge --Dm=20000 --quota=1:1 --Nm=40000 athaliana_lyrata.qa\n\n``--merge`` asks for chaining, and the distance cutoff ``--Dm=20000`` for\nextending the chain; ``--quota=1:1`` turns on the quota-based screening (and\nasks for one-to-one match), and the overlap cutoff ``--Nm=40000``. The reason\nto specify an overlap cutoff is because the quota-based screening is based on\n1D block overlap. Sometimes due to the over-chaining, two blocks will only\n*slightly* overlap. Therefore the distance ``40000`` is how much *slight*\noverlap we tolerate.\n\nFinally you can get the screened ``.maf`` file by doing::\n\n    maf_utils.py athaliana_lyrata.qa athaliana_lyrata.maf\n\nYour final screened ``.maf`` file is called ``athaliana_lyrata.maf.filtered``.\nHint: you can compare the original and filtered ``.maf`` using Miller lab's\n`Gmaj \u003chttp://globin.cse.psu.edu/dist/gmaj/\u003e`_ tool.\n\nFind quota-screened paralogous blocks\n:::::::::::::::::::::::::::::::::::::::::\nFirst we need to figure out how to get the input data. See the last two\nsections for preparing data from BLAST and BLASTZ. Then we can do something\nlike the following::\n\n    quota_align.py --format=raw --merge --Dm=20 --min_size=5 --self --quota=2:2 grape_grape.raw\n\nThe reason for setting up ``--quota=2:2`` is because grape has\n`paleo-hexaploidy event\n\u003chttp://www.nature.com/nature/journal/v449/n7161/full/nature06148.html\u003e`_.\nTherefore many regions will have 3 copies, but we need to remove the self\nmatch. Therefore we should do ``2:2`` instead. ``--self`` option may be turned\non for finding paralogous blocks, when you have reduced the redundancies in\nyour ``.qa`` file (note that self-match is symmetric across diagonal). The\nreason for that is in the self-matching case, the constraints on the union of\nthe constraints on **both** axis, rather than on each axis separately.\n\nFor a lineage that has tetraploidy event (genome doubling), using the example\nof brachypodium (which has undergone an ancient pan-grass tetraploidy), we can\ndo::\n\n    quota_align.py --format=raw --merge --Dm=20 --self --quota=1:1 brachy_brachy.raw\n\nNote in this case, ``--quota=1:1`` since we have most regions in 2 copies, but\nwe need to ignore the self match. Therefore the rule is when searching\nparalogous blocks (always do ``--quota=x:x``, where ``x`` is the multiplicity\nminus 1).\n\nFormat block order for GRIMM analysis\n:::::::::::::::::::::::::::::::::::::\nThis is so far only supported when ``--quota=1:1``. For example::\n\n    quota_align.py --merge --quota=1:1 athaliana_lyrata.qa\n    cluster_utils.py --print_grimm athaliana_lyrata.qa.filtered\n\nThe script will print this::\n\n    \u003egenome X\n    1 2 3 4 5 6 7 8 9 10 11$\n    12 13 14 15 16 17 18 19$\n    20 21 22 23 24 25 26 27 28 29 30 31$\n    32 33 34 35 36$\n    37 38 39 40 41$\n    42 43 44 45 46 47 48 49 50$\n    51 52 53 54 55 56 57 58$\n    59 60 61$\n    62 63$\n    \u003egenome Y\n    -1 2 -3 4 -6 -7 5 8 10 9 11 -14 13 -12 15 16 17 18 -19$\n    37 38 24 -25 26 29 28 -30 -27 31 32 33 -34 35 36$\n    -21 -20 22 23 39 40 41$\n    -50 49 -48 44 46 -45 47 63 -62 -55 -54 53 -52 51$\n    -42 43 56 57 -58 -59 60 -61$\n\nThis is the input format for Glenn Tesler's `GRIMM\n\u003chttp://grimm.ucsd.edu/GRIMM/\u003e`_ software. You can either run it locally or on\ntheir `website \u003chttp://nbcr.sdsc.edu/GRIMM/grimm.cgi\u003e`_.\n\n\nPre- and post-processing\n------------------------------------\nThere are a few utility scripts included in ``scripts/`` folder.\n\nGFF to BED\n::::::::::::::::::::\nMost annotation groups only provide ``.gff`` file (see `gff format\n\u003chttp://genome.ucsc.edu/FAQ/FAQformat.html#format3\u003e`_) for the annotation of\ngene models. I often convert the ``.gff`` file to a simpler ``.bed`` format\n(see `bed format \u003chttp://genome.ucsc.edu/FAQ/FAQformat.html#format1\u003e`_). You\ncan do the following to create the ``.bed`` file (``BCBio`` module required)::\n\n    gff_to_bed.py athaliana.gff \u003eathaliana.bed\n\nThis will get protein-coding models and put these in the ``.bed`` format.\n``.gff`` file must be **gff3-compatible**, otherwise you have to write\ncustomized parser (in fact, this is recommended as most ``.gff`` file for\ngenome projects are not compatible). ``.bed`` format is required for doing\nBLAST filtering, see below.\n\nBLAST filtering\n::::::::::::::::::::\nThe integer programming solver cannot solve large problem instance (say \u003e60000\nvariables), this mostly will not happen if we filter our anchors carefully\n(removing redundant and weak anchors). To filter the BLAST results before\nchaining, using the ``blast_to_raw.py`` shipped in this package. Say you have\nBLAST file (tabular format) ready. You need to do::\n\n    blast_to_raw.py athaliana_grape.blastp --qbed=athaliana.bed --sbed=grape.bed --tandem_Nmax=10 --cscore=.5\n\nThis will convert the BLAST file into the ``.raw`` formatted file that\n``quota_align.py`` can understand (use ``--format=raw``). For your convenience,\nseveral BLAST filters are also implemented in ``blast_to_raw.py``. Notice these\nBLAST filters are **optional**.\n\n- Remove local dups\n\nOption ``--tandem_Nmax=10`` will group the local dups that are within 10 gene\ndistance. When this option is on, ``blast_to_raw.py`` will\nwrite new ``.nolocaldups.bed`` file, these will substitute your original\n``.bed`` file from now on.\n\n- Remove repetitive matches\n\nFor genes that have many hits, we will adjust the evalue::\n\n    adjusted_evalue(A, B) = evalue(A, B) ** ((counts_of_blast / counts_of_genes) / (counts(A) + counts(B)))\n\n- Use the cscore filtering\n\nOption ``--cscore=.5`` will keep only the hits that have a good score.\nSee reference for cscore in the supplementary of `sea anemone\npaper \u003chttp://www.sciencemag.org/cgi/content/abstract/317/5834/86\u003e`_. C-score\nbetween gene A and B is defined::\n\n    cscore(A, B) = score(A, B)/max(best score of A, best score of B)\n\nTypically, after the ``blast_to_raw.py``, we can do the ``quota_align.py``\ndirectly::\n\n    quota_align.py --format=raw --merge --Dm=20 --min_size=5 --quota=4:1 athaliana_grape.raw\n\nPlot dot plot\n:::::::::::::::::::::\nTo visualize the ``quota-align.py`` result, all you need is the\n``.qa.filtered`` result, and two ``.bed`` file (remember if you have removed\nlocal dups above, make sure you use the ``.nolocaldups.bed``). As an example::\n\n    qa_plot.py --qbed=athaliana.nolocaldups.bed --sbed=grape.nolocaldups.bed athaliana_grape.qa.filtered\n\nThis will generate a dot plot that you can stare to spot any problem. Below is\nan example of athaliana-grape dot plot when quota of ``4:1`` is enforced\n(meaning that there are expected ``4`` athaliana regions mapping to ``1`` grape\nregion).\n\n.. image:: http://lh3.ggpht.com/_srvRoIok9Xs/S6gz7Plyw-I/AAAAAAAAA2s/koz29tPJt8M/s800/athaliana_grape.qa.png\n    :alt: sample dotplot\n\nThe result of quota-based screening can be compared to the raw blast result.\nUsing the ``blast_plot.py`` in ``script`` folder. The syntax is similar to\n``qa_plot``, only on differernt input format::\n\n    blast_plot.py --qbed=athaliana.bed --sbed=grape.bed athaliana_grape.blastp\n\nSummary\n--------\nThe following is shell script ``run.sh`` that can be used from a BLAST output\nto the dot plot figure. Please note that you need to modify the path and params\nto make it work on your machine::\n\n    #!/bin/bash\n\n    # quota-alignment folder\n    QA=${HOME}/projects/quota-alignment/\n    # query species\n    SA=brapa\n    ### target species\n    SB=athaliana\n\n    # filter blast results (note that it needs to be tab-delimited blast m8 format)\n    ${QA}/scripts/blast_to_raw.py ../blast/${SA}_${SB}.blastz --qbed=${SA}.bed --sbed=${SB}.bed --tandem_Nmax=10 --cscore=0.7\n    # run the quota-based screening\n    ${QA}/quota_align.py --format=raw --merge --Dm=30 --min_size=5 --quota=3:1 ../blast/${SA}_${SB}.raw\n    # visualize result as dot plot\n    ${QA}/scripts/qa_plot.py --qbed=${SA}.nolocaldups.bed --sbed=${SB}.nolocaldups.bed ../blast/${SA}_${SB}.raw.filtered\n\n\nReference\n---------\nTang et al. (2011) *Screening synteny blocks in pairwise genome comparisons\nthrough integer programming.* [ `BMC Bioinformatics\n\u003chttp://www.biomedcentral.com/1471-2105/12/102\u003e`_ ]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftanghaibao%2Fquota-alignment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftanghaibao%2Fquota-alignment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftanghaibao%2Fquota-alignment/lists"}