{"id":26701461,"url":"https://github.com/clavellab/pocpbenchmark","last_synced_at":"2026-01-27T08:02:51.612Z","repository":{"id":283478365,"uuid":"943330528","full_name":"ClavelLab/pocpbenchmark","owner":"ClavelLab","description":"Nextflow workflow for benchmarking proteins alignment tools for improved genus delineation using the Percentage Of Conserved Proteins (POCP) ","archived":false,"fork":false,"pushed_at":"2025-11-15T12:58:42.000Z","size":2212,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-30T22:54:55.484Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Nextflow","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ClavelLab.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-05T14:35:46.000Z","updated_at":"2025-11-15T12:58:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"bbc32b41-c1d4-4436-baeb-6b9a90afb5c0","html_url":"https://github.com/ClavelLab/pocpbenchmark","commit_stats":null,"previous_names":["clavellab/pocpbenchmark"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/ClavelLab/pocpbenchmark","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClavelLab%2Fpocpbenchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClavelLab%2Fpocpbenchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClavelLab%2Fpocpbenchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClavelLab%2Fpocpbenchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ClavelLab","download_url":"https://codeload.github.com/ClavelLab/pocpbenchmark/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClavelLab%2Fpocpbenchmark/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28809336,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T07:41:26.337Z","status":"ssl_error","status_checked_at":"2026-01-27T07:41:08.776Z","response_time":168,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-27T01:20:01.198Z","updated_at":"2026-01-27T08:02:51.607Z","avatar_url":"https://github.com/ClavelLab.png","language":"Nextflow","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Introduction\n\n**TL;DR**: `ClavelLab/pocpbenchmark` is a bioinformatics best-practice analysis pipeline for benchmarking proteins alignment tools for improved genus delineation using the Percentage Of Conserved Proteins (POCP).\n\nGenus delineation can be done using Percentage Of Conserved Proteins (POCP), but the original implementation ([Qin, Q.L et al. (2014). *J Bacteriol*](https://doi.org/10.1128/JB.01688-14)) using BLASTP is slow.\nHere we benchmark here different tools that should be faster than the BLASTP implementation, but we will first evaluate whether the accurracy of the POCP calculation is not sacrificed in the name of computational performance. We rely on the curated taxonomy and the publicly available genomes of the [Genome Taxonomy Database](https://gtdb.ecogenomic.org/).\n\nThe pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from [nf-core/modules](https://github.com/nf-core/modules) in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!\n\nOur work is published in _PeerJ_\n\u003e Pauvert C, Hitch TCA, Clavel T. 2025. Fast and robust estimate of bacterial genus novelty using the percentage of conserved proteins with unique matches (POCPu) PeerJ 13:e20259 https://doi.org/10.7717/peerj.20259\n\n## Pipeline summary\n\n1. Shorlist the genomes of the GTDB that have a valid name, are representative genome, belongs to a family with at least two genera, and to a genus with at least ten genomes.\n2. Compute the total numbers of proteins for each of the shortlisted proteomes makde available by the GTDB.\n3. Create a list of many-versus-many proteins comparisons to be ran within bacterial family and never with itself.\n4. Run the actual sequence comparisons using the following tools: blastp (with and without database creation), diamond and MMseqs2. \n5. Compute two types of POCP for each comparison and each tool based on different filtering strategies of the matches:\n    - POCP: using only matches with an e-value \u003c 1e−5, a sequence identity \u003e 40%, and an alignable region of the query protein sequence \u003e 50%. This is the original implementation described in the paper and used in [Protologger](https://github.com/thh32/Protologger/) for instance.\n    - POCPu: same as POCP but using only the unique matches of the query sequences, in the same manner as Martin Hölzer [POCP](https://github.com/hoelzer/pocp/tree/1.1.1) implementation.\n6. Plot the resulting POCP values against the expected POCP values from blastp\n7. Evaluate the genus delineation potential of different POCP implementation based on the GTDB taxonomy of the shortlisted genomes.\n\n\n## How to run the benchmark\n\nDue to exponentially large numbers of pairwise comparisons and Java Heap Space errors, we made the decision to run the benchmark within family.\nTherefore, a preparatory phase of the workflow is needed to list the necessary shortlisted genomes by family.\n\n### Preparatory workflow\n\n\n1. Install [`Nextflow`](https://www.nextflow.io/docs/latest/getstarted.html#installation) (`\u003e=22.10.1`)\n\n2. Install any of [`Docker`](https://docs.docker.com/engine/installation/), [`Singularity`](https://www.sylabs.io/guides/3.0/user-guide/) (you can follow [this tutorial](https://singularity-tutorial.github.io/01-installation/)), [`Podman`](https://podman.io/), [`Shifter`](https://nersc.gitlab.io/development/shifter/how-to-use/) or [`Charliecloud`](https://hpc.github.io/charliecloud/) for full pipeline reproducibility _(you can use [`Conda`](https://conda.io/miniconda.html) both to install Nextflow itself and also to manage software within pipelines. Please only use it within pipelines as a last resort; see [docs](https://nf-co.re/usage/configuration#basic-configuration-profiles))_.\n\n3. Download the workflow\n\n   ```bash\n   nextflow pull ClavelLab/pocpbenchmark\n   ```\n\n4. Ensure the correct configuration of the workflow\n\nNote that some form of configuration will be needed so that Nextflow knows how to fetch the required software. This is usually done in the form of a config profile (`YOURPROFILE` in the example command above). You can chain multiple config profiles in a comma-separated string.\n\n   \u003e - The pipeline comes with config profiles called `docker`, `singularity`, `podman`, `shifter`, `charliecloud` and `conda` which instruct the pipeline to use the named tool for software management. For example, `-profile test,docker`.\n   \u003e - Please check [nf-core/configs](https://github.com/nf-core/configs#documentation) to see if a custom config file to run nf-core pipelines already exists for your Institute. If so, you can simply use `-profile \u003cinstitute\u003e` in your command. This will enable either `docker` or `singularity` and set the appropriate execution settings for your local compute environment.\n   \u003e - If you are using `singularity`, please use the [`nf-core download`](https://nf-co.re/tools/#downloading-pipelines-for-offline-use) command to download images first, before running the pipeline. Setting the [`NXF_SINGULARITY_CACHEDIR` or `singularity.cacheDir`](https://www.nextflow.io/docs/latest/singularity.html?#singularity-docker-hub) Nextflow options enables you to store and re-use the images from a central location for future pipeline runs.\n   \u003e - If you are using `conda`, it is highly recommended to use the [`NXF_CONDA_CACHEDIR` or `conda.cacheDir`](https://www.nextflow.io/docs/latest/conda.html) settings to store the environments in a central location for future pipeline runs.\n\n\nNote that the default parameters should be the following.\n\n```\nparams {\n   gtdb_proteins_dir = \"https://data.gtdb.ecogenomic.org/releases/release214/214.0/genomic_files_reps/gtdb_proteins_aa_reps_r214.tar.gz\"\n   gtdb_metadata_tsv = \"https://data.gtdb.ecogenomic.org/releases/release214/214.0/bac120_metadata_r214.tar.gz\"\n   valid_names_tsv   = \"https://raw.githubusercontent.com/thh32/Protologger/master/DSMZ-latest.tab\"\n\n}\n```\nNote that if you already have GTDB database downloaded, you should specifify the path to the data, see [`winogradsky.config`](conf/winogradsky.config) for an example.\n\n5. Run the preparatory workflow\n\n```bash\nnextflow run main.nf -entry PREPBENCHMARK -profile winogradsky,docker --outdir benchmark-gtdb\n```\n\nThis workflow will create the file `benchmark-gtdb/families_to_run.txt`, which contains the Nextflow commands to be run for each family in the shortlist.\nThe file contains for instance the command below:\n\n```bash\nnextflow run main.nf -entry POCPBENCHMARK -profile winogradsky,docker --outdir benchmark-gtdb-f__Alteromonadaceae --family_shortlist benchmark-gtdb/split_per_family/f__Alteromonadaceae.csv\n```\n\nEspecially, the file already specific correctly the name of the output directory (`--outdir benchmark-gtdb-f__Alteromonadaceae`) as well as the shortlist of genomes for the family (`--family_shortlist benchmark-gtdb/split_per_family/f__Alteromonadaceae.csv`)\n\n\n### Benchmark workflow\n\n#### Full benchmark\n\nThe full benchmark workflow will then run all 10 methods for the selected family.\n\n```bash\n# Optional: git checkout master\nnextflow run main.nf -entry POCPBENCHMARK -profile winogradsky,docker --outdir benchmark-gtdb-f__Alteromonadaceae --family_shortlist benchmark-gtdb/split_per_family/f__Alteromonadaceae.csv\nnextflow run main.nf -entry POCPBENCHMARK -profile winogradsky,docker --outdir benchmark-gtdb-f__Streptomycetaceae --family_shortlist benchmark-gtdb/split_per_family/f__Streptomycetaceae.csv\n```\n\n\n#### Only recommended method\n\nA leaner workflow can be run with only the recommended method from our analysis: DIAMOND_VERYSENSITIVE.\n\n```bash\ngit checkout only-pocp-replacement\nnextflow run main.nf -entry POCPBENCHMARK -profile winogradsky,docker --outdir benchmark-gtdb-f__Alteromonadaceae --family_shortlist benchmark-gtdb/split_per_family/f__Alteromonadaceae.csv\n```\n\n### Selected output files for the analyses\n\nA selection of output files generated by the workflow are used downstream for the analyses.\nThese files are encapsulated in an archive with the command below:\n\n```bash\n./make_pocp_results_archive.sh benchmark-gtdb-f__Alteromonadaceae\n```\n\nThis command will create `benchmark-gtdb-f__Alteromonadaceae.zip`, and can be applied to all the families that were run for the benchmark.\nAll the archives created for the article are made available on Zenodo at: \u003chttps://doi.org/10.5281/zenodo.14974869\u003e\n\n### Analyses, figures and manuscript workflow\n\nThe code for the analyses, figure creations and manuscript is available at: \u003chttps://github.com/ClavelLab/pocpbenchmark_manuscript\u003e\n\n\n## Credits\n\nClavelLab/pocpbenchmark was originally written by [Charlie Pauvert](https://github.com/cpauvert) and [Thomas C.A. Hitch](https://github.com/thh32).\n\n\n## Citations\n\nAn extensive list of references for the tools used by the pipeline can be found in the [`CITATIONS.md`](CITATIONS.md) file.\n\nThis pipeline uses code and infrastructure developed and maintained by the [nf-core](https://nf-co.re) community, reused here under the [MIT license](https://github.com/nf-core/tools/blob/master/LICENSE).\n\n\u003e **The nf-core framework for community-curated bioinformatics pipelines.**\n\u003e\n\u003e Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso \u0026 Sven Nahnsen.\n\u003e\n\u003e _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclavellab%2Fpocpbenchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclavellab%2Fpocpbenchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclavellab%2Fpocpbenchmark/lists"}