{"id":38728742,"url":"https://github.com/mazzalab/fastqwiper","last_synced_at":"2026-01-17T11:22:41.276Z","repository":{"id":57428671,"uuid":"325529896","full_name":"mazzalab/fastqwiper","owner":"mazzalab","description":"An ensemble method to recover corrupted FASTQ files, drop or fix pesky lines, remove unpaired reads, and settle reads interleaving.","archived":false,"fork":false,"pushed_at":"2025-01-30T16:31:03.000Z","size":912,"stargazers_count":29,"open_issues_count":3,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-23T04:57:54.060Z","etag":null,"topics":["bioinformatics","corrupted","fastq","fix","ngs","recovery"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mazzalab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-30T11:12:41.000Z","updated_at":"2025-06-26T20:49:48.000Z","dependencies_parsed_at":"2023-10-16T10:10:42.199Z","dependency_job_id":"c7d11411-e908-424f-870e-762c7eb5bc5a","html_url":"https://github.com/mazzalab/fastqwiper","commit_stats":{"total_commits":192,"total_committers":2,"mean_commits":96.0,"dds":0.01041666666666663,"last_synced_commit":"b2bddca4b4cb88807cf3919d7fe09fdca76cd933"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/mazzalab/fastqwiper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzalab%2Ffastqwiper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzalab%2Ffastqwiper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzalab%2Ffastqwiper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzalab%2Ffastqwiper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mazzalab","download_url":"https://codeload.github.com/mazzalab/fastqwiper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazzalab%2Ffastqwiper/sbom","scorecard":{"id":630645,"data":{"date":"2025-08-11","repo":{"name":"github.com/mazzalab/fastqwiper","commit":"f9324b3db1597cf49259e6b36a7e4ec0b4d6937e"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.4,"checks":[{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":0,"reason":"Found 1/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/buildall_and_publish.yml:1","Warn: no topLevel permission defined: .github/workflows/code_coverage.yml:1","Warn: no topLevel permission defined: .github/workflows/conda_reusable.yml:1","Warn: no topLevel permission defined: .github/workflows/docker_reusable.yml:1","Warn: no topLevel permission defined: .github/workflows/pypi_reusable.yml:1","Warn: no topLevel permission defined: .github/workflows/singularity_reusable.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU General Public License v3.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/code_coverage.yml:18: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/code_coverage.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/code_coverage.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/code_coverage.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/code_coverage.yml:36: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/code_coverage.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/conda_reusable.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/conda_reusable.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/conda_reusable.yml:29: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/conda_reusable.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/docker_reusable.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/docker_reusable.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/docker_reusable.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/docker_reusable.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/docker_reusable.yml:27: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/docker_reusable.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/docker_reusable.yml:30: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/docker_reusable.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/docker_reusable.yml:36: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/docker_reusable.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/pypi_reusable.yml:25: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/pypi_reusable.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/pypi_reusable.yml:33: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/pypi_reusable.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/singularity_reusable.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/singularity_reusable.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/singularity_reusable.yml:18: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/singularity_reusable.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/singularity_reusable.yml:25: update your workflow using https://app.stepsecurity.io/secureworkflow/mazzalab/fastqwiper/singularity_reusable.yml/main?enable=pin","Warn: containerImage not pinned by hash: .devcontainer/Dockerfile:1: pin your Docker image by updating condaforge/mambaforge to condaforge/mambaforge@sha256:050753d138b6708128c9bb45055a9d943f06363f32f63a96f7dd5f304ada52a6","Info:   0 out of   5 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of  10 third-party GitHubAction dependencies pinned","Info:   0 out of   1 containerImage dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 4 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-21T07:49:28.695Z","repository_id":57428671,"created_at":"2025-08-21T07:49:28.695Z","updated_at":"2025-08-21T07:49:28.695Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28506915,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T10:25:30.148Z","status":"ssl_error","status_checked_at":"2026-01-17T10:25:29.718Z","response_time":85,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","corrupted","fastq","fix","ngs","recovery"],"created_at":"2026-01-17T11:22:40.547Z","updated_at":"2026-01-17T11:22:41.252Z","avatar_url":"https://github.com/mazzalab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FastqWiper \n[![Build](https://github.com/mazzalab/fastqwiper/actions/workflows/buildall_and_publish.yml/badge.svg)](https://github.com/mazzalab/fastqwiper/actions/workflows/buildall_and_publish.yml) [![codecov](https://codecov.io/gh/mazzalab/fastqwiper/graph/badge.svg?token=5V4AQTK619)](https://codecov.io/gh/mazzalab/fastqwiper) [![GitHub issues](https://img.shields.io/github/issues-raw/mazzalab/fastqwiper)](https://github.com/mazzalab/fastqwiper/issues) \n\n[![Anaconda-Server Badge](https://anaconda.org/bfxcss/fastqwiper/badges/version.svg)](https://anaconda.org/bfxcss/fastqwiper) [![Anaconda-Server Badge](https://anaconda.org/bfxcss/fastqwiper/badges/latest_release_date.svg)](https://anaconda.org/bfxcss/fastqwiper) [![Anaconda-Server Badge](https://anaconda.org/bfxcss/fastqwiper/badges/platforms.svg)](https://anaconda.org/bfxcss/fastqwiper) [![Anaconda-Server Badge](https://anaconda.org/bfxcss/fastqwiper/badges/downloads.svg)](https://anaconda.org/bfxcss/fastqwiper)\n\n[![PyPI version](https://badge.fury.io/py/fastqwiper.svg)](https://badge.fury.io/py/fastqwiper) [![PyPI pyversions](https://img.shields.io/pypi/pyversions/fastqwiper.svg)](https://pypi.python.org/pypi/fastqwiper/) ![PyPI - Downloads](https://img.shields.io/pypi/dm/fastqwiper)\n\n[![Docker](https://badgen.net/badge/icon/docker?icon=docker\u0026label)](https://hub.docker.com/r/mazzalab/fastqwiper) ![Docker Pulls](https://img.shields.io/docker/pulls/mazzalab/fastqwiper)\n\n`FastqWiper` **recovers** corrupted `fastq.gz`, **drops** or **fixes** pesky lines, **removes** unpaired reads, and **settles** reads interleaving in FASTQ files.\n\n* Compatibility: Python ≥3.10, \u003c3.13\n* OS: Windows, Linux, Mac OS (Snakemake workflows run in Windows only through Docker for Windows)\n* Contributions: [bioinformatics@css-mendel.it](bioinformatics@css-mendel.it)\n* Docker: https://hub.docker.com/r/mazzalab/fastqwiper\n* Singularity: https://cloud.sylabs.io/library/mazzalab/fastqwiper/fastqwiper.sif\n* Bug report: [https://github.com/mazzalab/fastqwiper/issues](https://github.com/mazzalab/fastqwiper/issues)\n\n\n## USAGE\n\u003cimg src=\"assets/fw_choice.svg\" width=\"450\"\u003e\u003c/img\u003e\n- \u003ccode style=\"color : greenyellow\"\u003e**Case 1.**\u003c/code\u003eYou have one or a couple (R1\u0026R2) of **computer readable** (meaning that the .gz files can be successfully decompressed or that the .fa/.fasta files can be viewed from the beginning to the EOF) FASTQ files which contain pesky, unformatted, uncompliant lines: Use *FastWiper* to clean them;\n- \u003ccode style=\"color : darkorange\"\u003e**Case 2.**\u003c/code\u003eYou have one or a couple (R1\u0026R2) of **computer readable** FASTQ files that you want to drop unpaired reads from or fix reads interleaving: Use the FastqWiper's *Snakemake workflows*;\n- \u003ccode style=\"color : orangered\"\u003e**Case 3.**\u003c/code\u003eYou have one `fastq.gz` file or a couple (R1\u0026R2) of `fastq.gz` files which are corrupted (**unreadable**, meaning that the .gz files cannot be successfully decompressed) and you want to recover healthy reads and reformat them: Use the FastqWiper's *Snakemake workflows*;\n\n\n## Installation\n### \u003ccode style=\"color : greenyellow\"\u003eCase 1\u003c/code\u003e\nThis requires you to install FastqWiper and therefore \u003cu\u003enot\u003c/u\u003e to use *workflows*. You can do it for all OSs:\n\n#### Use Conda\n\n```\nconda create -n fastqwiper python=3.11\nconda activate fastqwiper\nconda install -c bfxcss -c conda-forge fastqwiper\n\nwipertools --help\n```\n*Hint: for an healthier experience, use* **mamba**\n\n\n#### Use Pypi\n```\npip install fastqwiper\n```\n\u003cbr/\u003e\n\n#### Usage\n\n`usage: wipertools [-h] {fastqwiper,splitfastq,summarygather} ...`\n```\npositional arguments:\n    fastqwiper          FastqWiper program\n    splitfastq          FASTQ splitter program\n    summarygather       Gatherer of the FastqWiper summaries\n\noptions:\n  -h, --help            show this help message and exit\n```\n```\nusage: wipertools fastqwiper [-h] -i FASTQ_IN -o FASTQ_OUT [-l [LOG_OUT]] [-f [LOG_FREQUENCY]] [-a [ALPHABET]]\n\noptions:\n  -i, --fastq_in TEXT          The input FASTQ file to be cleaned  [required]\n  -o, --fastq_out TEXT         The wiped FASTQ file                [required]\n  -l, --log_frequency INTEGER  The number of reads you want to print a status message. Default: 500000\n  -f, --log_out TEXT           The file name of the final quality report summary. Print on the screen if not specified\n  -a, --alphabet               Allowed character in the SEQ line. Default: ACGTN\n  -h, --help                   Show this message and exit.\n```\n\u003cbr/\u003e\nFastqWiper accepts \u003cb\u003estrictly readable\u003c/b\u003e `*.fastq` or `*.fastq.gz` files in input.\n\n\n### \u003ccode style=\"color : darkorange\"\u003eCase 2\u003c/code\u003e \u0026 \u003ccode style=\"color : orangered\"\u003eCase 3\u003c/code\u003e\nThere are \u003cb\u003eQUICK\u003c/b\u003e and a \u003cb\u003eSLOW\u003c/b\u003e methods to configure `FastqWiper`'s workflows.\n\n\n#### One quick way (Docker)\n1. Pull the Docker image from DockerHub:\n\n`docker pull mazzalab/fastqwiper`\n\n2. Once downloaded the image, type:\n\nCMD: `docker run --rm -ti --name fastqwiper -v \"YOUR_LOCAL_PATH_TO_DATA_FOLDER:/fastqwiper/data\" mazzalab/fastqwiper paired 8 sample 33 ACGTN 500000`\n\n#### Another quick way (Singularity)\n1. Pull the Singularity image from the Cloud Library:\n\n`singularity pull library://mazzalab/fastqwiper/fastqwiper.sif`\n\n2. Once downloaded the image (e.g., fastqwiper.sif_2024.2.104.sif), type:\n\nCMD `singularity run --bind YOUR_LOCAL_PATH_TO_DATA_FOLDER:/fastqwiper/data --writable-tmpfs fastqwiper.sif_2024.2.104.sif paired 8 sample 33 ACGTN 500000`\n\nIf you want to bind the `.singularity` cache folder and the `logs` folder, you can omit `--writable-tmpfs`, create the folders `.singularity` and `logs` (`mkdir .singularity logs`) on the host system, and use this command instead:\n\nCMD: `singularity run --bind YOUR_LOCAL_PATH_TO_DATA_FOLDER/:/fastqwiper/data --bind YOUR_LOCAL_PATH_TO_.SNAKEMAKE_FOLDER/:/fastqwiper/.snakemake --bind YOUR_LOCAL_PATH_TO_LOGS_FOLDER/:/fastqwiper/logs fastqwiper.sif_2024.2.104.sif paired 8 sample 33 ACGTN 500000`\n\nFor both **Docker** and **Singularity**:\n\n- `YOUR_LOCAL_PATH_TO_DATA_FOLDER` is the path of the folder where the fastq.gz files to be wiped are located;\n- `paired` triggers the cleaning of R1 and R2. Alternatively, `single` will trigger the wipe of individual FASTQ files;\n- `8` is the number of your choice of computing cores to be spawned (1 = triggers sequential execution; \u003e1 triggers parallel execution)\n- `sample` is part of the names of the FASTQ files to be wiped. \u003cb\u003eBe aware\u003c/b\u003e that: for \u003cb\u003epaired-end\u003c/b\u003e files (e.g., \"sample_R1.fastq.gz\" and \"sample_R2.fastq.gz\"), your files must finish with `_R1.fastq.gz` and `_R2.fastq.gz`. Therefore, the argument to pass is everything before these texts: `sample` in this case. For \u003cb\u003esingle end\u003c/b\u003e/individual files (e.g., \"excerpt_R1_001.fastq.gz\"), your file must end with the string `.fastq.gz`; the preceding text, i.e., \"excerpt_R1_001\" in this case, will be the text to be passed to the command as an argument. \n- `33` (optional) is the ASCII offset (33=Sanger, 64=old Solexa)\n- `ACGTN` (optional) is the allowed alphabet in the SEQ line of the FASTQ file\n- `500000` (optional) is the log frequency (# reads)\n\n### \u003ccode style=\"color : red\"\u003eThe slow way (Linux \u0026 Mac OS)\u003c/code\u003e\nTo enable the use of preconfigured [pipelines](https://github.com/mazzalab/fastqwiper/tree/main/pipeline), you need to install **Snakemake**. The recommended way to install Snakemake is via Conda, because it enables **Snakemake** to [handle software dependencies of your workflow](https://snakemake.readthedocs.io/en/stable/snakefiles/deployment.html#integrated-package-management).\nHowever, the default conda solver is slow and often hangs. Therefore, we recommend installing [Mamba](https://github.com/mamba-org/mamba) as a drop-in replacement via\n\n`conda install -c conda-forge mamba`\n\nif you have anaconda/miniconda already installed, or directly installing `Mambaforge` as described [here](https://github.com/conda-forge/miniforge#mambaforge).\n\n\nThen, create and activate a clean environment as above:\n\n```\nmamba create -n fastqwiper python=3.11\nmamba activate fastqwiper\n```\nFinally, install the Snakemake dependency:\n\n```\nmamba install -c bioconda snakemake\n```\n\n\n#### Usage\nClone the FastqWiper repository in a folder of your choice and enter it:\n\n```\ngit clone https://github.com/mazzalab/fastqwiper.git\ncd fastqwiper\n```\n\nIt contains, in particular, a folder `data` containing the fastq files to be processed, a folder `pipeline` containing the released pipelines and a folder `fastqwiper` with the source files of `FastqWiper`. \u003cbr/\u003e\nInput files to be processed must be copied into the **data** folder.\n\nCurrently, to run the `FastqWiper` pipelines, the following packages need to be installed manually:\n\n### required packages:\n[gzrt](https://github.com/arenn/gzrt) (Linux build from source [instructions](https://github.com/arenn/gzrt/blob/master/README.build), Ubuntu install [instructions](https://howtoinstall.co/en/gzrt), Mac OS install [instructions](https://formulae.brew.sh/formula/gzrt))\n\n[BBTools](https://jgi.doe.gov/data-and-tools/software-tools/bbtools/) (install [instructions](https://jgi.doe.gov/data-and-tools/software-tools/bbtools/bb-tools-user-guide/installation-guide/))\n\nIf installed from source, `gzrt` scripts need to be put on PATH. `bbmap` must be installed in the root folder of FastqWiper, as the image below\n\n![FastqWiper folder yierarchy](assets/hierarchy.png)\n\n### Commands:\nCopy the fastq files you want to fix in the `data` folder.\n\n\u003ccode style=\"color : orange\"\u003e**N.b.**: In all commands above, you will pass the name of the sample to be analyzed to the workflow through the config argument: `sample_name`. Remember that your fastq files' names must finish with `_R1.fastq.gz` and `_R2.fastq.gz`, for paired fastq files, and with `.fastq.gz`, for individual fastq files, and, therefore, the text to be assigned to the variable `sample_name` must be everything before them. E.g., if your files are `my_sample_R1.fastq.gz` and `my_sample_R2.fastq.gz`, then `--config sample_name=my_sample`.\u003c/code\u003e\n\n\n#### Paired-end files\n\n- **Get a dry run** of a pipeline (e.g., `fix_wipe_pairs_reads_sequential.smk`):\u003cbr /\u003e\n`snakemake --config sample_name=my_sample qin=33 alphabet=ACGTN log_freq=1000 -s pipeline/fix_wipe_pairs_reads_sequential.smk --use-conda --cores 4 -np`\n\n- **Generate the planned DAG**:\u003cbr /\u003e\n`snakemake --config sample_name=my_sample qin=33 alphabet=ACGTN log_freq=1000 -s pipeline/fix_wipe_pairs_reads_sequential.smk --dag | dot -Tpdf \u003e dag.pdf`\u003cbr /\u003e \u003cbr /\u003e\n\u003cimg src=\"https://github.com/mazzalab/fastqwiper/blob/main/assets/dag_paired_sequential.svg?raw=true\" width=\"400\"\u003e\n\n- **Run the pipeline** (n.b., during the first execution, Snakemake will download and install some required remote packages and may take longer). The number of computing cores can be tuned accordingly:\u003cbr /\u003e\n`snakemake --config sample_name=my_sample alphabet=ACGTN log_freq=1000 -s pipeline/fix_wipe_pairs_reads_sequential.smk --use-conda --cores 2`\n\nFixed files will be copied in the `data` folder and will be suffixed with the string `_fixed_wiped_paired_interleaving`.\nWe remind that the `fix_wipe_pairs_reads_sequential.smk` and `fix_wipe_pairs_reads_parallel.smk` pipelines perform the following actions:\n- execute `gzrt` on corrupted fastq.gz files (i.e., that cannot be unzipped because of errors) and recover readable reads;\n- execute `FastqWiper` on recovered reads to make them compliant with the FASTQ format (source: [Wipipedia](https://en.wikipedia.org/wiki/FASTQ_format))\n- execute `Trimmomatic` on wiped reads to remove residual unpaired reads\n- execute `BBmap (repair.sh)` on paired reads to fix the correct interleaving and sort fastq files.  \n\n#### Single-end files\n`fix_wipe_single_reads_parallel.smk` and `fix_wipe_single_reads_sequential.smk` will not execute `trimmomatic` and BBmap's `repair.sh`.\n\n- **Get a dry run** of a pipeline (e.g., `fix_wipe_single_reads_sequential.smk`):\u003cbr /\u003e\n`snakemake --config sample_name=my_sample alphabet=ACGTN log_freq=1000 -s pipeline/fix_wipe_single_reads_sequential.smk --use-conda --cores 2 -np`\n\n- **Generate the planned DAG**:\u003cbr /\u003e\n`snakemake --config sample_name=my_sample alphabet=ACGTN log_freq=1000 -s pipeline/fix_wipe_single_reads_sequential.smk --dag | dot -Tpdf \u003e dag.pdf`\u003cbr /\u003e\u003cbr /\u003e\n\u003cimg src=\"https://github.com/mazzalab/fastqwiper/blob/main/assets/dag_single_sequential.svg?raw=true\" width=\"200\"\u003e\n\n- **Run the pipeline** (n.b., The number of computing cores can be tuned accordingly):\u003cbr /\u003e\n`snakemake --config sample_name=my_sample alphabet=ACGTN log_freq=1000 -s pipeline/fix_wipe_single_reads_sequential.smk --use-conda --cores 2`\n  \n# Author\n**Tommaso Mazza**  \n[![X](https://img.shields.io/badge/X-%23000000.svg?style=for-the-badge\u0026logo=X\u0026logoColor=white)](https://twitter.com/irongraft) [![LinkedIn](https://img.shields.io/badge/linkedin-%230077B5.svg?style=for-the-badge\u0026logo=linkedin\u0026logoColor=white)](https://www.linkedin.com/in/tommasomazza/)\n\nLaboratory of Bioinformatics\u003c/br\u003e\nFondazione IRCCS Casa Sollievo della Sofferenza\u003c/br\u003e\nViale Regina Margherita 261 - 00198 Roma IT\u003c/br\u003e\nTel: +39 06 44160526 - Fax: +39 06 44160548\u003c/br\u003e\nE-mail: t.mazza@operapadrepio.it\u003c/br\u003e\nWeb page: http://www.css-mendel.it\u003c/br\u003e\nWeb page: http://bioinformatics.css-mendel.it\u003c/br\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazzalab%2Ffastqwiper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmazzalab%2Ffastqwiper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazzalab%2Ffastqwiper/lists"}