{"id":26472658,"url":"https://github.com/getwilds/ww-sourmash","last_synced_at":"2025-08-21T07:11:08.112Z","repository":{"id":267754325,"uuid":"902245094","full_name":"getwilds/ww-sourmash","owner":"getwilds","description":"Self-contained WDL workflow that uses Sourmash to perform similarity searches between sequence files.","archived":false,"fork":false,"pushed_at":"2024-12-12T07:53:53.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-19T21:49:27.671Z","etag":null,"topics":["wdl-workflow","wilds-wdl"],"latest_commit_sha":null,"homepage":"","language":"WDL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/getwilds.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-12T07:35:55.000Z","updated_at":"2025-02-24T22:24:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"f1e2f2db-ef6d-4b46-b9b8-1012f17fdf4b","html_url":"https://github.com/getwilds/ww-sourmash","commit_stats":null,"previous_names":["getwilds/ww-sourmash"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/getwilds/ww-sourmash","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getwilds%2Fww-sourmash","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getwilds%2Fww-sourmash/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getwilds%2Fww-sourmash/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getwilds%2Fww-sourmash/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/getwilds","download_url":"https://codeload.github.com/getwilds/ww-sourmash/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getwilds%2Fww-sourmash/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271441676,"owners_count":24760343,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-21T02:00:08.990Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["wdl-workflow","wilds-wdl"],"created_at":"2025-03-19T21:44:08.350Z","updated_at":"2025-08-21T07:11:08.104Z","avatar_url":"https://github.com/getwilds.png","language":"WDL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ww-sourmash: Sourmash Sketch and Search Workflow\n[![Project Status: Experimental – Useable, some support, not open to feedback, unstable API.](https://getwilds.org/badges/badges/experimental.svg)](https://getwilds.org/badges/#experimental)\n\n## Overview\nThis workflow uses Sourmash to generate MinHash signatures for sequence files and perform similarity searches between them. It's implemented in WDL (Workflow Description Language) and designed to run with Cromwell or other WDL-compatible workflow engines.\n\n## Prerequisites\n- Cromwell or another WDL-compatible workflow engine\n- Docker (the workflow uses the `getwilds/sourmash:4.8.2` container)\n- Sufficient disk space for your sequence files and their signatures\n\n## Workflow Structure\nThe workflow consists of two main steps:\n1. **SketchBothSequences**: Generates MinHash signatures for both query and database sequences\n2. **RunSourmashSearch**: Performs similarity search between the generated signatures\n\n## Input Files\nCreate a JSON file (e.g., `inputs.json`) with the following structure:\n```json\n{\n    \"SourmashSketchAndSearch.query_fastq\": \"/path/to/query.fastq.gz\",\n    \"SourmashSketchAndSearch.database_fasta\": \"/path/to/database.fa.gz\"\n}\n```\n\n## Workflow Options\nCreate a JSON file (e.g., `options.json`) to configure workflow execution:\n```json\n{\n    \"workflow_failure_mode\": \"ContinueWhilePossible\",\n    \"write_to_cache\": true,\n    \"read_from_cache\": true,\n    \"default_runtime_attributes\": {\n        \"maxRetries\": 1\n    },\n    \"final_workflow_outputs_dir\": \"/path/to/outputs/\",\n    \"use_relative_output_paths\": true\n}\n```\n\n### Options Explained\n- `workflow_failure_mode`: Determines how the workflow handles task failures\n- `write_to_cache`: Enables caching of task outputs for future runs\n- `read_from_cache`: Allows reuse of cached outputs from previous runs\n- `maxRetries`: Number of times to retry failed tasks\n- `final_workflow_outputs_dir`: Directory for final workflow outputs\n- `use_relative_output_paths`: Maintains relative path structure in output directory\n\n## Default Parameters\nThe workflow includes several default parameters that can be overridden in your inputs JSON:\n- `ksize`: \"31\" (k-mer size for sketching and searching)\n- `threshold`: 0.08 (minimum similarity threshold for reporting matches)\n- `sketch_type`: \"dna\" (molecule type: dna, protein, dayhoff, hp, or nucleotide)\n- `scaled`: true (use scaled MinHash)\n- `scale_factor`: 1000 (scale factor for scaled MinHash)\n\n## Running the Workflow\n\n### With Cromwell\n```bash\njava -jar cromwell.jar run \\\n    sourmash-search-workflow.wdl \\\n    -i inputs.json \\\n    -o options.json\n```\n\n### Workflow Outputs\nThe workflow produces three main outputs:\n1. Query sequence signature file (`.sig`)\n2. Database sequence signature file (`.sig`)\n3. Search results file (`search_results.csv`)\n\n## Resource Requirements\nDefault resource allocations per task:\n- **SketchBothSequences**:\n  - Memory: 4GB\n  - CPU: 2\n  - Disk: 50GB SSD\n- **RunSourmashSearch**:\n  - Memory: 4GB\n  - CPU: 1\n  - Disk: 50GB SSD\n\nAdjust these values in the WDL file based on your input data sizes and computational resources.\n\n## Troubleshooting\n\n### Common Issues\n1. **Insufficient Disk Space**: Increase `disk_size_gb` in task runtime sections\n2. **Memory Errors**: Adjust `memory_gb` based on input file sizes\n3. **Docker Pulling Failures**: Ensure access to Docker Hub and correct image version\n\n### Task Retry Behavior\n- Tasks will retry once on failure (configurable via `maxRetries`)\n- The workflow continues executing possible tasks if one fails (`ContinueWhilePossible` mode)\n\n## Contributing\nFeel free to submit issues and enhancement requests to improve this workflow.\n\n## License\nDistributed under the MIT License. See `LICENSE` for details.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetwilds%2Fww-sourmash","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgetwilds%2Fww-sourmash","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetwilds%2Fww-sourmash/lists"}