{"id":19749495,"url":"https://github.com/mskcc/pluto-cwl","last_synced_at":"2025-04-30T09:31:18.125Z","repository":{"id":39718158,"uuid":"276108242","full_name":"mskcc/pluto-cwl","owner":"mskcc","description":"CWL workflows for helix filter scripts","archived":false,"fork":false,"pushed_at":"2023-07-13T20:29:55.000Z","size":1360,"stargazers_count":1,"open_issues_count":26,"forks_count":6,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-05-12T00:46:17.985Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mskcc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-06-30T13:34:15.000Z","updated_at":"2024-05-12T00:46:17.985Z","dependencies_parsed_at":"2023-01-30T17:31:28.289Z","dependency_job_id":null,"html_url":"https://github.com/mskcc/pluto-cwl","commit_stats":null,"previous_names":[],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mskcc%2Fpluto-cwl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mskcc%2Fpluto-cwl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mskcc%2Fpluto-cwl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mskcc%2Fpluto-cwl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mskcc","download_url":"https://codeload.github.com/mskcc/pluto-cwl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224204836,"owners_count":17273192,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T02:26:46.534Z","updated_at":"2024-11-12T02:26:47.060Z","avatar_url":"https://github.com/mskcc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pluto-cwl\r\n\r\n**P**ost-processing \u0026 **L**ightweight **U**pdates **T**o pipeline **O**utput\r\n\r\nCWL files and workflows to accompany the [helix_filters_01](https://github.com/mskcc/helix_filters_01) repo. Supported by infrastructure in the [`pluto`](https://github.com/mskcc/pluto) submodule.\r\n\r\n# Installation \u0026 Setup\r\n\r\nClone this repo with \r\n\r\n```\r\ngit clone --recursive https://github.com/mskcc/pluto-cwl.git\r\ncd pluto-cwl\r\n```\r\n\r\nInstall dependencies for the repo with the command:\r\n\r\n```\r\nmake install \r\n```\r\n\r\nThis will checkout the included `git` submodules and install a local `conda` with extra dependencies.\r\n\r\nUse this command to activate the installed environment for running workflows:\r\n\r\n```\r\n. env.juno.sh toil\r\n```\r\n\r\nThis will:\r\n\r\n- update your environment to use the `cwltool` and `toil` installed in the local `conda`\r\n- (if running on Juno HPC) update your environment with Toil variables needed to run on Juno\r\n- (if running on Juno HPC) upate your environment to use pre-cached Singularity containers located on Juno\r\n\r\n\r\n# Run a CWL\r\n\r\nThe primary entry point for the workflow is [`cwl/workflow_with_facets.cwl`](https://github.com/mskcc/pluto-cwl/blob/master/cwl/workflow_with_facets.cwl). \r\n\r\nYou can run a CWL included in this repo by using the wrapper scripts bundled in the `pluto` submodule; \r\n\r\n- [`pluto/run-cwltool.sh`](https://github.com/mskcc/pluto/blob/master/run-cwltool.sh) for simple use cases\r\n- [`pluto/run-toil.sh`](https://github.com/mskcc/pluto/blob/master/run-toil.sh) if parallel processing and HPC (LSF) useage is required\r\n\r\n## Test Suite\r\n\r\nDevelopment and testing takes place via the test suite.\r\n\r\nThe included test suite can be run with:\r\n\r\n```\r\nmake test\r\n```\r\n\r\nIt typically takes about 45 minutes to run all included tests\r\n\r\n- NOTE: tests require data sets that are pre-saved on the `juno` server\r\n\r\nSome very large integration tests are skipped by default. To include all tests, export the environment variable `LARGE_TESTS=True` or include it in the command line invocation. You can also change the CWL engine from `cwltool` to `toil`, among other settings, the same way. For example;\r\n\r\n```\r\nLARGE_TESTS=True CWL_ENGINE=Toil PRINT_COMMAND=True TMP_DIR=/scratch USE_LSF=True make test\r\n```\r\n\r\nAvailable environment variable settings are derived from the [`pluto.settings`](https://github.com/mskcc/pluto/blob/master/settings.py) submodule.\r\n\r\n### Parallel Test Suite\r\n\r\nAn extra recipe is included which can run the tests in parallel, for example to run 8 tests at once you can use this command:\r\n\r\n```\r\nmake parallel-test\r\n```\r\n\r\n### Single Test\r\n\r\nFor development purposes, it is helpful to be able to run only a specific test case, or subset of tests.\r\n\r\nYou can run just the script with the tests you are interested in, such as;\r\n\r\n```\r\npython tests/test_workflow_cwl.py\r\n```\r\n\r\nYou can further select which test case(s) from the script you wish to run by adding their labels as args;\r\n\r\n```\r\npython tests/test_workflow_cwl.py TestClassName\r\n\r\npython tests/test_workflow_cwl.py TestClassName.test_function\r\n```\r\n\r\nThis can be combined with the environment variables described above (such as `LARGE_TESTS`, `PRINT_COMMAND`, `KEEP_TMP`, etc.).\r\n \r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmskcc%2Fpluto-cwl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmskcc%2Fpluto-cwl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmskcc%2Fpluto-cwl/lists"}