{"id":23191919,"url":"https://github.com/moold/paralleltask","last_synced_at":"2025-09-04T07:35:27.081Z","repository":{"id":43829727,"uuid":"228567988","full_name":"moold/ParallelTask","owner":"moold","description":"A simple and lightweight parallel task engine","archived":false,"fork":false,"pushed_at":"2024-08-12T03:43:24.000Z","size":131,"stargazers_count":35,"open_issues_count":1,"forks_count":7,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-07-10T04:21:20.248Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moold.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-17T08:22:09.000Z","updated_at":"2025-06-13T08:45:36.000Z","dependencies_parsed_at":"2025-01-10T09:30:57.403Z","dependency_job_id":"2de9c071-509a-49b2-9544-f1ddae336a49","html_url":"https://github.com/moold/ParallelTask","commit_stats":{"total_commits":51,"total_committers":1,"mean_commits":51.0,"dds":0.0,"last_synced_commit":"abef4b845b9b45e89d4c42d19e234a2dd55aa41e"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/moold/ParallelTask","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FParallelTask","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FParallelTask/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FParallelTask/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FParallelTask/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moold","download_url":"https://codeload.github.com/moold/ParallelTask/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FParallelTask/sbom","scorecard":{"id":659307,"data":{"date":"2025-08-11","repo":{"name":"github.com/moold/ParallelTask","commit":"abef4b845b9b45e89d4c42d19e234a2dd55aa41e"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":0,"reason":"Found 0/6 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU General Public License v3.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-21T15:36:50.213Z","repository_id":43829727,"created_at":"2025-08-21T15:36:50.214Z","updated_at":"2025-08-21T15:36:50.214Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273573594,"owners_count":25129881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-04T02:00:08.968Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-18T12:19:26.161Z","updated_at":"2025-09-04T07:35:27.039Z","avatar_url":"https://github.com/moold.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Paralleltask\nParalleltask is a simple and lightweight parallel task engine. It can launch a given number of tasks from a batch of independent tasks, and keep this number of running tasks until all tasks are completed.\n\n## Why Paralleltask?\nSuppose you have dozens or hundreds of independent tasks that can run in parallel (non-independent tasks can be put together to form an independent task group). Due to the limitation of computing resources, you cannot run all tasks at the same time. Of course, it is not realistic to run one by one, so you want to run a specific number of tasks at the same time, and keep this number of running tasks (once a task is completed, start a new task) until all tasks are completed.\n\n* zero configuration, no dependencies, no prior knowledge required, easy to install and use.\n\n* support breakpoint resume, automatically re-execute failed tasks and ignore successful tasks.\n\n* automatically kill submitted tasks once the main program receives a termination signal (`Ctrl+C`).\n\n* support multiple task scheduling systems, such as `LOCAL`, `SGE`, `PBS`, `SLURM` and `LSF`.\n\n* automatically convert relative path to absolute path in shell scripts.\n\n* support python 2 and 3.\n\n## Installation\n```\npip install paralleltask\n```\n\nIf you prefer to use the [drmaa](https://github.com/pygridtools/drmaa-python) library, instead of using commands (such as `qsub`) to submit and control tasks, see [here](https://github.com/moold/ParallelTask/blob/master/DRMAA.md) to install `drmaa`.\n\n## Usage\n```\nparalleltask work.sh\n```\n\nThe only required input file `work.sh` is a command collection file, which contains all the task commands that need to be run. A task command can be one or more lines, and an independent task group can be set by the parameter `--lines`. Using `paralleltask -h` for details.\n\n## Testing\n```bash\n# produce a work file: test.sh\nfor i in {1..50};do echo \"sleep ${i} \u0026\u0026 echo \\\"I am subtask${i}\\\"\" \u003e\u003e test.sh;done;\n# run\nparalleltask test.sh\n```\nA demo file `test.sh` is [here](https://github.com/moold/ParallelTask/blob/master/test/test.sh)\n\n\n## Configuration \nIf you want to change some of the default settings, you can pass parameters (use `paralleltask -h` for details) or directly edit the configure template file `cluster.cfg`.\n\n***Note***: Paralleltask will replace `{mem}`, `{cpu}`, `{bash}`, `{out}`, `{err}`, `{script}` and `{job_id}` with specific values needed for each jobs, see the configure template [file](https://github.com/moold/ParallelTask/blob/master/src/paralleltask/cluster.cfg) for details.\n\n## Getting Help\n\nFeel free to raise an issue at the [issue page](https://github.com/moold/ParallelTask/issues), and welcome to [pull request](https://github.com/moold/ParallelTask/pulls) if you find a bug or have an idea for this project.\n\n## Star\nYou can track updates by tab the `Star` button on the upper-right corner at the [github page](https://github.com/moold/ParallelTask).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoold%2Fparalleltask","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoold%2Fparalleltask","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoold%2Fparalleltask/lists"}