{"id":16369692,"url":"https://github.com/dcsunset/concurrent-executor","last_synced_at":"2025-08-08T02:09:50.812Z","repository":{"id":65690944,"uuid":"595784812","full_name":"DCsunset/concurrent-executor","owner":"DCsunset","description":"Executing multiple commands concurrently using Python asyncio","archived":false,"fork":false,"pushed_at":"2023-06-15T01:01:49.000Z","size":87,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-05T13:47:20.026Z","etag":null,"topics":["asyncio","cli","command","concurrent","python","ssh"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DCsunset.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-01-31T20:01:26.000Z","updated_at":"2023-03-24T05:41:18.000Z","dependencies_parsed_at":"2023-07-26T09:13:45.076Z","dependency_job_id":null,"html_url":"https://github.com/DCsunset/concurrent-executor","commit_stats":{"total_commits":50,"total_committers":1,"mean_commits":50.0,"dds":0.0,"last_synced_commit":"67eb05ffebd00e079024200881f3f4fa6d53d255"},"previous_names":["dcsunset/concurrent-ssh"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/DCsunset/concurrent-executor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DCsunset%2Fconcurrent-executor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DCsunset%2Fconcurrent-executor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DCsunset%2Fconcurrent-executor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DCsunset%2Fconcurrent-executor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DCsunset","download_url":"https://codeload.github.com/DCsunset/concurrent-executor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DCsunset%2Fconcurrent-executor/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264538021,"owners_count":23624426,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","cli","command","concurrent","python","ssh"],"created_at":"2024-10-11T02:56:03.158Z","updated_at":"2025-07-10T06:07:28.650Z","avatar_url":"https://github.com/DCsunset.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# concurrent-executor\n\n[![PyPI](https://img.shields.io/pypi/v/concurrent-executor)](https://pypi.org/project/concurrent-executor/)\n\nExecuting multiple commands concurrently using Python asyncio\n\n## Installation\n\n```sh\npip install concurrent-executor\n# Or from the latest GitHub version\npip install git+https://github.com/DCsunset/concurrent-executor\n\n```\n\n## CLI Usage\n\nAll the following tools can handle signals as follows:\n\n* Upon receiving one `SIGINT` (including keyboard interrupt) or `SIGTERM`, the program will send `SIGTERM` to all spawned processes.\n* Upon receiving more than one of them, the program will instead send `SIGKILL` to kill all spawned processes.\n\n### cssh\n\n`cssh` is a command line tool provided by this package.\nIt is used to executing commands concurrently on remote servers vis SSH.\n\nUse `-H` or `--hosts` to specify the hosts to run the commands on:\n\n```sh\ncssh -H host1 host2 ... host_n -- command\n# pass extra ssh options\ncssh -o=\"-q -4\" -H host1 host2 ... host_n -- command\n# read hosts from file\ncssh -f hosts.txt -- command\n# string interpolation (to include host name in command by {0})\ncssh -H host1 host2 -- command --host {0}\n# pipe to stdin\necho \"yes\" | cssh -H host1 host2 -- command \n# Pipe file content to stdin\ncssh -H host1 host2 -- command \u003c input.txt\n```\n\nNote that `--` is necessary to separate the options and the command.\nFor `-o/--options` to work correctly, use `=` to prevent it from being parsed as another option.\n\nThe standard input (stdin) of the `cssh` process is piped to the stdin of every spawned processes.\n\nFor more details, see `cssh -h`.\n\n### cexec\n\n`cexec` is another command line tool provided by this package.\nIt is used to execute arbitrary shell commands concurrently using template (string interpolation in Python).\n\nThe command itself can container placeholder in strings: (See [Python string interpolation](https://peps.python.org/pep-0498/) for more detail.)\n\n```sh\n# The variables are a, b, c in the template command\n# This command creates 3 directories and write to a file in each directory\ncexec -V a b c -- \"mkdir {0} \u0026\u0026 echo 1 \u003e {0}/out\"\n# Read variables from a file\ncexec -f vars.txt -- \"mkdir {0} \u0026\u0026 echo 1 \u003e {0}/out\"\n# Run different commands directly\ncexec -V \"cmd1\" \"cmd2\" \"cmd3\" -- \"{}\"\n# pipe to stdin\necho \"yes\" | cexec -V a b c -- \"cat -\"\n```\n\nFor more details, see `cexec -h`.\n\n### Library\n\nIt can also be used as a library:\n\n```python\nimport asyncio\nfrom concurrent_executor.executor import SshExecutor\n\nasync def main():\n  hosts = [\"host1\", \"host2\"]\n  executor = SshExecutor(hosts)\n  # running concurrently\n  await executor.run(\"some_command --test\")\n\n  # access stdout for all hosts (or stderr)\n  async for index, out in executor.stdout:\n    print(f\"{host[index]}: {out}\")\n\n  # wait until all finished\n  ret_codes = await executor.wait()\n  \nasyncio.run(main())\n```\n\nSee more usage in `concurrent_executor/cli.py`.\n\n## Development\n\nTo set up the development environment,\nfirst clone this repo.\n\nThen it's recommended to use`venv`:\n\n```sh\n# suppose PWD is the root dir of the repo\npython -m venv venv\n# activate the environment``\nsource venv/bin/activate\npip install -r requirements.txt\n```\n\nTo deactivate, run `deactivate`.\n\n\n## License\n\nThis project is licensed under AGPL-3.0. Copyright notice:\n\n    concurrent-executor\n    Copyright (C) 2023 DCsunset\n\n    This program is free software: you can redistribute it and/or modify\n    it under the terms of the GNU Affero General Public License as published\n    by the Free Software Foundation, either version 3 of the License, or\n    (at your option) any later version.\n\n    This program is distributed in the hope that it will be useful,\n    but WITHOUT ANY WARRANTY; without even the implied warranty of\n    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\n    GNU Affero General Public License for more details.\n\n    You should have received a copy of the GNU Affero General Public License\n    along with this program.  If not, see \u003chttps://www.gnu.org/licenses/\u003e.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdcsunset%2Fconcurrent-executor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdcsunset%2Fconcurrent-executor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdcsunset%2Fconcurrent-executor/lists"}