{"id":13453954,"url":"https://github.com/themattrix/bash-concurrent","last_synced_at":"2025-05-16T18:07:03.849Z","repository":{"id":64874806,"uuid":"49480062","full_name":"themattrix/bash-concurrent","owner":"themattrix","description":"Bash function to run tasks in parallel and display pretty output as they complete.","archived":false,"fork":false,"pushed_at":"2019-04-16T14:50:50.000Z","size":91,"stargazers_count":1250,"open_issues_count":4,"forks_count":55,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-04-12T16:58:43.472Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/themattrix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-01-12T06:32:17.000Z","updated_at":"2025-03-21T16:31:12.000Z","dependencies_parsed_at":"2022-12-19T02:31:15.433Z","dependency_job_id":null,"html_url":"https://github.com/themattrix/bash-concurrent","commit_stats":null,"previous_names":[],"tags_count":26,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themattrix%2Fbash-concurrent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themattrix%2Fbash-concurrent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themattrix%2Fbash-concurrent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themattrix%2Fbash-concurrent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/themattrix","download_url":"https://codeload.github.com/themattrix/bash-concurrent/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254582905,"owners_count":22095518,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T08:00:49.702Z","updated_at":"2025-05-16T18:07:03.828Z","avatar_url":"https://github.com/themattrix.png","language":"Shell","readme":"# Concurrent ![version: 2.4.0](https://img.shields.io/badge/version-2.4.0-green.svg?style=flat-square) ![language: bash](https://img.shields.io/badge/language-bash-blue.svg?style=flat-square) ![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg?style=flat-square) [![Travis](https://img.shields.io/travis/themattrix/bash-concurrent.svg?style=flat-square)](https://travis-ci.org/themattrix/bash-concurrent)\n\nA Bash function to run tasks in parallel and display pretty output as they complete.\n\n[![asciicast](https://asciinema.org/a/34219.png)](https://asciinema.org/a/34219)\n\n\n## Examples\n\nRun three tasks concurrently:\n\n```bash\nconcurrent \\\n    - 'My long task'   sleep 10 \\\n    - 'My medium task' sleep 5  \\\n    - 'My short task'  sleep 1\n```\n\nRun three tasks sequentially:\n\n```bash\nconcurrent \\\n    - 'My long task'   sleep 10 \\\n    - 'My medium task' sleep 5  \\\n    - 'My short task'  sleep 1  \\\n    --sequential\n```\n\nStart the medium task *after* the short task succeeds:\n\n```bash\nconcurrent \\\n    - 'My long task'   sleep 10 \\\n    - 'My medium task' sleep 5  \\\n    - 'My short task'  sleep 1  \\\n    --require 'My short task'   \\\n    --before  'My medium task'\n```\n\nStart the short task after *both* other tasks succeed:\n\n```bash\nconcurrent \\\n    - 'My long task'   sleep 10 \\\n    - 'My medium task' sleep 5  \\\n    - 'My short task'  sleep 1  \\\n    --require 'My long task'    \\\n    --require 'My medium task'  \\\n    --before  'My short task'\n```\n\nSame as above, but shorter:\n\n```bash\nconcurrent \\\n    - 'My long task'   sleep 10 \\\n    - 'My medium task' sleep 5  \\\n    - 'My short task'  sleep 1  \\\n    --require-all --before 'My short task'\n```\n\nStart the medium task *and* the long task after the short task succeeds:\n\n```bash\nconcurrent \\\n    - 'My long task'   sleep 10 \\\n    - 'My medium task' sleep 5  \\\n    - 'My short task'  sleep 1  \\\n    --require 'My short task'   \\\n    --before  'My medium task'  \\\n    --before  'My long task'\n```\n\nSame as above, but shorter:\n\n```bash\nconcurrent \\\n    - 'My long task'   sleep 10 \\\n    - 'My medium task' sleep 5  \\\n    - 'My short task'  sleep 1  \\\n    --require 'My short task' --before-all\n```\n\nRun the first two tasks concurrently,\n*and then* the second two tasks concurrently,\n*and then* the final three tasks concurrently.\n\n```bash\nconcurrent \\\n    - 'Task 1'  sleep 3 \\\n    - 'Task 2'  sleep 3 \\\n    --and-then \\\n    - 'Task 3'  sleep 3 \\\n    - 'Task 4'  sleep 3 \\\n    --and-then \\\n    - 'Task 5'  sleep 3 \\\n    - 'Task 6'  sleep 3 \\\n    - 'Task 7'  sleep 3\n```\n\nIf your command has a `-` argument, you can use a different task delimiter:\n\n```bash\nconcurrent \\\n    + 'My long task'   wget -O - ... \\\n    + 'My medium task' sleep 5  \\\n    + 'My short task'  sleep 1\n```\n\nYou can display extra information at the end of each task's status line by\nechoing to `fd 3`.\n\n```bash\nmy_task() {\n    ...\n    echo \"(extra info)\" \u003e\u00263\n    ...\n}\n```\n\nTake a look at [`demo.sh`](demo.sh) for more involved examples.\n\n\n## Dry Run\n\nIf you have a lot of dependencies between tasks, it's generally a good idea to\nperform a dry-run to ensure that the tasks are ordered as expected. Set the\n`CONCURRENT_DRY_RUN` environment variable to perform a dry-run.\n\n\n## Forking Limit\n\nBy default, `concurrent` allows up to 50 concurrently-running tasks.\nSet the `CONCURRENT_LIMIT` environment variable to override this limit.\n\nA neat trick is to set the limit to 1, essentially forcing a `--sequential`\nrun, but with existing tasks between dependencies taken into account.\n\nA limit less than 1 is treated as no limit.\n\n\n## Compact Display\n\nIf the number of tasks exceed the terminal height, the \"compact display\" will\nbe activated. It can also be explicitly activated by setting the\n`CONCURRENT_COMPACT` environment variable to anything other than `0`.\n\nIn this mode, each task is represented by a single character instead of an\nentire line. An execution summary is displayed above the tasks.\n\n[![asciicast](https://asciinema.org/a/37290.png)](https://asciinema.org/a/37290)\n\n\n## Non-Interactive Mode\n\nIf the output is not attached to a tty, the \"non-interactive\" mode will be\nactivated. It can also be explicitly activated by setting the\n`CONCURRENT_NONINTERACTIVE` environment variable to anything other than `0`.\n\nIn this mode, each task is displayed as soon as it completes. Colors are\nalso disabled.\n\n\n## Logging\n\nBy default, logs for each task will be created in `./logs/\u003ctimestamp\u003e/`.\nFor example:\n\n    $ ls .logs/2016-02-02@00:09:07\n    0. Creating VM (0).log\n    1. Creating ramdisk (0).log\n    2. Enabling swap (0).log\n    3. Populating VM with world data (1).log\n    4. Spigot: Pulling docker image for build (1).log\n    5. Spigot: Building JAR (skip).log\n    6. Pulling remaining docker images (skip).log\n    7. Launching services (skip).log\n\n\nTo change this directory, set `CONCURRENT_LOG_DIR` before calling `concurrent`.\n\n\n## Failure Demo\n\n[![asciicast](https://asciinema.org/a/34217.png)](https://asciinema.org/a/34217)\n\n\n## Interrupted Demo\n\n[![asciicast](https://asciinema.org/a/34218.png)](https://asciinema.org/a/34218)\n\n\n## Requirements\n\n- bash \u003e= 4.2 (for `declare -g`)\n- cat\n- cp\n- date\n- mkdir\n- mkfifo\n- mktemp\n- mv\n- sed\n- tail\n- tput\n\n\n## Change Log\n\n- **2.4.0**\n  - *New:* Gracefully switches to non-interactive mode when a tty is not present or if `CONCURRENT_NONINTERACTIVE` is non-zero.\n- **2.3.3**\n  - *Fix:* Cursor no longer hidden when running nested.\n- **2.3.2**\n  - *Fix:* Failing tasks with no output now exit with the correct status (credit: @uluyol).\n- **2.3.1**\n  - *Fix:* Now clearing to end of line when printing extra status info after a task (credit: @fragmede).\n- **2.3.0**\n  - *New:* Concurrency limit defaults to 50, unless overridden by `CONCURRENT_LIMIT`.\n  - *New:* If the number of tasks exceed the terminal height (or `CONCURRENT_COMPACT` is set), each task will be displayed as a single character instead of a taking up an entire line.\n  - *New:* Cursor now hidden while running.\n  - *Fix:* Greatly improved speed of event loop. Especially noticeable for large numbers of tasks.\n  - *Fix:* Namespaced `command_*` and `prereq_*` arrays so that they don't carry into the tasks.\n- **2.2.1**\n  - *Fix:* Tasks not allowed to read from stdin.\n- **2.2.0**\n  - *New:* Instances of concurrent can be nested without breaking.\n  - *New:* Set custom log dir with `CONCURRENT_LOG_DIR`.\n  - *Fix:* Works under Cygwin (special thanks to @FredDeschenes).\n  - *Fix:* No longer requires GNU sed (gsed) on OS X.\n  - *Fix:* Animation now uses a single process.\n  - *Fix:* Extra status info is now merely bold instead of bold/white, which should be more visible on light terminal backgrounds.\n- **2.1.0**\n  - *New:* New `--and-then` flag for dividing tasks into groups. All tasks in a group run concurrently, but all must complete before the next group may start (inspiration: [fooshards on Reddit](https://www.reddit.com/r/programming/comments/42n64o/concurrent_bash_function_to_run_tasks_in_parallel/czbxnrh)).\n  - *Fix:* Removed extra backslashes in README (credit: [bloody-albatross on Reddit](https://www.reddit.com/r/programming/comments/42n64o/concurrent_bash_function_to_run_tasks_in_parallel/czbrtjg))\n- **2.0.1**\n  - *Fix:* `kill` is a bash builtin (credit: @ScoreUnder)\n  - *Fix:* Require GNU sed on OS X (credit: @kumon)\n  - *Fix:* Static analysis with shellcheck on push via Travis CI (credit: @xzovy)\n  - *Fix:* Cleaner signal handling.\n  - *Fix:* Simplified event loop.\n- **2.0.0**\n  - *New:* Tasks can now display status updates by echoing to fd 3.\n- **1.6.0**\n  - *New:* `CONCURRENT_DRY_RUN` environment variable runs `sleep 3` instead of actual commands (and prints message).\n- **1.5.2**\n  - *Fix:* Requirement loops disallowed.\n- **1.5.1**\n  - *Fix:* Task is not allowed to require itself directly.\n- **1.5.0**\n  - *New:* First argument is now the task delimiter.\n- **1.4.1**\n  - *Fix:* Namespaced previously-missed function.\n- **1.4.0**\n  - *New:* New `--require-all` and `--before-all` flags.\n  - *Fix:* Namespaced all concurrent-related functions and variables.\n  - *Fix:* Unsetting all concurrent-related functions and variables in the task's context.\n  - *Fix:* Enforcing foreground in an interactive shell.\n- **1.3.0**\n  - *New:* New `--sequential` flag, for when each task requires the previous.\n- **1.2.0**\n  - *New:* Running tasks have an animated cursor.\n  - *Fix:* Enforcing bash version 4.3.\n  - *Fix:* Echo is re-enabled even if an internal error occurs.\n- **1.1.6**\n  - *Fix:* Enforcing bash version 4.\n- **1.1.5**\n  - *Fix:* Tasks now use original `$PWD` and `$OLDPWD`.\n- **1.1.4**\n  - *Fix:* Tasks now use original `$SHELLOPTS` and `$BASHOPTS`.\n- **1.1.3**\n  - *Fix:* Sanitizing forward slashes from log names.\n- **1.1.2**\n  - *Fix:* Ensuring task status file exists even if an internal error occurs.\n- **1.1.1**\n  - *Fix:* Task command may now have arguments starting with `-`.\n- **1.1.0**\n  - *New:* Gracefully handling SIGINT.\n  - *Fix:* Works on OS X too.\n- **1.0.0**\n  - Initial working release.\n","funding_links":[],"categories":["Shell"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthemattrix%2Fbash-concurrent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthemattrix%2Fbash-concurrent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthemattrix%2Fbash-concurrent/lists"}