{"id":20489486,"url":"https://github.com/nodys/clussh","last_synced_at":"2025-04-13T16:33:04.708Z","repository":{"id":85345690,"uuid":"112624188","full_name":"nodys/clussh","owner":"nodys","description":"Stream task execution, through ssh, to one or many host, one or many times, in parallel or in series.","archived":false,"fork":false,"pushed_at":"2018-05-09T16:05:12.000Z","size":800,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-27T07:21:46.094Z","etag":null,"topics":["ldjson","ndjson","ssh","stream","workers"],"latest_commit_sha":null,"homepage":"https://nodys.github.io/clussh","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nodys.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-11-30T14:57:10.000Z","updated_at":"2019-12-03T08:35:44.000Z","dependencies_parsed_at":"2023-03-13T04:54:38.499Z","dependency_job_id":null,"html_url":"https://github.com/nodys/clussh","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodys%2Fclussh","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodys%2Fclussh/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodys%2Fclussh/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nodys%2Fclussh/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nodys","download_url":"https://codeload.github.com/nodys/clussh/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248744062,"owners_count":21154800,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ldjson","ndjson","ssh","stream","workers"],"created_at":"2024-11-15T17:13:05.604Z","updated_at":"2025-04-13T16:33:04.681Z","avatar_url":"https://github.com/nodys.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Clussh\n[![Build Status](https://travis-ci.org/nodys/clussh.svg?branch=master)](https://travis-ci.org/nodys/clussh) [![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com) [![Clussh on npm](https://img.shields.io/npm/v/clussh.svg)](https://www.npmjs.com/package/clussh)\n\nStream task execution, through ssh, to one or many host, one or many times, in parallel or in series.\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://nodys.github.io/clussh/img/clussh-shell.gif\" alt=\"clussh\"\u003e\n\u003c/p\u003e\n\n## Features:\n\n- Distribute tasks upon many hosts\n- Stream task from stdin\n- Stream task outputs and progression stats to ldjson stream\n- Support scaling, parallel execution and retry, network failure and host unavailability\n\n## Installation\n\n```bash\nnpm install -g clussh\n```\n\nThis will install three command-line cli:\n- clussh       - The main command line\n- clussh-board - A streamable clussh progress board\n- clussh-log   - Pretty print clussh logs\n\n*Pre-requirement: [NodeJS](https://nodejs.org/) and its awesome [universe](https://node.cool)*\n\n## Usages \u0026 exemples\n\n```bash\n# Run \"Hello $(hostname)\" on the default worker ssh://yourusername@localhost\nclussh\n\n# Run given command on default worker\nclussh --cmd 'echo \"Hello world\"'\nclussh --cmd 'echo \"Hello world\"' --scale 10 # Teen times\nclussh --cmd 'echo \"Hello world\"' --scale 10 --concurrency 2 # Two in parallel\n\n# Run given bash script on default worker\nclussh ./install.sh\n\n# Run given bash script on to agents (with a concurrency of 2 for agent-b)\n# (NB: for ssh cooking \u0026 baking reasons, the shell script must ends with an exit statement)\nclussh -w ssh://user@agent-a.local -w ssh://user@agent-b.local?concurrency=2 ./runme.sh\n\n# Pipe task from line delimited json stream\necho '\n{ \"id\": \"task-a\", \"cmd\": \"echo hello; sleep 1\" }\n{ \"id\": \"task-b\", \"cmd\": \"echo hello; sleep 2\", \"scale\": 10 }\n{ \"id\": \"task-c\", \"script\": \"./runme.sh\", args: [\"--option\", \"value\"] }\n{ \"id\": \"task-d\", \"script\": \"./runme.sh\", \"worker\": \"ssh://only@on.host\" }\n' | clussh -w ssh://user@agent-a -w ssh://user@agent-b\n\n# You can pipe infinite line delemited json stream (eg. event logs for a file wacher...)\ninfinite-stream | clussh\n\n# Print human readable logs\nclussh | clussh-log\n\n# Print nice-looking progress board\nclussh | clussh-board\n\n# Print human readable logs\nclussh | clussh-log\n\n# Save logs for further exploration (and still display the board)\nclussh | tee output.log | clussh-board\ntail -f output.log | clussh-log   # Another console...\ntail -f output.log | clussh-board # Another console...\ntail -f output.log | ndjson-filter 'd.type === \"fail\"' | prettyldjson # Another console...\n```\n\n\n## Command line usage and configuration\n\n```bash\nclussh [options] [script filepath]\n\nOptions:\n  --help             Show help                                         [boolean]\n  --version          Show version number                               [boolean]\n  --retry, -r        How many retry for each task          [number] [default: 0]\n  --timeout, -t      Task timeout in millisecond or parseable ms time (eg. 1h,\n                     2d or `3 days`)                   [string] [default: \"10d\"]\n  --worker, -w       Repeatable worker uri list\n                                   [array] [default: \"ssh://jponchon@localhost\"]\n  --concurrency, -c  Concurrency per worker                [number] [default: 1]\n  --scale, -s        Default scale                         [number] [default: 1]\n  --cmd              Command to execute                                 [string]\n  --script           Script to execute (please ensure proper exit)      [string]\n  --identity, -i     List ssh identity files to use        [array] [default: []]\n```\n\n\nClussh can be configured with a `.clusshrc` file (see https://www.npmjs.com/package/rc)\n\n```json\n{\n  \"worker\": [\n    \"ssh://master@agent-a.local\",\n    \"ssh://master@agent-b.local\",\n    \"ssh://master@agent-b.local?concurrency=2\"\n  ],\n  \"concurrency\": 4,\n  \"retry\": 10\n}\n```\n\nAbove, one worker uri overide the default clussh configuration using url query string (`?concurrency=2` for host `agent-b.local`). This will result in two workers for `agent-b.local`: one with the default concurrency (4) and another with a concurrency of 2.\n\n\n## Log message format\n\nClussh output line delemited json stream consumable by `clussh-board` or `clussh-log` or any other line delemited json tools (see [prettyldjson](https://www.npmjs.com/package/prettyldjson) for pretty print and [ndjson-cli](https://www.npmjs.com/package/ndjson-cli) for map, reduce, filtering, etc.)\n\n*TODO: Fields documentation \u0026 messages types*\n\n## API\n\n```js\nconst clussh = require('clussh')\n\nconst clusshStream = clussh({\n  cmd: 'hello',\n  worker: [ 'ssh://foo@bar.com', 'ssh://bar@foo.com' ],\n  retry: 3,\n  concurrency: 2\n})\n\nclusshStream.pipe(process.stdout)\n```\n\n---\n\nLicense: [MIT](./LICENSE) - Novadiscovery\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodys%2Fclussh","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnodys%2Fclussh","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodys%2Fclussh/lists"}