https://github.com/nodys/clussh
Stream task execution, through ssh, to one or many host, one or many times, in parallel or in series.
https://github.com/nodys/clussh
ldjson ndjson ssh stream workers
Last synced: 17 days ago
JSON representation
Stream task execution, through ssh, to one or many host, one or many times, in parallel or in series.
- Host: GitHub
- URL: https://github.com/nodys/clussh
- Owner: nodys
- License: mit
- Created: 2017-11-30T14:57:10.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-05-09T16:05:12.000Z (almost 7 years ago)
- Last Synced: 2025-03-27T07:21:46.094Z (about 1 month ago)
- Topics: ldjson, ndjson, ssh, stream, workers
- Language: JavaScript
- Homepage: https://nodys.github.io/clussh
- Size: 781 KB
- Stars: 6
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Clussh
[](https://travis-ci.org/nodys/clussh) [](https://standardjs.com) [](https://www.npmjs.com/package/clussh)
Stream task execution, through ssh, to one or many host, one or many times, in parallel or in series.
## Features:
- Distribute tasks upon many hosts
- Stream task from stdin
- Stream task outputs and progression stats to ldjson stream
- Support scaling, parallel execution and retry, network failure and host unavailability
## Installation
```bash
npm install -g clussh
```
This will install three command-line cli:
- clussh - The main command line
- clussh-board - A streamable clussh progress board
- clussh-log - Pretty print clussh logs
*Pre-requirement: [NodeJS](https://nodejs.org/) and its awesome [universe](https://node.cool)*
## Usages & exemples
```bash
# Run "Hello $(hostname)" on the default worker ssh://yourusername@localhost
clussh
# Run given command on default worker
clussh --cmd 'echo "Hello world"'
clussh --cmd 'echo "Hello world"' --scale 10 # Teen times
clussh --cmd 'echo "Hello world"' --scale 10 --concurrency 2 # Two in parallel
# Run given bash script on default worker
clussh ./install.sh
# Run given bash script on to agents (with a concurrency of 2 for agent-b)
# (NB: for ssh cooking & baking reasons, the shell script must ends with an exit statement)
clussh -w ssh://[email protected] -w ssh://[email protected]?concurrency=2 ./runme.sh
# Pipe task from line delimited json stream
echo '
{ "id": "task-a", "cmd": "echo hello; sleep 1" }
{ "id": "task-b", "cmd": "echo hello; sleep 2", "scale": 10 }
{ "id": "task-c", "script": "./runme.sh", args: ["--option", "value"] }
{ "id": "task-d", "script": "./runme.sh", "worker": "ssh://[email protected]" }
' | clussh -w ssh://user@agent-a -w ssh://user@agent-b
# You can pipe infinite line delemited json stream (eg. event logs for a file wacher...)
infinite-stream | clussh
# Print human readable logs
clussh | clussh-log
# Print nice-looking progress board
clussh | clussh-board
# Print human readable logs
clussh | clussh-log
# Save logs for further exploration (and still display the board)
clussh | tee output.log | clussh-board
tail -f output.log | clussh-log # Another console...
tail -f output.log | clussh-board # Another console...
tail -f output.log | ndjson-filter 'd.type === "fail"' | prettyldjson # Another console...
```
## Command line usage and configuration
```bash
clussh [options] [script filepath]
Options:
--help Show help [boolean]
--version Show version number [boolean]
--retry, -r How many retry for each task [number] [default: 0]
--timeout, -t Task timeout in millisecond or parseable ms time (eg. 1h,
2d or `3 days`) [string] [default: "10d"]
--worker, -w Repeatable worker uri list
[array] [default: "ssh://jponchon@localhost"]
--concurrency, -c Concurrency per worker [number] [default: 1]
--scale, -s Default scale [number] [default: 1]
--cmd Command to execute [string]
--script Script to execute (please ensure proper exit) [string]
--identity, -i List ssh identity files to use [array] [default: []]
```
Clussh can be configured with a `.clusshrc` file (see https://www.npmjs.com/package/rc)
```json
{
"worker": [
"ssh://[email protected]",
"ssh://[email protected]",
"ssh://[email protected]?concurrency=2"
],
"concurrency": 4,
"retry": 10
}
```
Above, one worker uri overide the default clussh configuration using url query string (`?concurrency=2` for host `agent-b.local`). This will result in two workers for `agent-b.local`: one with the default concurrency (4) and another with a concurrency of 2.
## Log message format
Clussh output line delemited json stream consumable by `clussh-board` or `clussh-log` or any other line delemited json tools (see [prettyldjson](https://www.npmjs.com/package/prettyldjson) for pretty print and [ndjson-cli](https://www.npmjs.com/package/ndjson-cli) for map, reduce, filtering, etc.)
*TODO: Fields documentation & messages types*
## API
```js
const clussh = require('clussh')
const clusshStream = clussh({
cmd: 'hello',
worker: [ 'ssh://[email protected]', 'ssh://[email protected]' ],
retry: 3,
concurrency: 2
})
clusshStream.pipe(process.stdout)
```
---
License: [MIT](./LICENSE) - Novadiscovery