https://github.com/jabolopes/par

Run commands from stdin in parallel
https://github.com/jabolopes/par

concurrency go golang parallel shell xargs

Last synced: 9 days ago
JSON representation

Run commands from stdin in parallel

• Host: GitHub
• URL: https://github.com/jabolopes/par
• Owner: jabolopes
• License: bsd-3-clause
• Created: 2023-05-30T19:05:36.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-12-17T19:23:13.000Z (about 1 year ago)
• Last Synced: 2024-06-19T21:01:15.539Z (6 months ago)
• Topics: concurrency, go, golang, parallel, shell, xargs
• Language: Go
• Homepage:
• Size: 4.88 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# par

Run commands from stdin in parallel.

Each command is a single line terminated by the newline character (`\n`).

By default, the commands are run in a shell. The default number of

parallel processes equals the number of CPUs (or cores).

The flag `-n` enables dry run mode: commands are printed to `stderr`

but not run.

The flag `-v` enables verbose mode: commands are printed to `stderr`

when run.

## Installation

```shell

$ go install github.com/jabolopes/par

```

## Example

Copy and convert a bunch of CR2 photographs to JPEG in parallel:

```shell

$ for i in *.CR2; do echo convert /run/media/$USER/EOS_DIGITAL/DCIM/100EOS5D/$i ~/fotos/$(basename $i | sed -n "s/CR2$/jpg/p"); done | par -v

```

## Alternatives

> Why not `xargs`?

I can never remember the `xargs` flags by heart and the defaults seem

to be opposite of what I'm trying to achieve in terms of

parallelization. Also, subshelling doesn't work as expected since the

subshelling occurs before `xargs` executes not after, therefore,

subshelling cannot use the items read by `xargs`. There's probably a

solution to that problem but I didn't figure it out.

> Why not GNU `parallel`?

In addition to the same subshelling problem as `xargs`, there's also

the issues [1](https://news.ycombinator.com/item?id=15319715),

[2](https://bugs.launchpad.net/ubuntu/+source/parallel/+bug/1779764),

and [3](https://github.com/NixOS/nixpkgs/issues/110584), etc.