Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/hexdigest/execpool
A pool that spins up a given number of processes in advance and attaches stdin and stdout when needed. Very similar to FastCGI but works for any command.
https://github.com/hexdigest/execpool
Last synced: 14 days ago
JSON representation
A pool that spins up a given number of processes in advance and attaches stdin and stdout when needed. Very similar to FastCGI but works for any command.
- Host: GitHub
- URL: https://github.com/hexdigest/execpool
- Owner: hexdigest
- License: mit
- Created: 2021-06-17T18:41:46.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2021-07-06T20:39:16.000Z (over 3 years ago)
- Last Synced: 2024-07-31T20:51:52.292Z (3 months ago)
- Language: Go
- Homepage:
- Size: 55.7 KB
- Stars: 22
- Watchers: 4
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-go - execpool - A pool built around exec.Cmd that spins up a given number of processes in advance and attaches stdin and stdout to them when needed. Very similar to FastCGI or Apache Prefork MPM but works for any command. (Goroutines / Search and Analytic Databases)
- awesome-go-extra - execpool - 06-17T18:41:46Z|2021-07-06T20:39:16Z| (Goroutines / Advanced Console UIs)
README
# ExecPool
[](https://github.com/hexdigest/execpool/blob/master/LICENSE)
[](https://github.com/hexdigest/execpool/actions/workflows/go.yml)
[](https://coveralls.io/github/hexdigest/execpool?branch=main)
[](https://goreportcard.com/report/github.com/hexdigest/execpool)
[](http://godoc.org/github.com/hexdigest/execpool)
[](https://github.com/hexdigest/execpool/releases/latest)
[](https://github.com/avelino/awesome-go#goroutines)
Why? Sometimes when you implement Go services you have to integrate with third party command line tools.
You can do it by simply using [exec.Command](https://golang.org/pkg/os/exec/#Command) but the problem with this approach is
that your service has to wait while a new process is being loaded into the memory and started. Sometimes this can drastically
increase the latency of your service. ExecPool works similarly to [FastCGI](https://en.wikipedia.org/wiki/FastCGI) but it can
wrap any regular process. It spins up a given number of processes in advance and when it's time to handle a request from the
user your service just attaches stdin to an existing process from the pool. Basically the execpool helps you trade memory for latency.
# Usage
```go
package main
import (
"fmt"
"io/ioutil"
"log"
"os/exec"
"strings"
"github.com/hexdigest/execpool"
)
func main() {
cmd := exec.Command("grep", "none")
//spin up 100 processes of grep
pool, err := execpool.New(cmd, 100)
rc := pool.Exec(strings.NewReader("this makes sense\nthis is nonesense"))
b, err := ioutil.ReadAll(rc)
if err != nil {
log.Fatalf("failed to read from stdout: %v", err)
}
// this is nonesense
fmt.Println(string(b))
}
```
# Benchmark
This benchmark compares "standard" approach with exec.Command and execpool.Pool by running
grep 100 times. For heavier processes you can expect a bigger difference.
```
make benchmark
goos: darwin
goarch: amd64
pkg: github.com/hexdigest/execpool
cpu: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
BenchmarkNew-8 100 941753 ns/op
BenchmarkCmd-8 100 2386990 ns/op
```