https://github.com/kilianpaquier/pooling
Easily dispatch functions (and subfunctions indefinitely) into a shared pool of routines in golang
https://github.com/kilianpaquier/pooling
golang parallel-computing queue-tasks recursive-algorithm
Last synced: 3 months ago
JSON representation
Easily dispatch functions (and subfunctions indefinitely) into a shared pool of routines in golang
- Host: GitHub
- URL: https://github.com/kilianpaquier/pooling
- Owner: kilianpaquier
- License: mit
- Created: 2024-02-24T10:33:45.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-10T16:24:28.000Z (3 months ago)
- Last Synced: 2025-03-10T17:31:57.753Z (3 months ago)
- Topics: golang, parallel-computing, queue-tasks, recursive-algorithm
- Language: Go
- Homepage: https://pkg.go.dev/github.com/kilianpaquier/pooling/pkg
- Size: 84 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Codeowners: CODEOWNERS
Awesome Lists containing this project
README
# pooling
---
- [How to use ?](#how-to-use-)
- [Features](#features)
## How to use ?
```sh
go get -u github.com/kilianpaquier/pooling@latest
```
## Features
The pooling package allows one to dispatch an infinite number of functions to be executed in parallel while still limiting the number of routines.
For that, pooling package takes advantage of ants pool library. A pooling Pooler can have multiple pools (with builder SetSizes) to dispatch sub functions into different pools of routines.
When sending a function into the pooler (with the appropriate channel), this function can itself send other functions into the pooler. It allows one to "split" functions executions (like iterating over a slice and each element handled in parallel).
```go
func main() {
log := logrus.WithContext(context.Background())
pooler, err := pooling.NewPoolerBuilder().
SetSizes(10, 500, ...). // each size will initialize a pool with given size
SetOptions(ants.WithLogger(log)).
Build()
if err != nil {
panic(err)
}
defer pooler.Close()
input := ReadFrom()
// Read function is blocking until input is closed
// and all running routines have ended
pooler.Read(input)
}
func ReadFrom() <-chan pooling.PoolerFunc {
input := make(chan pooling.PoolerFunc)
go func() {
// close input to stop blocking function Read once all elements are sent to input
defer close(input)
// do something populating input channel
for i := range 100 {
input <- HandleInt(i)
}
}()
return input
}
func HandleInt(i int) pooling.PoolerFunc {
return func(funcs chan<- pooling.PoolerFunc) {
// you may handle the integer whichever you want
// funcs channel is present to dispatch again some elements into a channel handled by the pooler
}
}
```