https://github.com/tomnomnom/gahttp
Async / concurrent HTTP requests for Go
https://github.com/tomnomnom/gahttp
Last synced: about 1 year ago
JSON representation
Async / concurrent HTTP requests for Go
- Host: GitHub
- URL: https://github.com/tomnomnom/gahttp
- Owner: tomnomnom
- Created: 2018-09-02T10:57:35.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2023-06-29T09:54:52.000Z (almost 3 years ago)
- Last Synced: 2025-03-18T07:51:31.556Z (about 1 year ago)
- Language: Go
- Size: 7.81 KB
- Stars: 50
- Watchers: 3
- Forks: 12
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# gahttp
Async/concurrent HTTP requests for Go with rate-limiting.
Work in progress.
## Example
```golang
package main
import (
"fmt"
"net/http"
"time"
"github.com/tomnomnom/gahttp"
)
func printStatus(req *http.Request, resp *http.Response, err error) {
if err != nil {
return
}
fmt.Printf("%s: %s\n", req.URL, resp.Status)
}
func main() {
p := gahttp.NewPipeline()
p.SetConcurrency(20)
p.SetRateLimit(time.Second * 1)
urls := []string{
"http://example.com",
"http://example.com",
"http://example.com",
"http://example.net",
"http://example.org",
}
for _, u := range urls {
p.Get(u, gahttp.Wrap(printStatus, gahttp.CloseBody))
}
p.Done()
p.Wait()
}
```
## TODO
* `DoneAndWait()` func?
* Helper for writing responses to channel? (e.g. `func ChanWriter() (chan *Response, procFn)`)
- For when you don't want to do the work concurrently
* Actually handle timeouts / provide context interface for cancellation etc?