https://github.com/miyamo2/r2
Proof of concept for the Go 1.22 range functions and provides a simple and easy-to-use interface for sending HTTP requests with retries.
https://github.com/miyamo2/r2
go golang polling rangefunc retry
Last synced: 27 days ago
JSON representation
Proof of concept for the Go 1.22 range functions and provides a simple and easy-to-use interface for sending HTTP requests with retries.
- Host: GitHub
- URL: https://github.com/miyamo2/r2
- Owner: miyamo2
- License: mit
- Created: 2024-08-01T03:34:43.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2024-09-08T02:14:07.000Z (9 months ago)
- Last Synced: 2025-05-08T23:44:36.089Z (27 days ago)
- Topics: go, golang, polling, rangefunc, retry
- Language: Go
- Homepage:
- Size: 150 KB
- Stars: 6
- Watchers: 1
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# r2 - __range__ over http __request__
[](https://pkg.go.dev/github.com/miyamo2/r2)
[](https://img.shields.io/github/go-mod/go-version/miyamo2/r2)
[](https://img.shields.io/github/v/release/miyamo2/r2)
[](https://codecov.io/gh/miyamo2/r2)
[](https://goreportcard.com/report/github.com/miyamo2/r2)
[](https://img.shields.io/github/license/miyamo2/r2?&color=blue)
**r2** is a proof of concept for the Go 1.22 range functions and provides a simple and easy-to-use interface for sending HTTP requests with retries.
## Quick Start
### Install
```sh
go get github.com/miyamo2/r2
```
### Setup `GOEXPERIMENT`
> [!IMPORTANT]
>
> If your Go project is Go 1.23 or higher, this section is not necessary.
```sh
go env -w GOEXPERIMENT=rangefunc
```
### Simple Usage
```go
url := "http://example.com"
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
r2.WithPeriod(time.Second),
}
for res, err := range r2.Get(ctx, url, opts...) {
if err != nil {
slog.WarnContext(ctx, "something happened.", slog.Any("error", err))
// Note: Even if continue is used, the iterator could be terminated.
// Likewise, if break is used, the request could be re-executed in the background once more.
continue
}
if res == nil {
slog.WarnContext(ctx, "response is nil")
continue
}
if res.StatusCode != http.StatusOK {
slog.WarnContext(ctx, "unexpected status code.", slog.Int("expect", http.StatusOK), slog.Int("got", res.StatusCode))
continue
}
buf, err := io.ReadAll(res.Body)
if err != nil {
slog.ErrorContext(ctx, "failed to read response body.", slog.Any("error", err))
continue
}
slog.InfoContext(ctx, "response", slog.String("response", string(buf)))
// There is no need to close the response body yourself as auto closing is enabled by default.
}
```
vs 'github.com/avast/retry-go'
```go
url := "http://example.com"
var buf []byte
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
type ErrTooManyRequests struct{
error
RetryAfter time.Duration
}
opts := []retry.Option{
retry.Attempts(3),
retry.Context(ctx),
// In r2, the delay is calculated with the backoff and jitter by default.
// And, if 429 Too Many Requests are returned, the delay is set based on the Retry-After.
retry.DelayType(
func(n uint, err error, config *Config) time.Duration {
if err != nil {
var errTooManyRequests ErrTooManyRequests
if errors.As(err, &ErrTooManyRequests) {
if ErrTooManyRequests.RetryAfter != 0 {
return ErrTooManyRequests.RetryAfter
}
}
}
return retry.BackOffDelay(n, err, config)
}),
}
// In r2, the timeout period per request can be specified with the `WithPeriod` option.
client := http.Client{
Timeout: time.Second,
}
err := retry.Do(
func() error {
res, err := client.Get(url)
if err != nil {
return err
}
if res == nil {
return fmt.Errorf("response is nil")
}
if res.StatusCode == http.StatusTooManyRequests {
retryAfter := res.Header.Get("Retry-After")
if retryAfter != "" {
retryAfterDuration, err := time.ParseDuration(retryAfter)
if err != nil {
return &ErrTooManyRequests{error: fmt.Errorf("429: too many requests")}
}
return &ErrTooManyRequests{error: fmt.Errorf("429: too many requests"), RetryAfter: retryAfterDuration}
}
return &ErrTooManyRequests{error: fmt.Errorf("429: too many requests")}
}
if res.StatusCode >= http.StatusBadRequest && res.StatusCode < http.StatusInternalServerError {
// In r2, client errors other than TooManyRequests are excluded from retries by default.
return nil
}
if res.StatusCode >= http.StatusInternalServerError {
// In r2, automatically retry if the server error response is returned by default.
return fmt.Errorf("5xx: server error response")
}
if res.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected status code: expected %d, got %d", http.StatusOK, res.StatusCode)
}
// In r2, the response body is automatically closed by default.
defer res.Body.Close()
buf, err = io.ReadAll(res.Body)
if err != nil {
slog.ErrorContext(ctx, "failed to read response body.", slog.Any("error", err))
return err
}
return nil
},
opts...,
)
if err != nil {
// handle error
}
slog.InfoContext(ctx, "response", slog.String("response", string(buf)))
```
### Features
| Feature | Description |
|-------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`Get`](https://github.com/miyamo2/r2?tab=readme-ov-file#get) | Send HTTP Get requests until the [termination condition](https://github.com/miyamo2/r2?tab=readme-ov-file#termination-conditions) is satisfied. |
| [`Head`](https://github.com/miyamo2/r2?tab=readme-ov-file#head) | Send HTTP Head requests until the [termination condition](https://github.com/miyamo2/r2?tab=readme-ov-file#termination-conditions) is satisfied. |
| [`Post`](https://github.com/miyamo2/r2?tab=readme-ov-file#post) | Send HTTP Post requests until the [termination condition](https://github.com/miyamo2/r2?tab=readme-ov-file#termination-conditions) is satisfied. |
| [`Put`](https://github.com/miyamo2/r2?tab=readme-ov-file#put) | Send HTTP Put requests until the [termination condition](https://github.com/miyamo2/r2?tab=readme-ov-file#termination-conditions) is satisfied. |
| [`Patch`](https://github.com/miyamo2/r2?tab=readme-ov-file#patch) | Send HTTP Patch requests until the [termination condition](https://github.com/miyamo2/r2?tab=readme-ov-file#termination-conditions) is satisfied. |
| [`Delete`](https://github.com/miyamo2/r2?tab=readme-ov-file#delete) | Send HTTP Delete requests until the [termination condition](https://github.com/miyamo2/r2?tab=readme-ov-file#termination-conditions) is satisfied. |
| [`PostForm`](https://github.com/miyamo2/r2?tab=readme-ov-file#postform) | Send HTTP Post requests with form until the [termination condition](https://github.com/miyamo2/r2?tab=readme-ov-file#termination-conditions) is satisfied. |
| [`Do`](https://github.com/miyamo2/r2?tab=readme-ov-file#do) | Send HTTP requests with the given method until the [termination condition](https://github.com/miyamo2/r2?tab=readme-ov-file#termination-conditions) is satisfied. |
#### Get
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
r2.WithPeriod(time.Second),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
#### Head
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
r2.WithPeriod(time.Second),
}
for res, err := range r2.Head(ctx, "https://example.com", opts...) {
// do something
}
```
#### Post
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
r2.WithPeriod(time.Second),
r2.WithContentType(r2.ContentTypeApplicationJson),
}
body := bytes.NewBuffer([]byte(`{"foo": "bar"}`))
for res, err := range r2.Post(ctx, "https://example.com", body, opts...) {
// do something
}
```
#### Put
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
r2.WithPeriod(time.Second),
r2.WithContentType(r2.ContentTypeApplicationJson),
}
body := bytes.NewBuffer([]byte(`{"foo": "bar"}`))
for res, err := range r2.Put(ctx, "https://example.com", body, opts...) {
// do something
}
```
#### Patch
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
r2.WithPeriod(time.Second),
r2.WithContentType(r2.ContentTypeApplicationJson),
}
body := bytes.NewBuffer([]byte(`{"foo": "bar"}`))
for res, err := range r2.Patch(ctx, "https://example.com", body, opts...) {
// do something
}
```
#### Delete
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
r2.WithPeriod(time.Second),
r2.WithContentType(r2.ContentTypeApplicationJson),
}
body := bytes.NewBuffer([]byte(`{"foo": "bar"}`))
for res, err := range r2.Delete(ctx, "https://example.com", body, opts...) {
// do something
}
```
#### PostForm
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
r2.WithPeriod(time.Second),
r2.WithContentType(r2.ContentTypeApplicationJson),
}
form := url.Values{"foo": []string{"bar"}}
for res, err := range r2.Post(ctx, "https://example.com", form, opts...) {
// do something
}
```
#### Do
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
r2.WithPeriod(time.Second),
r2.WithContentType(r2.ContentTypeApplicationJson),
}
body := bytes.NewBuffer([]byte(`{"foo": "bar"}`))
for res, err := range r2.Do(ctx, http,MethodPost, "https://example.com", body, opts...) {
// do something
}
```
#### Termination Conditions
- Request succeeded and no termination condition is specified by `WithTerminateIf`.
- Condition that specified in `WithTerminateIf` is satisfied.
- Response status code is a `4xx Client Error` other than `429: Too Many Request`.
- Maximum number of requests specified in `WithMaxRequestAttempts` is reached.
- Exceeds the deadline for the `context.Context` passed in the argument.
- When the for range loop is interrupted by break.
### Options
**r2** provides the following request options
| Option | Description | Default |
|-------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------|
| [`WithMaxRequestAttempts`](https://github.com/miyamo2/r2?tab=readme-ov-file#withmaxrequesttimes) | The maximum number of requests to be performed.If less than or equal to 0 is specified, maximum number of requests does not apply. | `0` |
| [`WithPeriod`](https://github.com/miyamo2/r2?tab=readme-ov-file#withperiod) | The timeout period of the per request.If less than or equal to 0 is specified, the timeout period does not apply. If `http.Client.Timeout` is set, the shorter one is applied. | `0` |
| [`WithInterval`](https://github.com/miyamo2/r2?tab=readme-ov-file#withinterval) | The interval between next request.By default, the interval is calculated by the exponential backoff and jitter.If response status code is 429(Too Many Request), the interval conforms to 'Retry-After' header. | `0` |
| [`WithTerminateIf`](https://github.com/miyamo2/r2?tab=readme-ov-file#withterminateif) | The termination condition of the iterator that references the response. | `nil` |
| [`WithHttpClient`](https://github.com/miyamo2/r2?tab=readme-ov-file#withhttpclient) | The client to use for requests. | `http.DefaultClient` |
| [`WithHeader`](https://github.com/miyamo2/r2?tab=readme-ov-file#withheader) | The custom http headers for the request. | `http.Header`(blank) |
| [`WithContentType`](https://github.com/miyamo2/r2?tab=readme-ov-file#withcontenttype) | The 'Content-Type' for the request. | `''` |
| [`WithAspect`](https://github.com/miyamo2/r2?tab=readme-ov-file#withaspect) | The behavior to the pre-request/post-request. | - |
| [`WithAutoCloseResponseBody`](https://github.com/miyamo2/r2?tab=readme-ov-file#withautocloseresponse) | Whether the response body is automatically closed.By default, this setting is enabled. | `true` |
#### WithMaxRequestAttempts
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithMaxRequestAttempts(3),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
#### WithPeriod
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithPeriod(time.Second),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
#### WithInterval
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithInterval(time.Second),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
#### WithTerminateIf
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithTerminateIf(func(res *http.Response, _ error) bool {
myHeader := res.Header.Get("X-My-Header")
return len(myHeader) > 0
}),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
#### WithHttpClient
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
var myHttpClient *http.Client = getMyHttpClient()
opts := []r2.Option{
r2.WithHttpClient(myHttpClient),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
#### WithHeader
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithHeader(http.Header{"X-My-Header": []string{"my-value"}}),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
#### WithContentType
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithContentType("application/json"),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
#### WithAspect
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithAspect(func(req *http.Request, do func(req *http.Request) (*http.Response, error)) (*http.Response, error) {
res, err := do(req)
res.StatusCode += 1
return res, err
}),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
#### WithAutoCloseResponseBody
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
opts := []r2.Option{
r2.WithAutoCloseResponseBody(true),
}
for res, err := range r2.Get(ctx, "https://example.com", opts...) {
// do something
}
```
### Advanced Usage
[Read more advanced usages](https://github.com/miyamo2/r2/blob/main/.doc/ADVANCED_USAGE.md)
## For Contributors
Feel free to open a PR or an Issue.
However, you must promise to follow our [Code of Conduct](https://github.com/miyamo2/r2/blob/main/CODE_OF_CONDUCT.md).
### Tree
```sh
.
├ .doc/ # Documentation
├ .github/
│ └ workflows/ # GitHub Actions Workflow
├ internal/ # Internal Package; Shared with sub-packages.
└ tests/
├ integration/ # Integration Test
└ unit/ # Unit Test
```
### Tasks
We recommend that this section be run with [`xc`](https://github.com/joerdav/xc).
#### setup:deps
Install `mockgen` and `golangci-lint`.
```sh
go install go.uber.org/mock/mockgen@latest
go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest
```
#### setup:goenv
Set `GOEXPERIMENT` to `rangefunc` if Go version is 1.22.
```sh
GOVER=$(go mod graph)
if [[ $GOVER == *"[email protected]"* ]]; then
go env -w GOEXPERIMENT=rangefunc
fi
```
#### setup:mocks
Generate mock files.
```sh
go mod tidy
go generate ./...
```
#### lint
```sh
golangci-lint run --fix
```
#### test:unit
Run Unit Test
```sh
cd ./tests/unit
go test -v -coverpkg=github.com/miyamo2/r2 ./... -coverprofile=coverage.out
```
#### test:integration
Run Integration Test
```sh
cd ./tests/integration
go test -v -coverpkg=github.com/miyamo2/r2 ./... -coverprofile=coverage.out
```
## License
**r2** released under the [MIT License](https://github.com/miyamo2/r2/blob/main/LICENSE)