{"id":23155935,"url":"https://github.com/hyp3rd/go-again","last_synced_at":"2026-04-04T11:13:27.860Z","repository":{"id":65349713,"uuid":"590204304","full_name":"hyp3rd/go-again","owner":"hyp3rd","description":"`go-again` is an high-performance and thread-safe retry library with fine-grained access to the configuration options.","archived":false,"fork":false,"pushed_at":"2024-03-02T16:14:03.000Z","size":59,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-09T03:18:25.590Z","etag":null,"topics":["go","retry","retry-fuctions","retry-library","retryer"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hyp3rd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"github":["hyp3rd"]}},"created_at":"2023-01-17T21:54:08.000Z","updated_at":"2023-09-13T12:01:22.000Z","dependencies_parsed_at":"2024-03-02T17:28:05.496Z","dependency_job_id":"f41abc93-0fff-45b5-b0b6-e86a6217d640","html_url":"https://github.com/hyp3rd/go-again","commit_stats":{"total_commits":36,"total_committers":2,"mean_commits":18.0,"dds":"0.16666666666666663","last_synced_commit":"e8aa389ebefa72980a81bb768dfd8760b505fdc7"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/hyp3rd/go-again","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyp3rd%2Fgo-again","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyp3rd%2Fgo-again/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyp3rd%2Fgo-again/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyp3rd%2Fgo-again/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hyp3rd","download_url":"https://codeload.github.com/hyp3rd/go-again/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyp3rd%2Fgo-again/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263735418,"owners_count":23503507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","retry","retry-fuctions","retry-library","retryer"],"created_at":"2024-12-17T21:11:59.155Z","updated_at":"2026-04-04T11:13:27.849Z","avatar_url":"https://github.com/hyp3rd.png","language":"Go","funding_links":["https://github.com/sponsors/hyp3rd"],"categories":[],"sub_categories":[],"readme":"# go-again\n\n[![Go](https://github.com/hyp3rd/go-again/actions/workflows/go.yml/badge.svg)][build-link] [![CodeQL](https://github.com/hyp3rd/go-again/actions/workflows/codeql.yml/badge.svg)][codeql-link]\n\n`go-again` provides:\n\n- `again`: a thread-safe retry helper with exponential backoff, jitter, timeout, cancellation, hooks, and a temporary-error registry.\n- `pkg/scheduler`: a lightweight HTTP scheduler with pluggable state storage (in-memory by default, SQLite built-in) that reuses the retrier for retryable requests and optional callbacks.\n\n## Status\n\nAs of February 27, 2026, the core retrier hardening work and the scheduler extension described in `PRD.md` are implemented and covered by tests, including race checks.\n\n## Features\n\n### Retrier (`github.com/hyp3rd/go-again`)\n\n- Configurable `MaxRetries`, `Interval`, `Jitter`, `BackoffFactor`, and `Timeout`\n- `Do` and `DoWithContext` retry APIs\n- Temporary error filtering via explicit error list and/or `Registry`\n- Retry-all behavior when no temporary errors are supplied and the registry is empty\n- Cancellation via caller context and `Retrier.Cancel()` / `Retrier.Stop()`\n- `Errors` trace (`Attempts`, `Last`) plus `Errors.Join()`\n- `DoWithResult[T]` helper\n- Optional `slog` logger and retry hooks\n\n### Scheduler (`github.com/hyp3rd/go-again/pkg/scheduler`)\n\n- Interval scheduling with `StartAt`, `EndAt`, and `MaxRuns`\n- HTTP request execution (`GET`, `POST`, `PUT`)\n- Retry integration via `RetryPolicy`\n- Optional callback with bounded response-body capture\n- URL validation by default (via `sectools`) with override/disable support\n- Custom HTTP client, logger, concurrency limit, and scheduler-state storage backend\n\n## Installation\n\n```bash\ngo get github.com/hyp3rd/go-again\n```\n\nRequires Go `1.26+` (see `go.mod`).\n\n## Retrier Quick Start\n\n```go\npackage main\n\nimport (\n \"context\"\n \"errors\"\n \"fmt\"\n \"net/http\"\n \"time\"\n\n again \"github.com/hyp3rd/go-again\"\n)\n\nfunc main() {\n retrier, err := again.NewRetrier(\n  context.Background(),\n  again.WithMaxRetries(3),                 // retries after the first attempt\n  again.WithInterval(100*time.Millisecond),\n  again.WithJitter(50*time.Millisecond),\n  again.WithTimeout(2*time.Second),\n )\n if err != nil {\n  panic(err)\n }\n\n retrier.Registry.LoadDefaults()\n retrier.Registry.RegisterTemporaryError(http.ErrAbortHandler)\n\n var attempts int\n errs := retrier.Do(context.Background(), func() error {\n  attempts++\n  if attempts \u003c 3 {\n   return http.ErrAbortHandler\n  }\n\n  return nil\n })\n defer retrier.PutErrors(errs)\n\n if errs.Last != nil {\n  fmt.Println(\"failed:\", errs.Last)\n  return\n }\n\n fmt.Println(\"success after attempts:\", attempts)\n _ = errors.Join(errs.Attempts...) // equivalent to errs.Join()\n}\n```\n\n### Retrier Notes\n\n- `MaxRetries` counts retries after the first attempt (`total attempts = MaxRetries + 1`).\n- If `temporaryErrors` is omitted and `Registry` has entries, the registry is used as the retry filter.\n- If `temporaryErrors` is omitted and the registry is empty, all errors are retried until success/timeout/cancel/max-retries.\n- `Do` checks cancellation between attempts. For long-running work, use `DoWithContext`.\n- `Cancel()` and `Stop()` cancel the retrier's internal lifecycle context; they are terminal for that retrier instance.\n\n## Context-Aware Retrying\n\nUse `DoWithContext` when the operation itself accepts a context and should stop promptly on cancellation:\n\n```go\n// assuming `retrier` was created as in the previous example\nctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)\ndefer cancel()\n\nerrs := retrier.DoWithContext(ctx, func(ctx context.Context) error {\n select {\n case \u003c-time.After(250 * time.Millisecond):\n  return nil\n case \u003c-ctx.Done():\n  return ctx.Err()\n }\n})\ndefer retrier.PutErrors(errs)\n```\n\nThe retryable function should observe `ctx.Done()`; if it ignores context cancellation, the work may continue running after the retrier returns.\n\n## Scheduler Quick Start\n\nThe scheduler runs jobs immediately when scheduled (or at `StartAt` if set), then continues every `Schedule.Every` until `MaxRuns`, `EndAt`, removal, or `Stop()`.\n\nRequest and callback URLs are validated by default using `sectools` (HTTPS only, no userinfo, and no private/localhost hosts unless configured otherwise).\n\n```go\npackage main\n\nimport (\n \"context\"\n \"net/http\"\n \"time\"\n\n again \"github.com/hyp3rd/go-again\"\n \"github.com/hyp3rd/go-again/pkg/scheduler\"\n)\n\nfunc main() {\n retrier, _ := again.NewRetrier(\n  context.Background(),\n  again.WithMaxRetries(5),\n  again.WithInterval(10*time.Millisecond),\n  again.WithJitter(10*time.Millisecond),\n  again.WithTimeout(5*time.Second),\n )\n\n s := scheduler.NewScheduler(\n  scheduler.WithConcurrency(8),\n )\n defer s.Stop()\n\n_, _ = s.Schedule(scheduler.Job{\n  Schedule: scheduler.Schedule{\n   Every:   1 * time.Minute,\n   MaxRuns: 1,\n  },\n  Request: scheduler.Request{\n   Method: http.MethodPost,\n   URL:    \"https://example.com/endpoint\",\n   Body:   []byte(`{\"ping\":\"pong\"}`),\n  },\n  Callback: scheduler.Callback{\n   URL: \"https://example.com/callback\",\n  },\n  RetryPolicy: scheduler.RetryPolicy{\n   Retrier:          retrier,\n   RetryStatusCodes: []int{http.StatusTooManyRequests, http.StatusInternalServerError},\n },\n})\n}\n```\n\n## Scheduler Examples\n\n### Example: Schedule Once + Callback\n\nRunnable version:\n\n```bash\ngo run ./__examples/scheduler\n```\n\nSource:\n[`__examples/scheduler/scheduler.go`](__examples/scheduler/scheduler.go)\n\n```go\ns := scheduler.NewScheduler(\n scheduler.WithHTTPClient(server.Client()),\n scheduler.WithURLValidator(nil), // allow local endpoints for example usage\n)\ndefer s.Stop()\n\njobID, err := s.Schedule(scheduler.Job{\n Schedule: scheduler.Schedule{Every: 10 * time.Millisecond, MaxRuns: 1},\n Request: scheduler.Request{Method: http.MethodGet, URL: server.URL + \"/target\"},\n Callback: scheduler.Callback{URL: server.URL + \"/callback\"},\n})\nif err != nil {\n panic(err)\n}\n\npayload := \u003c-callbackCh\nfmt.Println(\"job:\", jobID, \"success:\", payload.Success, \"status:\", payload.StatusCode)\n```\n\n### Example: Query Status and History\n\n```go\n// after Schedule(...)\nstatus, ok := s.JobStatus(jobID)\nif ok {\n fmt.Println(\"state:\", status.State, \"runs:\", status.Runs, \"active:\", status.ActiveRuns)\n}\n\nhistory, ok := s.JobHistory(jobID)\nif ok {\n for _, run := range history {\n  fmt.Println(\"run#\", run.Sequence, \"status:\", run.Payload.StatusCode, \"success:\", run.Payload.Success)\n }\n}\n\nfiltered := s.QueryJobStatuses(scheduler.JobStatusQuery{\n States: []scheduler.JobState{scheduler.JobStateRunning, scheduler.JobStateScheduled},\n Offset: 0,\n Limit:  50,\n})\nfmt.Println(\"filtered statuses:\", len(filtered))\n\nrecentRuns, ok := s.QueryJobHistory(jobID, scheduler.JobHistoryQuery{\n FromSequence: 10,\n Limit:        5,\n})\nif ok {\n fmt.Println(\"recent retained runs:\", len(recentRuns))\n}\n```\n\n### Example: Durable Scheduler State with SQLite\n\nRunnable version:\n\n```bash\ngo run ./__examples/scheduler_sqlite\n```\n\nSource:\n[`__examples/scheduler_sqlite/scheduler_sqlite.go`](__examples/scheduler_sqlite/scheduler_sqlite.go)\n\n```go\ndbPath := filepath.Join(os.TempDir(), \"go-again-scheduler-example.db\")\nstorage, err := scheduler.NewSQLiteJobsStorageWithOptions(\n dbPath,\n scheduler.WithSQLiteHistoryMaxAge(24*time.Hour),\n scheduler.WithSQLiteHistoryMaxRowsPerJob(100),\n)\nif err != nil {\n panic(err)\n}\ndefer storage.Close()\n\ns := scheduler.NewScheduler(\n scheduler.WithJobsStorage(storage),\n scheduler.WithURLValidator(nil),\n)\ndefer s.Stop()\n\njobID, err := s.Schedule(scheduler.Job{\n Schedule: scheduler.Schedule{Every: 20 * time.Millisecond, MaxRuns: 1},\n Request: scheduler.Request{Method: http.MethodGet, URL: target.URL},\n})\nif err != nil {\n panic(err)\n}\nfmt.Println(\"scheduled job:\", jobID)\n\npruned, err := storage.PruneHistory()\nif err != nil {\n panic(err)\n}\nfmt.Println(\"pruned rows:\", pruned)\n```\n\n### Example: Fail-Closed Scheduler Construction\n\nUse `NewSchedulerWithError(...)` when constructor-time URL validator initialization errors must fail startup.\n\n```go\ns, err := scheduler.NewSchedulerWithError(\n scheduler.WithConcurrency(8),\n)\nif err != nil {\n // fail startup instead of warning + degraded mode\n return err\n}\ndefer s.Stop()\n```\n\n### Scheduler Options\n\n- `WithHTTPClient(client)` sets the HTTP client used for requests and callbacks.\n- `WithLogger(logger)` sets the scheduler logger.\n- `WithConcurrency(n)` limits concurrent executions when `n \u003e 0`.\n- `WithJobsStorage(storage)` sets pluggable scheduler state storage (active jobs plus status/history; default: in-memory).\n- `WithHistoryLimit(limit)` sets retained per-job history length (default `20`).\n- `WithURLValidator(validator)` overrides URL validation. Pass `nil` to disable validation.\n- `NewSchedulerWithError(...)` returns constructor errors (including startup state reconciliation failures and default URL validator initialization failure).\n\n### Scheduler Behavior Notes\n\n- Supported methods for requests and callbacks: `GET`, `POST`, `PUT`.\n- Callbacks are skipped when `Callback.URL` is empty.\n- Callback method defaults to `POST`.\n- `Callback.MaxBodyBytes` defaults to `4096`.\n- `Request.Timeout` and `Callback.Timeout` apply per HTTP request/callback (not the schedule lifetime).\n- If `RetryPolicy.Retrier` is nil, the scheduler creates a default retrier and loads registry defaults.\n- Calling `Schedule` after `Stop()` returns `scheduler.ErrSchedulerStopped`.\n- `Schedule` returns `scheduler.ErrStorageOperation` when required scheduler-state persistence fails.\n- `NewSchedulerWithError(...)` should be preferred for fail-closed startup behavior in security-sensitive paths.\n- `JobCount()` and `JobIDs()` provide lightweight read-only scheduler introspection.\n- `JobStatus(id)`, `JobStatuses()`, and `JobHistory(id)` provide status and retained run history snapshots.\n- `QueryJobStatuses(JobStatusQuery)` adds ID/state filters with pagination (`Offset`, `Limit`) over status snapshots.\n- `QueryJobHistory(id, JobHistoryQuery)` adds history filtering (`FromSequence`) and tail limiting (`Limit`) while preserving ascending sequence order.\n- Default `InMemoryJobsStorage` is process-local; use `WithJobsStorage(...)` for custom durable/backed storage.\n- `NewSQLiteJobsStorage(path)` provides a built-in durable storage implementation for `WithJobsStorage(...)`; call `Close()` when finished.\n- `NewSQLiteJobsStorageWithOptions(path, ...)` configures SQLite retention controls: `WithSQLiteHistoryMaxAge(duration)`, `WithSQLiteHistoryMaxRowsPerJob(n)`, and `WithSQLiteHistoryRetention(...)`.\n- `SQLiteJobsStorage.PruneHistory()` and `PruneHistoryWithRetention(...)` provide manual pruning for periodic cleanup jobs.\n- SQLite retention is also applied on write for new history records (age-based and max-rows-per-job), in addition to scheduler `WithHistoryLimit`.\n- On scheduler startup, recovered active-job registrations from storage are reconciled: `scheduled`/`running` states are marked `canceled`, then active-job IDs are cleared. Jobs are not auto-resumed.\n- Non-fatal storage write failures during runtime transitions are logged (warn) and execution continues.\n- Non-fatal request/callback response body read/close failures are logged (warn) and execution continues.\n- `NewSchedulerWithError(...)` fails constructor-time reconciliation errors; `NewScheduler()` logs a warning and continues.\n- `NewScheduler()` logs a warning and continues if default URL validator initialization fails; use `NewSchedulerWithError()` to fail closed.\n\n### Custom URL Validation (Allow Local HTTPS)\n\n```go\nvalidator, _ := validate.NewURLValidator(\n validate.WithURLAllowPrivateIP(true),\n validate.WithURLAllowLocalhost(true),\n validate.WithURLAllowIPLiteral(true),\n)\n\ns := scheduler.NewScheduler(\n scheduler.WithURLValidator(validator),\n)\n```\n\n### Disable URL Validation (Allow Non-HTTPS)\n\n```go\ns := scheduler.NewScheduler(\n scheduler.WithURLValidator(nil),\n)\n```\n\n## Examples\n\nRun the example programs directly:\n\n```bash\ngo run ./__examples/chan\ngo run ./__examples/context\ngo run ./__examples/scheduler\ngo run ./__examples/scheduler_sqlite\ngo run ./__examples/timeout\ngo run ./__examples/validate\n```\n\n## Development Commands\n\n```bash\nmake test\nmake test-race\nmake lint\nmake sec\n```\n\nBenchmark (direct Go command):\n\n```bash\ngo test -bench=. -benchtime=3s -benchmem -run=^$ -memprofile=mem.out ./...\n```\n\n## Known Limitations / Gaps\n\n- `Scheduler.Stop()` cancels the scheduler lifecycle; the same instance is not intended to be reused afterward.\n- `Retrier.Cancel()` / `Retrier.Stop()` are terminal for the retrier instance.\n- `DoWithContext` can only stop work promptly if the retryable function respects the provided context.\n- `NewScheduler()` (non-error constructor) intentionally degrades to warning-only behavior if default URL validator initialization fails; use `NewSchedulerWithError()` when you need constructor-time failure.\n\n## Performance\n\n`go-again` adds retry orchestration overhead but is designed to keep allocations low. See the benchmark in `tests/retrier_test.go` and run the benchmark command above in your environment for current numbers.\n\n## Documentation\n\n- API docs: \u003chttps://pkg.go.dev/github.com/hyp3rd/go-again\u003e\n- Product/status notes: `PRD.md`\n\n## License\n\nThe code and documentation in this project are released under Mozilla Public License 2.0.\n\n## Author\n\nI'm a surfer, a crypto trader, and a software architect with 15 years of experience designing highly available distributed production environments and developing cloud-native apps in public and private clouds. Feel free to hook me up on [LinkedIn](https://www.linkedin.com/in/francesco-cosentino/).\n\n[build-link]: https://github.com/hyp3rd/go-again/actions/workflows/go.yml\n[codeql-link]: https://github.com/hyp3rd/go-again/actions/workflows/codeql.yml\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyp3rd%2Fgo-again","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhyp3rd%2Fgo-again","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyp3rd%2Fgo-again/lists"}