{"id":50907853,"url":"https://github.com/namuan/sqlite-durable-workflow","last_synced_at":"2026-06-16T07:01:29.785Z","repository":{"id":361407246,"uuid":"1254333277","full_name":"namuan/sqlite-durable-workflow","owner":"namuan","description":"Durable workflow engine powered by SQLite — event-sourced, exactly-once checkpointing, no external dependencies","archived":false,"fork":false,"pushed_at":"2026-05-30T14:45:12.000Z","size":448,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-30T15:07:15.548Z","etag":null,"topics":["durable-workflows","event-sourcing","go","orchestration","sqlite","workflow-automation","workflow-engine"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/namuan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-30T12:41:59.000Z","updated_at":"2026-05-30T14:49:57.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/namuan/sqlite-durable-workflow","commit_stats":null,"previous_names":["namuan/sqlite-durable-workflow"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/namuan/sqlite-durable-workflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namuan%2Fsqlite-durable-workflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namuan%2Fsqlite-durable-workflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namuan%2Fsqlite-durable-workflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namuan%2Fsqlite-durable-workflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/namuan","download_url":"https://codeload.github.com/namuan/sqlite-durable-workflow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namuan%2Fsqlite-durable-workflow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34393305,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-16T02:00:06.860Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["durable-workflows","event-sourcing","go","orchestration","sqlite","workflow-automation","workflow-engine"],"created_at":"2026-06-16T07:01:26.505Z","updated_at":"2026-06-16T07:01:29.770Z","avatar_url":"https://github.com/namuan.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sqlite-durable-workflow\n\nSQLite-native durable workflow engine for Go. Eliminates the need for a\ndedicated orchestration service by using SQLite as the sole source of truth\nfor workflow execution, persistence, recovery, scheduling, and coordination.\n\n## Design\n\n![](assets/c4-a.png)\n\n- **SQLite is the orchestrator** — no external queue, broker, or service\n- **Event-sourced** — every state transition is an immutable event\n- **Workers are stateless** — crash, restart, scale horizontally\n- **Exactly-once checkpointing** — enforced by primary key constraints\n- **Automatic recovery** — expired leases and event replay handle failures\n\n## Project structure\n\n![](assets/c4-b.png)\n\n## Install (Go library)\n\n```bash\ngo get github.com/nnn/sqlite-durable-workflow\n```\n\n## Quick start (Go library)\n\n```go\nimport durable \"github.com/nnn/sqlite-durable-workflow\"\n\nengine, _ := durable.New(\"workflows.db\")\ndefer engine.Close()\n\nwf, _ := engine.CreateWorkflow(\"order-123\", \"order_processing\")\nwf, _ = engine.AcquireLease(\"worker-1\", 30*time.Second)\nengine.CompleteStep(wf.WorkflowID, \"worker-1\", 1, input, output)\nengine.CompleteWorkflow(wf.WorkflowID, \"worker-1\")\n```\n\nRun the full example:\n\n```bash\ngo run ./cmd/example/\n```\n\n## HTTP server\n\nThe engine can run as a standalone JSON API server so workers in any language\ncan participate.\n\n```bash\ngo run ./cmd/server/ -db workflows.db -addr :8080\n```\n\n| Flag       | Default        | Description                                                  |\n|------------|----------------|--------------------------------------------------------------|\n| `-db`      | `workflows.db` | Path to SQLite database                                      |\n| `-addr`    | `:8080`        | HTTP listen address                                          |\n| `-tick`    | `1s`           | Scheduler poll interval                                      |\n| `-lease`   | `30s`          | Default lease duration                                       |\n| `-api-key` | _(empty)_      | Require `Bearer` token matching this value (empty = no auth) |\n\n### Authentication\n\nWhen the `-api-key` flag is set, all requests must include an `Authorization` header:\n\n```\nAuthorization: Bearer \u003cyour-api-key\u003e\n```\n\nRequests without a valid key receive `401 Unauthorized`.\n\n### Endpoints\n\n| Method | Path                               | Description                    |\n|--------|------------------------------------|--------------------------------|\n| `POST` | `/workflows`                       | Create a workflow              |\n| `GET`  | `/workflows/{id}`                  | Get workflow by ID             |\n| `POST` | `/workflows/{id}/complete`         | Mark workflow COMPLETED        |\n| `POST` | `/workflows/{id}/fail`             | Mark workflow FAILED           |\n| `POST` | `/workflows/{id}/waiting`          | Mark workflow WAITING          |\n| `POST` | `/leases/acquire`                  | Acquire next eligible lease    |\n| `POST` | `/leases/{id}/renew`               | Renew lease                    |\n| `POST` | `/leases/{id}/release`             | Release lease                  |\n| `POST` | `/steps/start`                     | Record step start              |\n| `POST` | `/steps/complete`                  | Checkpoint completed step      |\n| `POST` | `/steps/fail`                      | Record step failure            |\n| `GET`  | `/steps/{id}`                      | Get all checkpoints            |\n| `POST` | `/signals`                         | Send a signal                  |\n| `POST` | `/signals/{id}/consume`            | Atomically consume next signal |\n| `GET`  | `/signals/{id}`                    | List signals                   |\n| `POST` | `/timers`                          | Create a timer                 |\n| `POST` | `/retry`                           | Schedule a retry               |\n| `GET`  | `/events/{id}`                     | Get event history              |\n| `GET`  | `/observability/failed`            | All FAILED workflows           |\n| `GET`  | `/observability/running`           | All RUNNING workflows          |\n| `GET`  | `/observability/queued`            | All QUEUED workflows           |\n| `GET`  | `/observability/waiting`           | All WAITING workflows          |\n| `GET`  | `/observability/avg-step-duration` | Average step duration          |\n\n## TypeScript SDK\n\n```bash\ncd sdks/typescript\nnpm install\n```\n\n### Quick start (TypeScript)\n\n```ts\nimport {Client} from \"@sqlite-durable-workflow/sdk\";\n\nconst client = new Client(\"http://localhost:8080\");\n\n// With API key, timeout, and retry\nconst client = new Client(\"http://localhost:8080\", {\n    apiKey: \"my-secret-key\",\n    timeout: 10_000,\n    maxRetries: 3,\n    retryDelay: 500,\n});\n\n// Create a workflow\nconst wf = await client.createWorkflow(\"order-123\", \"order_processing\");\n\n// Acquire a lease and execute a step\nconst claimed = await client.acquireLease({worker_id: \"worker-1\"});\nif (claimed) {\n    await client.completeStep(claimed.workflow_id, \"worker-1\", 1,\n        {order_id: \"123\"},\n        {status: \"validated\"}\n    );\n    await client.completeWorkflow(claimed.workflow_id, \"worker-1\");\n}\n\n// Observability\nconst failed = await client.queryFailed();\nconst events = await client.getEvents(\"order-123\");\n```\n\n### Run the example\n\n```bash\n# Terminal 1: start the server\ngo run ./cmd/server/\n\n# Terminal 2: run the TypeScript example\ncd sdks/typescript \u0026\u0026 npx tsx examples/basic.ts\n```\n\n### TypeScript API\n\n| Method                                          | Description                    |\n|-------------------------------------------------|--------------------------------|\n| `createWorkflow(id, type)`                      | Create a new QUEUED workflow   |\n| `getWorkflow(id)`                               | Fetch a workflow by ID         |\n| `completeWorkflow(id, worker)`                  | Mark workflow COMPLETED        |\n| `failWorkflow(id, worker)`                      | Mark workflow FAILED           |\n| `markWaiting(id, worker)`                       | Pause workflow                 |\n| `acquireLease(cfg)`                             | Claim next eligible workflow   |\n| `renewLease(id, cfg)`                           | Extend lease                   |\n| `releaseLease(id, worker)`                      | Return to QUEUED               |\n| `startStep(id, worker, step, input)`            | Record step start              |\n| `completeStep(id, worker, step, input, output)` | Checkpoint completed step      |\n| `failStep(id, worker, step, input, error)`      | Record step failure            |\n| `getSteps(id)`                                  | Fetch checkpoints              |\n| `sendSignal(id, type, payload)`                 | Persist signal                 |\n| `consumeSignal(id)`                             | Atomically consume next signal |\n| `getSignals(id, unconsumedOnly)`                | List signals                   |\n| `createTimer(id, wakeAt)`                       | Schedule a wake-up             |\n| `retryStep(id, worker, step, count, cfg)`       | Schedule retry                 |\n| `getEvents(id)`                                 | Fetch event history            |\n| `queryFailed()`                                 | All FAILED workflows           |\n| `queryRunning()`                                | All RUNNING workflows          |\n| `queryQueued()`                                 | All QUEUED workflows           |\n| `queryWaiting()`                                | All WAITING workflows          |\n| `avgStepDuration()`                             | Average step duration          |\n\n### TypeScript types\n\n| Type             | Description                                                     |\n|------------------|-----------------------------------------------------------------|\n| `Workflow`       | Workflow record                                                 |\n| `WorkflowEvent`  | Immutable event                                                 |\n| `WorkflowStep`   | Step checkpoint                                                 |\n| `Signal`         | Async signal                                                    |\n| `Timer`          | Scheduled timer                                                 |\n| `RetryConfig`    | Retry configuration (fixed / linear / exponential)              |\n| `LeaseConfig`    | Lease parameters                                                |\n| `ClientOptions`  | Client config: `apiKey`, `timeout`, `maxRetries`, `retryDelay`  |\n| `WorkflowStatus` | `\"QUEUED\" \\| \"RUNNING\" \\| \"WAITING\" \\| \"COMPLETED\" \\| \"FAILED\"` |\n\n## Go API reference\n\n### Engine lifecycle\n\n| Method                         | Description                                     |\n|--------------------------------|-------------------------------------------------|\n| `New(dbPath) (*Engine, error)` | Open/create database and initialize schema      |\n| `Close() error`                | Close the database                              |\n| `DB() *sql.DB`                 | Access underlying connection for custom queries |\n\n### Workflows\n\n| Method                                        | Description                        |\n|-----------------------------------------------|------------------------------------|\n| `CreateWorkflow(id, type) (*Workflow, error)` | Create a new QUEUED workflow       |\n| `GetWorkflow(id) (*Workflow, error)`          | Fetch a workflow by ID             |\n| `CompleteWorkflow(id, worker) error`          | Mark workflow COMPLETED            |\n| `FailWorkflow(id, worker) error`              | Mark workflow FAILED               |\n| `MarkWaiting(id, worker) error`               | Pause workflow (timer/signal wait) |\n\n### Leases\n\n| Method                                              | Description                        |\n|-----------------------------------------------------|------------------------------------|\n| `AcquireLease(worker, duration) (*Workflow, error)` | Claim next eligible workflow       |\n| `RenewLease(id, worker, duration) error`            | Extend lease for long-running work |\n| `ReleaseLease(id, worker) error`                    | Return workflow to QUEUED          |\n\n### Step execution\n\n| Method                                                | Description                                 |\n|-------------------------------------------------------|---------------------------------------------|\n| `StartStep(id, worker, step, input) error`            | Record step start + STEP_STARTED event      |\n| `CompleteStep(id, worker, step, input, output) error` | Checkpoint step, advance cursor, emit event |\n| `FailStep(id, worker, step, input, err) error`        | Record step failure                         |\n| `GetSteps(id) ([]WorkflowStep, error)`                | Fetch all checkpoints                       |\n\n### Signals\n\n| Method                                             | Description                    |\n|----------------------------------------------------|--------------------------------|\n| `SendSignal(id, type, payload) (*Signal, error)`   | Persist async signal           |\n| `ConsumeSignal(id) (*Signal, error)`               | Atomically consume next signal |\n| `GetSignals(id, unconsumedOnly) ([]Signal, error)` | List signals                   |\n\n### Timers\n\n| Method                                    | Description         |\n|-------------------------------------------|---------------------|\n| `CreateTimer(id, wakeAt) (*Timer, error)` | Schedule a wake-up  |\n| `GetPendingTimers(id) ([]Timer, error)`   | List unfired timers |\n\n### Retry\n\n| Method                                                  | Description                                 |\n|---------------------------------------------------------|---------------------------------------------|\n| `RetryStep(id, worker, step, count, cfg) (bool, error)` | Schedule retry or return false if exhausted |\n\nBuilt-in backoff policies: `FixedBackoff`, `LinearBackoff`, `ExponentialBackoff`.\n\n### Scheduler\n\n| Method                                      | Description                                |\n|---------------------------------------------|--------------------------------------------|\n| `NewScheduler(engine, interval) *Scheduler` | Create background scheduler                |\n| `Run(ctx)`                                  | Blocking loop — fire timers, expire leases |\n| `Tick()`                                    | Single manual tick (for tests)             |\n\n### Observability\n\n| Method                                       | Description                     |\n|----------------------------------------------|---------------------------------|\n| `QueryFailed() ([]Workflow, error)`          | All FAILED workflows            |\n| `QueryRunning() ([]Workflow, error)`         | All RUNNING workflows           |\n| `QueryQueued() ([]Workflow, error)`          | All QUEUED workflows            |\n| `QueryWaiting() ([]Workflow, error)`         | All WAITING workflows           |\n| `AvgStepDuration() (map[int]float64, error)` | Average step duration by number |\n| `GetEvents(id) ([]WorkflowEvent, error)`     | Full event history              |\n\n## Testing\n\n```bash\nmake test          # run all tests (Go + TypeScript)\nmake test-go       # Go tests only (with race detector)\nmake test-ts       # TypeScript integration tests\n```\n\n### Go test suite (180 tests + 20 benchmarks)\n\n| File                            | Tests | Coverage |\n|---------------------------------|-------|----------|\n| `engine_test.go`                | 25    | Core lifecycle, leases, step checkpointing, signals, timers, retry, events, observability |\n| `engine_schema_test.go`         | 16    | Table existence, column nullability, indexes, defaults, PRAGMAs, foreign keys, STRICT enforcement |\n| `engine_state_machine_test.go`  | 30    | All 5 states reachable, valid/invalid transitions, version increments, FIFO ordering, lease lifecycle, boundary ID values |\n| `engine_concurrency_test.go`    | 15    | Multi-goroutine lease races, worker pools, concurrent signal/timer/step operations, mixed-operation stress |\n| `engine_edge_cases_test.go`     | 30    | Nil/boundary payloads, large step numbers, nonexistent workflows, empty queues, timer edge cases, JSON round-trip |\n| `engine_recovery_test.go`       | 10    | Crash with lease held, mid-step recovery, multi-workflow survival, signal/timer persistence, reopen, rolled-back transactions |\n| `engine_event_sourcing_test.go` | 15    | Sequence continuity, all event types emitted, monotonicity, multi-workflow isolation, state replay from events |\n| `engine_complex_scenarios_test.go` | 12 | Full saga with retry, signal-as-trigger, timer continuation, chained timers, interleaved workflows, lease transfer |\n| `engine_scheduler_test.go`      | 14    | Timer firing, future timer protection, lease expiry, continuous run, cancellation, multiple timers/leases, idle ticks |\n| `engine_benchmarks_test.go`     | 20    | Create, acquire, complete step, signal, timer throughput; concurrent variants; full lifecycle; scheduler overhead |\n\nAll Go tests run with `-race` (Go race detector) enabled.\n\n### TypeScript integration tests (22 tests)\n\nEnd-to-end tests in `sdks/typescript/tests/sdk.test.ts` spawn a real `cmd/server/` process and exercise every API endpoint. Vitest runner with 15-second timeouts.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnamuan%2Fsqlite-durable-workflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnamuan%2Fsqlite-durable-workflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnamuan%2Fsqlite-durable-workflow/lists"}