{"id":51058673,"url":"https://github.com/oborchers/pg-durable-evaluation","last_synced_at":"2026-06-22T23:30:53.027Z","repository":{"id":363125157,"uuid":"1262048322","full_name":"oborchers/pg-durable-evaluation","owner":"oborchers","description":"Independent local evaluation of Microsoft's pg_durable PostgreSQL extension as a durable workflow engine","archived":false,"fork":false,"pushed_at":"2026-06-07T16:02:56.000Z","size":43,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-07T16:09:45.341Z","etag":null,"topics":["database-migrations","docker","durable-workflows","job-scheduler","observability","pg-durable","postgres","postgres-extension","postgres-http","postgresql","psql","sql","workflow-engine"],"latest_commit_sha":null,"homepage":null,"language":"PLpgSQL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oborchers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-07T14:01:04.000Z","updated_at":"2026-06-07T16:03:00.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/oborchers/pg-durable-evaluation","commit_stats":null,"previous_names":["oborchers/pg-durable-evaluation"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/oborchers/pg-durable-evaluation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oborchers%2Fpg-durable-evaluation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oborchers%2Fpg-durable-evaluation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oborchers%2Fpg-durable-evaluation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oborchers%2Fpg-durable-evaluation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oborchers","download_url":"https://codeload.github.com/oborchers/pg-durable-evaluation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oborchers%2Fpg-durable-evaluation/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34669839,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-22T02:00:06.391Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database-migrations","docker","durable-workflows","job-scheduler","observability","pg-durable","postgres","postgres-extension","postgres-http","postgresql","psql","sql","workflow-engine"],"created_at":"2026-06-22T23:30:52.847Z","updated_at":"2026-06-22T23:30:53.020Z","avatar_url":"https://github.com/oborchers.png","language":"PLpgSQL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pg_durable Evaluation\n\nIndependent local evaluation of Microsoft's `pg_durable` PostgreSQL extension as a Postgres-native durable workflow/job engine.\n\nThis is a public companion repo for a technical blog post. It records what worked, what failed, and what needs more scrutiny before production use.\n\nThe harness builds the upstream extension from a pinned source checkout, runs it in Docker, and executes SQL scenarios that cover:\n\n- asynchronous fan-out aggregation\n- job dispatch and SQL function calls\n- durable variables and result substitution\n- HTTP/API calls that simulate an AI transformer step\n- signal-driven human approval\n- restart resilience across a PostgreSQL container restart\n- trigger-started workflows, transaction semantics, load/concurrency, retry behavior, DSL test strategy, and migration/deployment hazards\n- observability and developer-experience notes for a future technical blog post\n\n## Headline Findings\n\n- `pg_durable` is promising for Postgres-centered workflows where the database already owns the state.\n- Local Docker execution worked, including a restart-resilience probe.\n- Triggers can start workflows, which makes Postgres-native row-state orchestration plausible.\n- The SQL DSL is powerful but quote-sensitive, so SQL-level tests matter.\n- SQL node failures did not auto-retry in the tested flaky-function scenario.\n- In-flight workflows call the SQL function body that exists when the node executes, not the body that existed when the workflow started.\n- `df.http()` has a restrictive native HTTP security model, but it does not govern arbitrary HTTP-capable SQL extensions such as `postgres-http`.\n- `df.metrics()` can overcount failed instances after rolled-back starts because lower-level orphan execution rows are counted.\n- There is no built-in secret store (`df.secrets` is specced but not implemented in this build). `df.setvar` substitutes a key into request headers, but the resolved key is persisted in plain text in `df.vars` and in `duroxide.history.event_data` (runtime history). The graph definition (`df.nodes`) stores only the placeholder.\n\n## Quick Start\n\n```bash\n./scripts/run_experiments.sh --rebuild\n```\n\nAfter the first build, rerun without `--rebuild`:\n\n```bash\n./scripts/run_experiments.sh\n```\n\nThe runner leaves a PostgreSQL container running as `pg_durable_poc` and writes run artifacts to `runs/\u003ctimestamp\u003e/`.\n\nRun the production-adjacent follow-up suite:\n\n```bash\n./scripts/run_practical_experiments.sh\n```\n\nConnect manually:\n\n```bash\ndocker exec -it pg_durable_poc psql -U postgres\n```\n\n## Current Result\n\nLatest complete run: `runs/20260607T114327Z`.\n\nAll planned happy-path experiments passed, and the negative-path tests intentionally produced two failed pg_durable instances for blocked HTTP destinations. Raw run artifacts are local-only and ignored by Git; the distilled evidence is in `summary.md`, `findings.md`, and `EXPERIMENT.md`.\n\nFollow-up postgres-http comparison run: `runs/20260607T122843Z-postgres-http`.\n\nThat follow-up installed the separate PostgreSQL `http` extension in a derived image and called it from inside pg_durable SQL nodes. It worked, but it also showed an important security boundary: pg_durable's native `df.http()` allowlist does not govern arbitrary HTTP-capable SQL extensions.\n\nPractical follow-up run: `runs/20260607T125709Z-practical`.\n\nThat run tested trigger-started workflows, load/concurrency, retry behavior, transaction semantics, SQL-level DSL tests, and migration/deployment behavior. The most important caveats: SQL exceptions did not auto-retry in this test, workflows started in an uncommitted transaction must not be waited on before commit, and in-flight workflows used the replaced version of a SQL function when the later node executed.\n\nVerified follow-up reports filed upstream:\n\n- metrics mismatch from rolled-back starts: https://github.com/microsoft/pg_durable/issues/213\n- retry/backoff evidence added to existing issue: https://github.com/microsoft/pg_durable/issues/155#issuecomment-4642853550\n- `df.http()` allowlist scope clarification: https://github.com/microsoft/pg_durable/issues/214\n\n## Repository Contents\n\n- `sql/` contains the individual experiment scenarios.\n- `scripts/` contains the Docker-backed runners.\n- `docker/postgres-http.Dockerfile` builds the derived image used for the `postgres-http` comparison.\n- `summary.md` is the high-level blog-oriented summary.\n- `findings.md` is the detailed technical evaluation.\n- `EXPERIMENT.md` describes the experiment matrix.\n- `docs/research-notes.md` captures source-documentation notes and gotchas.\n\n## Upstream Pin\n\nSee `upstream.lock` for the exact source revision used by this PoC. The upstream source is cloned under `vendor/pg_durable/` for local builds and intentionally ignored by Git.\n\n## Notes For Readers\n\nThe harness builds from source, so the first run can take several minutes. Docker Hub authentication may matter for base-image pulls. The local image uses pg_durable's test-domain HTTP feature so the scenarios can call public test endpoints without secrets.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foborchers%2Fpg-durable-evaluation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foborchers%2Fpg-durable-evaluation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foborchers%2Fpg-durable-evaluation/lists"}