{"id":51459778,"url":"https://github.com/scttfrdmn/substrate","last_synced_at":"2026-07-06T04:30:36.073Z","repository":{"id":345650377,"uuid":"1175599197","full_name":"scttfrdmn/substrate","owner":"scttfrdmn","description":"Event-sourced AWS emulator for deterministic testing of AI-generated infrastructure code","archived":false,"fork":false,"pushed_at":"2026-06-10T05:37:13.000Z","size":1878,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-10T06:28:10.959Z","etag":null,"topics":["aws","aws-cdk","aws-emulator","boto3","cloudformation","devops","event-sourcing","go","golang","infrastructure-as-code","integration-testing","localstack-alternative","mock","terraform","testing"],"latest_commit_sha":null,"homepage":"https://scttfrdmn.github.io/substrate/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scttfrdmn.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-07T23:22:15.000Z","updated_at":"2026-06-10T05:36:49.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/scttfrdmn/substrate","commit_stats":null,"previous_names":["scttfrdmn/substrate"],"tags_count":108,"template":false,"template_full_name":null,"purl":"pkg:github/scttfrdmn/substrate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scttfrdmn%2Fsubstrate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scttfrdmn%2Fsubstrate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scttfrdmn%2Fsubstrate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scttfrdmn%2Fsubstrate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scttfrdmn","download_url":"https://codeload.github.com/scttfrdmn/substrate/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scttfrdmn%2Fsubstrate/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35178402,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-06T02:00:07.184Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-cdk","aws-emulator","boto3","cloudformation","devops","event-sourcing","go","golang","infrastructure-as-code","integration-testing","localstack-alternative","mock","terraform","testing"],"created_at":"2026-07-06T04:30:34.975Z","updated_at":"2026-07-06T04:30:36.058Z","avatar_url":"https://github.com/scttfrdmn.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Substrate\n\n\u003e **The test harness for AI-generated infrastructure.**\n\u003e Deterministic. Time-travel debuggable. Cost-visible.\n\n[![CI](https://github.com/scttfrdmn/substrate/actions/workflows/ci.yml/badge.svg)](https://github.com/scttfrdmn/substrate/actions/workflows/ci.yml)\n[![Go Report Card](https://goreportcard.com/badge/github.com/scttfrdmn/substrate)](https://goreportcard.com/report/github.com/scttfrdmn/substrate)\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)\n\n## What is Substrate?\n\nSubstrate is an **event-sourced AWS emulator** for testing the infrastructure\ncode that drives AWS — CloudFormation, CDK, Terraform, and any SDK or CLI call —\n**deterministically, offline, and with cost visibility**, before you deploy to a\nreal account.\n\nIt models **what is observable through an AWS API call** — request/response\nshapes, resource state and how it transitions over a simulated clock, error\ncodes, and seedable outcomes — *not* what software inside a resource does. It\nnever runs your user-data, your Lambda code, an inference, or a training job;\nthose are recorded as intent with a **seedable** result. That boundary is what\nmakes every run reproducible: API observations can be recorded as events and\nreplayed identically, whereas a real workload's timing, scheduling, and I/O\ncannot.\n\n**Use it two ways:**\n- **As a server** — run `substrate`, point any AWS SDK/CLI at `http://localhost:4566`.\n  This is how most consumers use it.\n- **As a Go test harness** — `import \".../emulator\"`, spin up an in-process server\n  or deploy a CloudFormation template directly, no HTTP needed.\n\n## The Problem\n\n```\nAI generates infrastructure code → ??? → Deploy to AWS → $$$\n                                    ^\n                             This is where you find out\n```\n\nAI generates infrastructure code *at volume*, inside generate → test → fix loops.\nThat changes what a test harness has to do — and it's where most AWS emulators\nfall down.\n\n## Why Substrate for AI-generated infrastructure\n\nA generate-and-verify loop has needs a human-paced workflow doesn't have:\n\n- **Determinism, because flakes mislead the loop.** A non-deterministic failure is\n  a *false signal an agent acts on* — wasting fix cycles chasing timing noise.\n  Substrate records every request as an immutable event over a simulated clock, so\n  a failing run is real and replays identically. No retry-until-green.\n- **Free, fast, offline, so you can run it on every generation.** No real account,\n  no provisioning latency, no spend, no blast radius. Suitable for tight inner\n  loops and CI on every candidate the model produces.\n- **Cost visibility as a machine-gradeable guardrail.** An AI has no instinct for\n  bill shock. Substrate prices each operation, so you get a projected dollar figure\n  *before* deploy — and can gate on it: `betty.Deploy(ctx, tmpl, Intent{MaxCost: 1.0})`\n  fails the run if generated infra would blow a budget.\n- **Seedable failure paths, to verify the error handling the code claims to have.**\n  Generated code is full of retry/poll/wait logic for capacity errors, throttling,\n  and terminal states. Substrate lets you seed exactly those outcomes on demand, so\n  the unhappy paths are actually exercised — not just assumed.\n\n## How it compares\n\nSubstrate is a **different tier** from container emulators and real accounts, not a\ndrop-in replacement. It trades workload-execution fidelity for determinism,\nreplayability, and cost insight — the fast inner-loop tier.\n\n| | **Substrate** | LocalStack | moto | Real AWS |\n|---|:---:|:---:|:---:|:---:|\n| Deterministic replay | ✅ | ❌ | partial | ❌ |\n| Time-travel debugging | ✅ | ❌ | ❌ | ❌ |\n| Cost visibility before deploy | ✅ | ❌ | ❌ | after the bill |\n| Seedable failure / capacity / timing paths | ✅ | partial | partial | ❌ |\n| Runs your actual workload code | ❌ *(by design)* | ✅ | ❌ | ✅ |\n| Language-agnostic (any SDK/CLI over HTTP) | ✅ | ✅ | Python-first | ✅ |\n| No account · offline · free | ✅ | ✅ | ✅ | ❌ |\n\nThe one ❌ is deliberate: Substrate models **what is observable through an AWS API\ncall**, not what software inside a resource does (see\n[Scope \u0026 Philosophy](docs/scope.md)). If you need to execute your Lambda's code or\nboot a real container, reach for LocalStack or a real account; if you need to test\n*how your infrastructure code drives and reacts to the AWS API* — fast,\ndeterministically, and on every AI-generated candidate — that's Substrate.\n\n## Core capabilities\n\n### 1. Deterministic reproducibility\n\nEvery AWS request is an immutable event over a simulated clock. Same inputs +\nsame seed = same outputs, every time — no flakes, and a failing run replays\nidentically for debugging.\n\n```go\nsession, _ := engine.StartRecording(ctx, \"test-lambda-timeout\")\nrunTests() // fails\nengine.StopRecording(ctx, session)\n\n// Replay it 1000 times — identical failure every time.\nfor range 1000 {\n    results, _ := engine.Replay(ctx, session.StreamID)\n}\n```\n\n### 2. Time-travel debugging\n\nStep backward through recorded request history and inspect service state at any\npoint — see exactly where a sequence of API calls diverged from what you expected.\n\n```go\nreplay := engine.Replay(ctx, \"failing-test\")\nengine.JumpToEvent(ctx, 87)                  // jump to the failure\nengine.StepBackward(ctx)                      // step back\nstate, _ := engine.InspectState(ctx, \"iam\")   // see what broke\n```\n\n### 3. Cost visibility before deploy\n\nReal AWS pricing tracked per operation. Know your monthly bill before it arrives.\n\n```\nTotal: $1,247.50/month\n\n  S3 PUT:   $875.00  (175M ops @ $0.005/1K)\n  Lambda:   $267.50  (10M invocations)\n\nWARNING: High S3 PUT rate — consider batching (save ~99%)\n```\n\n### 4. Seedable outcomes (API-surface scope)\n\nDeterminism doesn't mean every test sees the same result. Substrate defaults to\nthe nominal success path; a test **seeds** an alternate outcome — an\n`InsufficientInstanceCapacity` on launch, a training job that comes back `Failed`\nwith a `CapacityError`, a specific query result — and the plugin returns it at\nrequest time, fully reproducibly. The failure, capacity, and timing paths that\nare rare or impossible to trigger against real AWS become trivial to test.\n\n## Quick Start (server)\n\nThe primary way to use Substrate is as a drop-in AWS endpoint.\n\n### Install\n\n```bash\ngo install github.com/scttfrdmn/substrate/cmd/substrate@latest\n```\n\nOr build from source / run with Docker:\n\n```bash\ngit clone https://github.com/scttfrdmn/substrate \u0026\u0026 cd substrate\nmake build          # produces ./bin/substrate\n\ndocker run -p 4566:4566 ghcr.io/scttfrdmn/substrate:latest\n```\n\n### Run the server\n\n```bash\nsubstrate server\n# Listening on :4566\n```\n\nConfiguration via `substrate.yaml` or environment variables (see\n[`substrate.yaml.example`](substrate.yaml.example)).\n\n### Point your AWS client at it\n\n\u003cdetails open\u003e\n\u003csummary\u003e\u003cstrong\u003eAWS CLI\u003c/strong\u003e\u003c/summary\u003e\n\n```bash\naws iam create-user --user-name alice \\\n    --endpoint-url http://localhost:4566 --region us-east-1 --no-sign-request\n```\n\nOr set a profile in `~/.aws/config`:\n\n```ini\n[profile substrate]\nregion = us-east-1\nendpoint_url = http://localhost:4566\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eGo SDK v2\u003c/strong\u003e\u003c/summary\u003e\n\n```go\ncfg, _ := config.LoadDefaultConfig(context.TODO(),\n    config.WithRegion(\"us-east-1\"),\n    config.WithBaseEndpoint(\"http://localhost:4566\"),\n    config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(\"test\", \"test\", \"\")),\n)\nclient := iam.NewFromConfig(cfg)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003ePython (boto3)\u003c/strong\u003e\u003c/summary\u003e\n\n```python\nimport boto3\n\nclient = boto3.client(\n    \"iam\", region_name=\"us-east-1\",\n    endpoint_url=\"http://localhost:4566\",\n    aws_access_key_id=\"test\", aws_secret_access_key=\"test\",\n)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eNode.js (AWS SDK v3)\u003c/strong\u003e\u003c/summary\u003e\n\n```javascript\nimport { IAMClient } from \"@aws-sdk/client-iam\";\n\nconst client = new IAMClient({\n  region: \"us-east-1\",\n  endpoint: \"http://localhost:4566\",\n  credentials: { accessKeyId: \"test\", secretAccessKey: \"test\" },\n});\n```\n\u003c/details\u003e\n\n## Use as a Go test harness\n\nFor Go code, the fastest path is `StartTestServer`, which spins up an in-process\nserver on a random port and registers a `t.Cleanup` to shut it down.\n\n```go\nimport \"github.com/scttfrdmn/substrate/emulator\"\n\nfunc TestMyInfra(t *testing.T) {\n    ts := emulator.StartTestServer(t)\n\n    cfg, _ := config.LoadDefaultConfig(context.Background(),\n        config.WithRegion(\"us-east-1\"),\n        config.WithBaseEndpoint(ts.URL),\n        config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(\"test\", \"test\", \"\")),\n    )\n    // Use cfg with any AWS SDK v2 client...\n}\n```\n\nOr deploy a CloudFormation template and validate it entirely in-process, no HTTP\nserver required, via the **Betty** client:\n\n```go\nimport \"github.com/scttfrdmn/substrate/emulator\"\n\nstore    := emulator.NewEventStore(emulator.EventStoreConfig{Enabled: true, Backend: \"memory\"})\nstate    := emulator.NewMemoryStateManager()\ntc       := emulator.NewTimeController(time.Now())\nregistry := emulator.NewPluginRegistry()\n\nbetty := emulator.NewBettyClient(registry, store, state, tc, logger)\n\nresult, _ := betty.Deploy(ctx, cfnTemplate, emulator.Intent{MaxCost: 1.0})\n\nsession, _ := betty.StartRecording(ctx, \"my-test\")\n// ... run operations against the emulator ...\nreport, _ := betty.StopRecording(ctx, session)\nfmt.Printf(\"status=%s cost=$%.4f\\n\", report.PassFail, report.Cost.Total)\n```\n\n\u003e The Go import path is `github.com/scttfrdmn/substrate/emulator`. Installing the\n\u003e CLI (`.../cmd/substrate@latest`) is unaffected.\n\nSee [`examples/betty_workflow/main.go`](examples/betty_workflow/main.go) for a\ncomplete runnable example.\n\n## Supported services\n\nSubstrate ships **63 built-in service plugins** spanning compute, storage,\nnetworking, databases, messaging, analytics, ML, security, and management:\n\n- **Compute \u0026 containers** — EC2, Lambda, ECS, ECR, Batch, EKS-adjacent\n- **Storage \u0026 data** — S3, EFS, FSx, DynamoDB, RDS, Redshift, ElastiCache, Timestream\n- **Networking** — VPC/EC2, ELBv2, Route 53, CloudFront, API Gateway (REST \u0026 HTTP)\n- **Messaging \u0026 streaming** — SQS, SNS, EventBridge, Kinesis, Firehose, MSK\n- **ML \u0026 analytics** — SageMaker, Bedrock Runtime, Athena, Glue, EMR Serverless, OpenSearch, QuickSight, HealthOmics\n- **Security \u0026 identity** — IAM, STS, KMS, Secrets Manager, ACM, Cognito, SSO, WAFv2, RAM\n- **Management \u0026 cost** — CloudWatch (+ Logs), CloudTrail, Organizations, Budgets, Cost Explorer, Service Quotas, SSM, Health, the CodeSuite, Step Functions, Backup, Transfer\n\nMany integrate with **Betty** for CloudFormation deployment. See the\n[Service Reference](docs/services.md) for the authoritative, per-operation list.\n\n### Known limitations\n\n- **Cross-service IAM enforcement** — policies are evaluated for IAM and STS;\n  per-operation enforcement for other services is partial.\n- **Persistence** — in-memory by default; SQLite available via\n  `EventStoreConfig{Backend: \"sqlite\"}`.\n- **Authentication** — SigV4 verification is opt-in (off by default for testing\n  ease); enable with `ServerOptions.VerifySignatures = true`.\n- **Workload internals are out of scope by design** — Substrate models the API\n  surface, not what runs inside a resource (see *What is Substrate?*).\n\n## Status\n\nCurrent release: **v0.68.0**. See [Releases](https://github.com/scttfrdmn/substrate/releases)\nand [CHANGELOG.md](CHANGELOG.md) for full history.\n\n## Documentation\n\n📖 **Full documentation: [scttfrdmn.github.io/substrate](https://scttfrdmn.github.io/substrate/)**\n\n- **[Getting Started](docs/getting-started.md)** — install, first test, 15-minute tutorial\n- **[Service Reference](docs/services.md)** — all 63 plugins with operation lists\n- **[Testing Guide](docs/testing-guide.md)** — `StartTestServer`, recording/replay, cost assertions\n- **[Endpoint Configuration](docs/endpoint-configuration.md)** — SDK and tool configuration\n\n## Development\n\n```bash\nmake test      # run tests with the race detector\nmake lint      # golangci-lint\nmake coverage  # coverage report\nmake build     # build the substrate binary\nmake e2e       # end-to-end tests\n```\n\nRequirements: Go 1.26+, [golangci-lint](https://golangci-lint.run/).\n\n## Contributing\n\nIssues and pull requests welcome. All work is tracked in\n[GitHub Issues](https://github.com/scttfrdmn/substrate/issues) and organised into\n[Milestones](https://github.com/scttfrdmn/substrate/milestones). `main` is\nprotected — changes land via pull request.\n\n## License\n\nApache 2.0 — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscttfrdmn%2Fsubstrate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscttfrdmn%2Fsubstrate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscttfrdmn%2Fsubstrate/lists"}