{"id":51908914,"url":"https://github.com/rafaesc/cloud-ledger","last_synced_at":"2026-07-27T02:01:33.447Z","repository":{"id":365336192,"uuid":"1266552646","full_name":"rafaesc/cloud-ledger","owner":"rafaesc","description":null,"archived":false,"fork":false,"pushed_at":"2026-07-01T16:01:54.000Z","size":2100,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-07-01T17:29:10.088Z","etag":null,"topics":["aws","awsxray","cicd","devcontainer","docker","dynamodb","foci","java","lambda","python","spring-boot","sqs","terraform"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rafaesc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-11T18:18:01.000Z","updated_at":"2026-07-01T17:17:58.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/rafaesc/cloud-ledger","commit_stats":null,"previous_names":["rafaesc/cloud-ledger"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/rafaesc/cloud-ledger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rafaesc%2Fcloud-ledger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rafaesc%2Fcloud-ledger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rafaesc%2Fcloud-ledger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rafaesc%2Fcloud-ledger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rafaesc","download_url":"https://codeload.github.com/rafaesc/cloud-ledger/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rafaesc%2Fcloud-ledger/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35933751,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-27T02:00:06.776Z","response_time":101,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","awsxray","cicd","devcontainer","docker","dynamodb","foci","java","lambda","python","spring-boot","sqs","terraform"],"created_at":"2026-07-27T02:01:32.725Z","updated_at":"2026-07-27T02:01:33.433Z","avatar_url":"https://github.com/rafaesc.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CloudLedger\n\n**An audit-grade, event-sourced financial ledger API on AWS — the primitive layer payment systems are built on.**\n\n[![Local E2E](https://github.com/rafaesc/cloud-ledger/actions/workflows/e2e.yml/badge.svg)](https://github.com/rafaesc/cloud-ledger/actions/workflows/e2e.yml)\n[![Deployed Prod](https://github.com/rafaesc/cloud-ledger/actions/workflows/deploy-prod.yml/badge.svg)](https://github.com/rafaesc/cloud-ledger/actions/workflows/deploy-prod.yml)\n![Java 21](https://img.shields.io/badge/Java-21-blue)\n![Spring Boot 4.1](https://img.shields.io/badge/Spring%20Boot-4.1-6db33f)\n![Python 3.12](https://img.shields.io/badge/Python-3.12-3776ab)\n![IaC Terraform](https://img.shields.io/badge/IaC-Terraform-7b42bc)\n![k6](https://img.shields.io/badge/Load%20Test-k6-7d64ff)\n\n![Local mirrors cloud](docs/f628d673-32e5-49db-865e-fc08c325aec9.png)\n\n## What it is\n\nA financial ledger must be tamper-proof, auditable, and correct under concurrent writes. CloudLedger\nsolves this with an **append-only Aurora event store** — balances are never updated in place; every\ndeposit, withdrawal, and transfer is an immutable fact (`MoneyDeposited`, `TransferDebited`, …)\nreplayed on read, so every state an account was ever in is reconstructable. Concurrent writes are\ncoordinated by an **optimistic-lock fence** (`WHERE version = :expected`) that surfaces contention as\na `409 Conflict` and a retry — never silent data loss.\n\nOn right-sized dev infrastructure it sustains **p99 \u003c 200 ms up to ~150 transfers/s** before the tail\nknee, with reads served sub-10 ms from a Redis write-through cache (DynamoDB fallback). Full latency\ncurve and load evidence: [ADR-004](docs/adrs/ADR-004-right-size-before-tuning.md). All infrastructure\nis **Terraform** — S3 backend + native lock, per-service KMS CMKs, `local`/`prod` environments.\n\n## Architecture\n\n![Architecture](docs/diagram-export-3-7-2026-2_27_11-a.-m..png)\n\nCommands append events to **Aurora** (atomic with the optimistic-lock fence) → the new balance is\n**write-through** to **Redis** → events publish to **SQS** post-commit (transactional-outbox fallback\nif SQS is down) → a **Lambda projector** builds the **DynamoDB** read model. Reads hit Redis (sub-ms)\nwith DynamoDB fallback; Aurora is never on the read path. Collector-less **OpenTelemetry → X-Ray**\ntraces every hop, propagated across the SQS boundary.\n\n*Aurora + Redis in private subnets; ECS in public subnets reaching AWS via VPC endpoints (no NAT).*\n\n## Demo\n\nRecorded on live AWS (a temporary deploy, since torn down for cost). **Videos in progress — links\ncoming soon:**\n\n- ▶️ **Concurrent conflict** — two simultaneous transfers → one `409` → retry succeeds, no double-spend\n- ▶️ **Projection rebuild** — delete the DynamoDB read model, watch it reconstruct 0→N from the Aurora event log\n- ▶️ **Breakpoint under load** — the CloudWatch dashboard saturating as k6 climbs past the knee\n\n![CloudWatch ops dashboard during the breakpoint climb](docs/img/cloudwatch-dashboard-breakpoint.png)\n\nTransfer latency vs. arrival rate (prod, 2 vCPU) — flat ~87 ms to 150 rps, p99 knee ~200 rps,\ncollapse ~300 rps ([ADR-004](docs/adrs/ADR-004-right-size-before-tuning.md)).\n\n## Demo scenarios\n\nAssumes the local stack from [Local development](#local-development) is already up\n(`docker compose up -d` + `local-bootstrap.sh` + set variables, API healthy on `http://localhost`). Get a token and\nopen an account once, then run any of the three scenarios below against it:\n\n```bash\nTOKEN=$(curl -s -X POST \"$TOKEN_URL\" \\\n  -H \"Content-Type: application/x-www-form-urlencoded\" \\\n  -d \"grant_type=client_credentials\u0026client_id=${CLIENT_ID}\u0026client_secret=${CLIENT_SECRET}\u0026scope=https://api.getcloudledger.com/write%20https://api.getcloudledger.com/read\" \\\n  | jq -r '.access_token')\n\nACCOUNT_ID=$(uuidgen)\ncurl -s -X POST http://localhost/v1/accounts -H \"Authorization: Bearer $TOKEN\" \\\n  -H \"Idempotency-Key: $(uuidgen)\" -H \"Content-Type: application/json\" \\\n  -d \"{\\\"accountId\\\":\\\"$ACCOUNT_ID\\\",\\\"currency\\\":\\\"USD\\\"}\"\n```\n\n**1. Idempotency** — replay a write with the same `Idempotency-Key` → the original `201`, no second event:\n```bash\nKEY=$(uuidgen)\ncurl -s -o /dev/null -w '%{http_code}\\n' -X POST http://localhost/v1/accounts/$ACCOUNT_ID/deposits \\\n  -H \"Authorization: Bearer $TOKEN\" -H \"Idempotency-Key: $KEY\" -H \"Content-Type: application/json\" \\\n  -d '{\"amount\":100}'\n# replay with the SAME key → identical 201, exactly one event in Aurora\ncurl -s -o /dev/null -w '%{http_code}\\n' -X POST http://localhost/v1/accounts/$ACCOUNT_ID/deposits \\\n  -H \"Authorization: Bearer $TOKEN\" -H \"Idempotency-Key: $KEY\" -H \"Content-Type: application/json\" \\\n  -d '{\"amount\":100}'\n```\n\n**2. Concurrent conflict** — two same-version writes race on the same account → one `201`, one\n`409 {\"error\":\"version_conflict\"}` → re-read + retry succeeds. Money neither lost nor created.\nPlain curl `\u0026` backgrounding races the two requests but doesn't guarantee they land on the exact\nsame in-memory version — for a real, repeatable proof of the `409` (fired at true concurrency via\n`http.batch`) run [`k6/scenarios/conflict.js`](k6/scenarios/conflict.js) instead:\n```bash\ncurl -s -o /dev/null -w '%{http_code}\\n' -X POST http://localhost/v1/accounts/$ACCOUNT_ID/withdrawals \\\n  -H \"Authorization: Bearer $TOKEN\" -H \"Idempotency-Key: $(uuidgen)\" -H \"Content-Type: application/json\" \\\n  -d '{\"amount\":10}' \u0026\ncurl -s -o /dev/null -w '%{http_code}\\n' -X POST http://localhost/v1/accounts/$ACCOUNT_ID/withdrawals \\\n  -H \"Authorization: Bearer $TOKEN\" -H \"Idempotency-Key: $(uuidgen)\" -H \"Content-Type: application/json\" \\\n  -d '{\"amount\":10}' \u0026\nwait\n```\n\n**3. CQRS projection** — the command returns on Aurora commit + Redis write-through (read-your-writes\nimmediately); the projector catches DynamoDB up within ~seconds. `GET /transactions` reflects it once\nprojected — no Aurora replay on the read path:\n```bash\ncurl -s http://localhost/v1/accounts/$ACCOUNT_ID/balance -H \"Authorization: Bearer $TOKEN\" | jq       # Redis write-through — fresh immediately\ncurl -s http://localhost/v1/accounts/$ACCOUNT_ID/transactions -H \"Authorization: Bearer $TOKEN\" | jq  # DynamoDB TXNS# — populated once the projector catches up\n```\n\n## Tech stack\n\n| Layer | Technology |\n|---|---|\n| API language / framework | Java 21 · Spring Boot 4.1 (hexagonal ports \u0026 adapters) |\n| Lambda language | Python 3.12 (`uv`-managed) — outbox-poller, projector, cleanup |\n| Event store | Aurora PostgreSQL — ACID: the optimistic-lock fence + event insert commit in one txn |\n| Async fan-out | SQS-first (KMS-encrypted, DLQ) + transactional-outbox fallback → Python projector λ |\n| Read model | DynamoDB single-table + GSI · ElastiCache Redis (write-through, 30-min TTL, circuit breaker) |\n| Auth | Cognito M2M (Client Credentials JWT); a separate `api/admin` scope for operator endpoints |\n| IaC | Terraform — S3 backend + native lock, per-service KMS CMKs, `local`/`prod` envs |\n| Observability | Collector-less OpenTelemetry (ADOT, SigV4) → X-Ray Transaction Search · CloudWatch dashboard |\n| Load \u0026 verification | k6 — smoke/conflict CI gates + load/read-balance/breakpoint sandbox runs |\n| CI/CD | GitHub Actions → ECR → rolling ECS deploy → k6 smoke + conflict gates |\n| Local dev | Docker Compose + Floci (LocalStack-compatible) · Testcontainers · pytest e2e |\n\n## Key engineering decisions\n\n- **Optimistic locking via a DB unique constraint — not `SERIALIZABLE`, not row locks** — a version\n  race loses atomically on `UNIQUE (aggregate_id, version)` and surfaces as a retryable `409`\n  instead of opaque serialization retries or hot-account queuing; proven under a real race by the\n  k6 conflict gate ([ADR-001](docs/adrs/ADR-001-optimistic-locking-db-constraint.md))\n- **SQS-first publish, transactional outbox demoted to fallback** — the classic outbox makes every\n  event pay the relay hop; publishing post-commit and writing outbox rows only when SQS fails keeps\n  projection lag at publish latency while the event store remains the recovery path\n  ([ADR-002](docs/adrs/ADR-002-sqs-first-outbox-fallback.md))\n- **Command-path Redis write-through over cache-aside** — the write path already holds the\n  authoritative balance at commit, so writing the snapshot through gives read-your-writes without\n  waiting on the async projector ([ADR-003](docs/adrs/ADR-003-command-path-redis-write-through.md))\n- **Right-size before tuning** — an X-Ray trace showed 2 ms queries inside a 3.5 s p99: contention\n  (10-connection pool on 0.5 vCPU), not slow work; 4× CPU + a fixed pool of 20 tripled the knee,\n  with virtual threads deliberately held as a separate experiment\n  ([ADR-004](docs/adrs/ADR-004-right-size-before-tuning.md))\n- **Collector-less OpenTelemetry → X-Ray** — ADOT SigV4-signs OTLP itself, so no collector sidecar\n  runs anywhere; trace context is persisted on outbox rows so the projector joins the *original*\n  trace across the SQS hop ([ADR-005](docs/adrs/ADR-005-collectorless-otel-to-xray.md))\n\nFor the longer-form walkthrough — how these five decisions connect, including the trace that led\nfrom ADR-005 to the ADR-004 performance fix — see\n[`docs/architecture-narrative.md`](docs/architecture-narrative.md).\n\n## Local development\n\n**Prerequisites:** Java 21 · Docker · Terraform · Python 3.12 + `uv` · k6\n\n```bash\ngit clone https://github.com/rafaesc/cloud-ledger \u0026\u0026 cd cloud-ledger\ndocker compose up -d                        # Floci: SQS, DynamoDB, Lambda, RDS, Cognito\nbash terraform/scripts/local-bootstrap.sh   # apply Terraform + migrations + build/push + boot API + Lambda images\n\n# Pull M2M creds from the local Terraform state (-chdir works from the repo root)\nexport CLIENT_ID=$(terraform -chdir=terraform/envs/local output -raw cognito_client_id)\nexport CLIENT_SECRET=$(terraform -chdir=terraform/envs/local output -json cognito_client_secret | jq -r .)\nexport BASE_URL=http://localhost            # Floci ALB on :80 — not the .elb.localhost output\nexport TOKEN_URL=\"http://localhost:4566/cognito-idp/oauth2/token\"\n\ncd api \u0026\u0026 ./gradlew test                    # unit + Testcontainers\ncd lambdas \u0026\u0026 uv run pytest                 # Lambda tests (moto)\ncd e2e \u0026\u0026 uv run pytest -v                  # full-stack e2e against the running API (http://localhost)\n```\n`local-bootstrap.sh` also builds and pushes the API image, and Floci runs it as an ECS task behind\nan ALB — no separate `./gradlew bootRun` needed. The API is live at `http://localhost` (port 80,\nnot `:8080`); confirm with `curl http://localhost/actuator/health`, the same check\n[`e2e.yml`](.github/workflows/e2e.yml) polls before running the e2e suite against it.\n\n### Running the k6 suite\n\nRequires the stack from the steps above already running (`docker compose up -d` +\n`local-bootstrap.sh`) — `CLIENT_ID`/`CLIENT_SECRET` come from the Cognito pool Terraform just\ncreated, so nothing here works standalone:\n\n```bash\ncurl -s -o /dev/null -w '%{http_code}\\n' http://localhost/actuator/health   # expect 200 before continuing\nBASE_URL=$BASE_URL PROJECTION_LAG_MS=60000 k6 run k6/scenarios/smoke.js     # CI gate: 1 VU x 10 transfers E2E\nBASE_URL=$BASE_URL PROJECTION_LAG_MS=60000 k6 run k6/scenarios/conflict.js  # correctness demo: one 201 + N-1 409s, no double-spend\n```\n\nMore details: [`k6/README.md`](k6/README.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frafaesc%2Fcloud-ledger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frafaesc%2Fcloud-ledger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frafaesc%2Fcloud-ledger/lists"}