https://github.com/jonmatum/serverless-second-brain

Serverless backend for a personal knowledge graph on AWS — DynamoDB, Lambda, Bedrock, API Gateway, CloudFront. Phases 1-2 deployed, 3-4 in progress.
https://github.com/jonmatum/serverless-second-brain

ai-agents api-gateway aws bedrock dynamodb eventbridge infrastructure-as-code knowledge-graph lambda mcp pkm second-brain serverless step-functions terraform

Last synced: 28 days ago
JSON representation

Serverless backend for a personal knowledge graph on AWS — DynamoDB, Lambda, Bedrock, API Gateway, CloudFront. Phases 1-2 deployed, 3-4 in progress.

Host: GitHub
URL: https://github.com/jonmatum/serverless-second-brain
Owner: jonmatum
License: mit
Created: 2026-03-20T23:25:13.000Z (3 months ago)
Default Branch: main
Last Pushed: 2026-04-24T04:02:13.000Z (about 2 months ago)
Last Synced: 2026-04-24T06:07:59.321Z (about 2 months ago)
Topics: ai-agents, api-gateway, aws, bedrock, dynamodb, eventbridge, infrastructure-as-code, knowledge-graph, lambda, mcp, pkm, second-brain, serverless, step-functions, terraform
Language: TypeScript
Size: 701 KB
Stars: 0
Watchers: 0
Forks: 1
Open Issues: 22
Metadata Files:
- Readme: README.md
- License: LICENSE
- Agents: AGENTS.md

Awesome Lists containing this project

README

          # Serverless Second Brain

> ⚠️ **Work in progress** — Phases 1–2 deployed and functional, Phases 3–4 not yet started.

Serverless backend for a personal knowledge graph on AWS. DynamoDB, Lambda, Bedrock, API Gateway, CloudFront, and Cognito.

From the essay: [From Prototype to Production: A Serverless Second Brain on AWS](https://jonmatum.com/essays/from-prototype-to-production-serverless-second-brain)

## Architecture

```mermaid

flowchart TB

  accTitle: Serverless second brain architecture

  accDescr: Three layers — Interface, Compute, Memory — with AWS services per block and two access doors for humans and AI agents

  User((User))

  Agent((AI Agent))

  subgraph Interface

    CF[CloudFront + S3]

    APIGW[API Gateway REST]

    GW[AgentCore Gateway]

  end

  subgraph Compute

    LCapture[Lambda Capture]

    LSearch[Lambda Search]

    LGraph[Lambda Graph]

    ACR[AgentCore Runtime]

    EB[EventBridge Scheduler]

    LSurface[Lambda Surfacing]

  end

  subgraph Memory

    DDB[(DynamoDB)]

    S3C[S3 Content]

  end

  subgraph AI

    BR[Bedrock Claude]

    BRE[Bedrock Embeddings]

  end

  User -- "browser" --> CF

  User -- "REST API" --> APIGW

  Agent -- "MCP" --> GW

  APIGW -- "/capture" --> LCapture

  APIGW -- "/search" --> LSearch

  APIGW -- "/graph" --> LGraph

  LCapture -- "async" --> LEnrich[Lambda Enrich]

  GW -- "MCP tools" --> LCapture

  GW -- "MCP tools" --> LSearch

  GW -- "MCP tools" --> LGraph

  ACR -- "reasons" --> BR

  EB -- "daily cron" --> LSurface

  LCapture -- "writes" --> DDB

  LCapture -- "stores MDX" --> S3C

  LCapture -- "classifies" --> BR

  LCapture -- "embeddings" --> BRE

  LSearch -- "queries" --> DDB

  LSearch -- "query embedding" --> BRE

  LGraph -- "reads edges" --> DDB

  LSurface -- "reads" --> DDB

  LSurface -- "notifies" --> SNS[SNS]

```

## Two doors, same brain

| Door | Protocol | Auth | Use case |

|---|---|---|---|

| Human | REST API (API Gateway) | Cognito JWT for writes, open reads | SPA, curl, integrations |

| Agent | MCP (AgentCore Gateway) | OAuth + semantic discovery | AI agents, Copilot, Claude |

Both doors call the same Lambda functions — the difference is protocol and auth.

## Cost

Scales to zero. No minimum costs beyond S3 storage.

| Load | Monthly cost |

|---|---|

| Idle (0 req/day) | ~$0.51 |

| Moderate (100 req/day) | ~$2.44 |

| High (1,000 req/day) | ~$11.21 |

Real cost data: [`docs/benchmarks/results.md`](docs/benchmarks/results.md)

## API

| Method | Endpoint | Description | Auth |

|---|---|---|---|

| `POST` | `/capture` | Ingest text → Bedrock classifies → DynamoDB | Cognito JWT |

| `GET` | `/search?q=` | Hybrid keyword + semantic search | None |

| `GET` | `/graph` | Full knowledge graph (nodes + edges) | None |

| `GET` | `/nodes/{id}` | Single node with edges and related nodes | None |

| `GET` | `/health` | Service health check | None |

Full API spec: [`.kiro/steering/api-spec.md`](.kiro/steering/api-spec.md)

## MCP tools (agent door)

| Tool | Lambda | Description |

|---|---|---|

| `read_node` | Graph | Read a single node by slug |

| `list_nodes` | Graph | List nodes with filters |

| `search` | Search | Semantic + keyword search |

| `add_node` | Capture | Create a new knowledge node |

| `connect_nodes` | Capture | Create an edge between nodes |

| `flag_stale` | Surfacing | Mark a node for review |

Full MCP spec: [`.kiro/steering/mcp-tools.md`](.kiro/steering/mcp-tools.md)

## Phased delivery

Each phase is independently deployable.

| Phase | Components | Status |

|---|---|---|

| 1 — Capture | Terraform foundation, DynamoDB, S3, Capture Lambda, API Gateway, Cognito auth | ✅ Deployed |

| 2 — Read | Search Lambda, Graph Lambda, CloudFront + S3 frontend | ✅ Deployed |

| 3 — Agent | AgentCore Gateway + Runtime, MCP tools, reasoning agent | 🔲 Not started |

| 4 — Proactive | EventBridge scheduler, Surfacing Lambda, SNS notifications | 🔲 Not started |

## Architecture Decision Records

Every significant technical decision is documented in [`docs/decisions/`](docs/decisions/):

| ADR | Decision | Status |

|---|---|---|

| [001](docs/decisions/001-lambda-packaging-and-framework.md) | Lambda packaging (zip) with no web framework | Accepted |

| [002](docs/decisions/002-mcp-write-safety.md) | Write safety — 6 controls for MCP agent mutations | Accepted |

| [003](docs/decisions/003-authentication-and-visibility.md) | Cognito authentication and visibility model | Accepted |

| [004](docs/decisions/004-dynamodb-single-table-design.md) | DynamoDB single-table design with 2 GSIs | Accepted |

| [005](docs/decisions/005-hybrid-search-strategy.md) | Hybrid keyword + semantic search | Accepted |

| [006](docs/decisions/006-step-functions-express-capture-pipeline.md) | Step Functions Express for capture pipeline | Superseded |

| [007](docs/decisions/007-agentcore-gateway-over-self-hosted-mcp.md) | AgentCore Gateway over self-hosted MCP server | Accepted |

| [008](docs/decisions/008-in-memory-embedding-scan.md) | In-memory embedding scan (temporary) | Accepted |

| [009](docs/decisions/009-spec-driven-development.md) | Spec-Driven Development approach | Accepted |

| [010](docs/decisions/010-bedrock-token-optimization.md) | Bedrock token optimization — recent slugs | Accepted |

| [011](docs/decisions/011-cloudfront-s3-static-hosting.md) | CloudFront + S3 over Vercel/Amplify | Accepted |

| [012](docs/decisions/012-github-actions-oidc.md) | GitHub Actions OIDC over static credentials | Accepted |

| [013](docs/decisions/013-vite-spa-over-nextjs.md) | Vite SPA over Next.js | Accepted |

| [014](docs/decisions/014-async-enrichment-pipeline.md) | Async enrichment — split capture into sync + async | Accepted |

> ADR-006 superseded: capture now uses a monolithic Lambda handler via API Gateway Lambda proxy integration. Step Functions infrastructure and step handlers have been fully removed.

## Prerequisites

- AWS account with Bedrock model access enabled:

  - `us.anthropic.claude-sonnet-4-20250514-v1:0`

  - `amazon.titan-embed-text-v2:0`

- Terraform >= 1.5

- Node.js 22.x

- GitHub repo with OIDC configured for AWS

## Project structure

```

infra/

  bootstrap/              → One-time Terraform state backend

  modules/                → Reusable Terraform modules

  environments/{dev,prod} → Environment roots

src/

  functions/{capture,enrich,search,graph,connect,flag,surfacing}/

  shared/                 → Types, clients, HTTP helpers, math, key prefixes

frontend/                 → React SPA (Vite + TailwindCSS)

scripts/                  → Migration, benchmarks, utilities

docs/decisions/           → ADRs

docs/benchmarks/          → Performance and cost data

.kiro/steering/           → SDD specifications

.github/workflows/        → CI/CD (GitHub Actions OIDC)

```

## Specifications

All contracts defined before code — [Spec-Driven Development](https://jonmatum.com/concepts/spec-driven-development):

- [`architecture.md`](.kiro/steering/architecture.md) — layers, services, cost constraints, phases

- [`dynamodb-schema.md`](.kiro/steering/dynamodb-schema.md) — single-table design, item types, GSIs

- [`api-spec.md`](.kiro/steering/api-spec.md) — REST endpoints, schemas, error codes

- [`mcp-tools.md`](.kiro/steering/mcp-tools.md) — MCP tool definitions, write safety rules

- [`event-schemas.md`](.kiro/steering/event-schemas.md) — EventBridge, SNS contracts

- [`terraform-conventions.md`](.kiro/steering/terraform-conventions.md) — naming, modules, state, security

- [`code-conventions.md`](.kiro/steering/code-conventions.md) — TypeScript, Lambda patterns, commits

## Development

```sh

# Bootstrap (one-time)

cd infra/bootstrap && terraform init && terraform apply

# Deploy

cd infra/environments/dev && terraform init && terraform apply

# Lambda development

cd src/functions/capture && npm install && npm run build

```

## License

MIT

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/jonmatum/serverless-second-brain

Awesome Lists containing this project

README