{"id":48129525,"url":"https://github.com/barney-w/surf","last_synced_at":"2026-04-04T16:36:42.714Z","repository":{"id":343107773,"uuid":"1175976595","full_name":"barney-w/surf","owner":"barney-w","description":"The open framework for extensible \u0026 grounded AI agent orchestration.","archived":false,"fork":false,"pushed_at":"2026-03-26T23:38:21.000Z","size":8922,"stargazers_count":1,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-27T02:12:03.490Z","etag":null,"topics":["agent-framework","agent-orchestration","ai-agent","ai-framework","azure","multi-agent"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/barney-w.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-08T12:55:42.000Z","updated_at":"2026-03-26T06:50:31.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/barney-w/surf","commit_stats":null,"previous_names":["barney-w/surf"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/barney-w/surf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barney-w%2Fsurf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barney-w%2Fsurf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barney-w%2Fsurf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barney-w%2Fsurf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/barney-w","download_url":"https://codeload.github.com/barney-w/surf/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barney-w%2Fsurf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31405706,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T10:20:44.708Z","status":"ssl_error","status_checked_at":"2026-04-04T10:20:06.846Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-framework","agent-orchestration","ai-agent","ai-framework","azure","multi-agent"],"created_at":"2026-04-04T16:36:42.203Z","updated_at":"2026-04-04T16:36:42.696Z","avatar_url":"https://github.com/barney-w.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/assets/surf.png\" alt=\"surf mascot\" width=\"120\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003cstrong\u003eThe open framework for AI agent orchestration.\u003c/strong\u003e\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  Build multi-agent systems that route queries to specialist agents,\u003cbr\u003e\n  ground every answer in your own knowledge base, and ship across\u003cbr\u003e\n  web, desktop, and mobile from a single codebase.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/barney-w/surf/actions/workflows/pr-checks.yml\"\u003e\u003cimg src=\"https://img.shields.io/github/actions/workflow/status/barney-w/surf/pr-checks.yml?label=CI\u0026style=flat-square\" alt=\"CI\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://www.python.org/\"\u003e\u003cimg src=\"https://img.shields.io/badge/Python-3.12+-3776AB?logo=python\u0026logoColor=white\u0026style=flat-square\" alt=\"Python\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/barney-w/surf-kit\"\u003e\u003cimg src=\"https://img.shields.io/badge/TypeScript-strict-3178C6?style=flat-square\" alt=\"TypeScript\"\u003e\u003c/a\u003e\n  \u003ca href=\"./LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/License-Apache_2.0-blue.svg?style=flat-square\" alt=\"License\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#quickstart\"\u003eQuickstart\u003c/a\u003e\u0026nbsp;\u0026nbsp;\u0026bull;\u0026nbsp;\u0026nbsp;\u003ca href=\"#how-it-works\"\u003eHow it works\u003c/a\u003e\u0026nbsp;\u0026nbsp;\u0026bull;\u0026nbsp;\u0026nbsp;\u003ca href=\"#what-you-get\"\u003eFeatures\u003c/a\u003e\u0026nbsp;\u0026nbsp;\u0026bull;\u0026nbsp;\u0026nbsp;\u003ca href=\"#agents\"\u003eAgents\u003c/a\u003e\u0026nbsp;\u0026nbsp;\u0026bull;\u0026nbsp;\u0026nbsp;\u003ca href=\"#deep-dive\"\u003eDeep Dive\u003c/a\u003e\u0026nbsp;\u0026nbsp;\u0026bull;\u0026nbsp;\u0026nbsp;\u003ca href=\"./CONTRIBUTING.md\"\u003eContributing\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n## Quickstart\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003ePrerequisites\u003c/strong\u003e\u003c/summary\u003e\n\n- [Python 3.12+](https://www.python.org/)\n- [uv](https://docs.astral.sh/uv/)\n- [just](https://just.systems/)\n- [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/) (logged in)\n- Azure subscription with OpenAI access\n\n\u003c/details\u003e\n\n### Setup\n\n```bash\naz login\ncd api \u0026\u0026 uv sync \u0026\u0026 cd ../ingestion \u0026\u0026 uv sync \u0026\u0026 cd ..\njust setup-dev          # deploy dev Azure resources + generate .env\njust dev                # start API with hot reload (auto-starts Postgres, runs migrations)\n```\n\n### Verify\n\n```bash\ncurl http://localhost:8090/api/v1/health\n```\n\n\u003e **Note:** RBAC role propagation can take a few minutes. If you get 403 errors, wait and retry.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eRun with DevUI / Web / Desktop\u003c/strong\u003e\u003c/summary\u003e\n\n```bash\njust devui              # interactive agent chat with tool call visibility — port 8091\njust web                # full SPA with auth, conversation history, debug panels — port 3000\njust desktop            # Tauri desktop app with native window management\n```\n\n\u003c/details\u003e\n\n---\n\n## How it works\n\n```mermaid\ngraph TD\n  web[\"Web / Desktop / Mobile\u003cbr/\u003esurf-kit + React\"]\n  nginx[\"nginx\u003cbr/\u003ereverse proxy\"]\n  api[\"FastAPI API\"]\n  coordinator[\"Coordinator Agent\u003cbr/\u003eclaude-haiku-4-5\"]\n  hr[\"HR Agent\u003cbr/\u003eclaude-sonnet-4-6\"]\n  it[\"IT Agent\u003cbr/\u003eclaude-sonnet-4-6\"]\n  website[\"Website Agent\u003cbr/\u003eclaude-sonnet-4-6\"]\n  rag[\"Azure AI Search\u003cbr/\u003eBM25 + Vector\"]\n  proofread[\"Proofreader\u003cbr/\u003eclaude-haiku-4-5\"]\n  qg[\"Quality Gate\"]\n  postgres[\"PostgreSQL\u003cbr/\u003econversations + feedback\"]\n  otel[\"OpenTelemetry\u003cbr/\u003eAzure Monitor / OTLP\"]\n  langfuse[\"Langfuse\u003cbr/\u003eLLM tracing\"]\n  keyvault[\"Key Vault\"]\n\n  web --\u003e|SSE| nginx --\u003e api\n  api --\u003e coordinator\n  coordinator --\u003e|handoff| hr\n  coordinator --\u003e|handoff| it\n  coordinator --\u003e|handoff| website\n  hr --\u003e rag\n  it --\u003e rag\n  website --\u003e rag\n  hr --\u003e qg --\u003e proofread\n  api --\u003e postgres\n  api --\u003e otel\n  api --\u003e langfuse\n  api --\u003e keyvault\n```\n\n---\n\n## What you get\n\n|                              |                                                                                                             |\n| ---------------------------- | ----------------------------------------------------------------------------------------------------------- |\n| **Zero-registration agents** | Subclass `DomainAgent` and the framework discovers, registers, and wires it automatically. No config files. |\n| **Auth-filtered routing**    | Agents are invisible to users who lack the required auth level. The coordinator can't even describe them.   |\n| **3-strategy RAG**           | Hybrid search with broadened-filter fallback, keyword-only rescue, and post-response quality gates.         |\n| **Prompt injection defence** | Four independent layers — domain-isolated RAG, structured JSON, quality gate, source-pollution guard.       |\n| **Multi-model routing**      | Haiku for fast coordinator decisions, Sonnet for specialist agents. Direct Anthropic or Azure AI Foundry.   |\n| **Ship everywhere**          | Web, desktop, and mobile from one React codebase via the shared `surf-kit` component library.               |\n\n---\n\n## Agents\n\n| Agent           | Purpose                                          | RAG Scope                | Model        | Auth Level        |\n| --------------- | ------------------------------------------------ | ------------------------ | ------------ | ----------------- |\n| **Coordinator** | Routes queries, synthesises multi-domain answers | Unscoped                 | Haiku (fast) | Public            |\n| **HR**          | Leave, onboarding, performance, L\u0026D policies     | `domain=hr`              | Sonnet       | Microsoft Account |\n| **IT**          | VPN, passwords, software, hardware, security     | `domain=it`              | Sonnet       | Organisational    |\n| **Website**     | Public-facing content, services, events          | `content_source=website` | Sonnet       | Public            |\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eAdding a new agent\u003c/strong\u003e\u003c/summary\u003e\n\n```python\n# api/src/agents/finance/agent.py\nclass FinanceAgent(DomainAgent):\n    @property\n    def name(self) -\u003e str:\n        return \"finance_agent\"\n\n    @property\n    def description(self) -\u003e str:\n        return \"Handles budget and procurement queries\"\n\n    @property\n    def rag_scope(self) -\u003e RAGScope:\n        return RAGScope(domain=\"finance\", document_types=[\"policy\", \"procedure\"])\n\n    @property\n    def system_prompt(self) -\u003e str:\n        return \"You are a finance specialist...\"\n```\n\nThat's it. No registration, no config changes. The framework discovers the subclass at startup, creates its RAG tool with domain-isolated filters, and adds it to the coordinator's handoff graph. See `api/src/agents/_base.py` for the full interface and `api/src/agents/_discovery.py` for the discovery mechanism.\n\n\u003c/details\u003e\n\n---\n\n## Deep Dive\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eProject Structure\u003c/strong\u003e\u003c/summary\u003e\n\n```\nsurf/\n  api/                  FastAPI backend — agents, orchestrator, RAG, middleware\n    src/\n      agents/           Domain agents + coordinator (auto-discovered)\n      orchestrator/     Workflow builder, PDF processing, middleware pipeline\n      rag/              Search execution, 3-strategy tool, quality gate\n      routes/           Chat, auth, user profile, admin, agent listing\n      services/         Conversation persistence, Graph API, streaming, response pipeline\n      middleware/       Auth, rate limiting, body limits, telemetry, input validation\n      config/           Settings with environment-aware validation\n    tests/\n      unit/             28 test modules (~7K lines)\n      security/         JWT bypass, prompt injection, conversation isolation\n      integration/      Multi-turn flows against real Postgres\n      eval/             LLM-judged response quality suite\n      load/             Locust load testing\n  web/                  React 19 + Vite 7 + TailwindCSS 4 frontend\n    src-tauri/          Tauri desktop app (Rust shell)\n  mobile/               React Native + Expo (iOS / Android)\n  ingestion/            Document pipeline — PDF, DOCX, TXT, CSV connectors\n  infra/                Azure IaC — 19 Bicep modules, 1,200+ lines\n    modules/            Application Insights custom module\n    environments/       dev / staging / prod parameter files\n    workbooks/          Azure Monitor telemetry workbook\n  data/                 Sample documents and ingestion manifests\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eArchitecture (SVG diagram)\u003c/strong\u003e\u003c/summary\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/architecture.svg\" alt=\"Surf Architecture Overview\" width=\"1100\" /\u003e\n\u003c/div\u003e\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eRAG Pipeline\u003c/strong\u003e\u003c/summary\u003e\n\nThe RAG tool (`api/src/rag/tools.py`) implements a multi-strategy search pipeline:\n\n1. **Primary hybrid search** — BM25 + vector (text-embedding-3-large) with domain-scoped OData filters\n2. **Broadened filter fallback** — relaxes non-identity filters when primary returns too few results\n3. **Keyword-only rescue** — drops vector search entirely for edge cases where embeddings miss\n\nAdditional pipeline features:\n\n- **LLM query rewriting** — rewrites conversational questions into keyword-rich search queries\n- **Chunk merging** — consecutive chunks from the same document are merged to give the LLM complete context\n- **Score normalisation** — normalises across BM25 and RRF score scales\n- **Quality gate** — post-response validation catches infrastructure errors, skipped searches, ignored results, and missing sources (`api/src/rag/quality_gate.py`)\n- **Source recovery** — extracts and deduplicates source references from raw agent output (`api/src/agents/_output.py`)\n- **Proofreading pass** — a fast Haiku model fixes generation artefacts before final delivery (`api/src/agents/_proofread.py`)\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eAPI Reference\u003c/strong\u003e\u003c/summary\u003e\n\n| Method   | Endpoint                                  | Description                                               |\n| -------- | ----------------------------------------- | --------------------------------------------------------- |\n| `POST`   | `/api/v1/chat`                            | Chat — returns JSON response                              |\n| `POST`   | `/api/v1/chat/stream`                     | Chat — Server-Sent Events with real-time streaming        |\n| `GET`    | `/api/v1/chat/{conversation_id}`          | Load conversation history                                 |\n| `DELETE` | `/api/v1/chat/{conversation_id}`          | Delete a conversation                                     |\n| `POST`   | `/api/v1/chat/{conversation_id}/feedback` | Record thumbs up/down + comment                           |\n| `GET`    | `/api/v1/agents`                          | List available agents (filtered by caller's auth level)   |\n| `POST`   | `/api/v1/auth/guest`                      | Issue a guest access token                                |\n| `GET`    | `/api/v1/me`                              | User profile (JWT claims + Graph API enrichment)          |\n| `GET`    | `/api/v1/me/photo`                        | User profile photo (via Graph API OBO)                    |\n| `GET`    | `/api/v1/conversations`                   | List conversations for the authenticated user             |\n| `GET`    | `/api/v1/health`                          | Health check (supports `?deep=true` for component checks) |\n| `GET`    | `/api/v1/admin/`                          | Dev-only conversation browser dashboard                   |\n\n#### SSE Event Protocol\n\n```\nphase(thinking) → agent(name) → phase(generating) → delta* → phase(verifying) →\nconfidence → verification → usage → done → [DONE]\n```\n\n- `:keepalive` comments every 5 seconds\n- `phase(waiting)` after 10 seconds of no output (e.g. during upstream 429 retry)\n- `debug` events with RAG search details (dev mode + `X-Surf-Debug` header)\n- `error` events with structured codes for client-side handling\n\n#### PDF Attachments\n\nThe chat endpoint accepts PDF file attachments with tiered processing (`api/src/orchestrator/pdf.py`):\n\n- **Tier 1 (direct vision)**: PDFs up to 30 pages are sent as native document content blocks\n- **Tier 2 (text extraction)**: Larger PDFs get text extracted and sent as text blocks\n- Size limit: 100 MB with decompression bomb protection\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eSecurity Model\u003c/strong\u003e\u003c/summary\u003e\n\nSurf implements defence-in-depth. The full model is documented in [`docs/security-model.md`](./docs/security-model.md).\n\n| Layer                 | Mechanism                                                                              | Location                                                                     |\n| --------------------- | -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- |\n| **Authentication**    | Entra ID (RS256 JWKS) + guest tokens (HS256 HMAC) + dev bypass                         | `api/src/middleware/auth.py`                                                 |\n| **Authorisation**     | 3-tier AuthLevel enum; agent graphs filtered per auth level                            | `api/src/agents/_base.py`, `api/src/orchestrator/builder.py`                 |\n| **Rate limiting**     | Per-user limits on every endpoint (slowapi)                                            | `api/src/middleware/rate_limit.py`                                           |\n| **Input validation**  | Message length cap (10K chars), control character stripping, body size limits          | `api/src/middleware/input_validation.py`, `api/src/middleware/body_limit.py` |\n| **Prompt injection**  | Domain-isolated RAG, structured JSON enforcement, quality gate, source-pollution guard | `api/src/rag/tools.py`, `api/src/services/streaming.py`                      |\n| **Production guards** | App refuses to start with auth disabled, debug on, wildcard CORS, or no Postgres SSL   | `api/src/main.py`                                                            |\n| **Data isolation**    | All queries scoped to `user_id`; CASCADE deletes; conversation TTL expiry              | `api/src/services/conversation.py`                                           |\n| **Secret management** | Key Vault for runtime secrets; managed identity for Azure services; OIDC for CI/CD     | `infra/main.bicep`                                                           |\n\nSecurity tests in `api/tests/security/` cover JWT bypass attempts, input injection vectors, and conversation isolation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eObservability\u003c/strong\u003e\u003c/summary\u003e\n\n| Signal          | Backend                                         | Detail                                                                                |\n| --------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------- |\n| **Traces**      | OpenTelemetry → Azure Monitor or OTLP collector | Spans across routes, agent handoffs, RAG search, persistence                          |\n| **Metrics**     | OTel histograms + counters                      | Chat duration, token usage (in/out per agent), quality gate triggers, rate limit hits |\n| **LLM tracing** | Langfuse v3                                     | Per-call tracing with cost tracking; local dev stack included in `docker-compose.yml` |\n| **Dashboards**  | Application Insights workbook                   | Pre-built telemetry workbook in `infra/workbooks/api-telemetry.json`                  |\n| **Alerts**      | Azure metric alerts                             | Container restart, 5xx rate, CPU threshold (all in `infra/main.bicep`)                |\n\nTelemetry configuration: `api/src/middleware/telemetry.py`. Langfuse integration: `api/src/middleware/langfuse_utils.py`.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eInfrastructure\u003c/strong\u003e\u003c/summary\u003e\n\nSurf's Azure infrastructure is defined in a single `infra/main.bicep` orchestrator (1,200+ lines) using [Azure Verified Modules](https://azure.github.io/Azure-Verified-Modules/):\n\n| Resource             | Module                               | Purpose                                              |\n| -------------------- | ------------------------------------ | ---------------------------------------------------- |\n| Log Analytics        | `avm/operational-insights/workspace` | OpenTelemetry traces + structured logs               |\n| Application Insights | `modules/application-insights.bicep` | APM, telemetry workbook                              |\n| Managed Identity     | `avm/managed-identity`               | App identity + CI identity (WIF)                     |\n| Azure OpenAI         | `avm/cognitive-services/account`     | text-embedding-3-large (ingestion only)              |\n| Azure AI Search      | `avm/search/search-service`          | Hybrid BM25 + vector retrieval                       |\n| Key Vault            | `avm/key-vault/vault`                | Secrets (API keys, client secrets, guest token HMAC) |\n| VNet + NSGs          | `avm/network/virtual-network`        | Private networking with subnet isolation             |\n| Private DNS Zones    | `avm/network/private-dns-zone`       | DNS for Search, Storage, OpenAI private endpoints    |\n| Storage              | `avm/storage/storage-account`        | Document blob storage for ingestion                  |\n| Container Registry   | `avm/container-registry`             | Container image hosting                              |\n| Container Apps       | Native Bicep resource                | API (0-3 replicas), web (nginx), ingestion (0-1)     |\n| Metric Alerts        | `avm/insights/metric-alert`          | Restart, 5xx, and CPU alerts                         |\n\nThree environments: `dev.bicepparam`, `staging.bicepparam`, `prod.bicepparam`.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eCI/CD\u003c/strong\u003e\u003c/summary\u003e\n\nBoth GitHub Actions and GitLab CI/CD pipelines are maintained:\n\n| Pipeline      | GitHub Actions                       | GitLab CI                     | Trigger                         |\n| ------------- | ------------------------------------ | ----------------------------- | ------------------------------- |\n| **API**       | `.github/workflows/api-ci.yml`       | `.gitlab/ci/api-ci.yml`       | Push to `main` (`api/**`)       |\n| **Web**       | `.github/workflows/web-ci.yml`       | `.gitlab/ci/web-ci.yml`       | Push to `main` (`web/**`)       |\n| **Ingestion** | `.github/workflows/ingestion-ci.yml` | `.gitlab/ci/ingestion-ci.yml` | Push to `main` (`ingestion/**`) |\n| **Infra**     | `.github/workflows/infra-deploy.yml` | `.gitlab/ci/infra-deploy.yml` | Push to `main` (`infra/**`)     |\n| **PR Checks** | `.github/workflows/pr-checks.yml`    | `.gitlab/ci/pr-checks.yml`    | Pull/merge request              |\n\nKey properties:\n\n- **Zero stored secrets** — GitHub uses OIDC federation; GitLab uses Workload Identity Federation via a dedicated CI managed identity provisioned in Bicep\n- **Path-filtered** — only relevant pipelines run per commit\n- **Security scanning** — Gitleaks secret scanning, pip-audit dependency auditing\n- **Docker builds** with BuildKit and multi-platform support\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eIngestion Pipeline\u003c/strong\u003e\u003c/summary\u003e\n\nThe ingestion service (`ingestion/`) transforms raw documents into searchable index entries:\n\n| Stage               | Description                                                                       |\n| ------------------- | --------------------------------------------------------------------------------- |\n| **Connectors**      | PDF (PyMuPDF), DOCX (python-docx), TXT, CSV parsers (`ingestion/src/connectors/`) |\n| **SharePoint sync** | Graph API integration for syncing files and pages to blob storage                 |\n| **Chunking**        | Token-aware text splitting with tiktoken                                          |\n| **Embedding**       | Azure OpenAI text-embedding-3-large via managed identity                          |\n| **Indexing**        | Azure AI Search with hybrid (BM25 + vector) index schema                          |\n| **Scheduling**      | Hourly indexer runs via Azure AI Search indexer pipeline                          |\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eTesting\u003c/strong\u003e\u003c/summary\u003e\n\n| Suite           | Location                   | What it covers                                                                                 |\n| --------------- | -------------------------- | ---------------------------------------------------------------------------------------------- |\n| **Unit**        | `api/tests/unit/`          | 28 modules — agents, routes, middleware, RAG tool, config, output parsing, telemetry, Langfuse |\n| **Security**    | `api/tests/security/`      | JWT bypass, prompt injection, conversation isolation                                           |\n| **Integration** | `api/tests/integration/`   | Multi-turn conversation flows against real Postgres                                            |\n| **Eval**        | `api/tests/eval/`          | LLM-judged response quality with dataset-driven parametrisation and weighted rubric scoring    |\n| **Load**        | `api/tests/load/`          | Locust load testing (`locustfile.py`)                                                          |\n| **Smoke**       | `web/playwright.config.ts` | Playwright browser smoke tests                                                                 |\n| **Ingestion**   | `ingestion/tests/`         | Connector and pipeline tests                                                                   |\n\nRun with: `just test` (unit + security), `just test-integration`, `just eval`, `just smoke`.\n\n\u003c/details\u003e\n\n---\n\n## Development\n\n| Command                 | Description                                                                    |\n| ----------------------- | ------------------------------------------------------------------------------ |\n| `just dev`              | Run API with hot reload (port 8090) — auto-starts Postgres and runs migrations |\n| `just devui`            | Launch DevUI — interactive agent chat with tool call tracing (port 8091)       |\n| `just web`              | Run web frontend (port 3000)                                                   |\n| `just desktop`          | Run Tauri desktop app                                                          |\n| `just test`             | Run unit + security tests                                                      |\n| `just test-integration` | Run integration tests against real Postgres                                    |\n| `just eval`             | Run LLM-judged eval suite                                                      |\n| `just smoke`            | Run Playwright smoke tests                                                     |\n| `just lint`             | Lint all Python code (ruff)                                                    |\n| `just typecheck`        | Type-check all Python code (pyright)                                           |\n| `just format`           | Format all Python code                                                         |\n| `just audit`            | Run pip-audit security scanning                                                |\n| `just otel`             | Start OpenTelemetry collector for local telemetry                              |\n| `just langfuse`         | Start local Langfuse trace viewer at http://localhost:3100                     |\n| `just admin`            | Open the dev admin dashboard                                                   |\n| `just ask \"question\"`   | Ask the dev agent about the codebase                                           |\n| `just ask-repl`         | Start interactive dev agent session                                            |\n| `just setup-dev`        | Deploy dev Azure resources + generate .env                                     |\n| `just teardown-dev`     | Delete dev Azure resources                                                     |\n| `just deploy`           | Deploy API + web containers to Azure                                           |\n| `just deploy-all`       | Deploy infrastructure + all containers                                         |\n\n---\n\n## Links\n\n|                     |                                                          |\n| ------------------- | -------------------------------------------------------- |\n| **Security Model**  | [docs/security-model.md](./docs/security-model.md)       |\n| **Desktop App**     | [docs/tauri-desktop-app.md](./docs/tauri-desktop-app.md) |\n| **Load Testing**    | [api/tests/load/README.md](./api/tests/load/README.md)   |\n| **Contributing**    | [CONTRIBUTING.md](./CONTRIBUTING.md)                     |\n| **Code of Conduct** | [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md)               |\n| **Security Policy** | [SECURITY.md](./SECURITY.md)                             |\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eTech Stack\u003c/strong\u003e\u003c/summary\u003e\n\n| Layer             | Technology                                                                             |\n| ----------------- | -------------------------------------------------------------------------------------- |\n| **API**           | Python 3.12, FastAPI 0.115+, Pydantic 2, agent-framework                               |\n| **LLM**           | Anthropic Claude (Haiku routing, Sonnet specialist) — direct API or Azure AI Foundry   |\n| **RAG**           | Azure AI Search (hybrid BM25 + vector), Azure OpenAI text-embedding-3-large            |\n| **Database**      | PostgreSQL 17 with Alembic migrations                                                  |\n| **Web**           | React 19, Vite 7, TailwindCSS 4, TypeScript strict                                     |\n| **Desktop**       | Tauri 2 (Rust shell + shared web frontend)                                             |\n| **Mobile**        | React Native + Expo 54, NativeWind                                                     |\n| **Shared UI**     | [surf-kit](https://github.com/barney-w/surf-kit) — hooks, theme, icons, agent protocol |\n| **Auth**          | Microsoft Entra ID (JWKS) + HMAC guest tokens + MSAL                                   |\n| **Observability** | OpenTelemetry, Azure Monitor, Langfuse v3                                              |\n| **Infra**         | Bicep (Azure Verified Modules), Container Apps, VNet, Key Vault                        |\n| **CI/CD**         | GitHub Actions + GitLab CI (OIDC / WIF, zero stored secrets)                           |\n| **Testing**       | pytest, Playwright, Locust, LLM eval judge                                             |\n| **Quality**       | ruff (lint + format), pyright (strict types), pip-audit, Gitleaks                      |\n\n\u003c/details\u003e\n\n---\n\n[Apache-2.0](./LICENSE)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbarney-w%2Fsurf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbarney-w%2Fsurf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbarney-w%2Fsurf/lists"}