{"id":47915096,"url":"https://github.com/hra42/or-observer","last_synced_at":"2026-04-04T05:32:10.728Z","repository":{"id":342926601,"uuid":"1169303198","full_name":"hra42/or-observer","owner":"hra42","description":"Openrouter Observer build on Webhooks","archived":false,"fork":false,"pushed_at":"2026-03-28T18:05:29.000Z","size":281,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-28T19:26:52.084Z","etag":null,"topics":["analytics","duckdb","llm","monitoring","observability","openrouter","self-hosted"],"latest_commit_sha":null,"homepage":"","language":"Svelte","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hra42.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-28T13:37:32.000Z","updated_at":"2026-03-28T18:05:34.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/hra42/or-observer","commit_stats":null,"previous_names":["hra42/or-observer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hra42/or-observer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hra42%2For-observer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hra42%2For-observer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hra42%2For-observer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hra42%2For-observer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hra42","download_url":"https://codeload.github.com/hra42/or-observer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hra42%2For-observer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31389382,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T04:26:24.776Z","status":"ssl_error","status_checked_at":"2026-04-04T04:23:34.147Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","duckdb","llm","monitoring","observability","openrouter","self-hosted"],"created_at":"2026-04-04T05:32:09.562Z","updated_at":"2026-04-04T05:32:10.714Z","avatar_url":"https://github.com/hra42.png","language":"Svelte","funding_links":[],"categories":[],"sub_categories":[],"readme":"# or-observer\n\nSelf-hosted LLM observability platform for tracking cost, performance, errors, and usage analytics from OpenRouter API calls via webhooks.\n\n## What It Does\n\nor-observer receives traces from OpenRouter via webhooks and provides:\n\n- Cost tracking and breakdowns by model or user\n- Request latency metrics (avg, p95, p99) with hourly aggregation\n- Trace inspection with filterable, paginated table and detail view\n- Dashboard and analytics UI\n\nTarget scale: \u003c 1K requests/day. Runs self-hosted with no external dependencies.\n\n## Features\n\n- Webhook receiver for OpenRouter broadcast OTLP JSON events\n- Cost dashboard: real-time cost cards, hourly trend chart, top models table\n- Trace explorer: filterable and paginated with per-trace metadata modal\n- Analytics: cost breakdown by model/user, latency percentile charts\n- Embedded DuckDB with DuckLake extension — fast columnar storage, single file\n- Full REST API with pagination and time-range filtering\n\n## Quick Start\n\n```bash\ndocker-compose up --build\n```\n\n- Backend: `http://localhost:8080`\n- Frontend: `http://localhost:5173`\n\nData persists in `./data/`. Send test webhooks to `http://localhost:8080/webhook`.\n\n## Manual Setup\n\n### Prerequisites\n\n- Go 1.26+\n- Node.js 18+\n\n### Backend\n\n```bash\ngo mod download\ngo run cmd/server/main.go\n```\n\nStarts on `:8080` by default.\n\n### Frontend\n\n```bash\ncd frontend\nnpm install\nnpm run dev       # dev server with hot reload\nnpm run build     # production build\nnpm run check     # type check\n```\n\nConnects to `http://localhost:8080` by default. Override with `VITE_API_URL`.\n\n## Configuration\n\n| Env Var | Default | Description |\n|---------|---------|-------------|\n| `DB_PATH` | `data/traces.duckdb` | DuckDB file path |\n| `ADDR` | `:8080` | Backend listen address |\n| `VITE_API_URL` | `http://localhost:8080` | API base URL (frontend only) |\n\n## API Reference\n\n### POST /webhook\n\nReceives OpenRouter broadcast OTLP JSON. Returns `200 OK` on success.\n\n```bash\ncurl -X POST http://localhost:8080/webhook \\\n  -H \"Content-Type: application/json\" \\\n  -d @sample_trace.json\n```\n\n### GET /health\n\n```json\n{\n  \"status\": \"ok\",\n  \"database\": \"connected\",\n  \"traces_ingested\": 42,\n  \"uptime_seconds\": 120\n}\n```\n\n### GET /api/traces\n\nPaginated trace list.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `limit` | int | Max results (default 50, max 500) |\n| `offset` | int | Pagination offset |\n| `user_id` | string | Filter by user (exact match) |\n| `model` | string | Filter by model (exact match) |\n| `start_date` | ISO 8601 | Filter by created_at \u003e= |\n| `end_date` | ISO 8601 | Filter by created_at \u003c= |\n\n```json\n{\n  \"total\": 1234,\n  \"limit\": 50,\n  \"offset\": 0,\n  \"traces\": [\n    {\n      \"id\": \"abc-123-span-456\",\n      \"trace_id\": \"abc-123\",\n      \"span_id\": \"span-456\",\n      \"model\": \"openai/gpt-4\",\n      \"prompt_tokens\": 100,\n      \"completion_tokens\": 50,\n      \"total_tokens\": 150,\n      \"cost\": 0.00315,\n      \"duration_ms\": 1234,\n      \"user_id\": \"user-789\",\n      \"created_at\": \"2026-02-28T14:30:00Z\"\n    }\n  ]\n}\n```\n\n### GET /api/metrics/hourly\n\nOn-demand hourly aggregation from raw traces.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `start` | ISO 8601 | Range start (default: 24h ago) |\n| `end` | ISO 8601 | Range end (default: now) |\n| `groupBy` | `model` \\| `user` | Dimension to group by (default: overall) |\n\n```json\n{\n  \"metrics\": [\n    {\n      \"hour\": \"2026-02-28T14:00:00Z\",\n      \"dimension\": \"openai/gpt-4\",\n      \"request_count\": 42,\n      \"avg_latency_ms\": 1500.0,\n      \"p95_latency_ms\": 2800.0,\n      \"p99_latency_ms\": 3200.0,\n      \"total_tokens\": 6300,\n      \"total_cost\": 0.132,\n      \"error_count\": 1\n    }\n  ]\n}\n```\n\n### GET /api/costs/breakdown\n\nCost summary by dimension and period.\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `groupBy` | `model` \\| `user` | Required. Dimension to group by |\n| `period` | `hourly` \\| `daily` \\| `overall` | Required. Lookback window |\n\n```json\n{\n  \"period\": \"daily\",\n  \"group_by\": \"model\",\n  \"breakdown\": [\n    {\n      \"dimension\": \"openai/gpt-4\",\n      \"request_count\": 340,\n      \"total_cost\": 12.45,\n      \"avg_cost\": 0.036,\n      \"total_tokens\": 45600\n    }\n  ]\n}\n```\n\n## How It Works\n\n1. **Ingest** — OpenRouter sends OTLP JSON to `POST /webhook`. The `openrouter-go` SDK deserializes the payload; the handler inserts each span into DuckDB.\n2. **Deduplicate** — Each trace is keyed by `trace_id + span_id`. Duplicate webhook deliveries are silently ignored.\n3. **Query** — API handlers compute metrics on-demand via SQL (no background aggregation worker needed at this scale).\n4. **Visualize** — SvelteKit frontend fetches from `/api/*` with TanStack Query, renders charts with Recharts.\n\n## Development\n\n### Run tests\n\n```bash\ngo test ./...\ngo test -v ./internal/handlers/...\n```\n\n### Inspect the database\n\n```bash\nduckdb data/traces.duckdb\n# then:\nSELECT COUNT(*) FROM lake.traces;\nSELECT model, SUM(cost) FROM lake.traces GROUP BY model ORDER BY 2 DESC;\n```\n\n## Database Schema\n\n**lake.traces**\n\n| Column | Type | Notes |\n|--------|------|-------|\n| id | VARCHAR | trace_id + \"-\" + span_id |\n| trace_id | VARCHAR | |\n| span_id | VARCHAR | |\n| span_name | VARCHAR | |\n| model | VARCHAR | e.g. `openai/gpt-4` |\n| status | VARCHAR | `ok` or error |\n| prompt_tokens | INTEGER | |\n| completion_tokens | INTEGER | |\n| total_tokens | INTEGER | |\n| cost | DECIMAL(10,6) | USD |\n| duration_ms | INTEGER | |\n| user_id | VARCHAR | |\n| session_id | VARCHAR | |\n| metadata | JSON | merged span + resource attributes |\n| created_at | TIMESTAMP | |\n| webhook_received_at | TIMESTAMP | |\n\n**lake.metrics_hourly** — pre-aggregated rollups (hour, model, user_id, request_count, latency percentiles, total_cost, error_count)\n\n**lake.errors** — error logs (id, trace_id, error_type, error_message, stacktrace, created_at)\n\n## Troubleshooting\n\n**Frontend can't reach backend** — check `VITE_API_URL`. In Docker, services use `http://backend:8080`.\n\n**Webhook returns 400** — validate JSON matches OpenRouter OTLP format; check server logs for parse errors.\n\n**Traces not appearing** — same `trace_id + span_id` pair is deduplicated silently; verify with `SELECT * FROM lake.traces` in the duckdb CLI.\n\n**DuckDB locked** — embedded DuckDB allows one writer at a time. Ensure only one backend process is running and no duckdb CLI session has the file open.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhra42%2For-observer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhra42%2For-observer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhra42%2For-observer/lists"}