{"id":47685332,"url":"https://github.com/chrisvogt/chronogrove","last_synced_at":"2026-04-02T14:47:51.869Z","repository":{"id":37697409,"uuid":"190331097","full_name":"chrisvogt/chronogrove","owner":"chrisvogt","description":"Ingests and shapes data from Spotify, Goodreads, Steam, Flickr, Discogs and other services for my personal site.","archived":false,"fork":false,"pushed_at":"2026-04-01T05:46:02.000Z","size":1774,"stargazers_count":5,"open_issues_count":16,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-01T05:47:07.164Z","etag":null,"topics":["personal-api"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chrisvogt.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-06-05T05:25:59.000Z","updated_at":"2026-04-01T05:46:04.000Z","dependencies_parsed_at":"2024-09-15T09:57:14.768Z","dependency_job_id":"34b6aea0-b801-45b2-b0ca-4e1a27980d78","html_url":"https://github.com/chrisvogt/chronogrove","commit_stats":null,"previous_names":["chrisvogt/chronogrove"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/chrisvogt/chronogrove","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisvogt%2Fchronogrove","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisvogt%2Fchronogrove/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisvogt%2Fchronogrove/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisvogt%2Fchronogrove/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chrisvogt","download_url":"https://codeload.github.com/chrisvogt/chronogrove/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisvogt%2Fchronogrove/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31308432,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-02T12:59:32.332Z","status":"ssl_error","status_checked_at":"2026-04-02T12:54:48.875Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["personal-api"],"created_at":"2026-04-02T14:47:47.513Z","updated_at":"2026-04-02T14:47:51.864Z","avatar_url":"https://github.com/chrisvogt.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align='center'\u003e\n  Chronogrove (\u003ca href='https://metrics.chrisvogt.me' title='Operator console'\u003emetrics.chrisvogt.me\u003c/a\u003e)\n\u003c/h1\u003e\n\n\u003cp align='center'\u003e\n  \u003ca href='https://github.com/chrisvogt/chronogrove/actions/workflows/ci.yml'\u003e\n    \u003cimg src='https://github.com/chrisvogt/chronogrove/actions/workflows/ci.yml/badge.svg?branch=main' alt='Continuous Integration badge' /\u003e\n  \u003c/a\u003e\n  \u003ca href='https://github.com/chrisvogt/chronogrove/actions/workflows/codeql.yml'\u003e\n    \u003cimg src='https://github.com/chrisvogt/chronogrove/actions/workflows/codeql.yml/badge.svg?branch=main' alt='CodeQL badge' /\u003e\n  \u003c/a\u003e\n  \u003ca href='https://codecov.io/gh/chrisvogt/chronogrove'\u003e\n    \u003cimg src='https://codecov.io/gh/chrisvogt/chronogrove/branch/main/graph/badge.svg?token=Hr0GpQiCu0' alt='Code coverage report badge.' /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n**Chronogrove** is the engine behind provider-backed widgets on [www.chrisvogt.me](https://www.chrisvogt.me): it syncs third-party accounts (Discogs, Steam, Instagram, Spotify, Goodreads, Flickr, and more), stores normalized widget documents, and serves them over a stable JSON API. Firebase is the reference runtime (Hosting + Cloud Functions + Firestore); the design stays portable enough to consider other hosts later.\n\nConsumer experiences today include the open-source [**Gatsby theme Chronogrove**](https://github.com/chrisvogt/gatsby-theme-chronogrove). The goal is for the same API to power other site integrations (WordPress and similar) and, over time, shareable **Web Components** (and other HTML-native building blocks) that call the public routes directly.\n\nThis repository holds the **backend and operator console** (schema browser, status checks, authenticated sync). The themed marketing site and MDX content live in the Gatsby theme and site repos above.\n\n## Quick start (first 5 minutes)\n\n1. **Install prerequisites**\n   - Node.js (version in [.nvmrc](./.nvmrc), currently 24+)\n   - pnpm (for example: `corepack enable \u0026\u0026 corepack prepare pnpm@10.32.1 --activate`)\n   - Firebase CLI (`pnpm add -g firebase-tools` or `npm install -g firebase-tools`)\n   - `firebase login`\n2. **Clone and install**\n   ```bash\n   git clone git@github.com:chrisvogt/chronogrove.git\n   cd chronogrove\n   pnpm install\n   ```\n3. **Set local env vars**\n   ```bash\n   cp functions/.env.template functions/.env.local\n   # edit functions/.env.local (at least CLIENT_API_KEY, CLIENT_AUTH_DOMAIN, CLIENT_PROJECT_ID)\n   ```\n4. **Run local dev (recommended)**\n   ```bash\n   pnpm run dev:full\n   ```\n5. **Open**\n   - App: `http://localhost:5173`\n   - Emulator UI: `http://127.0.0.1:4000`\n\nIf `/api` calls fail in local dev, the Functions emulator is usually not reachable.\n\n## What this project does\n\n- Fetches and serves widget data for: Spotify, Steam, Goodreads, Instagram, Discogs, Flickr, and GitHub.\n- Supports scheduled sync jobs plus manual admin-triggered sync.\n- Uses Firebase Auth (Google, email/password, phone) with HTTP-only session cookies and JWT fallback.\n- Runs locally with Firebase emulators.\n- Serves the Next.js operator dashboard at [metrics.chrisvogt.me](https://metrics.chrisvogt.me).\n\n\u003e Note: `github` is a readable widget provider, but **not** part of the scheduled/manual sync queue.\n\n## Architecture at a glance\n\nThis service backs widgets on [www.chrisvogt.me](https://www.chrisvogt.me) and any client using the same API contract (for example the [Gatsby theme](https://github.com/chrisvogt/gatsby-theme-chronogrove)). Each diagram is intentionally focused on one path. For queue semantics and job document fields, see [docs/SYNC_JOB_QUEUE.md](docs/SYNC_JOB_QUEUE.md).\n\n### 1) Public widget reads\n\nUnauthenticated widget reads from Firestore-backed content.\n\n```mermaid\nflowchart LR\n  site[www.chrisvogt.me or theme] --\u003e fn[Cloud Functions\u003cbr/\u003eGET /api/widgets/:provider]\n  fn --\u003e fs[(Firestore\u003cbr/\u003eusers/.../widget-content)]\n```\n\n### 2) Scheduled sync (planner + worker)\n\nPlanner enqueues one job per syncable provider. Worker claims queued jobs and runs provider sync.\n\n```mermaid\nflowchart TB\n  subgraph sched[Cloud Scheduler]\n    p[runSyncPlanner · default schedule]\n    w[runSyncWorker · every 15 min]\n  end\n  p --\u003e plan[planSyncJobs]\n  plan --\u003e q[(Firestore · sync_jobs)]\n  w --\u003e next[runNextSyncJob]\n  next --\u003e q\n  next --\u003e job[processSyncJob + provider sync]\n  job --\u003e apis[Platform APIs]\n  job --\u003e docs[(Firestore · widget documents)]\n```\n\n### 3) Operator console manual sync\n\n[metrics.chrisvogt.me](https://metrics.chrisvogt.me) uses Firebase Auth + session cookie. Manual sync runs inline (enqueue -\u003e claim -\u003e process) instead of waiting for worker cadence.\n\n```mermaid\nflowchart TB\n  admin[Operator console] --\u003e auth[Firebase Auth]\n  admin --\u003e sess[POST /api/auth/session]\n  admin --\u003e sync[GET /api/widgets/sync/:provider]\n  admin --\u003e stream[GET .../sync/:provider/stream SSE]\n  sync --\u003e fn[runSyncForProvider]\n  stream --\u003e fn\n  fn --\u003e q[(sync_jobs)]\n  fn --\u003e job[processSyncJob]\n  job --\u003e out[Platform APIs + widget writes]\n```\n\n### Key request flows\n\n| Flow | Description |\n|------|-------------|\n| **Widget reads** | `GET /api/widgets/:provider` (public, cached). Reads provider widget document from Firestore and returns it. |\n| **Scheduled sync** | `runSyncPlanner` enqueues queue jobs; `runSyncWorker` periodically claims and executes queued jobs. |\n| **Manual sync** | Authenticated `GET /api/widgets/sync/:provider` (JSON) or `GET /api/widgets/sync/:provider/stream` (SSE). Both use the same queue + inline processing path. |\n| **Auth** | Dashboard signs in with Firebase Auth and creates a session cookie through `POST /api/auth/session`. Protected routes accept session cookie or JWT. |\n\n## Monorepo layout\n\nThis repository is a pnpm workspace with:\n\n- `hosting/`: Next.js dashboard (static export)\n- `functions/`: Firebase Cloud Functions backend\n\nTurborepo runs workspace scripts from the root and caches work.\n\n**Use repo root for commands** (do not run per-package installs).\n\n## Commands (repo root)\n\n| Command | What it does |\n|--------|----------------|\n| `pnpm install` | Install dependencies for root and both packages. |\n| `pnpm run dev` | Run Next.js dev server on `localhost:5173`. Expects Functions emulator to be running for `/api` calls. |\n| `pnpm run dev:full` | Run Firebase emulators + Next dev together (uses `firebase emulators:start` and `pnpm run dev`). |\n| `pnpm run build` | Run workspace builds via Turborepo (`hosting` export and `functions` TypeScript build). |\n| `pnpm run lint` | Run workspace lint tasks (currently functions ESLint). |\n| `pnpm run test` | Run workspace tests. |\n| `pnpm run test:coverage` | Run tests with coverage. |\n| `pnpm run deploy:all` | Guard env + build + deploy default Firebase targets. |\n| `pnpm run deploy:hosting` | Build and deploy only Firebase Hosting. |\n| `pnpm run deploy:functions` | Guard env + deploy only Functions (Firebase predeploy still builds functions). |\n\n\u003e Use `pnpm run deploy:all` (with `run`). `pnpm deploy` is a pnpm command, not this project's deploy flow.\n\n## Local development\n\n### Option A (recommended): hot reload dashboard + emulators\n\nOne terminal:\n\n```bash\npnpm run dev:full\n```\n\nOr split terminals:\n\n```bash\n# Terminal 1\nfirebase emulators:start --only functions,auth\n\n# Terminal 2\npnpm run dev\n```\n\nOpen `http://localhost:5173`.\n\n### Option B: full Firebase-like local serving\n\n```bash\npnpm run build\nfirebase emulators:start --only hosting,functions,auth\n```\n\nOpen the hosting URL (for example `http://metrics.dev-chrisvogt.me:8084`).\n\n### Emulator URLs\n\n| Service | URL |\n|---------|-----|\n| Emulator UI | `http://127.0.0.1:4000` |\n| Hosting | `http://127.0.0.1:8084` (or configured host) |\n| Functions | `http://127.0.0.1:5001` |\n| Auth | `http://127.0.0.1:9099` |\n| Firestore | `http://127.0.0.1:8080` |\n\n## Environment variables\n\nFor local development:\n\n```bash\ncp functions/.env.template functions/.env.local\n```\n\nSet at minimum:\n\n- `CLIENT_API_KEY`\n- `CLIENT_AUTH_DOMAIN`\n- `CLIENT_PROJECT_ID`\n\nOptional examples:\n\n- `NODE_ENV=development`\n- `GEMINI_API_KEY` (if AI summary features are enabled)\n\n### Important env safety notes\n\n- Never commit `functions/.env.local`.\n- Avoid `functions/.env` during normal development; Firebase can deploy values from that file into Functions.\n\n## API surface (high-level)\n\n### Public widget reads\n\n- `GET /api/widgets/:provider` where `provider` is one of:\n  - `discogs`, `flickr`, `github`, `goodreads`, `instagram`, `spotify`, `steam`\n\n### Protected sync endpoints\n\n- `GET /api/widgets/sync/:provider` (JSON)\n- `GET /api/widgets/sync/:provider/stream` (SSE)\n\nSyncable `provider` values are:\n\n- `discogs`, `flickr`, `goodreads`, `instagram`, `spotify`, `steam`\n\n### Auth/config endpoints\n\n- `POST /api/auth/session`\n- `POST /api/auth/logout`\n- `GET /api/client-auth-config`\n- `GET /api/firebase-config` (compat alias)\n\n## Hosting and backend notes\n\n### Hosting rewrites (`firebase.json`)\n\n1. `/api/**` rewrites to Cloud Function `app`.\n2. No SPA catch-all rewrite. Static exported routes are served directly; unmatched routes return exported `404.html`.\n\n### Backend details (`functions/`)\n\n- Provider-neutral bootstrap wires runtime/config/store/auth adapters.\n- Current implementation uses Firebase runtime/auth/document adapters.\n- Functions source is TypeScript; build output is `functions/lib/`.\n\n## Testing\n\nFrom repo root:\n\n```bash\npnpm run test\npnpm run test:coverage\n```\n\nFunctions watch mode:\n\n```bash\npnpm --filter chronogrove-functions run test:watch\n```\n\n## Deployment\n\nFrom repo root:\n\n```bash\npnpm run build\npnpm run deploy:all\npnpm run deploy:hosting\npnpm run deploy:functions\n```\n\n## Additional docs\n\nReference docs under [`docs/`](docs/):\n\n| Document | What it covers |\n|----------|----------------|\n| [docs/SYNC_JOB_QUEUE.md](docs/SYNC_JOB_QUEUE.md) | `sync_jobs` queue behavior (planner, worker, manual sync, states, summary metrics). |\n| [docs/SESSION_COOKIES.md](docs/SESSION_COOKIES.md) | Session cookie model, `/api/auth/session`, JWT fallback, security properties. |\n| [docs/MULTI_TENANT_ARCHITECTURE_PLAN.md](docs/MULTI_TENANT_ARCHITECTURE_PLAN.md) | Migration plan from single-tenant env config toward user-scoped storage and sync. |\n\n## Contributing\n\n1. Fork the repository.\n2. Create a feature branch (`git checkout -b feature/amazing-feature`).\n3. Install and configure local env.\n4. Run tests (`pnpm run test`).\n5. Ensure builds pass (`pnpm run build`).\n6. Open a pull request.\n\n## Copyright \u0026 License\n\nCopyright © 2020-2026 [Chris Vogt](https://www.chrisvogt.me). Released under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrisvogt%2Fchronogrove","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchrisvogt%2Fchronogrove","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrisvogt%2Fchronogrove/lists"}