{"id":51528535,"url":"https://github.com/thenoblet/prompthire-backend","last_synced_at":"2026-07-09T00:30:37.385Z","repository":{"id":356823190,"uuid":"1234202585","full_name":"thenoblet/prompthire-backend","owner":"thenoblet","description":"FastAPI service that generates three role-specific interview questions for a 30-minute screening, given a job title.","archived":false,"fork":false,"pushed_at":"2026-05-10T00:20:40.000Z","size":18,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-10T00:27:04.808Z","etag":null,"topics":["ai","fastapi","instructor","litellm","postgresql","pydantic"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thenoblet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-09T22:00:35.000Z","updated_at":"2026-05-10T00:23:51.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/thenoblet/prompthire-backend","commit_stats":null,"previous_names":["thenoblet/prompthire-backend"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/thenoblet/prompthire-backend","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoblet%2Fprompthire-backend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoblet%2Fprompthire-backend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoblet%2Fprompthire-backend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoblet%2Fprompthire-backend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thenoblet","download_url":"https://codeload.github.com/thenoblet/prompthire-backend/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thenoblet%2Fprompthire-backend/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35282897,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-08T02:00:06.796Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","fastapi","instructor","litellm","postgresql","pydantic"],"created_at":"2026-07-09T00:30:35.639Z","updated_at":"2026-07-09T00:30:37.375Z","avatar_url":"https://github.com/thenoblet.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PromptHire Backend\n\nFastAPI service that generates three role-specific interview questions for a 30-minute screening, given a job title. Built with a strict layered architecture, resilient LLM access (retries + multi-model fallback), Postgres-backed caching and rate limiting, and a typed wire envelope shared with the [PromptHire frontend](https://github.com/thenoblet/prompthire-frontend).\n\n## What this service does\n\n- Accepts `POST /api/v1/generate` with a job role (e.g. `\"Senior Backend Engineer\"`).\n- Returns exactly three structured questions: `{ category, question, rationale }`.\n- Caches identical roles for 24 hours, so repeat traffic doesn't hit the LLM.\n- Applies layered rate limits (per-IP/min, per-IP/day, global/day) sized for free-tier provider quotas.\n- Falls back through a configurable model chain when a provider rate-limits, times out, or auth-fails — so a free-tier outage doesn't take the demo down.\n- Audits every attempt (success, schema failure, upstream error, cache hit) to a `generations` table for post-hoc analysis.\n\n## Tech stack\n\n| Layer | Choice | Why |\n|---|---|---|\n| Web framework | **FastAPI** + Uvicorn | Async-first, strong Pydantic integration, lifespan-managed app state. |\n| Validation / config | **Pydantic v2** + `pydantic-settings` | Wire DTOs and env-driven config in one type system. |\n| LLM access | **litellm** + **instructor** | Provider-agnostic SDK; structured outputs enforced via Pydantic schemas. |\n| Resilience | **tenacity** | Exponential-backoff retries on transient provider errors. |\n| Storage | **PostgreSQL 16** via async **SQLAlchemy 2.0** + **asyncpg** | Async-friendly ORM, atomic `INSERT … ON CONFLICT DO UPDATE` for counters. |\n| Migrations | **Alembic** | Async-engine env, hand-written migrations for predictability. |\n| Tooling | **ruff** (lint + format), `pip` + `requirements.txt` | Single tool for style; no Poetry/uv ceremony. |\n\n## Resilience model\n\nTwo stacked layers of fault tolerance around every LLM call:\n\n```\n┌───────────────────────────────────────────────────────────────────┐\n│   Outer:  fallback chain over models                              │\n│           Gemini → OpenRouter A → OpenRouter B → ...              │\n│   Inner:  tenacity retries (3 attempts, 2s/4s/8s backoff)         │\n│           on RateLimitError / Timeout / APIConnectionError        │\n└───────────────────────────────────────────────────────────────────┘\n```\n\n- A model failure (any classified or unknown error) advances the fallback chain.\n- The model that *actually answered* is recorded in the audit row (not just the primary).\n- All retries exhausted across all models → `502` with a typed error envelope.\n\nPlus a Postgres-backed question cache: identical normalised role + model → cached 24h, no LLM call.\n\n## API contract\n\n### Endpoints\n\n| Method | Path | Notes |\n|---|---|---|\n| `GET` | `/healthz` | Liveness probe; reports DB connectivity. No rate limit. |\n| `POST` | `/api/v1/generate` | Generate three questions. Rate-limited per IP + global daily cap. |\n\n### Request\n\n```json\nPOST /api/v1/generate\nContent-Type: application/json\n\n{ \"role\": \"Senior Backend Engineer\" }\n```\n\n`role` is `min_length=1, max_length=200, strip_whitespace=True`.\n\n### Successful response\n\n```json\n{\n  \"data\": {\n    \"questions\": [\n      { \"category\": \"...\", \"question\": \"...\", \"rationale\": \"...\" },\n      { \"category\": \"...\", \"question\": \"...\", \"rationale\": \"...\" },\n      { \"category\": \"...\", \"question\": \"...\", \"rationale\": \"...\" }\n    ]\n  },\n  \"meta\": null\n}\n```\n\nAlways exactly three questions. Schema enforced via instructor's `response_model`.\n\n### Error response (any non-2xx)\n\n```json\n{\n  \"error\": {\n    \"message\": \"user-facing string\",\n    \"code\": \"STABLE_TAG\",\n    \"error_code\": null,\n    \"severity\": \"error\" | \"warning\",\n    \"retryable\": true | false,\n    \"reference_id\": \"9f3c…\",\n    \"details\": null\n  }\n}\n```\n\n`reference_id` is a fresh UUID per error and is logged at the same severity, so support reports correlate to log lines. `details` carries per-field info on validation errors.\n\n### Error code reference\n\n| Source | HTTP | `code` | `retryable` |\n|---|---|---|---|\n| Pydantic request validation | 422 | `VALIDATION_ERROR` | `false` |\n| Per-IP rate limit (min or day) exceeded | 429 | `RATE_LIMITED` | `true` |\n| Global daily cap reached | 503 | `SERVICE_AT_CAPACITY` | `false` |\n| All models in chain exhausted retries | 502 | `UPSTREAM_ERROR` | `true` |\n| All models produced malformed output | 502 | `BAD_SHAPE` | `true` |\n| DB unreachable on critical path | 503 | `DB_UNAVAILABLE` | `true` |\n| Service misconfigured | 500 | `CONFIG_ERROR` | `false` |\n| Unhandled exception | 500 | `INTERNAL_ERROR` | `false` |\n\n## Project layout\n\n```\nsrc/app/\n├── main.py                      # FastAPI factory, lifespan (DatabaseManager + LLMClient on app.state)\n│\n├── core/                        # Cross-cutting; no business logic. Leaf module.\n│   ├── config.py                # Pydantic-Settings; required + optional env vars\n│   ├── exceptions.py            # Domain error hierarchy with HTTP codes \u0026 severities\n│   ├── error_handlers.py        # FastAPI exception handlers → ErrorResponse envelope\n│   ├── rate_limit.py            # Multi-window FastAPI dependency\n│   └── deps.py                  # SessionDep / LLMClientDep / QuestionServiceDep\n│\n├── routers/                     # HTTP layer; thin\n│   ├── health.py                # GET /healthz\n│   └── v1/                      # Versioned API\n│       └── generate.py          # POST /api/v1/generate\n│\n├── services/                    # Use case orchestration\n│   └── question_service.py      # Cache → cap check → LLM → counter → audit\n│\n├── repositories/                # Postgres-backed persistence per table\n│   ├── audit_repository.py      # Writes to `generations`\n│   ├── cache_repository.py      # Reads/writes `question_cache` (normalize_role lives here)\n│   └── rate_limit_repository.py # Three windows: minute, day, global\n│\n├── infrastructure/              # External integrations; the only place these libs are imported\n│   ├── database.py              # DatabaseManager: engine + sessionmaker + lifecycle\n│   ├── llm.py                   # LLMClient: litellm + instructor + tenacity + fallback chain\n│   └── prompts.py               # System + user prompt templates\n│\n├── models/                      # Domain types and ORM tables\n│   ├── question.py              # Question (frozen dataclass) — domain shape\n│   └── db/                      # SQLAlchemy ORM models — owned by persistence layer\n│       ├── base.py\n│       ├── generation.py        # Audit row\n│       ├── question_cache.py    # 24h LLM-response cache\n│       ├── rate_limit_bucket.py # Per-IP per-minute counter\n│       ├── rate_limit_daily.py  # Per-IP per-day counter\n│       └── global_daily_count.py# Service-wide per-day counter\n│\n└── schemas/                     # All Pydantic models live here\n    ├── generate.py              # Wire DTOs for /api/v1/generate\n    ├── health.py                # Wire DTO for /healthz\n    ├── response.py              # Generic ApiResponse[T] + ErrorResponse envelope\n    └── llm.py                   # LLMQuestion / LLMQuestions — instructor binds against this\n```\n\n### Dependency rule (enforced by code review)\n\n```\nrouters   →  services  →  repositories ──┐\n   ↓            ↓                          ├──→  asyncpg, SQLAlchemy\nschemas      models      infrastructure ──┘     litellm, instructor, tenacity\n   ↓            ↓               ↓\n              core            core\n```\n\n- `routers` never import litellm or SQLAlchemy.\n- `repositories` never import FastAPI.\n- `infrastructure` is the only ring that imports external SDKs.\n- `schemas` and `models` don't import each other; translation happens at the router boundary.\n- `core` is leaf — everything imports from it; it imports nothing else from `app/`.\n\n## Database schema\n\n| Table | Purpose | Key |\n|---|---|---|\n| `generations` | Audit log of every `/generate` attempt (success, cache hit, schema failure, upstream error). Stores model, latency, JSONB questions or error summary. | `id BIGSERIAL` |\n| `question_cache` | TTL-bound cache of LLM responses keyed by `sha256(model + \":\" + normalized_role)`. Includes `hit_count` for hot-role analysis. | `role_hash` |\n| `rate_limit_buckets` | Per-IP per-minute counter (atomic UPSERT). | `(ip, route, window_start)` |\n| `rate_limit_daily` | Per-IP per-day counter. | `(ip, route, day)` |\n| `global_daily_count` | Service-wide per-day counter. Cache hits do **not** count against this. | `(day, route)` |\n\nMigration `0001_initial.py` creates `generations` + `rate_limit_buckets`. Migration `0002_cache_and_rate_limit_v2.py` adds the three new tables and extends `generations.status` to allow `'cache_hit'`.\n\n## Setup\n\n### Local development\n\n```bash\n# 1. Clone and create venv\npython3.11 -m venv .venv\nsource .venv/bin/activate\n\n# 2. Install deps\npip install -r requirements.txt -r requirements-dev.txt\npip install -e . --no-deps                       # src-layout: register the `app` package\n\n# 3. Local Postgres for dev (compose ships a healthchecked postgres:16 container)\ndocker compose up -d postgres\n\n# 4. Configure env\ncp .env.example .env                             # edit: LITELLM_MODEL, provider key,\n                                                 #       DB_HOST/DB_PORT/DB_USER/DB_PASSWORD/DB_NAME\n                                                 # see Configuration section below\n\n# 5. Apply migrations  (only needed for local dev — Docker handles this in production)\nalembic upgrade head\n\n# 6. Run\nuvicorn app.main:app --reload                    # listens on :8000\n```\n\nVerify:\n\n```bash\ncurl -s http://localhost:8000/healthz | jq\n# → {\"data\": {\"status\": \"ok\", \"db\": \"ok\"}, \"meta\": null}\n\ncurl -s -X POST http://localhost:8000/api/v1/generate \\\n  -H \"content-type: application/json\" \\\n  -d '{\"role\": \"Senior Backend Engineer\"}' | jq\n```\n\n### Docker / production\n\nThe container is **self-deploying**: `entrypoint.sh` runs `alembic upgrade head` and then `exec`s the CMD (uvicorn). Pull the new image, start it, and the schema migrates itself before the first request lands.\n\n```bash\ndocker build -t prompthire-backend:dev .\ndocker run --rm \\\n  -e LITELLM_MODEL=gemini/gemini-2.5-flash \\\n  -e GEMINI_API_KEY=... \\\n  -e DB_HOST=postgres -e DB_PORT=5432 \\\n  -e DB_USER=postgres -e DB_PASSWORD=... -e DB_NAME=prompthire \\\n  -p 8000:8000 \\\n  prompthire-backend:dev\n```\n\nWhat that one `docker run` does:\n\n1. `entrypoint.sh` runs `alembic upgrade head` — idempotent; no-op if already at head.\n2. Logs `[entrypoint] Migrations applied. Starting application: …`.\n3. `exec`s into uvicorn so `SIGTERM` from `docker stop` reaches the app directly and shutdown is graceful.\n\nIf a migration fails, the container exits non-zero and the orchestrator's restart policy applies. **Trade-off worth knowing:** in a multi-instance deploy (e.g. multiple replicas behind a load balancer) every replica races to apply the same migration. Acceptable for single-instance Dokploy deploys; for multi-instance the right answer is a separate \"migrate\" job in the deploy pipeline. See `entrypoint.sh` for the full script.\n\n**Operator escape hatches** (when you need to bypass the auto-migrate or roll back):\n\n```bash\n# drop into a shell, no migrations\ndocker run --rm -it --entrypoint /bin/sh prompthire-backend:dev\n\n# roll back one migration without starting the app\ndocker run --rm --entrypoint alembic prompthire-backend:dev downgrade -1\n\n# the entrypoint still runs first, but CMD is overridable for one-off ops\ndocker run --rm prompthire-backend:dev alembic current\n```\n\n## Configuration\n\nAll config is environment-driven via `pydantic-settings`. See `.env.example` for the full list with comments. The notable ones:\n\n| Var | Required | Default | Notes |\n|---|---|---|---|\n| `LITELLM_MODEL` | yes | — | Primary model id (e.g. `gemini/gemini-2.5-flash`). |\n| `LITELLM_FALLBACK_MODELS` | no | `\"\"` | Comma-separated chain. Tried in order if primary exhausts retries. |\n| `\u003cPROVIDER\u003e_API_KEY` | yes | — | E.g. `GEMINI_API_KEY`, `OPENROUTER_API_KEY`. Loaded by `python-dotenv` into `os.environ` at startup so litellm picks them up. |\n| `DB_HOST` | yes | — | Postgres hostname (e.g. `localhost`, or service name in a Docker network). |\n| `DB_PORT` | no | `5432` | Postgres TCP port. |\n| `DB_USER` | yes | — | Postgres username. |\n| `DB_PASSWORD` | yes | — | Postgres password. URL-encoded automatically when the connection string is composed, so special characters are safe. |\n| `DB_NAME` | yes | — | Postgres database name. |\n| `RATE_LIMIT_PER_MIN` | no | `5` | Per-IP per-minute cap. |\n| `RATE_LIMIT_PER_DAY` | no | `20` | Per-IP per-day cap. |\n| `GLOBAL_DAILY_CAP` | no | `200` | Service-wide cap on LLM calls per UTC day (cache hits exempt). |\n| `CACHE_ENABLED` | no | `true` | Operator kill-switch for the question cache. |\n| `CACHE_TTL_HOURS` | no | `24` | Cache entry lifetime. |\n| `LITELLM_TIMEOUT_SECONDS` | no | `30` | Hard ceiling on a single LLM call. |\n| `CORS_ORIGINS` | no | `\"\"` | Comma-separated allowlist; empty = same-origin only. |\n| `TRUST_FORWARDED_FOR` | no | `false` | Trust the first hop of `X-Forwarded-For` for rate-limit IP keying. Enable only behind a known proxy. |\n| `LOG_LEVEL` | no | `INFO` | Standard Python logging level. |\n\n### Multi-model fallback example\n\n```bash\nLITELLM_MODEL=gemini/gemini-2.5-flash\nLITELLM_FALLBACK_MODELS=openrouter/poolside/laguna-m.1:free,openrouter/meta-llama/llama-3.1-8b-instruct:free\nGEMINI_API_KEY=...\nOPENROUTER_API_KEY=...\n```\n\nWhen Gemini's free tier 429s, the request transparently rolls over to the first OpenRouter model; if that also fails, it tries the second. The audit row records which model actually answered.\n\n## Engineering decisions worth a closer look\n\nIf a reviewer is scrolling, these are the spots that demonstrate non-obvious thinking — start here:\n\n- **`src/app/infrastructure/llm.py`** — multi-model fallback wrapping per-model tenacity retries; `_root_cause` walks the full `__cause__`/`__context__` chain because instructor wraps provider errors at varying depths.\n- **`src/app/services/question_service.py`** — fixed-order pipeline (cache → cap → LLM → counter → cache write → audit) with explicit non-fatal failure modes documented inline.\n- **`src/app/core/exceptions.py`** — domain errors carry their own `code`/`http_status`/`severity`/`retryable`, so the wire envelope is a near-mechanical translation in `error_handlers.py`.\n- **`src/app/repositories/cache_repository.py`** — `normalize_role` order matters (NFKC → strip → lowercase → collapse whitespace → strip control chars); `sha256(model + \":\" + normalized_role)` so model swaps invalidate cache automatically.\n- **`src/app/repositories/rate_limit_repository.py`** — three atomic UPSERT counters (minute / day / global) with `RETURNING count` so we never read-then-write.\n- **`src/app/schemas/response.py`** — single `ApiResponse[T]` + `ErrorResponse` envelope used by every route; consistent shape regardless of payload.\n- **`src/app/main.py`** — composition root; only place where `DatabaseManager` and `LLMClient` are constructed; loads `.env` into `os.environ` so litellm sees provider keys.\n- **`entrypoint.sh` + `Dockerfile`** — self-deploying container: migrations run on container start before uvicorn. `exec \"$@\"` so `SIGTERM` reaches the app directly. The CMD remains overridable for operator one-offs.\n\n## Frontend integration\n\nThe [frontend](../prompthire-frontend) `lib/generator.ts` posts to `/api/v1/generate` and unwraps the `ApiResponse[T]` / `ErrorResponse` envelope into a typed `GeneratorError` (with `severity` / `retryable` / `referenceId`). In dev, Vite proxies `/api/*` to `http://localhost:8000`. Same-origin in prod via nginx.\n\n## Things deliberately *not* in this version\n\nThese were considered and deferred — listed here so a reviewer knows they were thought about, not missed:\n\n- **Tests.** No pytest harness in this revision. Manual smoke verification via TestClient/curl. Adding a behaviour-driven test pass is a tracked next step.\n- **Auth.** Public endpoint by design; per-IP + global rate limits + budget cap are the only guards.\n- **Sliding-window rate limit.** Fixed-window is sufficient for the threat model and single-instance deploy.\n- **Token / cost tracking.** Free-tier providers; no dollars to count. The audit row carries enough to retrofit later.\n- **Observability beyond logs.** Stdlib logging with structured-ish messages; metrics / traces / structlog deferred.\n- **Streaming responses.** Three-question payload is small; streaming would be theatre.\n- **Distributed cache / rate-limit.** Postgres scales horizontally; Redis is the obvious upgrade path if traffic warrants.\n\n## License\n\nFor demonstration / portfolio use.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthenoblet%2Fprompthire-backend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthenoblet%2Fprompthire-backend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthenoblet%2Fprompthire-backend/lists"}