{"id":37081902,"url":"https://github.com/dualeai/hpke-http","last_synced_at":"2026-02-06T20:08:09.553Z","repository":{"id":332166834,"uuid":"1128458589","full_name":"dualeai/hpke-http","owner":"dualeai","description":"End-to-end encryption for HTTP APIs using RFC 9180 HPKE. Drop-in middleware for FastAPI, aiohttp, and httpx.","archived":false,"fork":false,"pushed_at":"2026-01-12T10:39:11.000Z","size":657,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-12T23:10:10.976Z","etag":null,"topics":["aiohttp","chacha20","cryptography","e2e","encryption","fastapi","hpke","http","rfc9180","sse","streaming","x25519"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dualeai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-05T17:03:27.000Z","updated_at":"2026-01-09T22:12:58.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/dualeai/hpke-http","commit_stats":null,"previous_names":["dualeai/hpke-http"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/dualeai/hpke-http","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dualeai%2Fhpke-http","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dualeai%2Fhpke-http/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dualeai%2Fhpke-http/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dualeai%2Fhpke-http/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dualeai","download_url":"https://codeload.github.com/dualeai/hpke-http/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dualeai%2Fhpke-http/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28416384,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiohttp","chacha20","cryptography","e2e","encryption","fastapi","hpke","http","rfc9180","sse","streaming","x25519"],"created_at":"2026-01-14T09:59:02.045Z","updated_at":"2026-02-06T20:08:09.537Z","avatar_url":"https://github.com/dualeai.png","language":"Python","readme":"# hpke-http\n\nEnd-to-end encryption for HTTP APIs using RFC 9180 HPKE (Hybrid Public Key Encryption). Drop-in middleware for FastAPI, aiohttp, and httpx.\n\n[![CI](https://github.com/dualeai/hpke-http/actions/workflows/test.yml/badge.svg)](https://github.com/dualeai/hpke-http/actions/workflows/test.yml)\n[![PyPI](https://img.shields.io/pypi/v/hpke-http)](https://pypi.org/project/hpke-http/)\n[![Downloads](https://img.shields.io/pypi/dm/hpke-http)](https://pypi.org/project/hpke-http/)\n[![Python](https://img.shields.io/pypi/pyversions/hpke-http)](https://pypi.org/project/hpke-http/)\n[![License](https://img.shields.io/pypi/l/hpke-http)](https://opensource.org/licenses/Apache-2.0)\n\n## Highlights\n\n- **Transparent** - Drop-in middleware, no application code changes\n- **End-to-end encryption** - Protects data even when TLS terminates at CDN or load balancer\n- **PSK binding** - Each request cryptographically bound to pre-shared key (API key)\n- **Replay protection** - Counter-based nonces prevent replay attacks\n- **RFC 9180 compliant** - Auditable, interoperable standard\n- **Memory-efficient** - Streams large file uploads with O(chunk_size) memory\n\n## Installation\n\n```bash\nuv add \"hpke-http[fastapi]\"       # Server\nuv add \"hpke-http[aiohttp]\"       # Client (aiohttp)\nuv add \"hpke-http[httpx]\"         # Client (httpx)\nuv add \"hpke-http[fastapi,zstd]\"  # + zstd compression (gzip fallback included)\n```\n\n## Quick Start\n\nStandard JSON requests, SSE (Server-Sent Events) streaming, and file uploads are transparently encrypted.\n\n### Server (FastAPI)\n\n```python\nfrom fastapi import FastAPI, Request\nfrom fastapi.responses import StreamingResponse\nfrom hpke_http.middleware.fastapi import HPKEMiddleware\nfrom hpke_http.constants import KemId\n\napp = FastAPI()\n\nasync def resolve_psk(scope: dict) -\u003e tuple[bytes, bytes]:\n    # Get derived PSK ID from X-HPKE-PSK-ID header (already decoded)\n    psk_id = scope.get(\"hpke_psk_id\")\n    # Look up API key by its derived ID (see \"PSK Authentication\" section)\n    record = await db.lookup_by_derived_id(psk_id)  # Returns {psk, tenant_id}\n    scope[\"tenant_id\"] = record[\"tenant_id\"]  # For authorization\n    return (record[\"psk\"], psk_id)\n\napp.add_middleware(\n    HPKEMiddleware,\n    private_keys={KemId.DHKEM_X25519_HKDF_SHA256: private_key},\n    psk_resolver=resolve_psk,\n)\n\n@app.post(\"/users\")\nasync def create_user(request: Request):\n    data = await request.json()  # Decrypted by middleware\n    return {\"id\": 123, \"name\": data[\"name\"]}  # Encrypted by middleware\n\n@app.get(\"/users/{user_id}\")\nasync def get_user(request: Request):\n    return {\"id\": 123, \"name\": \"Alice\"}  # Encrypted by middleware\n\n@app.post(\"/chat\")\nasync def chat(request: Request):\n    data = await request.json()\n\n    async def generate():\n        yield b\"event: progress\\ndata: {\\\"step\\\": 1}\\n\\n\"\n        yield b\"event: complete\\ndata: {\\\"result\\\": \\\"done\\\"}\\n\\n\"\n\n    return StreamingResponse(generate(), media_type=\"text/event-stream\")\n```\n\n### Client (aiohttp)\n\n```python\nimport hashlib\nimport aiohttp\nfrom hpke_http.middleware.aiohttp import HPKEClientSession\n\n# Derive PSK ID from API key (see \"PSK Authentication\" section)\npsk_id = hashlib.sha256(api_key).digest()\n\nasync with HPKEClientSession(\n    base_url=\"https://api.example.com\",\n    psk=api_key,        # \u003e= 32 bytes\n    psk_id=psk_id,      # Derived from key, not tenant ID\n    # compress=True,           # Compression (zstd preferred, gzip fallback)\n    # require_encryption=True, # Raise if server responds unencrypted\n    # release_encrypted=True,  # Free encrypted bytes after decryption (saves memory)\n) as session:\n    # POST with JSON body\n    async with session.post(\"/users\", json={\"name\": \"Alice\"}) as resp:\n        user = await resp.json()\n\n    # SSE streaming\n    async with session.post(\"/chat\", json={\"prompt\": \"Hello\"}) as resp:\n        async for chunk in session.iter_sse(resp):\n            print(chunk)  # b\"event: progress\\ndata: {...}\\n\\n\"\n\n    # GET (bodyless) - response is still encrypted\n    async with session.get(\"/users/123\") as resp:\n        user = await resp.json()\n\n    # File upload - streams with O(chunk_size) memory\n    form = aiohttp.FormData()\n    form.add_field(\"file\", open(\"large.pdf\", \"rb\"), filename=\"large.pdf\")\n    async with session.post(\"/upload\", data=form) as resp:\n        result = await resp.json()\n```\n\n### Client (httpx)\n\n```python\nimport hashlib\nfrom hpke_http.middleware.httpx import HPKEAsyncClient\n\n# Derive PSK ID from API key (see \"PSK Authentication\" section)\npsk_id = hashlib.sha256(api_key).digest()\n\nasync with HPKEAsyncClient(\n    base_url=\"https://api.example.com\",\n    psk=api_key,        # \u003e= 32 bytes\n    psk_id=psk_id,      # Derived from key, not tenant ID\n    # compress=True,           # Compression (zstd preferred, gzip fallback)\n    # require_encryption=True, # Raise if server responds unencrypted\n    # release_encrypted=True,  # Free encrypted bytes after decryption (saves memory)\n) as client:\n    # POST with JSON body\n    resp = await client.post(\"/users\", json={\"name\": \"Alice\"})\n    user = resp.json()\n\n    # SSE streaming\n    resp = await client.post(\"/chat\", json={\"prompt\": \"Hello\"})\n    async for chunk in client.iter_sse(resp):\n        print(chunk)  # b\"event: progress\\ndata: {...}\\n\\n\"\n\n    # GET (bodyless) - response is still encrypted\n    resp = await client.get(\"/users/123\")\n    user = resp.json()\n\n    # File upload - streams with O(chunk_size) memory\n    resp = await client.post(\"/upload\", files={\"file\": open(\"large.pdf\", \"rb\")})\n    result = resp.json()\n```\n\n## Documentation\n\n- [RFC 9180 - HPKE](https://datatracker.ietf.org/doc/rfc9180/)\n- [RFC 7748 - X25519](https://datatracker.ietf.org/doc/rfc7748/)\n- [RFC 5869 - HKDF](https://datatracker.ietf.org/doc/rfc5869/)\n- [RFC 8439 - ChaCha20-Poly1305](https://datatracker.ietf.org/doc/rfc8439/)\n- [RFC 8878 - Zstandard](https://datatracker.ietf.org/doc/rfc8878/) (preferred compression)\n- [RFC 1952 - Gzip](https://datatracker.ietf.org/doc/rfc1952/) (fallback compression, always available)\n- [RFC 9110 - HTTP Semantics](https://datatracker.ietf.org/doc/rfc9110/) (Accept-Encoding negotiation)\n\n## Cipher Suite\n\n| Component | Algorithm | ID |\n| --------- | --------- | ------ |\n| KEM (Key Encapsulation) | DHKEM(X25519, HKDF-SHA256) | 0x0020 |\n| KDF (Key Derivation) | HKDF-SHA256 | 0x0001 |\n| AEAD (Authenticated Encryption) | ChaCha20-Poly1305 | 0x0003 |\n| Mode | PSK (Pre-Shared Key) | 0x01 |\n\n## PSK Authentication\n\nHPKE PSK mode binds each request to a pre-shared key. This requires two values:\n\n| Value | What it is | Example |\n|-------|------------|---------|\n| **PSK** | The secret key material | API key bytes, `b\"sk_live_7f3a9c...\"` |\n| **PSK ID** | Identifies *which* PSK to use | `SHA256(api_key)` — 32 bytes recommended, min 1 byte |\n\n\u003e **Data model:** One tenant typically has *many* API keys (dev/prod, per-service, per-team-member). The PSK ID identifies the specific key, not the tenant.\n\n### Security Considerations\n\n[RFC 9180 §9.4](https://www.rfc-editor.org/rfc/rfc9180.html#section-9.4) warns that `psk_id` **\"might be considered sensitive, since, in a given application context, [it] might identify the sender.\"**\n\nThe `X-HPKE-PSK-ID` header is sent in plaintext (only base64url-encoded, not encrypted). [RFC 9257](https://www.rfc-editor.org/rfc/rfc9257.html) documents the risks:\n\n| Risk | Description |\n|------|-------------|\n| **Passive linkability** | Observers correlate connections using the same PSK ID |\n| **Traffic analysis** | Identify specific API keys/users by their identifier |\n| **Active suppression** | Targeted blocking based on observed identifiers |\n\n### Mitigation: Derive PSK ID from the Key\n\n**Derive `psk_id` from the PSK itself** ([RFC 9180 §9.4](https://www.rfc-editor.org/rfc/rfc9180.html#section-9.4)):\n\n```mermaid\nsequenceDiagram\n    participant C as Client\n    participant S as Server\n\n    Note over C: psk_id = SHA256(psk)\n    C-\u003e\u003eC: Encrypt body with (psk, psk_id)\n    C-\u003e\u003eS: POST /api\u003cbr/\u003eX-HPKE-PSK-ID: \u003cderived_id\u003e\n    S-\u003e\u003eS: Lookup PSK by derived_id\n    S-\u003e\u003eS: Decrypt with (psk, psk_id)\n    S--\u003e\u003eC: Encrypted response\n```\n\n### Implementation\n\n**Client** — derive PSK ID from key:\n\n```python\nimport hashlib\n\napi_key = b\"sk_live_7f3a9c...\"  # Your API key (\u003e= 32 bytes)\n# Derive PSK ID from the key itself\npsk_id = hashlib.sha256(api_key).digest()\n\nasync with HPKEClientSession(\n    base_url=\"https://api.example.com\",\n    psk=api_key,\n    psk_id=psk_id,\n) as client:\n    await client.post(\"/api\", json=data)\n```\n\n**Server** — store derived ID when key created, lookup on request:\n\n```python\nimport hashlib\n\n# Key creation: store derived_id → {psk, tenant_id}\nderived_id = hashlib.sha256(api_key).digest()\ndb.store(derived_id, {\"psk\": api_key, \"tenant_id\": tenant_id})\n\n# psk_resolver: lookup by derived_id from header\nasync def resolve_psk(scope: dict) -\u003e tuple[bytes, bytes]:\n    derived_id = scope.get(\"hpke_psk_id\")\n    record = await db.lookup(derived_id)\n    scope[\"tenant_id\"] = record[\"tenant_id\"]\n    return (record[\"psk\"], derived_id)\n```\n\n## Wire Format\n\n### Request/Response (Chunked Binary)\n\nSee [Header Modifications](#header-modifications) for when headers are added.\n\n```text\nHeaders:\n  X-HPKE-Enc: \u003cbase64url(32B ephemeral key)\u003e\n  X-HPKE-Stream: \u003cbase64url(4B session salt)\u003e\n  X-HPKE-PSK-ID: \u003cbase64url(derived key ID, 32B recommended)\u003e\n\nBody (repeating chunks):\n┌───────────┬────────────┬─────────────────────────────────┐\n│ Length(4B)│ Counter(4B)│ Ciphertext (N + 16B tag)        │\n│ big-endian│ big-endian │ encrypted: encoding_id || data  │\n└───────────┴────────────┴─────────────────────────────────┘\nOverhead: 24B/chunk (4B length + 4B counter + 16B tag)\n```\n\n### SSE Event\n\n```text\nevent: enc\ndata: \u003cbase64(counter_be32 || ciphertext)\u003e\nDecrypted: raw SSE chunk (e.g., \"event: progress\\ndata: {...}\\n\\n\")\n```\n\nUses standard base64 (not base64url) - SSE data fields allow +/= characters.\n\n## Compression (Optional)\n\nZstd reduces bandwidth by **40-95%** for JSON/text. Enable with `compress=True` on both client and server. Payloads \u003c 64 bytes skip compression. See [Compression table](#compression) for algorithm priority.\n\n## Pitfalls\n\n```python\n# PSK too short\nHPKEClientSession(psk=b\"short\", psk_id=...)     # InvalidPSKError\nHPKEClientSession(psk=secrets.token_bytes(32), psk_id=...)  # \u003e= 32 bytes\n\n# PSK ID must be derived from the key (see \"PSK Authentication\" section)\npsk_id = hashlib.sha256(api_key).digest()\nHPKEClientSession(psk=api_key, psk_id=psk_id)   # Correct\n\n# SSE missing content-type (won't use SSE format)\nreturn StreamingResponse(gen())                                  # Binary format (wrong for SSE)\nreturn StreamingResponse(gen(), media_type=\"text/event-stream\")  # SSE format (correct)\n\n# Standard responses work automatically - no special handling needed\nreturn {\"data\": \"value\"}  # Auto-encrypted as binary chunks\n```\n\n## Limits\n\n| Resource | Limit | Applies to |\n| -------- | ----- | ---------- |\n| HPKE messages/context | 2^96-1 | All |\n| Chunks/session | 2^32-1 | All |\n| PSK minimum | 32 bytes | All |\n| PSK ID minimum | 1 byte | All |\n| Chunk size | 64KB | All |\n| Binary chunk overhead | 24B (length + counter + tag) | Requests \u0026 standard responses |\n| SSE event buffer | 64MB (configurable) | SSE only |\n\n\u003e **Note:** SSE is text-only (UTF-8). Binary data must be base64-encoded (+33% overhead).\n\n## HTTP Compatibility\n\n### Protocol Support\n\n| Feature | Supported | Notes |\n| ------- | --------- | ----- |\n| HTTP/1.1 | Yes | Chunked transfer encoding for streaming |\n| HTTP/2 | Yes | Native framing (chunked encoding forbidden by spec) |\n| HTTP/3 | Yes | QUIC streams, same semantics as HTTP/2 |\n| WebSockets | No | Different protocol, not applicable |\n\n### HTTP Methods\n\nHPKE key exchange happens on every request, including bodyless methods like GET and DELETE.\n\n| Method | Typical Use | Request Body | Response |\n| ------ | ----------- | ------------ | -------- |\n| POST | Create | Encrypted | Encrypted |\n| PUT | Replace | Encrypted | Encrypted |\n| PATCH | Update | Encrypted | Encrypted |\n| DELETE | Remove | Encrypted (if body) | Encrypted |\n| GET | Read | No body | Encrypted |\n| HEAD | Metadata | No body | Headers only (no body per HTTP spec) |\n| OPTIONS | Preflight | No body | Encrypted |\n\n### Response Encryption (Server)\n\n| Content-Type | Wire Format | Memory |\n| ------------ | ----------- | ------ |\n| Any non-SSE | Length-prefixed 64KB chunks | O(64KB) buffer |\n| `text/event-stream` | Base64 SSE events | O(event size) |\n\n### Response Decryption (Client)\n\n| Content-Type | API | Memory | Delivery |\n| ------------ | --- | ------ | -------- |\n| Any non-SSE | `resp.json()`, `resp.content` | O(response size) | After full download |\n| `text/event-stream` | `async for chunk in iter_sse(resp)` | O(event size) | As events arrive |\n\n\u003e Use `release_encrypted=True` to free encrypted buffer after decryption (reduces peak memory).\n\n### Compression\n\n| Algorithm | Request | Response | Priority |\n| --------- | ------- | -------- | -------- |\n| Zstd (RFC 8878) | Yes | Yes | 1 (preferred) |\n| Gzip (RFC 1952) | Yes | Yes | 2 (fallback) |\n| Identity | Yes | Yes | 3 (no compression) |\n\nAuto-negotiated via `Accept-Encoding` header on discovery endpoint (`/.well-known/hpke-keys`).\n\n#### Why HTTP-Level Compression Doesn't Help\n\nDisable gzip/brotli on CDN/LB for HPKE endpoints. Ciphertext is incompressible—HTTP compression wastes CPU. Use `compress=True` on the client instead (compresses before encryption).\n\n## Encryption Scope\n\n### What IS Encrypted\n\n| Component | Encrypted | Format |\n| --------- | --------- | ------ |\n| Request body | Yes | Binary chunks |\n| Response body | Yes | Binary chunks or SSE events |\n\n### What is NOT Encrypted\n\n| Component | Visible to | Reason |\n| --------- | ---------- | ------ |\n| URL path | Network | Routing requires plaintext |\n| Query parameters | Network | Part of URL |\n| HTTP method | Network | Protocol requirement |\n| HTTP headers | Network | Routing, caching, auth |\n| Status code | Network | Protocol requirement |\n| TLS metadata | Network | Transport layer |\n\n### Header Modifications\n\n| Header | Request | Response | Reason |\n| ------ | ------- | -------- | ------ |\n| `Content-Type` | Set to `application/octet-stream` (if body) | Preserved | Encrypted body is binary |\n| `Content-Length` | Auto (chunked, if body) | Removed | Size changes after encryption |\n| `X-HPKE-Enc` | Always | - | Ephemeral public key |\n| `X-HPKE-Stream` | Always | Added | Session salt for nonces |\n| `X-HPKE-PSK-ID` | Always | - | Derived PSK identifier (see [PSK Authentication](#psk-authentication)) |\n| `X-HPKE-Encoding` | Added (if compressed) | - | Compression algorithm |\n| `X-HPKE-Content-Type` | Added (if body) | - | Original Content-Type for server parsing |\n\n### Security Boundary\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│ TLS Encrypted (transport)                                   │\n│  ┌───────────────────────────────────────────────────────┐  │\n│  │ HTTP Layer (visible to CDN/LB/proxies)                │  │\n│  │  • Method: POST                                       │  │\n│  │  • URL: /api/chat                                     │  │\n│  │  • Headers: Authorization, X-HPKE-*, Content-Type     │  │\n│  │  ┌─────────────────────────────────────────────────┐  │  │\n│  │  │ HPKE Encrypted (end-to-end)                     │  │  │\n│  │  │  • Request body: {\"prompt\": \"Hello\"}            │  │  │\n│  │  │  • Response body: {\"response\": \"Hi!\"}           │  │  │\n│  │  │  • SSE events: event: done\\ndata: {...}\\n\\n     │  │  │\n│  │  └─────────────────────────────────────────────────┘  │  │\n│  └───────────────────────────────────────────────────────┘  │\n└─────────────────────────────────────────────────────────────┘\n```\n\n## Low-Level API\n\nDirect access to HPKE seal/open operations:\n\n```python\nfrom hpke_http.hpke import seal_psk, open_psk\n\n# pk_r: recipient public key, sk_r: recipient secret key\n# psk/psk_id: pre-shared key and identifier, aad: additional authenticated data\nenc, ct = seal_psk(pk_r, b\"info\", psk, psk_id, b\"aad\", b\"plaintext\")\npt = open_psk(enc, sk_r, b\"info\", psk, psk_id, b\"aad\", ct)\n```\n\n## Security\n\nUses OpenSSL constant-time implementations via `cryptography` library.\n\n- [Security Policy](./SECURITY.md) - Vulnerability reporting\n- [SBOM](https://github.com/dualeai/hpke-http/releases) - Software Bill of Materials (CycloneDX format) attached to releases\n\n## Contributing\n\nContributions welcome! Please open an issue first to discuss changes.\n\n```bash\nmake install      # Setup venv\nmake test         # Run tests\nmake lint         # Format and lint\n```\n\n## License\n\n[Apache-2.0](https://opensource.org/licenses/Apache-2.0)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdualeai%2Fhpke-http","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdualeai%2Fhpke-http","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdualeai%2Fhpke-http/lists"}