{"id":25426780,"url":"https://github.com/wesh92/fastapi-websockets-llm-example","last_synced_at":"2026-04-28T00:32:05.190Z","repository":{"id":275997934,"uuid":"927113306","full_name":"wesh92/fastapi-websockets-llm-example","owner":"wesh92","description":"Example of Websockets Implementation in FastAPI/Python","archived":false,"fork":false,"pushed_at":"2025-02-06T18:59:32.000Z","size":199,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-20T07:49:17.383Z","etag":null,"topics":["ai","fastapi","fastapi-template","llm","openrouter","python","websocket"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wesh92.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-04T12:33:04.000Z","updated_at":"2025-06-08T15:47:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"986a4259-c012-4408-a7b1-d1203c6a6d4a","html_url":"https://github.com/wesh92/fastapi-websockets-llm-example","commit_stats":null,"previous_names":["wesh92/fastapi-websockets-llm-example"],"tags_count":0,"template":true,"template_full_name":null,"purl":"pkg:github/wesh92/fastapi-websockets-llm-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wesh92%2Ffastapi-websockets-llm-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wesh92%2Ffastapi-websockets-llm-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wesh92%2Ffastapi-websockets-llm-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wesh92%2Ffastapi-websockets-llm-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wesh92","download_url":"https://codeload.github.com/wesh92/fastapi-websockets-llm-example/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wesh92%2Ffastapi-websockets-llm-example/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32361477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-27T20:07:02.737Z","status":"ssl_error","status_checked_at":"2026-04-27T20:07:00.910Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","fastapi","fastapi-template","llm","openrouter","python","websocket"],"created_at":"2025-02-17T00:22:01.539Z","updated_at":"2026-04-28T00:32:05.177Z","avatar_url":"https://github.com/wesh92.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FastAPI WebSocket OpenRouter Template\n\nA robust FastAPI-based WebSocket server template that integrates with OpenRouter's API for real-time AI model interactions. This project provides a production-ready foundation for building WebSocket-based applications that require AI model integration.\n\n## Features\n\nThis template includes:\n\n- Real-time WebSocket communication with AI models through OpenRouter\n- Robust error handling and logging\n- Docker containerization\n- Health check endpoints\n- Structured API documentation\n- Authentication support\n- Configurable routing system\n- State management for WebSocket connections\n\n## Prerequisites\n\n- Python 3.13 or higher\n- Docker and Docker Compose (for containerized deployment)\n- OpenRouter API key (stored in secrets.toml)\n\n## Dependencies\n\nCore dependencies include:\n\n- FastAPI (\u003e= 0.115.6) - Web framework for building APIs\n- aiohttp (\u003e= 3.11.11) - Async HTTP client/server framework\n- uvicorn (\u003e= 0.34.0) - ASGI server implementation\n- pydantic (\u003e= 2.10.4) - Data validation using Python type annotations\n- gunicorn (\u003e= 23.0.0) - WSGI HTTP server\n- python-multipart (\u003e= 0.0.20) - Streaming multipart parser\n- JWT authentication libraries:\n  - PyJWT[crypto] (\u003e= 2.10.1)\n  - passlib (\u003e= 1.7.4)\n  - bcrypt (\u003e= 4.2.1)\n\n## Project Structure\n\n```\n├── routes/\n│   ├── openrouter_websocket/\n│   │   ├── openrouter_models.py      # Pydantic models for OpenRouter\n│   │   ├── openrouter_service.py     # OpenRouter API communication service\n│   │   └── openrouter_websocket_controller.py  # WebSocket endpoint controller\n├── internal/\n│   ├── auth/                         # Authentication components\n│   ├── health/                       # Health check endpoints\n│   └── dependencies/                 # Shared dependencies\n├── documentation/                    # API documentation\n├── docker-compose.yml               # Docker Compose configuration\n├── main.py                         # Application entry point\n└── pyproject.toml                  # Project metadata and dependencies\n```\n\n## Configuration\n\n### Environment Setup\n\n1. Create a `secrets.toml` file in the project root:\n```toml\nSECRET_KEY = \"your-openssl-rand-hex-32-key\"\nOPENROUTER_SECRET = \"your-openrouter-api-key\"\n```\n\n### Docker Configuration\n\nThe included `docker-compose.yml` provides:\n- Port mapping (8000:8000)\n- Volume mounting for development\n- Health check configuration\n- Automatic container restart\n- Named network for service communication\n\n## Usage\n\n### Starting the Server\n\nUsing Docker:\n```bash\ndocker-compose up -d\n```\n\nUsing Python directly:\n```bash\nuvicorn main:app --host 0.0.0.0 --port 8000\n```\n\n### WebSocket Connection\n\nConnect to the WebSocket endpoint at:\n```\nws://localhost:8000/websockets/openrouter\n```\n\nYou can test with a platform like `WebSocketKing`.\n\n### Message Format\n\nMessages to the WebSocket should follow this format:\n```json\n{\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Write a simple python script to add 3 numbers.\"\n        }\n    ],\n    \"model\": \"google/gemini-flash-1.5\",\n    \"temperature\": 1,\n    \"top_p\": 1\n}\n```\n\n## API Documentation\n\nAccess the API documentation at:\n```\nhttp://localhost:8000/docs\n```\nNOTE: There is currently no documentation there for the WS although the category is there. TBD.\n\n## Error Handling\n\nThe system includes comprehensive error handling:\n- WebSocket connection management\n- API communication errors\n- Message parsing errors\n- Service initialization failures\n\nAll errors are logged with appropriate detail levels and returned to the client with clear error messages.\n\n## Health Monitoring\n\nHealth checks are available at:\n```\nhttp://localhost:8000/health\n```\n\nThe Docker container includes automated health checking every 90 seconds.\n\n## Development\n\n### Adding New Routes\n\n1. Create a new module in the `routes` directory\n2. Define your router using FastAPI's `APIRouter`\n3. Add the router configuration in `main.py`\n4. Update the OpenAPI documentation as needed\n\n### Code Style\n\nThe project uses ruff for code formatting and linting. Configuration is provided in `pyproject.toml`.\nThis is setup using `uv` and I recommend using that.\n\n## Security Considerations\n\n- API keys are stored in `secrets.toml` (not version controlled). You should probably use GH Secrets or some other provider.\n- WebSocket connections include state tracking\n- Authentication middleware available for protected routes\n- Headers are sanitized in logs","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwesh92%2Ffastapi-websockets-llm-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwesh92%2Ffastapi-websockets-llm-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwesh92%2Ffastapi-websockets-llm-example/lists"}