{"id":29486403,"url":"https://github.com/nazdridoy/llm-proxy","last_synced_at":"2026-05-19T03:18:00.133Z","repository":{"id":304382251,"uuid":"1018633437","full_name":"nazdridoy/llm-proxy","owner":"nazdridoy","description":"A robust, transparent REST API proxy service for LLM APIs with profile-based configuration and comprehensive logging.","archived":false,"fork":false,"pushed_at":"2025-07-12T17:35:08.000Z","size":31,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-12T19:33:07.716Z","etag":null,"topics":["ai-gateway","azure-openai","gateway","gemini-openai","llm","llm-gateway","openai","openai-proxy"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nazdridoy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-12T17:27:52.000Z","updated_at":"2025-07-12T17:33:44.000Z","dependencies_parsed_at":"2025-07-12T19:33:12.867Z","dependency_job_id":"ff15cfa6-9a01-4e9b-9192-d5c74a5f4aef","html_url":"https://github.com/nazdridoy/llm-proxy","commit_stats":null,"previous_names":["nazdridoy/llm-proxy"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/nazdridoy/llm-proxy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fllm-proxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fllm-proxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fllm-proxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fllm-proxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nazdridoy","download_url":"https://codeload.github.com/nazdridoy/llm-proxy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nazdridoy%2Fllm-proxy/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265419611,"owners_count":23761846,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-gateway","azure-openai","gateway","gemini-openai","llm","llm-gateway","openai","openai-proxy"],"created_at":"2025-07-15T08:00:40.122Z","updated_at":"2026-05-19T03:18:00.119Z","avatar_url":"https://github.com/nazdridoy.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM API Proxy\n\nA robust, transparent REST API proxy service for LLM APIs with profile-based configuration and comprehensive logging.\n\n## Features\n\n- **Profile-based Configuration**: Support multiple LLM API configurations via JSON files\n- **Version Management**: Multiple versions per profile for different API configurations\n- **Transparent Proxying**: Forward requests to various LLM APIs (OpenAI, Azure OpenAI, etc.)\n- **Comprehensive Logging**: Log all requests/responses with sanitized sensitive data\n- **Streaming Support**: Handle streaming responses for chat completions\n- **Error Handling**: Robust error handling with detailed logging\n- **Security**: CORS, Helmet, and request sanitization\n- **Health Monitoring**: Built-in health check and configuration endpoints\n\n## Project Structure\n\n```\nllm-proxy/\n├── configs/                 # Configuration files\n│   ├── openai.json          # OpenAI API configurations\n│   ├── azure.json           # Azure OpenAI configurations\n│   └── *.example.json       # Example configurations\n├── logs/                    # Log files (auto-created)\n├── src/\n│   ├── modules/\n│   │   ├── config.js        # Configuration loader\n│   │   ├── logger.js        # Logging service\n│   │   └── proxy.js         # Core proxy logic\n│   └── middleware/\n│       └── error.js         # Error handling\n├── index.js                 # Main application\n├── package.json\n├── env.example              # Environment variables template\n└── README.md\n```\n\n## Quick Start\n\n### 1. Installation\n\n```bash\n# Clone the repository\ngit clone \u003crepository-url\u003e\ncd llm-proxy\n\n# Install dependencies\nnpm install\n\n# Copy environment file\ncp env.example .env\n```\n\n### 2. Configuration\n\nCreate configuration files in the `configs/` directory:\n\n**`configs/openai.json`:**\n```json\n{\n  \"v1\": {\n    \"baseUrl\": \"https://api.openai.com/v1\",\n    \"apiKey\": \"your-openai-api-key-v1\",\n    \"description\": \"OpenAI API v1 configuration\"\n  },\n  \"v5\": {\n    \"baseUrl\": \"https://api.openai.com/v1\",\n    \"apiKey\": \"your-openai-api-key-v5\",\n    \"description\": \"OpenAI API v5 configuration\"\n  }\n}\n```\n\n**`configs/azure.json`:**\n```json\n{\n  \"v1\": {\n    \"baseUrl\": \"https://your-azure-resource.openai.azure.com/openai/deployments/your-deployment\",\n    \"apiKey\": \"your-azure-api-key\",\n    \"description\": \"Azure OpenAI v1 configuration\"\n  }\n}\n```\n\n### 3. Environment Variables\n\nEdit `.env` file:\n```bash\n# Server Configuration\nPORT=3000\nNODE_ENV=development\n\n# Logging Configuration\nLOG_LEVEL=info\nLOG_MAX_SIZE=10m\nLOG_MAX_FILES=5\n\n# Security Configuration\nCORS_ORIGIN=*\n```\n\n### 4. Start the Server\n\n```bash\n# Development mode\nnpm run dev\n\n# Production mode\nnpm start\n```\n\nThe server will start on `http://localhost:3000`\n\n## API Usage\n\n### URL Structure\n\n```\n/:\u003cprofile\u003e/:\u003cversion\u003e/*\n```\n\n- `:\u003cprofile\u003e`: Configuration profile name (e.g., `openai`, `azure`)\n- `:\u003cversion\u003e`: Version identifier (e.g., `v1`, `v5`)\n- `/*`: Path to forward to the target API\n\n### Examples\n\n#### Chat Completions\n\n```bash\ncurl http://localhost:3000/openai/v5/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"gpt-4\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n  }'\n```\n\n#### List Models\n\n```bash\ncurl http://localhost:3000/openai/v5/models \\\n  -H \"Content-Type: application/json\"\n```\n\n#### Azure OpenAI\n\n```bash\ncurl http://localhost:3000/azure/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"gpt-4\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n  }'\n```\n\n### Health Check\n\n```bash\ncurl http://localhost:3000/health\n```\n\nResponse:\n```json\n{\n  \"status\": \"healthy\",\n  \"timestamp\": \"2024-01-15T10:30:00.000Z\",\n  \"uptime\": 123.456,\n  \"version\": \"1.0.0\",\n  \"session\": {\n    \"id\": \"b8fae66f\",\n    \"startTime\": \"2024-01-15T10:30:00.000Z\",\n    \"duration\": 123456\n  }\n}\n```\n\n### Session Information\n\n```bash\ncurl http://localhost:3000/session\n```\n\nResponse:\n```json\n{\n  \"session\": {\n    \"sessionId\": \"b8fae66f\",\n    \"sessionStartTime\": \"2024-01-15T10:30:00.000Z\",\n    \"sessionDuration\": 123456,\n    \"logFilePath\": \"/path/to/logs/20250115103000_b8fae66f.log\"\n  },\n  \"timestamp\": \"2024-01-15T10:30:00.000Z\"\n}\n```\n\n### Configuration Info\n\n```bash\ncurl http://localhost:3000/config\n```\n\nResponse:\n```json\n{\n  \"profiles\": {\n    \"openai\": {\n      \"versions\": [\"v1\", \"v5\"],\n      \"description\": \"Configuration for openai profile\"\n    },\n    \"azure\": {\n      \"versions\": [\"v1\"],\n      \"description\": \"Configuration for azure profile\"\n    }\n  },\n  \"timestamp\": \"2024-01-15T10:30:00.000Z\"\n}\n```\n\n## Logging\n\nLogs are automatically created in the `logs/` directory with session-based logging. Each server session creates a single log file with the format:\n```\nYYYYMMDDHHMMSS_\u003csessionID\u003e.log\n```\n\nExample: `20250704062359_b8fae66f.log`\n\n### Session-Based Logging\n\n- **Single File Per Session**: All API interactions within a server session are logged to one file\n- **Persistent Session ID**: Each server instance gets a unique session ID that persists from start to shutdown\n- **Comprehensive Coverage**: All requests, responses, errors, and session events are logged together\n- **Easy Analysis**: Review all interactions for a specific server run in one place\n\n### Log Format\n\nEach log entry contains:\n- Timestamp\n- Session ID and session start time\n- Request details (method, URL, headers, body)\n- Target URL\n- Response details (status, headers, body)\n- Duration\n- Request ID\n- Profile and version information\n- Session events (start, shutdown, errors)\n\n### Log Sanitization\n\nSensitive data is automatically redacted:\n- Authorization headers\n- API keys\n- Cookies\n- Set-cookie headers\n\n## Error Handling\n\nThe proxy handles various error scenarios:\n\n- **404**: Profile or version not found\n- **500**: Invalid configuration\n- **502**: Upstream service unavailable\n- **504**: Upstream service timeout\n- **400**: Invalid request data\n\nAll errors include:\n- Error message\n- Request ID for tracking\n- Timestamp\n- Stack trace (in development mode)\n\n## Security Features\n\n- **CORS Protection**: Configurable CORS policies\n- **Helmet**: Security headers\n- **Request Sanitization**: Automatic removal of sensitive headers\n- **Rate Limiting**: Built-in rate limiting support\n- **Request ID Tracking**: Unique request IDs for all requests\n\n## Configuration Management\n\n### Adding New Profiles\n\n1. Create a new configuration file: `configs/\u003cprofile\u003e.json`\n2. Define versions with baseUrl and apiKey\n3. Restart the server\n\n### Updating Configurations\n\n1. Edit the configuration file\n2. Restart the server to reload configurations\n\n### Configuration Validation\n\nThe proxy validates all configurations on startup:\n- Required fields: `baseUrl`, `apiKey`\n- URL format validation\n- API key format validation\n\n## Development\n\n### Running in Development Mode\n\n```bash\nnpm run dev\n```\n\nThis uses nodemon for automatic restarts on file changes.\n\n### Environment Variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `PORT` | 3000 | Server port |\n| `NODE_ENV` | development | Environment mode |\n| `LOG_LEVEL` | info | Logging level |\n| `LOG_MAX_SIZE` | 50m | Max log file size |\n| `LOG_MAX_FILES` | 10 | Max log files to keep |\n| `CORS_ORIGIN` | * | CORS origin |\n| `BODY_LIMIT` | 50mb | Request body size limit |\n\n### Testing\n\n```bash\n# Test health endpoint\ncurl http://localhost:3000/health\n\n# Test configuration endpoint\ncurl http://localhost:3000/config\n\n# Test session information\ncurl http://localhost:3000/session\n\n# Test proxy with OpenAI\ncurl http://localhost:3000/openai/v5/models\n\n# Test session logging\nnpm run test:logging\n```\n\n## Production Deployment\n\n### Docker (Recommended)\n\n```dockerfile\nFROM node:18-alpine\n\nWORKDIR /app\n\nCOPY package*.json ./\nRUN npm ci --only=production\n\nCOPY . .\n\nEXPOSE 3000\n\nCMD [\"npm\", \"start\"]\n```\n\n### Environment Setup\n\n1. Set `NODE_ENV=production`\n2. Configure proper CORS origins\n3. Set up log rotation\n4. Use environment variables for sensitive data\n5. Set up monitoring and alerting\n\n### Monitoring\n\n- Health check endpoint: `/health`\n- Configuration endpoint: `/config`\n- Log files in `logs/` directory\n- Request ID tracking for debugging\n\n## Troubleshooting\n\n### Common Issues\n\n1. **Configuration not found**: Check config file format and location\n2. **API key errors**: Verify API keys in configuration files\n3. **CORS errors**: Check CORS_ORIGIN environment variable\n4. **Timeout errors**: Increase timeout in proxy configuration\n\n### Debug Mode\n\nSet `LOG_LEVEL=debug` for detailed logging.\n\n### Log Analysis\n\nLogs are in JSON format for easy parsing:\n```bash\n# View current session logs\ntail -f logs/$(ls -t logs/*.log | head -1)\n\n# Search for errors in current session\ngrep \"error\" logs/$(ls -t logs/*.log | head -1)\n\n# List all session logs\nls -la logs/*.log\n\n# View specific session log\ncat logs/20250704062359_b8fae66f.log\n```\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests if applicable\n5. Submit a pull request\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnazdridoy%2Fllm-proxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnazdridoy%2Fllm-proxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnazdridoy%2Fllm-proxy/lists"}