{"id":28510816,"url":"https://github.com/teabranch/open-bedrock-server","last_synced_at":"2026-03-16T13:01:18.884Z","repository":{"id":297403253,"uuid":"996568833","full_name":"teabranch/open-bedrock-server","owner":"teabranch","description":"A unified, provider-agnostic chat completions API server that seamlessly integrates OpenAI and AWS Bedrock through a single endpoint with intelligent format detection and conversion.","archived":false,"fork":false,"pushed_at":"2025-06-23T11:07:15.000Z","size":614,"stargazers_count":12,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-23T19:15:05.468Z","etag":null,"topics":["ai","aws","bedrock","converse-api","openai","python","responses-api"],"latest_commit_sha":null,"homepage":"https://teabranch.github.io/open-bedrock-server/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/teabranch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-05T06:19:12.000Z","updated_at":"2025-07-28T00:14:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"6a00afb5-660c-49c2-a10e-987dba0079d7","html_url":"https://github.com/teabranch/open-bedrock-server","commit_stats":null,"previous_names":["teabranch/open-amazon-chat-completions-server","teabranch/open-bedrock-server"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/teabranch/open-bedrock-server","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teabranch%2Fopen-bedrock-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teabranch%2Fopen-bedrock-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teabranch%2Fopen-bedrock-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teabranch%2Fopen-bedrock-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/teabranch","download_url":"https://codeload.github.com/teabranch/open-bedrock-server/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/teabranch%2Fopen-bedrock-server/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30508039,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-14T14:51:43.155Z","status":"ssl_error","status_checked_at":"2026-03-14T14:47:41.964Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","aws","bedrock","converse-api","openai","python","responses-api"],"created_at":"2025-06-08T23:06:15.996Z","updated_at":"2026-03-16T13:01:18.876Z","avatar_url":"https://github.com/teabranch.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Open Bedrock Server\n\nA unified, provider-agnostic chat completions API server that seamlessly integrates OpenAI and AWS Bedrock through a single endpoint with intelligent format detection and conversion.\n\n## 🌟 Why We Created This Project\n\nWe created Open Bedrock Server to simplify the transition between different Large Language Models (LLMs). Our goal was to provide a unified endpoint that reduces the complexity of managing multiple configurations and API nuances. We wanted something simple that would work seamlessly, allowing developers to switch between LLM providers with minimal effort.\n\n## 🚀 Key Features\n\n- **🔄 Unified Endpoint**: Single `/v1/chat/completions` endpoint handles all providers and formats\n- **🧠 Smart Format Detection**: Automatically detects OpenAI, Bedrock Claude, and Bedrock Titan formats\n- **🔀 Seamless Conversion**: Convert between any format combination (OpenAI ↔ Bedrock Claude ↔ Bedrock Titan)\n- **⚡ Model-Based Routing**: Automatic provider selection based on model ID patterns\n- **📁 File Query System**: Upload, process, and query files in chat completions with automatic content extraction\n- **🌊 Streaming Support**: Real-time streaming with format preservation\n- **🛡️ Enterprise Ready**: Authentication, logging, error handling, and monitoring\n- **💻 Full CLI**: Interactive chat, server management, and configuration tools\n- **📚 OpenAI Compatible**: Drop-in replacement for OpenAI Chat Completions API\n\n## 📋 Table of Contents\n\n- [Quick Start](#quick-start)\n- [Unified Endpoint](#unified-endpoint)\n- [File Query System](#file-query-system)\n- [Format Support](#format-support)\n- [Installation](#installation)\n- [Configuration](#configuration)\n- [Usage Examples](#usage-examples)\n- [CLI Reference](#cli-reference)\n- [API Documentation](#api-documentation)\n- [Architecture](#architecture)\n- [Contributing](#contributing)\n\n## 🚀 Quick Start\n\n### 1. Installation\n\n```bash\n# Clone the repository\ngit clone https://github.com/teabranch/open-bedrock-server.git\ncd open-bedrock-server\n\n# Install with uv (recommended)\nuv pip install -e .\n\n# Or with pip\npip install -e .\n```\n\n### 2. Configuration\n\n```bash\n# Interactive configuration setup\nbedrock-chat config set\n\n# Or manually create .env file\n# Create .env file with your configuration\n# Edit .env with your API keys\n```\n\n### 3. Start Server\n\n```bash\n# Start the server\nbedrock-chat serve --host 0.0.0.0 --port 8000\n\n# Server will be available at:\n# 🌐 API: http://localhost:8000\n# 📚 Docs: http://localhost:8000/docs\n```\n\n### 4. Test the Unified Endpoint\n\n```bash\n# OpenAI format → OpenAI response\ncurl -X POST \"http://localhost:8000/v1/chat/completions\" \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -d '{\n    \"model\": \"gpt-4o-mini\",\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Hello!\"}\n    ]\n  }'\n\n# OpenAI format → Bedrock Claude response\ncurl -X POST \"http://localhost:8000/v1/chat/completions\" \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -d '{\n    \"model\": \"anthropic.claude-3-haiku-20240307-v1:0\",\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Hello!\"}\n    ]\n  }'\n\n# Bedrock Claude format → OpenAI response\ncurl -X POST \"http://localhost:8000/v1/chat/completions\" \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -d '{\n    \"anthropic_version\": \"bedrock-2023-05-31\",\n    \"model\": \"anthropic.claude-3-haiku-20240307-v1:0\",\n    \"max_tokens\": 1000,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Hello!\"}\n    ]\n  }'\n```\n\n## 🔄 Unified Endpoint\n\n### Single Endpoint for Everything\n\nThe `/v1/chat/completions` endpoint is the **only endpoint you need**. It:\n\n1. **Auto-detects** your input format (OpenAI, Bedrock Claude, Bedrock Titan)\n2. **Routes** to the appropriate provider based on model ID\n3. **Converts** between formats as needed\n4. **Streams** responses in real-time when requested\n5. **Returns** responses in your preferred format\n\n### How It Works\n\n```mermaid\ngraph LR\n    A[Any Format Request] --\u003e B[Format Detection]\n    B --\u003e C[Model-Based Routing]\n    C --\u003e D[Provider API Call]\n    D --\u003e E[Format Conversion]\n    E --\u003e F[Unified Response]\n    \n    style A fill:#e1f5fe\n    style F fill:#e8f5e8\n```\n\n### Model-Based Routing\n\nThe server automatically routes requests based on model ID patterns:\n\n| Model Pattern | Provider | Examples |\n|---------------|----------|----------|\n| `gpt-*` | OpenAI | `gpt-4o-mini`, `gpt-3.5-turbo` |\n| `text-*` | OpenAI | `text-davinci-003` |\n| `dall-e-*` | OpenAI | `dall-e-3` |\n| `anthropic.*` | Bedrock | `anthropic.claude-3-haiku-20240307-v1:0` |\n| `amazon.*` | Bedrock | `amazon.titan-text-express-v1` |\n| `ai21.*`, `cohere.*`, `meta.*` | Bedrock | Various Bedrock models |\n\n## 📁 File Query System\n\n### Overview\n\nUpload, process, and query files directly in your chat completions. The system automatically extracts and formats content from various file types, making file data available as context for your AI conversations.\n\n### Quick Example\n\n```bash\n# 1. Upload a file\ncurl -X POST \"http://localhost:8000/v1/files\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -F \"file=@data.csv\" \\\n  -F \"purpose=assistants\"\n\n# Response: {\"id\": \"file-abc123\", \"filename\": \"data.csv\", ...}\n\n# 2. Query the file in chat\ncurl -X POST \"http://localhost:8000/v1/chat/completions\" \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -d '{\n    \"model\": \"gpt-4o-mini\",\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Analyze the data in this file\"}\n    ],\n    \"file_ids\": [\"file-abc123\"]\n  }'\n```\n\n### Supported File Types\n\n| Format | MIME Type | Auto-Processing |\n|--------|-----------|-----------------|\n| Text | `text/plain` | ✅ Full content |\n| CSV | `text/csv` | ✅ Structured data with headers |\n| JSON | `application/json` | ✅ Formatted structure |\n| HTML | `text/html` | ✅ Text extraction |\n| XML | `application/xml` | ✅ Formatted structure |\n| Markdown | `text/markdown` | ✅ Full content |\n\n### File Operations\n\n#### Upload Files\n\n```bash\nPOST /v1/files\n```\n\nUpload files to S3 storage with metadata extraction.\n\n#### List Files\n\n```bash\nGET /v1/files\nGET /v1/files?purpose=assistants\n```\n\nList uploaded files with optional filtering.\n\n#### Get File Metadata\n\n```bash\nGET /v1/files/{file_id}\n```\n\nRetrieve file information and metadata.\n\n#### Download File Content\n\n```bash\nGET /v1/files/{file_id}/content\n```\n\nDownload original file content.\n\n#### Delete Files\n\n```bash\nDELETE /v1/files/{file_id}\n```\n\nRemove files from storage.\n\n### Chat Integration\n\nAdd `file_ids` to any chat completion request to include file content as context:\n\n```json\n{\n  \"model\": \"gpt-4o-mini\",\n  \"messages\": [\n    {\"role\": \"user\", \"content\": \"What patterns do you see in this data?\"}\n  ],\n  \"file_ids\": [\"file-abc123\", \"file-def456\"]\n}\n```\n\nThe system automatically:\n\n1. **Retrieves** file content from S3\n2. **Processes** content based on file type\n3. **Formats** for optimal AI consumption\n4. **Prepends** to your user message as context\n\n### Configuration\n\nAdd S3 configuration to your `.env` file:\n\n```env\n# Required for file operations\nS3_FILES_BUCKET=your-s3-bucket-name\nAWS_REGION=us-east-1\n\n# AWS authentication (choose one method)\nAWS_ACCESS_KEY_ID=your-access-key\nAWS_SECRET_ACCESS_KEY=your-secret-key\n# OR\nAWS_PROFILE=your-aws-profile\n```\n\n### File Processing Examples\n\n**CSV Data Processing:**\n\n```ini\n=== UPLOADED FILES CONTEXT ===\n\n📄 **File: sales_data.csv** (text/csv, 1.2KB)\nCreated: 2024-12-09T14:23:01Z\n\n**Processed Content:**\nDate,Product,Sales,Revenue\n2024-01-01,Widget A,150,$1500.00\n2024-01-02,Widget B,200,$2000.00\n...\n```\n\n**JSON Configuration:**\n\n```js\n=== UPLOADED FILES CONTEXT ===\n\n📄 **File: config.json** (application/json, 500B)\nCreated: 2024-12-09T14:23:01Z\n\n**Processed Content:**\n{\n  \"database\": {\n    \"host\": \"localhost\",\n    \"port\": 5432\n  },\n  \"features\": [\"auth\", \"logging\"]\n}\n```\n\n### Health Check\n\nMonitor file service status:\n\n```bash\ncurl \"http://localhost:8000/v1/files/health\"\n```\n\nResponse includes S3 connectivity and credential validation.\n\n## 🔀 Format Support\n\n### All Format Combinations Supported\n\n| Input Format | Output Format | Use Case | Streaming |\n|-------------|---------------|----------|-----------|\n| OpenAI | OpenAI | Standard OpenAI usage | ✅ |\n| OpenAI | Bedrock Claude | OpenAI clients → Claude responses | ✅ |\n| OpenAI | Bedrock Titan | OpenAI clients → Titan responses | ✅ |\n| Bedrock Claude | OpenAI | Claude clients → OpenAI responses | ✅ |\n| Bedrock Claude | Bedrock Claude | Native Claude usage | ✅ |\n| Bedrock Titan | OpenAI | Titan clients → OpenAI responses | ✅ |\n| Bedrock Titan | Bedrock Titan | Native Titan usage | ✅ |\n\n### Format Examples\n\n**OpenAI Format:**\n\n```json\n{\n  \"model\": \"gpt-4o-mini\",\n  \"messages\": [\n    {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  \"temperature\": 0.7,\n  \"max_tokens\": 1000,\n  \"stream\": false\n}\n```\n\n**Bedrock Claude Format:**\n\n```json\n{\n  \"anthropic_version\": \"bedrock-2023-05-31\",\n  \"model\": \"anthropic.claude-3-haiku-20240307-v1:0\",\n  \"max_tokens\": 1000,\n  \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}],\n  \"temperature\": 0.7,\n  \"stream\": false\n}\n```\n\n**Bedrock Titan Format:**\n\n```json\n{\n  \"model\": \"amazon.titan-text-express-v1\",\n  \"inputText\": \"User: Hello!\\n\\nBot:\",\n  \"textGenerationConfig\": {\n    \"maxTokenCount\": 1000,\n    \"temperature\": 0.7\n  }\n}\n```\n\n## 📦 Installation\n\n### Prerequisites\n\n- Python 3.12+\n- OpenAI API key (for OpenAI models)\n- AWS credentials (for Bedrock models)\n\n### Install Options\n\n```bash\n# Option 1: Development installation\ngit clone https://github.com/teabranch/open-bedrock-server.git\ncd open-bedrock-server\nuv pip install -e .\n\n# Option 2: Direct from GitHub\npip install git+https://github.com/teabranch/open-bedrock-server.git\n\n# Option 3: Local wheel\npip install dist/open_bedrock_server-*.whl\n```\n\n### Verify Installation\n\n```bash\n# Check CLI is available\nbedrock-chat --help\n\n# Check version\nbedrock-chat config show\n```\n\n## ⚙️ Configuration\n\n### Environment Variables\n\nCreate a `.env` file with your configuration:\n\n```env\n# Required: Server Authentication\nAPI_KEY=your-secret-api-key\n\n# Required: OpenAI (if using OpenAI models)\nOPENAI_API_KEY=sk-your-openai-api-key\n\n# Required: S3 File Storage (if using file query features)\nS3_FILES_BUCKET=your-s3-bucket-name\n\n# Required: AWS (if using Bedrock models)\n# Option 1: Static credentials\nAWS_ACCESS_KEY_ID=your-aws-access-key\nAWS_SECRET_ACCESS_KEY=your-aws-secret-key\nAWS_REGION=us-east-1\n\n# Option 2: AWS Profile (alternative to static credentials)\nAWS_PROFILE=your-aws-profile\nAWS_REGION=us-east-1\n\n# Option 3: AWS Role Assumption (for cross-account access or enhanced security)\nAWS_ROLE_ARN=arn:aws:iam::123456789012:role/MyBedrockRole\nAWS_EXTERNAL_ID=your-external-id  # Optional, for cross-account role assumption\nAWS_ROLE_SESSION_NAME=bedrock-server-session  # Optional\nAWS_ROLE_SESSION_DURATION=3600  # Optional, session duration in seconds\nAWS_REGION=us-east-1\n\n# Option 4: Web Identity Token (for OIDC/Kubernetes service accounts)\nAWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token\nAWS_ROLE_ARN=arn:aws:iam::123456789012:role/MyWebIdentityRole\nAWS_REGION=us-east-1\n\n# Optional: Defaults\nDEFAULT_OPENAI_MODEL=gpt-4o-mini\nLOG_LEVEL=INFO\nCHAT_SERVER_URL=http://localhost:8000\nCHAT_API_KEY=your-secret-api-key\n```\n\n### AWS Authentication Methods\n\nThe server supports multiple AWS authentication methods with automatic detection and fallback. Choose the method that best fits your deployment scenario:\n\n#### 1. Static Credentials (Development/Testing)\n\nDirect AWS access keys - simplest but least secure:\n\n```env\nAWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\nAWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\nAWS_SESSION_TOKEN=your-session-token  # Optional, for temporary credentials\nAWS_REGION=us-east-1\n```\n\n**Use cases:** Local development, testing, CI/CD pipelines\n**Security:** ⚠️ Store securely, rotate regularly\n\n#### 2. AWS Profile (Local Development)\n\nUses AWS CLI configured profiles:\n\n```env\nAWS_PROFILE=your-aws-profile\nAWS_REGION=us-east-1\n```\n\n**Setup:**\n\n```bash\n# Configure AWS CLI profile\naws configure --profile your-aws-profile\n# Or use AWS SSO\naws configure sso --profile your-aws-profile\n```\n\n**Use cases:** Local development with multiple AWS accounts\n**Security:** ✅ Credentials managed by AWS CLI\n\n#### 3. Role Assumption (Cross-Account/Enhanced Security)\n\nAssume an IAM role using base credentials:\n\n```env\n# Base credentials (required for role assumption)\nAWS_PROFILE=base-profile\n# OR\nAWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE\nAWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\n\n# Role to assume\nAWS_ROLE_ARN=arn:aws:iam::123456789012:role/BedrockAccessRole\nAWS_EXTERNAL_ID=unique-external-id  # Optional, for cross-account scenarios\nAWS_ROLE_SESSION_NAME=bedrock-server-session  # Optional\nAWS_ROLE_SESSION_DURATION=3600  # Optional, 900-43200 seconds (default: 3600)\nAWS_REGION=us-east-1\n```\n\n**Role Trust Policy Example:**\n\n```json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Principal\": {\n        \"AWS\": \"arn:aws:iam::SOURCE-ACCOUNT:user/username\"\n      },\n      \"Action\": \"sts:AssumeRole\",\n      \"Condition\": {\n        \"StringEquals\": {\n          \"sts:ExternalId\": \"unique-external-id\"\n        }\n      }\n    }\n  ]\n}\n```\n\n**Use cases:** Cross-account access, temporary elevated permissions, security compliance\n**Security:** ✅ Time-limited sessions, audit trail, least privilege\n\n#### 4. Web Identity Token (OIDC/Kubernetes)\n\nFor containerized environments with OIDC providers:\n\n```env\nAWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token\nAWS_ROLE_ARN=arn:aws:iam::123456789012:role/EKSBedrockRole\nAWS_ROLE_SESSION_NAME=bedrock-server-session  # Optional\nAWS_REGION=us-east-1\n```\n\n**Kubernetes ServiceAccount Example:**\n\n```yaml\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n  name: bedrock-server\n  annotations:\n    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/EKSBedrockRole\n```\n\n**Use cases:** EKS, GitHub Actions, GitLab CI, other OIDC providers\n**Security:** ✅ No long-term credentials, automatic rotation\n\n#### 5. Instance Profile/Default Chain (Production)\n\nNo configuration needed - uses AWS default credential chain:\n\n```env\n# Only region required\nAWS_REGION=us-east-1\n```\n\n**Credential Chain Order:**\n\n1. Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)\n2. Shared credentials file (`~/.aws/credentials`)\n3. Shared config file (`~/.aws/config`)\n4. IAM instance profiles (EC2)\n5. ECS task roles\n6. EKS service account roles\n\n**Use cases:** EC2 instances, ECS tasks, Lambda functions\n**Security:** ✅ No credential management, automatic rotation\n\n### Authentication Priority and Fallback\n\nThe server uses this priority order for authentication:\n\n1. __Static Credentials__ (`AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY`)\n2. __AWS Profile__ (`AWS_PROFILE`)\n3. __Role Assumption__ (`AWS_ROLE_ARN` with base credentials)\n4. __Web Identity Token__ (`AWS_WEB_IDENTITY_TOKEN_FILE`)\n5. **Default Credential Chain** (instance profiles, etc.)\n\n### AWS Authentication Troubleshooting\n\n#### Common Issues and Solutions\n\n**❌ \"Role assumption requires base AWS credentials\"**\n\n```bash\n# Problem: AWS_ROLE_ARN set but no base credentials\n# Solution: Add base credentials\nAWS_PROFILE=your-profile  # OR\nAWS_ACCESS_KEY_ID=your-key\nAWS_SECRET_ACCESS_KEY=your-secret\n```\n\n**❌ \"The config profile (profile-name) could not be found\"**\n\n```bash\n# Problem: Profile doesn't exist\n# Solution: Configure the profile\naws configure --profile profile-name\n# Or check existing profiles\naws configure list-profiles\n```\n\n**❌ \"Access denied when assuming role\"**\n\n```bash\n# Problem: Role trust policy or permissions issue\n# Solution: Check role trust policy allows your principal\n# Verify role has bedrock:* permissions\n```\n\n**❌ \"Web identity token file not found\"**\n\n```bash\n# Problem: Token file path incorrect\n# Solution: Verify the token file exists\nls -la /var/run/secrets/eks.amazonaws.com/serviceaccount/token\n```\n\n**❌ \"You must specify a region\"**\n\n```bash\n# Problem: AWS_REGION not set\n# Solution: Set the region\nAWS_REGION=us-east-1\n```\n\n#### Testing Authentication\n\n```bash\n# Test your AWS configuration\nbedrock-chat config test-aws\n\n# Or manually test with AWS CLI\naws sts get-caller-identity --profile your-profile\naws bedrock list-foundation-models --region us-east-1\n```\n\n### Required AWS Permissions\n\nYour AWS credentials need these minimum permissions for Bedrock:\n\n```json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"bedrock:InvokeModel\",\n        \"bedrock:InvokeModelWithResponseStream\",\n        \"bedrock:ListFoundationModels\"\n      ],\n      \"Resource\": \"*\"\n    }\n  ]\n}\n```\n\nFor role assumption, also add:\n\n```json\n{\n  \"Effect\": \"Allow\",\n  \"Action\": \"sts:AssumeRole\",\n  \"Resource\": \"arn:aws:iam::*:role/YourBedrockRole\"\n}\n```\n\n### Interactive Configuration\n\n```bash\n# Run interactive setup\nbedrock-chat config set\n\n# View current configuration (sensitive values masked)\nbedrock-chat config show\n\n# Set specific values\nbedrock-chat config set --key OPENAI_API_KEY --value sk-your-key\n\n# Test AWS authentication\nbedrock-chat config test-aws\n```\n\n\u003e 📖 __For detailed AWS authentication documentation, see [AWS Authentication Guide](docs/AWS_AUTHENTICATION.md)__\n\n## 💡 Usage Examples\n\n### Basic Chat Completion\n\n```python\nimport httpx\n\n# OpenAI format request\nresponse = httpx.post(\n    \"http://localhost:8000/v1/chat/completions\",\n    headers={\n        \"Content-Type\": \"application/json\",\n        \"Authorization\": \"Bearer your-api-key\"\n    },\n    json={\n        \"model\": \"gpt-4o-mini\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n    }\n)\n\nprint(response.json())\n```\n\n### Format Conversion\n\n```python\n# Send OpenAI format, get Bedrock Claude response\nresponse = httpx.post(\n    \"http://localhost:8000/v1/chat/completions?target_format=bedrock_claude\",\n    headers={\n        \"Content-Type\": \"application/json\",\n        \"Authorization\": \"Bearer your-api-key\"\n    },\n    json={\n        \"model\": \"gpt-4o-mini\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n    }\n)\n\n# Response will be in Bedrock Claude format\nclaude_response = response.json()\nprint(claude_response[\"content\"][0][\"text\"])\n```\n\n### Streaming\n\n```python\nimport httpx\n\n# Streaming request\nwith httpx.stream(\n    \"POST\",\n    \"http://localhost:8000/v1/chat/completions\",\n    headers={\n        \"Content-Type\": \"application/json\",\n        \"Authorization\": \"Bearer your-api-key\"\n    },\n    json={\n        \"model\": \"gpt-4o-mini\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story\"}],\n        \"stream\": True\n    }\n) as response:\n    for line in response.iter_lines():\n        if line.startswith(\"data: \"):\n            data = line[6:]  # Remove \"data: \" prefix\n            if data != \"[DONE]\":\n                chunk = json.loads(data)\n                if chunk[\"choices\"][0][\"delta\"].get(\"content\"):\n                    print(chunk[\"choices\"][0][\"delta\"][\"content\"], end=\"\")\n```\n\n### Tool Calling\n\n```python\n# Tool calling with OpenAI format\nresponse = httpx.post(\n    \"http://localhost:8000/v1/chat/completions\",\n    headers={\n        \"Content-Type\": \"application/json\",\n        \"Authorization\": \"Bearer your-api-key\"\n    },\n    json={\n        \"model\": \"gpt-4o-mini\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"What's the weather in London?\"}],\n        \"tools\": [\n            {\n                \"type\": \"function\",\n                \"function\": {\n                    \"name\": \"get_weather\",\n                    \"description\": \"Get current weather\",\n                    \"parameters\": {\n                        \"type\": \"object\",\n                        \"properties\": {\n                            \"location\": {\"type\": \"string\"}\n                        },\n                        \"required\": [\"location\"]\n                    }\n                }\n            }\n        ],\n        \"tool_choice\": \"auto\"\n    }\n)\n\n# Check for tool calls in response\nif response.json()[\"choices\"][0][\"message\"].get(\"tool_calls\"):\n    print(\"Model wants to call a tool!\")\n```\n\n## 💻 CLI Reference\n\n### Core Commands\n\n```bash\n# Interactive chat session\nbedrock-chat chat --model gpt-4o-mini\n\n# Start server\nbedrock-chat serve --host 0.0.0.0 --port 8000\n\n# Configuration management\nbedrock-chat config set\nbedrock-chat config show\n\n# List available models\nbedrock-chat models\n\n# Get help\nbedrock-chat --help\nbedrock-chat COMMAND --help\n```\n\n### Chat Session\n\n```bash\n# Start interactive chat\nbedrock-chat chat --model gpt-4o-mini --stream\n\n# Chat with custom settings\nbedrock-chat chat \\\n  --model anthropic.claude-3-haiku-20240307-v1:0 \\\n  --temperature 0.8 \\\n  --max-tokens 500\n\n# Chat with custom server\nbedrock-chat chat \\\n  --server-url https://my-server.com \\\n  --api-key my-key\n```\n\n### Server Management\n\n```bash\n# Development server with auto-reload\nbedrock-chat serve --reload --log-level debug\n\n# Production server\nbedrock-chat serve \\\n  --host 0.0.0.0 \\\n  --port 8000 \\\n  --workers 4 \\\n  --env-file production.env\n```\n\n## 📚 API Documentation\n\n### Interactive Documentation\n\nWhen the server is running, visit:\n\n- **Swagger UI**: http://localhost:8000/docs\n- **ReDoc**: http://localhost:8000/redoc\n- **OpenAPI Schema**: http://localhost:8000/openapi.json\n\n### Available Endpoints\n\n| Endpoint | Method | Description |\n|----------|--------|-------------|\n| `/v1/chat/completions` | POST | **Unified chat completions endpoint** |\n| `/v1/models` | GET | List available models |\n| `/health` | GET | General health check |\n| `/v1/chat/completions/health` | GET | Unified endpoint health check |\n\n### Authentication\n\nAll endpoints require authentication via the `Authorization` Bearer token:\n\n```bash\ncurl -H \"Authorization: Bearer your-api-key\" http://localhost:8000/v1/models\n```\n\n## 🏗️ Architecture\n\n### Unified Architecture Overview\n\n```mermaid\ngraph TD\n    A[Client Request] --\u003e B[/v1/chat/completions]\n    B --\u003e C{Format Detection}\n    \n    C --\u003e D1[OpenAI Format]\n    C --\u003e D2[Bedrock Claude Format]\n    C --\u003e D3[Bedrock Titan Format]\n    \n    D1 --\u003e E{Model-Based Routing}\n    D2 --\u003e F[Format Conversion] --\u003e E\n    D3 --\u003e F\n    \n    E --\u003e G1[OpenAI Service]\n    E --\u003e G2[Bedrock Service]\n    \n    G1 --\u003e H1[OpenAI API]\n    G2 --\u003e H2[AWS Bedrock API]\n    \n    H1 --\u003e I[Response Processing]\n    H2 --\u003e I\n    \n    I --\u003e J{Target Format?}\n    J --\u003e K1[OpenAI Response]\n    J --\u003e K2[Bedrock Claude Response]\n    J --\u003e K3[Bedrock Titan Response]\n    \n    K1 --\u003e L[Client Response]\n    K2 --\u003e L\n    K3 --\u003e L\n```\n\n### Key Components\n\n1. **Format Detection**: Automatically identifies input format\n2. **Model-Based Routing**: Routes to appropriate provider based on model ID\n3. **Service Layer**: Abstracts provider-specific implementations\n4. **Adapter Layer**: Handles format conversions\n5. **Strategy Pattern**: Manages different model families within providers\n\n### Design Principles\n\n- **Single Responsibility**: Each component has a clear, focused purpose\n- **Open/Closed**: Easy to extend with new providers or formats\n- **Dependency Inversion**: High-level modules don't depend on low-level details\n- **Interface Segregation**: Clean, minimal interfaces between components\n\n## 🧪 Testing\n\n### Test Categories\n\nThe project includes several categories of tests:\n\n- **Unit Tests**: Fast, isolated tests that don't make external API calls\n- **Integration Tests**: Tests that make real API calls to external services\n- __External API Tests__: Tests marked with `external_api` that are skipped in CI\n\n### Running Tests\n\n```bash\n# Run all tests (excluding external API tests)\nuv run pytest\n\n# Run all tests including external API tests (requires credentials)\nuv run pytest -m \"not external_api or external_api\"\n\n# Run only unit tests (no external API calls)\nuv run pytest -m \"not external_api\"\n\n# Run only OpenAI integration tests (requires OPENAI_API_KEY)\nuv run pytest -m \"openai_integration\"\n\n# Run only AWS/Bedrock integration tests (requires AWS credentials)\nuv run pytest -m \"aws_integration\"\n\n# Run all external API tests (requires all credentials)\nuv run pytest -m \"external_api\"\n\n# Run with coverage\nuv run pytest --cov=src --cov-report=html\n\n# Run specific test categories\nuv run pytest tests/cli/  # CLI tests\nuv run pytest tests/core/ # Core functionality tests\nuv run pytest tests/adapters/ # Adapter tests\nuv run pytest tests/utils/ # Utility tests\n```\n\n### External API Test Configuration\n\nExternal API tests require real credentials and are automatically skipped when:\n\n- __OpenAI Tests__: `OPENAI_API_KEY` environment variable is not set\n- __AWS/Bedrock Tests__: AWS authentication is not configured (no `AWS_REGION`, `AWS_PROFILE`, or `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`)\n\n#### Setting up OpenAI Integration Tests\n\n```bash\nexport OPENAI_API_KEY=\"sk-your-openai-api-key\"\nexport TEST_OPENAI_MODEL=\"gpt-4o\"  # Optional, defaults to gpt-4o\nuv run pytest -m \"openai_integration\"\n```\n\n#### Setting up AWS/Bedrock Integration Tests\n\n```bash\n# Option 1: Using AWS Profile\nexport AWS_PROFILE=\"your-aws-profile\"\nexport AWS_REGION=\"us-east-1\"\n\n# Option 2: Using Access Keys\nexport AWS_ACCESS_KEY_ID=\"your-access-key\"\nexport AWS_SECRET_ACCESS_KEY=\"your-secret-key\"\nexport AWS_REGION=\"us-east-1\"\n\n# Option 3: Using Role Assumption\nexport AWS_PROFILE=\"your-base-profile\"\nexport AWS_ROLE_ARN=\"arn:aws:iam::123456789012:role/YourRole\"\nexport AWS_EXTERNAL_ID=\"your-external-id\"  # Optional\nexport AWS_REGION=\"us-east-1\"\n\n# Run AWS tests\nuv run pytest -m \"aws_integration\"\n```\n\n### GitHub Actions CI/CD\n\n- __Regular CI__: Runs unit tests only (`pytest -m \"not external_api\"`)\n- **Integration Tests**: Manual workflow for running external API tests with credentials\n   - Can be triggered manually from GitHub Actions tab\n   - Supports running OpenAI tests, AWS tests, or both\n   - Uses repository secrets for API keys and credentials\n\n### Test Coverage\n\n- ✅ **152 tests passing** with comprehensive coverage\n- ✅ **Format detection** and conversion\n- ✅ **Model-based routing** logic\n- ✅ **Streaming** functionality\n- ✅ **Error handling** and edge cases\n- ✅ **Authentication** and security\n- ✅ **CLI commands** and configuration\n- ✅ **External API integration** (when credentials available)\n\n## 🚀 Deployment\n\n### Docker Deployment\n\n```bash\n# Build image\ndocker build -t open-bedrock-server .\n\n# Run container\ndocker run -p 8000:8000 \\\n  -e API_KEY=your-api-key \\\n  -e OPENAI_API_KEY=sk-your-key \\\n  open-bedrock-server\n```\n\n### Production Considerations\n\n- **Environment Variables**: Use secure secret management\n- **Load Balancing**: Multiple server instances behind a load balancer\n- **Monitoring**: Implement health checks and metrics collection\n- **Rate Limiting**: Configure appropriate rate limits for your use case\n- **SSL/TLS**: Use HTTPS in production environments\n\n## 🤝 Contributing\n\nWe welcome contributions! Please see our [Development Guide](docs/development.md) for details on how to extend, customize, or contribute to the project.\n\n### Development Setup\n\n```bash\n# Clone and setup\ngit clone https://github.com/teabranch/open-bedrock-server.git\ncd open-bedrock-server\n\n# Install development dependencies\nuv pip install -e \".[dev]\"\n\n# Run tests\npython -m pytest\n\n# Run linting\nruff check src tests\nmypy src\n```\n\n### Adding New Providers\n\n1. Implement `AbstractLLMService` interface\n2. Add model ID patterns to `LLMServiceFactory`\n3. Create format conversion adapters\n4. Add comprehensive tests\n5. Update documentation\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## 🔗 Links\n\n- **Documentation**: [docs/](docs/)\n- **API Reference**: [docs/api-reference.md](docs/api-reference.md)\n- **CLI Reference**: [docs/cli-reference.md](docs/cli-reference.md)\n- **Architecture Guide**: [docs/architecture.md](docs/architecture.md)\n- **Issues**: [GitHub Issues](https://github.com/teabranch/open-bedrock-server/issues)\n\n---\n\n**Open Bedrock Server** - Unifying LLM providers through intelligent format detection and seamless conversion. 🚀\n\n**Cite This Project**\n\n```js\n@software{open-bedrock-server,\n  author = {TeaBranch},\n  title = {open-bedrock-server: Unifying LLM providers through intelligent format detection and seamless conversion},\n  year = {2025},\n  publisher = {GitHub},\n  journal = {GitHub Repository},\n  howpublished = {\\url{https://github.com/teabranch/open-bedrock-server}},\n  commit = {use the commit hash you're working with}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteabranch%2Fopen-bedrock-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fteabranch%2Fopen-bedrock-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteabranch%2Fopen-bedrock-server/lists"}