{"id":18155387,"url":"https://github.com/jparkerweb/bedrock-proxy-endpoint","last_synced_at":"2026-04-01T18:58:34.345Z","repository":{"id":237495604,"uuid":"794554797","full_name":"jparkerweb/bedrock-proxy-endpoint","owner":"jparkerweb","description":"🔀 Bedrock Proxy Endpoint  ⇢ Spin up your own custom OpenAI API server endpoint for easy AWS Bedrock inference (using standard baseUrl, and apiKey params)","archived":false,"fork":false,"pushed_at":"2026-03-19T23:44:30.000Z","size":958,"stargazers_count":16,"open_issues_count":0,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-20T14:24:34.356Z","etag":null,"topics":["api","aws-bedrock","endpoint","inference","inference-api","llama3","llm","mistral","mixtral","openai-api","proxy","serverless","wrapper"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jparkerweb.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":"jparkerweb","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2024-05-01T13:00:04.000Z","updated_at":"2026-03-19T23:44:33.000Z","dependencies_parsed_at":"2025-12-26T03:06:04.858Z","dependency_job_id":null,"html_url":"https://github.com/jparkerweb/bedrock-proxy-endpoint","commit_stats":{"total_commits":31,"total_committers":1,"mean_commits":31.0,"dds":0.0,"last_synced_commit":"08d6d934639ae44d02861bde3ddf3a39bc9efa27"},"previous_names":["jparkerweb/bedrock-tunnel-endpoint","jparkerweb/bedrock-proxy-endpoint"],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/jparkerweb/bedrock-proxy-endpoint","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jparkerweb%2Fbedrock-proxy-endpoint","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jparkerweb%2Fbedrock-proxy-endpoint/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jparkerweb%2Fbedrock-proxy-endpoint/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jparkerweb%2Fbedrock-proxy-endpoint/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jparkerweb","download_url":"https://codeload.github.com/jparkerweb/bedrock-proxy-endpoint/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jparkerweb%2Fbedrock-proxy-endpoint/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31290987,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","aws-bedrock","endpoint","inference","inference-api","llama3","llm","mistral","mixtral","openai-api","proxy","serverless","wrapper"],"created_at":"2024-11-02T04:10:51.972Z","updated_at":"2026-04-01T18:58:34.336Z","avatar_url":"https://github.com/jparkerweb.png","language":"JavaScript","funding_links":["https://ko-fi.com/jparkerweb"],"categories":[],"sub_categories":[],"readme":"# 🔀 Bedrock Proxy Endpoint\r\nSpin up your own custom OpenAI API server endpoint for easy AWS Bedrock LLM text inference (using standard `baseUrl`, and `apiKey` params)\r\n\r\n![bedrock-proxy-endpoint](https://raw.githubusercontent.com/jparkerweb/bedrock-proxy-endpoint/refs/heads/main/.readme/bedrock-proxy-endpoint.jpg)\r\n\r\n### Maintained by\r\n\u003ca href=\"https://www.equilllabs.com\"\u003e\r\n  \u003cimg src=\"https://raw.githubusercontent.com/jparkerweb/eQuill-Labs/refs/heads/main/src/static/images/logo-text-outline.png\" alt=\"eQuill Labs\" height=\"40\" style=\"max-height: 40px\"\u003e\r\n\u003c/a\u003e\r\n\r\n---\r\n\r\n### Why Bedrock Proxy Endpoint?\r\n\r\nAre you are stuck with using AWS Bedrock for all LLM text inference, but you want to keep your application platform agnostic?  Are you tired of figuring out how to format your LLM inference calls to work with the Bedrock SDK? Are you going crazy with all the differences between configuration from model to model?\r\n\r\n- `Bedrock Proxy Endpoint` makes it easy to continue using the OpenAI API client that you are use to by standing up your own OpenAI text compatible endpoint that will proxy all your calls to Bedrock in a compatible way.\r\n\r\n- Great for getting existing OpenAI API compatible applications working with AWS Bedrock.\r\n\r\n\r\n### Prerequisites\r\n\r\nBefore getting started, make sure you have the following installed:\r\n\r\n- Node.js (version 12 or higher)\r\n- npm (Node Package Manager)\r\n\r\n### Installation\r\n\r\n1. Clone the repository:\r\n\r\n    ```bash\r\n    git clone https://github.com/jparkerweb/bedrock-proxy-endpoint.git\r\n    ```\r\n\r\n2. Navigate to the project directory:\r\n\r\n    ```bash\r\n    cd bedrock-proxy-endpoint\r\n    ```\r\n\r\n3. Install the dependencies:\r\n\r\n    ```bash\r\n    npm ci\r\n    ```\r\n\r\n### Docker Usage\r\n\r\nYou can run Bedrock Proxy Endpoint using Docker in two ways:\r\n\r\n#### Option 1: Using Pre-built Image from GitHub Container Registry\r\n\r\nPull and run the latest image from GHCR:\r\n\r\n```bash\r\n# Pull the latest image\r\ndocker pull ghcr.io/jparkerweb/bedrock-proxy-endpoint:latest\r\n\r\n# Run with environment file\r\ndocker run -d \\\r\n  --name bedrock-proxy-endpoint \\\r\n  -p 88:88 \\\r\n  --env-file .env \\\r\n  ghcr.io/jparkerweb/bedrock-proxy-endpoint:latest\r\n\r\n# Or run with inline environment variables\r\ndocker run -d \\\r\n  --name bedrock-proxy-endpoint \\\r\n  -p 88:88 \\\r\n  -e HTTP_ENABLED=true \\\r\n  -e HTTP_PORT=88 \\\r\n  -e CONSOLE_LOGGING=true \\\r\n  ghcr.io/jparkerweb/bedrock-proxy-endpoint:latest\r\n```\r\n\r\n#### Option 2: Build and Run Locally\r\n\r\nBuild the Docker image locally and run it:\r\n\r\n```bash\r\n# Build the image locally\r\ndocker build -t bedrock-proxy-endpoint .\r\n\r\n# Run with environment file\r\ndocker run -d \\\r\n  --name bedrock-proxy-endpoint \\\r\n  -p 88:88 \\\r\n  --env-file .env \\\r\n  bedrock-proxy-endpoint\r\n\r\n# Or run with custom port mapping\r\ndocker run -d \\\r\n  --name bedrock-proxy-endpoint \\\r\n  -p 8080:8080 \\\r\n  -e HTTP_PORT=8080 \\\r\n  --env-file .env \\\r\n  bedrock-proxy-endpoint\r\n```\r\n\r\n#### Docker Compose Example\r\n\r\nCreate a `docker-compose.yml` file for easier management:\r\n\r\n```yaml\r\nversion: '3.8'\r\nservices:\r\n  bedrock-proxy-endpoint:\r\n    image: ghcr.io/jparkerweb/bedrock-proxy-endpoint:latest\r\n    # Or for local build: build: .\r\n    ports:\r\n      - \"88:88\"\r\n    env_file:\r\n      - .env\r\n    restart: unless-stopped\r\n    healthcheck:\r\n      test: [\"CMD\", \"node\", \"-e\", \"const http = require('http'); const req = http.request({host: 'localhost', port: process.env.HTTP_PORT || 3000, timeout: 2000}, (res) =\u003e process.exit(res.statusCode === 200 ? 0 : 1)); req.on('error', () =\u003e process.exit(1)); req.end();\"]\r\n      interval: 30s\r\n      timeout: 10s\r\n      retries: 3\r\n```\r\n\r\nThen run with:\r\n```bash\r\ndocker-compose up -d\r\n```\r\n\r\n### Configuration\r\n\r\n* Update the `.env` file in the root directory of the project with the following\r\n  environment variables based on your desired configuration:\r\n\r\n    | key                        | value type | example                   | notes                          |\r\n    |----------------------------|------------|---------------------------|--------------------------------|\r\n    | CONSOLE_LOGGING            | boolean    | false                     | Show realtime logs             |\r\n    | HTTP_ENABLED               | boolean    | true                      | Start a HTTP server            |\r\n    | HTTP_PORT                  | integer    | 80                        | HTTP server port               |\r\n    | MAX_REQUEST_BODY_SIZE      | string     | 50mb                      | Maximum size for request body  |\r\n    | HTTPS_ENABLED              | boolean    | false                     | Start a HTTPS server           |\r\n    | HTTPS_PORT                 | integer    | 443                       | HTTPS server port              |\r\n    | HTTPS_KEY_PATH             | string     | ./path/mykey.key          | Path to key file for HTTPS     |\r\n    | HTTPS_CERT_PATH            | string     | ./path/mycert.pem         | Path to cert file for HTTPS    |\r\n    | IP_RATE_LIMIT_ENABLED      | boolean    | true                      | Enable rate limiting by IP     |\r\n    | IP_RATE_LIMIT_WINDOW_MS    | integer    | 60000                     | Window in milliseconds         |\r\n    | IP_RATE_LIMIT_MAX_REQUESTS | integer    | 100                       | Max requests per IP per window |\r\n\r\n### Authentication\r\n\r\n`Bedrock Proxy` authenticates with AWS via `IAM`. Since the OpenAI API intance accpets an API Key we will utilize this value to hold your credentials. Construct your `apiKey` for inference in the next step following this format:\r\n\r\n- `${AWS_REGION}.${AWS_ACCESS_KEY_ID}.${AWS_SECRET_ACCESS_KEY}`\r\n- example `apiKey` value:  \r\n  `us-west-2.AKIAWSXXXXXXXXXXX.YYYYYYYYYYYYYYYYYYYYYYYYY`\r\n\r\n### Usage\r\n\r\n- Start the server via: `node server`  \r\n\r\n  \u003cimg src=\"docs/console.png\"\u003e  \r\n\r\n  You are now ready to make a standard chat completions to the server.\r\n\r\n- Important values\r\n  - `baseUrl`: Root address of server based on your `.env` configuration.\r\n  - `apiKey`: Descibed in the *Authentication* section above.\r\n  - `messages`: Array of objects in role / content format.\r\n  - `model`: This is the `modelName` from the list of supported models found on the `Bedrock Wrapper` README file [here](https://github.com/jparkerweb/bedrock-wrapper?tab=readme-ov-file#supported-models); The `/models` enpoint of this server will also return a list of supported models.\r\n  - `include_thinking_data`: _Optional_ boolean parameter that when set to `true` will include the model's thinking process in the response (only used with thinking models such as `Claude-3-7-Sonnet-Thinking`).\r\n  - `use_converse_api`: _Optional_ boolean parameter that when set to `true` will use AWS Bedrock's Converse API instead of the Invoke API. The Converse API provides a unified request/response format across all models, better conversation management, and cleaner multimodal handling. Defaults to `false` for backward compatibility.\r\n  - `stop` or `stop_sequences`: _Optional_ parameter to specify text sequences that will cause the model to stop generating (e.g., `[\"STOP\", \"END\"]`).\r\n\r\n### Example OpenAI API Call\r\nLook at the example folder for complete examples of how to use the server:\r\n- `example.js` - Basic text completion example\r\n- `example-vision.js` - Vision model example with image processing (image can be passed as a base64 string or a URL)\r\n- `example-stop-sequences.js` - Stop sequences example demonstrating how to control where generation stops\r\n\r\n```javascript\r\nimport OpenAI from 'openai';\r\n\r\nconst messages = [\r\n    {\r\n        role: \"system\",\r\n        content: \"You are a helpful AI assistant that follows instructions extremely well. Answer the user questions accurately.\",\r\n    },\r\n    {\r\n        role: \"user\",\r\n        content: \"Describe why the OpenAI API standard is so great. Limit your response to five sentences.\",\r\n    },\r\n    {\r\n        role: \"assistant\",\r\n        content: \"\",\r\n    },\r\n];\r\n\r\nconst baseURL = \"http://localhost\"; // URL of the Bedrock Proxy Endpoint\r\nconst apiKey = `${AWS_REGION}.${AWS_ACCESS_KEY_ID}.${AWS_SECRET_ACCESS_KEY}` // Your AWS Creds / API Key\r\n\r\nconst openai = new OpenAI({\r\n    baseURL: baseURL,\r\n    apiKey: apiKey,\r\n});\r\n\r\nasync function main() {\r\n    try {\r\n        const chatCompletion = await openai.chat.completions.create({\r\n            messages: messages,\r\n            model: \"Claude-4-5-Sonnet\",\r\n            max_tokens: 2048,\r\n            temperature: 0.4,\r\n            top_p: 0.7,\r\n            stream: true,\r\n            include_thinking_data: false,  // Set to true for thinking models like \"Claude-4-5-Sonnet-Thinking\"\r\n            use_converse_api: true,  // Set to true to use the unified Converse API instead of Invoke API (default: false)\r\n        });\r\n\r\n        if (chatCompletion) {\r\n            for await (const chunk of chatCompletion) {\r\n                const response = chunk.choices[0]?.delta?.content || \"\";\r\n                process.stdout.write(response);\r\n            }\r\n        }\r\n    } catch (error) {\r\n        console.error('Error:', error);\r\n    } finally {\r\n        process.exit(0);\r\n    }\r\n}\r\n\r\nmain();\r\n```\r\n\r\n### Root Info Page\r\n\r\nPoint your browser to the root of your endpoint server to view the info page: (example: `http://localhost`)  \r\n\r\n\u003cimg src=\"docs/bedrock-proxy-endpoint.png\" style=\"max-width:700px\"\u003e\r\n\r\n### Note\r\n\r\nAlternativly you can incorporate 🪨 \u003ca href=\"https://github.com/jparkerweb/bedrock-wrapper\" target=\"bedrockWrapper\"\u003e`Bedrock Wrapper`\u003c/a\u003e core directly into your code base. If you would like to explore that option checkout the npm package here: https://www.npmjs.com/package/bedrock-wrapper\r\n\r\n---\r\n\r\nPlease consider sending me a tip to support my work 😀\r\n# [🍵 tip me here](https://ko-fi.com/jparkerweb)\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjparkerweb%2Fbedrock-proxy-endpoint","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjparkerweb%2Fbedrock-proxy-endpoint","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjparkerweb%2Fbedrock-proxy-endpoint/lists"}