{"id":25859563,"url":"https://github.com/gpt4thewin/docker-nginx-openai-api-cache","last_synced_at":"2025-03-01T21:01:37.188Z","repository":{"id":151200599,"uuid":"622556221","full_name":"gpt4thewin/docker-nginx-openai-api-cache","owner":"gpt4thewin","description":"A nginx and docker built reverse proxy server to cache the slow expensive requests to the openai api.","archived":false,"fork":false,"pushed_at":"2024-09-14T17:09:31.000Z","size":8,"stargazers_count":52,"open_issues_count":0,"forks_count":13,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-09-15T02:44:47.715Z","etag":null,"topics":["cache-server","docker","nginx","openai","reverse-proxy"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gpt4thewin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-02T13:19:26.000Z","updated_at":"2024-09-14T17:09:34.000Z","dependencies_parsed_at":"2024-02-29T21:25:36.297Z","dependency_job_id":"72fc2c40-6a67-4e1c-8c46-eccab8810bb6","html_url":"https://github.com/gpt4thewin/docker-nginx-openai-api-cache","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gpt4thewin%2Fdocker-nginx-openai-api-cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gpt4thewin%2Fdocker-nginx-openai-api-cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gpt4thewin%2Fdocker-nginx-openai-api-cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gpt4thewin%2Fdocker-nginx-openai-api-cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gpt4thewin","download_url":"https://codeload.github.com/gpt4thewin/docker-nginx-openai-api-cache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241424420,"owners_count":19960749,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache-server","docker","nginx","openai","reverse-proxy"],"created_at":"2025-03-01T21:01:36.182Z","updated_at":"2025-03-01T21:01:37.181Z","avatar_url":"https://github.com/gpt4thewin.png","language":null,"funding_links":[],"categories":["Others"],"sub_categories":[],"readme":"# Docker Nginx OpenAI API Cache Reverse proxy\n\nThis project is a simple Docker Nginx project that serves as a cache for the OpenAI API.\n\nnginx here is preconfigured to work on OpenAI API.\n\n### Features:\n\n- **Works with any client** that allows you to configure the server address (as it acts as a reverse proxy)\n- Caches the response of the support endpoints. The key of cache is built from the request uri and body\n- Caches the response for the POST requests only\n- Use multiple LLM providers\n- Returns an \"X-Cache-Status\" header indicating whether the response was served from cache or not\n\n### Supported endpoints:\n\n- POST /v1/* (default)\n- POST /openai/* (openai)\n- POST /openrouter/* (openrouter)\n- POST /anthropic/* (anthropic)\n- POST /groq/* (groq)\n- POST /mistral/* (mistral)\n- POST /cohere/* (cohere)\n- POST /ai21/* (ai21)\n\n## Getting Started\n\n### Prerequisites\n\n- Docker\n- Docker compose\n\n### Installation\n\n1. Clone the repository:\n\n```\ngit clone https://github.com/gpt4thewin/docker-nginx-openai-api-cache.git\ncd docker-nginx-openai-api-cache\n```\n\n2. Start the container:\n\n```\ndocker-compose up -d\n```\n\n3. Test the server\n\nSetup your credentials:\n```\nOPENAI_API_KEY=\"....\"\n```\n\nRun this 2 times or more:\n```\ncurl -s -o /dev/null -w \"%{http_code}\" http://localhost:81/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $OPENAI_API_KEY\" \\\n  -d '{\n  \"model\": \"gpt-3.5-turbo\",\n  \"messages\": [\n    {\n      \"role\": \"user\",\n      \"content\": \"Hello there !\"\n    }\n  ],\n  \"temperature\": 0,\n  \"max_tokens\": 228,\n  \"top_p\": 1,\n  \"frequency_penalty\": 0,\n  \"presence_penalty\": 0\n}'\n```\n\n4. Check the logs\n\n```\ndocker-compose logs\n```\n\nThe last lines should show something like this\n```\nopenai-cache-proxy  | 172.28.0.1 - - [29/Feb/2024:19:59:49 +0000] \"POST /v1/chat/completions HTTP/1.1\" 200 494 \"-\" \"curl/7.80.0\" Cache: MISS\nopenai-cache-proxy  | 172.28.0.1 - - [29/Feb/2024:19:59:52 +0000] \"POST /v1/chat/completions HTTP/1.1\" 200 494 \"-\" \"curl/7.80.0\" Cache: HIT\n```\n\n5. Stop the container:\n\n```\ndocker-compose down\n```\n\n### Usage\n\nSet your client's API server address to `http://localhost:81/v1`\nOnce the containers are running, you can use the OpenAI API through the cache by sending requests to the supported URIs.\n\nURIs that are supported will be forwarded, unless they are cached. \nURIs that are not supported will be forwarded normally.\n\n### Configuration\n\nThe cache is configured using the `nginx.conf`. You can modify this file to change the cache settings or add additional URIs.\n\n## Contributing\n\nContributions are welcome! Please submit a pull request or open an issue if you encounter any problems or have suggestions for improvements.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgpt4thewin%2Fdocker-nginx-openai-api-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgpt4thewin%2Fdocker-nginx-openai-api-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgpt4thewin%2Fdocker-nginx-openai-api-cache/lists"}