{"id":17526693,"url":"https://github.com/chand1012/openai-cf-workers-ai","last_synced_at":"2025-10-11T15:06:19.983Z","repository":{"id":198927644,"uuid":"701820767","full_name":"chand1012/openai-cf-workers-ai","owner":"chand1012","description":"Replacing OpenAI's API with Cloudflare AI.","archived":false,"fork":false,"pushed_at":"2024-05-06T15:14:43.000Z","size":67,"stargazers_count":266,"open_issues_count":0,"forks_count":63,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-07-30T04:50:28.710Z","etag":null,"topics":["cloudflare-ai","cloudflare-d1","cloudflare-worker","cloudflare-workers","workers-kv"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chand1012.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-07T16:50:48.000Z","updated_at":"2025-07-29T03:22:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"1a1a55e1-a86e-4e4c-a09c-137eeb0d2562","html_url":"https://github.com/chand1012/openai-cf-workers-ai","commit_stats":null,"previous_names":["chand1012/openai-cf-workers-ai"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/chand1012/openai-cf-workers-ai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chand1012%2Fopenai-cf-workers-ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chand1012%2Fopenai-cf-workers-ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chand1012%2Fopenai-cf-workers-ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chand1012%2Fopenai-cf-workers-ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chand1012","download_url":"https://codeload.github.com/chand1012/openai-cf-workers-ai/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chand1012%2Fopenai-cf-workers-ai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279007599,"owners_count":26084333,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudflare-ai","cloudflare-d1","cloudflare-worker","cloudflare-workers","workers-kv"],"created_at":"2024-10-20T15:02:12.748Z","updated_at":"2025-10-11T15:06:19.956Z","avatar_url":"https://github.com/chand1012.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# \u003ch1 align=\"center\"\u003e⚡️ OpenAI for Workers AI 🧠\u003c/h1\u003e\n\n### \u003cp align=\"center\"\u003eSimple, quick, and dirty implementation of OpenAI's API on Cloudflare's new Workers AI platform.\u003c/p\u003e\n\n## Why?\n\nI think that in the near future, smaller, cheaper LLMs will be a legitimate competitor to OpenAI's GPT-3.5 and GPT-4 APIs. Most developers will not want to rewrite their entire codebase in order to use these up-and-coming models. I also think that Cloudflare Workers are a neat way to host AI and APIs, so I implemented the OpenAI API on Workers AI. This allows developers to use the OpenAI SDKs with the new LLMs without having to rewrite all of their code. This code, as is Workers AI, is not production ready but will be semi-regularly updated with new features as they roll out to Workers AI.\n\n## Compatibility\n\nHere are all the APIs I would like to implement or have implemented that are currently possible with the Workers AI platform.\n\n* [x] Completions\n* [x] Chat Completions\n* [x] Audio Transcription\n* [x] Embeddings\n* [x] Audio Translation\n  + Uses Whisper to transcribe, Llama 2 to identify the language, and m2m-100 to translate.\n* [x] Images\n  + [x] Image Generation\n* [ ] Files\n  + Needed for Assistants.\n  + Use a D1 database for metadata, R2 for the actual file.\n* [ ] Assistants\n  + [ ] Assistants\n    - Store assistants in a D1 database.\n    - File support may be just limited to text files if there is any at all.\n  + [ ] Threads\n    - Use a D1 database to store threads. Relate them to an assistant.\n  + [ ] Messages\n    - Store messages in a D1 database. Relate them to a thread.\n  + [ ] Runs\n    - Use a queue to handle runs. Get messages from a D1 database, return results to database.\n\nHere are the APIs that I would like to implement but are not currently possible with the Workers AI platform.\n\n* [ ] Fine Tuning\n* [ ] Images\n  + [ ] Image Editing\n  + [ ] Image Variants\n\nHere are the APIs that could probably be implemented but I don't have the need to implement them.\n\n* [ ] Moderation\n  + Use Llama 2 to classify. May be difficult to prompt engineer.\n\n## Deploying\n\nFirst, clone the repository.\n\n```bash\ngit clone https://github.com/chand1012/openai-cf-workers-ai\ncd openai-cf-workers-ai\n```\n\nThen modify the `CLOUDFLARE_ACCOUNT_ID` variable in the `wrangler.toml` file to match your Cloudflare Account ID.\n\n```toml\n[vars]\nCLOUDFLARE_ACCOUNT_ID = \"c8c30db3dddc4ad31065d336368c7905\" # replace with your own.\n```\n\nNext, install the dependencies and deploy to your account. If you are not logged in to wrangler, you will be prompted to log in.\n\n```bash\nyarn\nyarn init-prod # only needs run the first time!!!\nyarn deploy\n```\n\nFinally, set the `ACCESS_TOKEN` and `CLOUDFLARE_API_TOKEN` variables using wrangler. To get your Cloudflare API token, go to the [Cloudflare dashboard](https://dash.cloudflare.com/profile/api-tokens) and create a new token with the `Workers AI` template. `ACCESS_TOKEN` can be any string you want, but I recommend creating a random string with `openssl rand -hex 32` .\n\n```bash\nwrangler secret put ACCESS_TOKEN # put the access token here\nwrangler secret put CLOUDFLARE_API_TOKEN # put the Cloudflare API token here\n```\n\nNow you're ready to use the API! You can find the URL in the output of the `yarn deploy` command.\n\n## Development\n\nBefore Testing, copy `.dev.vars.example` to `.dev.vars` and populate the variables with the appropriate values. See [Deploying](#deploying) for more information.\n\nAs of 07/10/2023 testing locally does not work. However, you can test remotely using the following command:\n\n```bash\nyarn init-dev # only needs run the first time!!!\nyarn dev\n```\n\nThis will start a local server that will proxy requests to your deployed API. You can then use the API as you normally would, but with the local server's URL instead of the deployed URL.\n\n## Usage\n\nSee the [OpenAI API docs](https://platform.openai.com/docs/api-reference/introduction) for more information on the API. Here's an example from the OpenAI docs:\n\n```bash\ncurl https://openai-cf.yourusername.workers.dev/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer \u003cAny string value you set.\u003e\" \\\n  -d '{\n    \"model\": \"@cf/meta/llama-2-7b-chat-int8\",\n    \"messages\": [\n      {\n        \"role\": \"system\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n# {\"id\":\"ccfbc7fc-d871-4139-90dc-e6c33fc7f275\",\"model\":\"@cf/meta/llama-2-7b-chat-int8\",\"created\":1696701894,\"object\":\"chat.completion\",\"choices\":[{\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"Hello there! *adjusts glasses* It's a pleasure to meet you. Is there something I can help you with or would you like to chat? I'm here to assist you in any way I can. 😊\"},\"finish_reason\":\"stop\"}],\"usage\":{\"prompt_tokens\":0,\"completion_tokens\":0,\"total_tokens\":0}}\n```\n\nIf you want to use this with the OpenAI Python or JavaScript SDK, you can use the following code, replace the base URL with your own. For example:\n\n```python\nimport openai\nopenai.api_base = 'https://openai-cf.yourusername.workers.dev/v1'\n\n# rest of code\n```\n\n```javascript\nimport OpenAI from 'openai';\n\nconst openai = new OpenAI({\n    baseURL: 'https://openai-cf.yourusername.workers.dev/v1',\n    ...\n});\n\n// rest of code\n```\n\n## Compromises\n\nThere were a few compromises I had to make in order to create the API.\n\nThe first is that the API does not count tokens, and will always return zero for the `usage` attribute in the return object. It will always return it for compatibility reasons, but until tokenization is added for the respective model, we cannot count tokens. Each model tokenizes differently, so we can't use tiktoken. It may be possible to tokenize using HuggingFace transformers, but that may take too long and not allow free users to deploy the API. More investigation is needed.\n\nStop tokens are also non-functional. There is no way to specify a stop reason or token with the current API. It will be ignored. Sometimes these stop tokens can leak through to the final response. This is a known issue and will be fixed in the future.\n\n## License\n\nLicensed under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchand1012%2Fopenai-cf-workers-ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchand1012%2Fopenai-cf-workers-ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchand1012%2Fopenai-cf-workers-ai/lists"}