{"id":20975132,"url":"https://github.com/timesurgelabs/promptproxy","last_synced_at":"2026-05-02T17:36:23.717Z","repository":{"id":229042163,"uuid":"775250448","full_name":"TimeSurgeLabs/promptproxy","owner":"TimeSurgeLabs","description":"Call many AIs from a single API.","archived":false,"fork":false,"pushed_at":"2024-03-28T15:31:40.000Z","size":291,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-06-23T22:09:39.838Z","etag":null,"topics":["ai","docker","huggingface","llama","llama2","llm","openai","openai-api","openai-api-proxy","vllm"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TimeSurgeLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-03-21T03:04:24.000Z","updated_at":"2025-01-21T13:35:23.000Z","dependencies_parsed_at":"2024-03-28T16:40:15.292Z","dependency_job_id":"7cf294a8-a8b1-457d-aa86-7c12b1e5e42d","html_url":"https://github.com/TimeSurgeLabs/promptproxy","commit_stats":null,"previous_names":["timesurgelabs/promptproxy"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TimeSurgeLabs/promptproxy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimeSurgeLabs%2Fpromptproxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimeSurgeLabs%2Fpromptproxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimeSurgeLabs%2Fpromptproxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimeSurgeLabs%2Fpromptproxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TimeSurgeLabs","download_url":"https://codeload.github.com/TimeSurgeLabs/promptproxy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TimeSurgeLabs%2Fpromptproxy/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262704654,"owners_count":23351111,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","docker","huggingface","llama","llama2","llm","openai","openai-api","openai-api-proxy","vllm"],"created_at":"2024-11-19T04:39:35.728Z","updated_at":"2025-10-18T11:02:03.738Z","avatar_url":"https://github.com/TimeSurgeLabs.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003ePromptProxy\u003c/h1\u003e\n\u003cp align=\"center\"\u003eCall many AIs with many system prompts from a single API.\u003c/p\u003e\n\n# Features\n* Proxies to any OpenAI Compatible API!\n  + [OpenAI API](https://platform.openai.com/docs/overview)\n  + [HuggingFace Dedicated Inference Endpoints](https://huggingface.co/docs/inference-endpoints/index)\n  + [Ollama](https://github.com/ollama/ollama/blob/main/docs/openai.md)\n  + [FastChat](https://github.com/lm-sys/FastChat/blob/main/docs/openai_api.md)\n  + [vLLM](https://docs.vllm.ai/en/latest/getting_started/quickstart.html#openai-compatible-server)\n  + And many more!\n* Store system prompts and use them in your API calls without having to store them in your code.\n* System Prompt, AI API, Model, and API key management via PocketBase Admin UI.\n* Per user API key management.\n* Per user request tracking.\n# Roadmap\n* [ ] Claude API support\n* [ ] Refactor for maintainability\n* [ ] Per user rate limits\n* [ ] Per user AI access control\n* [ ] Per user prompt access control\n* [ ] Proper frontend UI\n* [ ] Streaming Support\n# Setup\n\n```sh\ngit clone https://github.com/TimeSurgeLabs/promptproxy.git\ncd promptproxy\ndocker build -t timesurgelabs/promptproxy .\ndocker run -p 8090:8090 -v $(PWD)/pb_data:/pb_data timesurgelabs/promptproxy\n```\n\nThen navigate to `http://localhost:8090/_` in your browser. Create an admin account. Now you can see the following tables:\n\n![Table List](docs/img/tables.png)\n\nFirst, create a new user by selecting the users table and hitting \"New Record\" in the top right. Then, fill out the form, then hit \"Create\" in the bottom right.\n\n![Creating Tom](docs/img/create_tom.png)\n\nNow that Tom is a user, he needs an API key. Go to the `keys` table and hit \"New Record\" in the top right. Then, fill out the form, then hit \"Create\" in the bottom right. I recommend creating a random key using OpenSSL (only works on Git Bash on Windows).\n\n```sh\nopenssl rand -base64 24\n```\n\n![API Key](docs/img/create_api_key.png)\n\nAfter that is done, we need to add some APIs to be used. Currently there is no way to restrict what users use which APIs, however it will track which user used which API and if they used any of the system prompts we store. For this example we're going to use the OpenAI API, however it will work with any API that is OpenAI compatible. Go to the `apis` table and hit \"New Record\" in the top right. Then, fill out the form, then hit \"Create\" in the bottom right.\n\n![OpenAI](docs/img/create_openai.png)\n\nNow you have to add some models. Go to the `models` table and hit \"New Record\" in the top right. Then, fill out the form, selecting the correct API for the model and filling out the correct model name, then hit \"Create\" in the bottom right. The new ID of your model will be the model ID you use in the API.\n\n![Model](docs/img/existing_model.png)\n\nFinally, we can add system prompts! These are prompts used by the API to alter the behavior of the AI models. Go to the `prompts` table and hit \"New Record\" in the top right. Then, fill out the form, selecting the correct model for the prompt and filling out the correct prompt, then hit \"Create\" in the bottom right. The new ID of your prompt will be the prompt ID you use in the API. Unlike models, prompts are not linked to specific models or APIs so you can experiment and use the same prompts across multiple models and APIs.\n\n![Prompt](docs/img/prompt.png)\n\n# Usage\n\nYou can call the API using the endpoints `/v1/completion` or `/v1/chat/completion` . Rather than putting a model like `gpt-4` , you'll put the database ID of the model. If you want to use a system prompt, put a colon after the model ID and then put the database ID of the prompt.\n\n```sh\ncurl -X POST \"http://localhost:8090/v1/completion\" -H \"accept: application/json\" -H \"Content-Type: application/json\" -d \"{\\\"model\\\":\\\"2nhjz8jg6goy6ll:hl7ssaft143mf8k\\\",\\\"prompt\\\":\\\"This is a test prompt.\\\"}\"\n```\n\nUsage is the same for chat completions, just using `messages` instead of the `prompt` endpoint. Note that for the OpenAI API, the models are not the same for both endpoints.\n\n```sh\ncurl -X POST \"http://localhost:8090/v1/chat/completion\" -H \"accept: application/json\" -H \"Content-Type: application/json\" -d '{\"model\":\"2nhjz8jg6goy6ll:hl7ssaft143mf8k\",\"messages\":[{\n      \"role\": \"user\",\n      \"content\": \"Hola mi nombre es brian y el tuyo?\"\n    },\n    {\n      \"role\": \"assistant\",\n      \"content\": \"spanish\"\n    },\n    {\n      \"role\": \"user\",\n      \"content\": \"Was für ein schönes Baby!\"\n    },\n    {\n      \"role\": \"assistant\",\n      \"content\": \"german\"\n    },\n    {\n      \"role\": \"user\",\n      \"content\": \"Ma voiture est en panne.\"\n    }]}''\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimesurgelabs%2Fpromptproxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimesurgelabs%2Fpromptproxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimesurgelabs%2Fpromptproxy/lists"}