Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/litongjava/go-llm-proxy
go-llm-proxy
https://github.com/litongjava/go-llm-proxy
Last synced: about 1 month ago
JSON representation
go-llm-proxy
- Host: GitHub
- URL: https://github.com/litongjava/go-llm-proxy
- Owner: litongjava
- License: mit
- Created: 2024-05-14T11:16:13.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2024-07-14T07:06:05.000Z (6 months ago)
- Last Synced: 2024-07-14T08:22:13.392Z (6 months ago)
- Language: Go
- Size: 17.6 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- License: LICENSE
Awesome Lists containing this project
README
# go-llm-proxy
## Introduction
`go-llm-proxy` is a service designed to proxy OpenAI API requests. It receives OpenAI API requests from clients and forwards them to the official OpenAI servers. This project solely acts as a proxy and does not store any data.
**Note**: You must obtain your own OpenAI API key.
## Quick Start
After deployment, the service is accessible at: `https://llm.maxkb.ai`
### Example Request
Use `curl` to send a request:
```shell
curl --location --request POST 'https://llm.maxkb.ai/openai/v1/chat/completions' \
--header 'Authorization: Bearer YOUR_OPENAI_API_KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
"messages": [
{
"role": "system",
"content": "Just say hi"
}
],
"model": "gpt-4o-mini",
"stream": true
}'
```
**Important**: Replace `YOUR_OPENAI_API_KEY` with your actual API key.
## Build
Compile the project using Go:
```sh
go build
```
## Run
After building, run the generated executable:
```sh
./go-llm-proxy
```
## Test
Use `curl` to test whether the proxy service correctly forwards requests and displays the returned data:
```sh
curl --location --request POST 'http://127.0.0.1:8888/openai/v1/chat/completions' \
--header 'Authorization: Bearer YOUR_OPENAI_API_KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
"messages": [
{
"role": "system",
"content": "Just say hi"
}
],
"model": "gpt-3.5-turbo",
"stream": true
}'
```
### Example Response
```shell
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"Hi"},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"!"},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" How"},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" can"},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" I"},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" assist"},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" you"},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" today"},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"?"},"logprobs":null,"finish_reason":null}]}
data:{"id":"chatcmpl-9P3fvvyk4IuCprCnvMytoKN8UtskC","object":"chat.completion.chunk","created":1715759355,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}
data:[DONE]
```
## Supported Models
Currently supported models include: [OpenAI Models](https://platform.openai.com/docs/models)
Please configure the specific model you wish to connect to via the proxy as needed.
## Contribution
Contributions from the community are welcome! Whether it's adding new features or improving existing ones, feel free to submit Pull Requests to our [GitHub repository](https://github.com/your-repo/go-llm-proxy).
## License
This project is licensed under the MIT License. See the [`LICENSE`](LICENSE) file for details.
## Security Notice
**Do not** expose your OpenAI API key in public repositories or shared environments. Ensure that your API key is securely stored to prevent unauthorized use.