https://github.com/ivanlulyf/bunny-llm
Deno LLM API Service
https://github.com/ivanlulyf/bunny-llm
chatgpt chatgpt-api cloudflare-workers-ai llm llm-serving
Last synced: 9 months ago
JSON representation
Deno LLM API Service
- Host: GitHub
- URL: https://github.com/ivanlulyf/bunny-llm
- Owner: IvanLuLyf
- License: mit
- Created: 2024-04-07T05:30:01.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-02-26T05:48:43.000Z (11 months ago)
- Last Synced: 2025-03-29T05:36:54.154Z (10 months ago)
- Topics: chatgpt, chatgpt-api, cloudflare-workers-ai, llm, llm-serving
- Language: TypeScript
- Homepage: https://bunny-llm.deno.dev
- Size: 132 KB
- Stars: 7
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Bunny-LLM
a Deno LLM API Service
## Usage
Set `base_url` to `https://bunny-llm.deno.dev/v1/` or your own endpoint.
Set env variable `BUNNY_API_TOKEN` and specific vendor tokens.
Model format `[vendor]:[model_name]`, for example `openai:gpt-4-turbo`
### Example
```python
from openai import OpenAI
client = OpenAI(
base_url='https://bunny-llm.deno.dev/v1/',
api_key='YOUR_BUNNY_API_TOKEN',
)
model_name = 'cloudflare:@cf/qwen/qwen1.5-0.5b-chat'
res = client.chat.completions.create(
model=model_name,
messages=[
{'role': 'user', 'content': 'Who are you?'}
],
)
print(res.choices[0].message.content)
```
### CloudFlare Workers AI
`vendor` is `cf` or `cloudflare`
`model` can refer to [Supported Models](https://developers.cloudflare.com/workers-ai/models/)
Set env variable `CF_ACCOUNT_ID` and `CF_API_TOKEN` or use the following algorithm.
```javascript
// JavaScript
api_key = encodeURIComponent(JSON.stringify({
'account': 'YOUR_CF_ACCOUNT_ID',
'token': 'YOUR_CF_API_TOKEN',
}))
```
```python
# python
import json
import urllib.parse
api_key=urllib.parse.quote(json.dumps({
'account': 'YOUR_CF_ACCOUNT_ID',
'token': 'YOUR_CF_API_TOKEN',
}))
```
### DashScope
`vendor` is `ali`
`model` can refer to [Supported Models](https://help.aliyun.com/zh/model-studio/developer-reference/models/)
`api_key` can set by `DASHSCOPE_API_KEY` or pass it directly [Get API KEY](https://dashscope.console.aliyun.com/apiKey)
### Groq
`vendor` is `groq`
`api_key` can set by `GROQ_API_KEY` or pass it directly [Get API KEY](https://console.groq.com/keys)
`model` can refer to [Supported Models](https://console.groq.com/docs/models)
## Deploy
### Vercel
[](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FIvanLuLyf%2Fbunny-llm)
### Netlify
[](https://app.netlify.com/start/deploy?repository=https://github.com/IvanLuLyf/bunny-llm/)