Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/la3rence/cf-llm-hono
Streaming LLM powered by Cloudflare AI
https://github.com/la3rence/cf-llm-hono
ai cloudflare hono
Last synced: 3 months ago
JSON representation
Streaming LLM powered by Cloudflare AI
- Host: GitHub
- URL: https://github.com/la3rence/cf-llm-hono
- Owner: la3rence
- Created: 2023-12-01T02:34:27.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-10-28T02:04:13.000Z (3 months ago)
- Last Synced: 2024-10-28T05:58:15.814Z (3 months ago)
- Topics: ai, cloudflare, hono
- Language: TypeScript
- Homepage:
- Size: 253 KB
- Stars: 5
- Watchers: 0
- Forks: 0
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Hono for Cloudflare Workers AI and ChatGPT
Creat the `wrangle.toml` file for cloudflare deployment:
```toml
name = "my-llm-api"
[ai]
binding = "AI"
[vars]
GPT_API_KEY = ""
GPT_BASE = ""
```
```sh
pnpm install
pnpm run deploy
```
## Models
```txt
"text-davinci-002-render-sha",
"gemini-pro",
"@cf/meta/llama-2-7b-chat-int8",
"@cf/mistral/mistral-7b-instruct-v0.1",
"@cf/meta/llama-2-7b-chat-fp16",
"@hf/thebloke/llama-2-13b-chat-awq",
"@hf/thebloke/zephyr-7b-beta-awq",
"@hf/thebloke/mistral-7b-instruct-v0.1-awq",
"@hf/thebloke/codellama-7b-instruct-awq"
and more...
```
## Vercel AI SDK
This project is fully compitable with [Vercel AI SDK](https://sdk.vercel.ai/docs/api-reference/use-chat).
## Reference
- [Hono Docs](https://hono.dev/top)