https://github.com/pond918/llm-bots
npm package of llm bots with tree-structured chat histories. can be deployed on local browser, or node server with shared sessions and isolated user states.
https://github.com/pond918/llm-bots
alpaca bard bingchat chatbot chatglm chatgpt claude ernie iflytek llm moss
Last synced: 2 months ago
JSON representation
npm package of llm bots with tree-structured chat histories. can be deployed on local browser, or node server with shared sessions and isolated user states.
- Host: GitHub
- URL: https://github.com/pond918/llm-bots
- Owner: pond918
- License: apache-2.0
- Created: 2023-05-25T23:51:32.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2025-03-24T07:18:48.000Z (3 months ago)
- Last Synced: 2025-03-24T08:31:18.788Z (3 months ago)
- Topics: alpaca, bard, bingchat, chatbot, chatglm, chatgpt, claude, ernie, iflytek, llm, moss
- Language: TypeScript
- Homepage:
- Size: 3.67 MB
- Stars: 1
- Watchers: 1
- Forks: 2
- Open Issues: 17
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# LLM bots
[](https://github.com/pond918/llm-bots/actions)
[](https://www.npmjs.com/package/@pond918/llm-bots)
[](https://snyk.io/test/github/pond918/llm-bots?targetFile=package.json)
[](https://github.com/prettier/prettier)
[](https://www.npmjs.com/package/@pond918/llm-bots)
[](https://app.fossa.com/projects/git%2Bgithub.com%2Fpond918%2Fllm-bots?ref=badge_shield)
[](https://makeapullrequest.com)
## What is it?
An npm package of many large language model (LLMs) client chat bots, e.g. ChatGPT, Bing Chat, bard, Alpaca, Vincuna, Claude, ChatGLM, MOSS, iFlytek Spark, ERNIE and more. You may register your own bots easily.
The package enables bots to support unlimited conversations. each conversation has tree-structured chat history ( each chat message has a `lastMsgId`).
Another use case is to isolate different users for the same bot on the same nodejs server. All user related data: session/conversation/chat history, etc. is stored in the specified BotStorage on different namespaces.
## Getting Started
### Install the library
```sh
npm install --save @pond918/llm-bots
```
### using the bots
```typescript
import { ChatDto, LLMBots } from '@pond918/llm-bots'
const bots = new LLMBots()
const claudeBot = bots.instance('vicuna-13b')
// vicuna-13b needn't token
const token = null
const ready = await claudeBot?.initSession(token)
if (ready) {
const resp = await claudeBot?.sendPrompt(new ChatDto('hi there. 1 word most'))
console.log(resp)
// stream response
claudeBot?.sendPrompt(new ChatDto('who is Gauss. 5 words most'), (msg) => console.log(msg))
}
```
### register a new bot
```typescript
import { LLMBots } from '@pond918/llm-bots'
// you may apply a custom user data storage
const bots = LLMBots.factory(storage);
// new bot will replace old bot with same name.
bots.register(new MyLLMBot());
const models = LLMBots.list();
console.log(Object.keys(models));
```
## bot state management
There are 3 state for an llm bot instance:
- llm config state: e.g. llm url. Stored as bot class properties.
- server session state: API tokens/login sessions. Stored in the session pool, may be shared among users.
- user data state: conversation/chat history. Stored in the provided storage, usually users isolated.
## TODOs
- [ ] namespaced/scoped storage
## License
This project is [Apache-2.0 Licensed](LICENSE).
## Credits
- bots implementation are based on [ChatAll](https://github.com/sunner/ChatALL). Respect!
- [NodeJS Starter ToolKit](https://github.com/vitorsalgado/create-nodejs-ts)