https://github.com/r2d4/llm.ts
Call any LLM with a single API. Zero dependencies.
https://github.com/r2d4/llm.ts
ai cohere huggingface llm llms openai
Last synced: about 1 month ago
JSON representation
Call any LLM with a single API. Zero dependencies.
- Host: GitHub
- URL: https://github.com/r2d4/llm.ts
- Owner: r2d4
- License: mit
- Created: 2023-05-03T02:07:24.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-05-09T21:20:25.000Z (over 2 years ago)
- Last Synced: 2025-05-08T23:33:37.869Z (5 months ago)
- Topics: ai, cohere, huggingface, llm, llms, openai
- Language: TypeScript
- Homepage:
- Size: 33.2 KB
- Stars: 210
- Watchers: 7
- Forks: 10
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# llm.ts
Call 30+ LLMs with a single API.
* Send multiple prompts to multiple LLMs and get the results back in a single response.
* Zero dependencies (under 10kB minified)
* Bring your own API keys
* Works anywhere (Node, Deno, browser)
### Install
#### npm
```bash
npm install llm.ts
```
#### yarn
```bash
yarn add llm.ts
```
### [Example](examples/simple)
```typescript
import { LLM, MODEL } from 'llm.ts';
(async function () {
await new LLM({
apiKeys: {
openAI: process.env.OPENAI_API_KEY ?? '',
cohere: process.env.COHERE_API_KEY ?? '',
huggingface: process.env.HF_API_TOKEN ?? '',
}
}).completion({
prompt: [
'Repeat the following sentence: "I am a robot."',
'Repeat the following sentence: "I am a human."',
],
model: [
// use the model name
'text-ada-001',
// or specify a specific provider
'cohere/command-nightly',
// or use enums to avoid typos
MODEL.HF_GPT2,
],
}).then(resp => {
console.log(resp);
})
})()
```
```json
{
"created": 1683079463217,
"choices": [
{
"text": "\n\nI am a robot.",
"index": 0,
"model": "text-ada-001",
"promptIndex": 0,
"created": 1683079462
},
{
"text": "\n\nI am a human.",
"index": 1,
"model": "text-ada-001",
"promptIndex": 1,
"created": 1683079462
},
{
"text": "\nI am a robot.",
"index": 2,
"model": "command-nightly",
"promptIndex": 0,
"created": 1683079463217
},
{
"text": "\nI am a human.",
"index": 3,
"model": "command-nightly",
"promptIndex": 1,
"created": 1683079463216
},
{
"text": " \"Is that your question? I was expecting the answer.\" \"Then why do you think you are being asked!\" 1. \"What are you?\" \"What are you?\" \"Why are you",
"index": 4,
"model": "gpt2",
"promptIndex": 0,
"created": 1683079463088
},
{
"text": " — this quote is most often cited in reference to the Qur'an. (e.g. Ibn `Allaahu `udayyyih, Al-Rai`an, Al",
"index": 5,
"model": "gpt2",
"promptIndex": 1,
"created": 1683079463091
}
]
}
```
### Models supported (want to add one, open a PR!):
* text-ada-001
* text-babbage-001
* text-curie-001
* text-davinci-002
* text-davinci-003
* cohere-command
* cohere-command-nightly
* cohere-command-light
* cohere-command-light-nightly
* gpt2
* bloom-1b
* bloom-3b
* bloom-7b1
* llama-7b
* llama-13b
* llama-30b
* llama-65b
* gptj-6b
* gptj-2.7b
* gpt-neo-125m
* gpt-neo-1.3b
* gpt-neo-20b
* cerebras-gpt-111m
* cerebras-gpt-1.3b
* cerebras-gpt-2.7b
* santacoder
* codegen-350m
* codegen-2b
* stablelm-tuned-3b
* stablelm-tuned-7b
* pythia-70m
* pythia-160m
* pythia-12b
* distilgpt2
### Hosting providers supported (want to add one, open a PR!):
* OpenAI
* Cohere
* HuggingFace
### License
MIT