https://github.com/spellcraftai/openai-streams
Tools for working with OpenAI streams in Node.js and TypeScript.
https://github.com/spellcraftai/openai-streams
openai streams
Last synced: 2 months ago
JSON representation
Tools for working with OpenAI streams in Node.js and TypeScript.
- Host: GitHub
- URL: https://github.com/spellcraftai/openai-streams
- Owner: SpellcraftAI
- License: mit
- Created: 2023-02-15T00:33:31.000Z (over 2 years ago)
- Default Branch: canary
- Last Pushed: 2023-07-20T20:15:40.000Z (almost 2 years ago)
- Last Synced: 2025-04-06T05:54:03.949Z (2 months ago)
- Topics: openai, streams
- Language: TypeScript
- Homepage: https://openai-streams.vercel.app
- Size: 343 KB
- Stars: 290
- Watchers: 2
- Forks: 26
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# OpenAI Streams
[**GitHub**](https://github.com/SpellcraftAI/openai-streams) |
[**NPM**](https://npmjs.com/package/openai-streams) |
[**Docs**](https://openai-streams.vercel.app)
> Now with ChatGPT API support! See [**Use with ChatGPT
> API**](#use-with-chatgpt-api). (Whisper coming soon!)
This library returns OpenAI API responses as streams only. Non-stream endpoints
like `edits` etc. are simply a stream with only one chunk update.
- Prioritizes streams, so you can display a completion as it arrives.
- Auto-loads `OPENAI_API_KEY` from `process.env`.
- One single function with inferred parameter type based on the endpoint you
provide.
Uses `ReadableStream` by default for browser, Edge Runtime, and Node 18+, with
a `NodeJS.Readable` version available at `openai-streams/node`.
### Installation
```bash
yarn add openai-streams
# -or-
npm i --save openai-streams
```
### Usage
```ts
await OpenAI(
/** 'completions', 'chat', etc. */
ENDPOINT,
/** max_tokens, temperature, messages, etc. */
PARAMS,
/** apiBase, apiKey, mode, controller, etc */
OPTIONS
);
```
1. **Set the `OPENAI_API_KEY` env variable** (or pass the `{ apiKey }` option).
The library will throw if it cannot find an API key. Your program will load
this at runtime from `process.env.OPENAI_API_KEY` by default, but you may
override this with the `{ apiKey }` option.
**IMPORTANT:** For security, you should only load this from a `process.env`
variable.
```ts
await OpenAI(
"completions",
{
/* endpoint params */
},
{ apiKey: process.env.MY_SECRET_API_KEY }
);
```
2. **Call the API via `await OpenAI(endpoint, params, options?)`.**
The `params` type will be inferred based on the `endpoint` you provide, i.e.
for the `"edits"` endpoint, `import('openai').CreateEditRequest` will be
enforced.
Example with `raw` streaming mode:
```ts
await OpenAI(
"chat",
{
messages: [
/* ... */
],
},
{ mode: "raw" }
);
```
#### Edge/Browser: Consuming streams in Next.js Edge functions
This will also work in the browser, but you'll need users to paste their OpenAI
key and pass it in via the `{ apiKey }` option.
```ts
import { OpenAI } from "openai-streams";
export default async function handler() {
const stream = await OpenAI("completions", {
model: "text-davinci-003",
prompt: "Write a happy sentence.\n\n",
max_tokens: 100,
});
return new Response(stream);
}
export const config = {
runtime: "edge",
};
```
#### Node: Consuming streams in Next.js API Route (Node)
If you cannot use an Edge runtime or want to consume Node.js streams for another
reason, use `openai-streams/node`:
```ts
import type { NextApiRequest, NextApiResponse } from "next";
import { OpenAI } from "openai-streams/node";
export default async function test(_: NextApiRequest, res: NextApiResponse) {
const stream = await OpenAI("completions", {
model: "text-davinci-003",
prompt: "Write a happy sentence.\n\n",
max_tokens: 25,
});
stream.pipe(res);
}
```
See the example in
[`example/src/pages/api/hello.ts`](https://github.com/SpellcraftAI/openai-streams/blob/master/example/src/pages/api/hello.ts).
#### Use with ChatGPT API
By default, with `mode = "tokens"`, you will receive just the message deltas.
For full events, use `mode = "raw"`.
See: https://platform.openai.com/docs/guides/chat/introduction
```ts
const stream = await OpenAI("chat", {
model: "gpt-3.5-turbo",
messages: [
{
role: "system",
content: "You are a helpful assistant that translates English to French.",
},
{
role: "user",
content: 'Translate the following English text to French: "Hello world!"',
},
],
});
```
In `tokens` mode, you will just receive the response chunks, which look like this
(separated with newlines for illustration):
```
Hello
!
How
can
I
assist
you
today
?
```
Use `mode = "raw"` for access to raw events.
### Notes
1. Internally, streams are often manipulated using generators via `for await
(const chunk of yieldStream(stream)) { ... }`. We recommend following this
pattern if you find it intuitive.