https://github.com/nishtahir/openai-pool

https://github.com/nishtahir/openai-pool

Last synced: 23 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/nishtahir/openai-pool
• Owner: nishtahir
• License: apache-2.0
• Created: 2024-01-27T21:39:31.000Z (11 months ago)
• Default Branch: main
• Last Pushed: 2024-01-27T21:40:28.000Z (11 months ago)
• Last Synced: 2024-04-16T12:46:50.160Z (9 months ago)
• Language: TypeScript
• Size: 11.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# openai-pool

A simple round robin pool for OpenAI clients.

## Usage

```js

import { OpenAIPoolBuilder } from "openai-pool";

const clientA = new OpenAI({ apiKey: "your key" });

const clientB = new OpenAI({ apiKey: "your key" });

const pool = new OpenAIPoolBuilder()

  .withClient(clientA)

  .withClient(clientB)

  .create();

// The pool mimics the OpenAI client API

// but invokations are round robin load balanced between client instances

// Uses clientA

pool.chat.completions.create({

  model: "gpt-3.5-turbo",

  messages: [],

});

// Uses clientB

pool.chat.completions.create({

  model: "gpt-3.5-turbo",

  messages: [],

});

// Uses clientA

pool.embeddings.create({

  model: "text-embedding-ada-002",

  input: [],

});

```