{"id":13453911,"url":"https://github.com/transitive-bullshit/OpenOpenAI","last_synced_at":"2025-03-24T01:32:12.646Z","repository":{"id":207486923,"uuid":"716449467","full_name":"transitive-bullshit/OpenOpenAI","owner":"transitive-bullshit","description":"Self-hosted version of OpenAI’s new stateful Assistants API","archived":false,"fork":false,"pushed_at":"2024-01-10T15:00:40.000Z","size":686,"stargazers_count":533,"open_issues_count":11,"forks_count":47,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-03-21T22:08:24.273Z","etag":null,"topics":["assistants","gpts","openai","openai-api","self-hosted"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/transitive-bullshit.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":".github/funding.yml","license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["transitive-bullshit"]}},"created_at":"2023-11-09T06:51:10.000Z","updated_at":"2025-03-20T17:32:40.000Z","dependencies_parsed_at":"2023-12-31T09:25:21.756Z","dependency_job_id":"bf2b6ea2-fb7e-4a15-8d7b-fa86a3b1bc21","html_url":"https://github.com/transitive-bullshit/OpenOpenAI","commit_stats":null,"previous_names":["transitive-bullshit/openopenai"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transitive-bullshit%2FOpenOpenAI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transitive-bullshit%2FOpenOpenAI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transitive-bullshit%2FOpenOpenAI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transitive-bullshit%2FOpenOpenAI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/transitive-bullshit","download_url":"https://codeload.github.com/transitive-bullshit/OpenOpenAI/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245194349,"owners_count":20575741,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["assistants","gpts","openai","openai-api","self-hosted"],"created_at":"2024-07-31T08:00:49.206Z","updated_at":"2025-03-24T01:32:07.636Z","avatar_url":"https://github.com/transitive-bullshit.png","language":"TypeScript","funding_links":["https://github.com/sponsors/transitive-bullshit"],"categories":["TypeScript"],"sub_categories":[],"readme":"# OpenOpenAI \u003c!-- omit in toc --\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg alt=\"Example usage\" src=\"/media/screenshot.jpg\"\u003e\n\u003c/p\u003e\n\n\u003cp\u003e\n  \u003ca href=\"https://github.com/transitive-bullshit/OpenOpenAI/actions/workflows/test.yml\"\u003e\u003cimg alt=\"Build Status\" src=\"https://github.com/transitive-bullshit/OpenOpenAI/actions/workflows/test.yml/badge.svg\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/transitive-bullshit/OpenOpenAI/blob/main/license\"\u003e\u003cimg alt=\"MIT License\" src=\"https://img.shields.io/badge/license-MIT-blue\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://prettier.io\"\u003e\u003cimg alt=\"Prettier Code Formatting\" src=\"https://img.shields.io/badge/code_style-prettier-brightgreen.svg\" /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n- [Intro](#intro)\n- [Why?](#why)\n- [Stack](#stack)\n- [Development](#development)\n  - [Environment Variables](#environment-variables)\n  - [Services](#services)\n  - [E2E Examples](#e2e-examples)\n    - [Custom Function Example](#custom-function-example)\n    - [Retrieval Tool Example](#retrieval-tool-example)\n  - [Server routes](#server-routes)\n- [TODO](#todo)\n- [License](#license)\n\n## Intro\n\n**This project is a self-hosted version of OpenAI's new stateful Assistants API.** 💪\n\nAll [API route definitions](./src/generated/oai-routes.ts) and [types](./src/generated/oai.ts) are **100% auto-generated** from OpenAI's official OpenAPI spec, so all it takes to switch between the official API and your custom API is changing the `baseURL`. 🤯\n\nThis means that all API parameters, responses, and types are wire-compatible with the official OpenAI API, and the fact that they're auto-generated means that it will be relatively easy to keep them in sync over time.\n\nHere's an example using the official Node.js `openai` package:\n\n```ts\nimport OpenAI from 'openai'\n\n// The only difference is the `baseURL` pointing to your custom API server 🔥\nconst openai = new OpenAI({\n  baseURL: 'http://localhost:3000'\n})\n\n// Since the custom API is spec-compliant with OpenAI, you can use the sdk normally 💯\nconst assistant = await openai.beta.assistants.create({\n  model: 'gpt-4-1106-preview',\n  instructions: 'You are a helpful assistant.'\n})\n```\n\n\u003cdetails\u003e\n\u003csummary\u003ePython example\u003c/summary\u003e\n\nHere's the same example using the official Python `openai` package:\n\n```py\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url: \"http://localhost:3000\"\n)\n\n# Now you can use the sdk normally!\n# (only file and beta assistant resources are currently supported)\n# You can even switch back and forth between the official and custom APIs!\nassistant = client.beta.assistants.create(\n    model=\"gpt-4-1106-preview\",\n    description=\"You are a helpful assistant.\"\n)\n```\n\n\u003c/details\u003e\n\nNote that this project is not meant to be a full recreation of the entire OpenAI API. Rather, **it is focused only on the stateful portions of the new Assistants API**. The following resource types are supported:\n\n- Assistants\n- AssistantFiles\n- Files\n- Messages\n- MessageFiles\n- Threads\n- Runs\n- RunSteps\n\nSee the official [OpenAI Assistants Guide](https://platform.openai.com/docs/assistants/how-it-works) for more info on how Assistants work.\n\n## Why?\n\nBeing able to run your own, custom OpenAI Assistants that are **100% compatible with the official OpenAI Assistants** unlocks all sorts of useful possibilities:\n\n- Using OpenAI Assistants with **custom models** (OSS ftw!) 💪\n- **Fully customizable RAG** via the built-in retrieval tool (LangChain and LlamaIndex integrations [coming soon](https://github.com/transitive-bullshit/OpenOpenAI/issues/2))\n- Using a **custom code interpreter** like [open-interpreter](https://github.com/KillianLucas/open-interpreter) 🔥\n- **Self-hosting / on-premise** deployments of Assistants\n- Full control over **assistant evals**\n- Developing \u0026 testing GPTs in fully **sandboxed environments**\n- Sandboxed testing of **custom Actions** before deploying to the OpenAI \"GPT Store\"\n\nMost importantly, if the OpenAI \"GPT Store\" ends up gaining traction with ChatGPT's 100M weekly active users, then **the ability to reliably run, debug, and customize OpenAI-compatible Assistants** will end up being incredibly important in the future.\n\nI could even imagine a future Assistant store which is fully compatible with OpenAI's GPTs, but instead of relying on OpenAI as the gatekeeper, it could be **fully or partially decentralized**. 💯\n\n## Stack\n\n- [Postgres](https://www.postgresql.org) - Primary datastore via [Prisma](https://www.prisma.io) ([schema file](./prisma/schema.prisma))\n- [Redis](https://redis.io) - Backing store for the async task queue used to process thread runs via [BullMQ](https://bullmq.io)\n- [S3](https://aws.amazon.com/s3) - Stores uploaded files\n  - Any S3-compatible storage provider is supported, such as [Cloudflare R2](https://developers.cloudflare.com/r2/)\n- [Hono](https://hono.dev) - Serves the REST API via [@hono/zod-openapi](https://github.com/honojs/middleware/tree/main/packages/zod-openapi)\n  - We're using the [Node.js](https://hono.dev/getting-started/nodejs) adaptor by default, but Hono supports many environments including CF workers, Vercel, Netlify, Deno, Bun, Lambda, etc.\n- [Dexter](https://github.com/dexaai/dexter) - Production RAG by [Dexa](https://dexa.ai)\n- [TypeScript](https://www.typescriptlang.org) 💕\n\n## Development\n\nPrerequisites:\n\n- [node](https://nodejs.org/en) \u003e= 18\n- [pnpm](https://pnpm.io) \u003e= 8\n\nInstall deps:\n\n```bash\npnpm install\n```\n\nGenerate the prisma types locally:\n\n```bash\npnpm generate\n```\n\n### Environment Variables\n\n```bash\ncp .env.example .env\n```\n\n- **Postgres**\n  - `DATABASE_URL` - Postgres connection string\n  - [On macOS](https://wiki.postgresql.org/wiki/Homebrew): `brew install postgresql \u0026\u0026 brew services start postgresql`\n  - You'll need to run `npx prisma db push` to set up your database according to our [prisma schema](./prisma/schema.prisma)\n- **OpenAI**\n  - `OPENAI_API_KEY` - OpenAI API key for running the underlying chat completion calls\n  - This is required for now, but depending on [how interested people are](https://github.com/transitive-bullshit/OpenOpenAI/issues/1), it won't be hard to add support for local models and other providers\n- **Redis**\n  - [On macOS](https://redis.io/docs/install/install-redis/install-redis-on-mac-os/): `brew install redis \u0026\u0026 brew services start redis`\n  - If you have a local redis instance running, the default redis env vars should work without touching them\n  - `REDIS_HOST` - Optional; defaults to `localhost`\n  - `REDIS_PORT` - Optional; defaults to `6379`\n  - `REDIS_USERNAME` - Optional; defaults to `default`\n  - `REDIS_PASSWORD` - Optional\n- **S3** - Required to use file attachments\n  - Any S3-compatible provider is supported, such as [Cloudflare R2](https://developers.cloudflare.com/r2/)\n  - Alterantively, you can use a local S3 server like [MinIO](https://github.com/minio/minio#homebrew-recommended) or [LocalStack](https://github.com/localstack/localstack)\n    - To run LocalStack on macOS: `brew install localstack/tap/localstack-cli \u0026\u0026 localstack start -d`\n    - To run MinIO macOS: `brew install minio/stable/minio \u0026\u0026 minio server /data`\n  - I recommend using Cloudflare R2, though – it's amazing and should be free for most use cases!\n  - `S3_BUCKET` - Required\n  - `S3_REGION` - Optional; defaults to `auto`\n  - `S3_ENDPOINT` - Required; example: `https://\u003cid\u003e.r2.cloudflarestorage.com`\n  - `ACCESS_KEY_ID` - Required ([cloudflare R2 docs](https://developers.cloudflare.com/r2/api/s3/tokens/))\n  - `SECRET_ACCESS_KEY` - Required ([cloudflare R2 docs](https://developers.cloudflare.com/r2/api/s3/tokens/))\n\n### Services\n\nThe app is composed of two services: a RESTful API **server** and an async task **runner**. Both services are stateless and can be scaled horizontally.\n\nThere are two ways to run these services locally. The quickest way is via `tsx`:\n\n```bash\n# Start the REST API server in one shell\nnpx tsx src/server\n\n# Start an async task queue runner in another shell\nnpx tsx src/runner\n```\n\nAlternatively, you can transpile the source TS to JS first, which is preferred for running in production:\n\n```bash\npnpm build\n\n# Start the REST API server in one shell\nnpx tsx dist/server\n\n# Start an async task queue runner in another shell\nnpx tsx dist/runner\n```\n\n### E2E Examples\n\n#### Custom Function Example\n\n[This example](./e2e/index.ts) contains an end-to-end assistant script which uses a custom `get_weather` function.\n\nYou can run it using the official [openai](https://github.com/openai/openai-node) client for Node.js against the default OpenAI API hosted at `https://api.openai.com/v1`.\n\n```bash\nnpx tsx e2e\n```\n\nTo run the same test suite against your local API, you can run:\n\n```bash\nOPENAI_API_BASE_URL='http://127.0.0.1:3000' npx tsx e2e\n```\n\nIt's pretty cool to see both test suites running the exact same Assistants code using the official OpenAI Node.js client – without any noticeable differences between the two versions. Huzzah! 🥳\n\n#### Retrieval Tool Example\n\n[This example](./e2e/retrieval.ts) contains an end-to-end assistant script which uses the built-in `retrieval` tool with this `readme.md` file as an attachment.\n\nYou can run it using the official [openai](https://github.com/openai/openai-node) client for Node.js against the default OpenAI API hosted at `https://api.openai.com/v1`.\n\n```bash\nnpx tsx e2e/retrieval.ts\n```\n\nTo run the same test suite against your local API, you can run:\n\n```bash\nOPENAI_API_BASE_URL='http://127.0.0.1:3000' npx tsx e2e/retrieval.ts\n```\n\nThe output will likely differ slightly due to differences in OpenAI's built-in retrieval implementation and [our default, naive retrieval implementation](./src/lib/retrieval.ts).\n\nNote that the [current `retrieval` implementation](https://github.com/transitive-bullshit/OpenOpenAI/blob/main/src/lib/retrieval.ts) only support text files like `text/plain` and markdown, as no preprocessing or conversions are done at the moment. We also use a very naive retrieval method at the moment which always returns the full file contents as opposed to pre-processing them and only returning the most semantically relevant chunks. See [this issue](https://github.com/transitive-bullshit/OpenOpenAI/issues/2) for more info.\n\n### Server routes\n\n```\nGET       /files\nPOST      /files\nDELETE    /files/:file_id\nGET       /files/:file_id\nGET       /files/:file_id/content\nGET       /assistants\nPOST      /assistants\nGET       /assistants/:assistant_id\nPOST      /assistants/:assistant_id\nDELETE    /assistants/:assistant_id\nGET       /assistants/:assistant_id/files\nGET       /assistants/:assistant_id/files\nPOST      /assistants/:assistant_id/files\nDELETE    /assistants/:assistant_id/files/:file_id\nGET       /assistants/:assistant_id/files/:file_id\nPOST      /threads\nGET       /threads/:thread_id\nPOST      /threads/:thread_id\nDELETE    /threads/:thread_id\nGET       /threads/:thread_id/messages\nPOST      /threads/:thread_id/messages\nGET       /threads/:thread_id/messages/:message_id\nPOST      /threads/:thread_id/messages/:message_id\nGET       /threads/:thread_id/messages/:message_id/files\nGET       /threads/:thread_id/messages/:message_id/files/:file_id\nGET       /threads/:thread_id/runs\nPOST      /threads/runs\nPOST      /threads/:thread_id/runs\nGET       /threads/:thread_id/runs/:run_id\nPOST      /threads/:thread_id/runs/:run_id\nPOST      /threads/:thread_id/runs/:run_id/submit_tool_outputs\nPOST      /threads/:thread_id/runs/:run_id/cancel\nGET       /threads/:thread_id/runs/:run_id/steps\nGET       /threads/:thread_id/runs/:run_id/steps/:step_id\nGET       /openapi\n```\n\nYou can view the server's auto-generated openapi spec by running the server and then visiting `http://127.0.0.1:3000/openapi`\n\n## TODO\n\n**Status**: All API routes have been tested side-by-side with the official OpenAI API and are working as expected. The only missing features at the moment are support for the built-in `code_interpreter` tool ([issue](https://github.com/transitive-bullshit/OpenOpenAI/issues/3)) and support for non-text files with the built-in `retrieval` tool ([issue](https://github.com/transitive-bullshit/OpenOpenAI/issues/2)). All other functionality should be fully supported and wire-compatible with the official API.\n\n**TODO**:\n\n- hosted demo (bring your own OpenAI API key?)\n- get hosted redis working\n- handle locking thread and messages\n  - not sure how this works exactly, but according to the [OpenAI Assistants Guide](https://platform.openai.com/docs/assistants/how-it-works/runs-and-run-steps), threads are locked while runs are being processed\n- built-in `code_interpreter` tool ([issue](https://github.com/transitive-bullshit/OpenOpenAI/issues/3))\n- support non-text files w/ built-in `retrieval` tool ([issue](https://github.com/transitive-bullshit/OpenOpenAI/issues/2))\n- openai uses prefix IDs for its resources, which would be great, except it's a pain to get working with Prisma ([issue](https://github.com/transitive-bullshit/OpenOpenAI/issues/7))\n- figure out why localhost resolution wasn't working for [#6](https://github.com/transitive-bullshit/OpenOpenAI/pull/6)\n- handle context overflows (truncation for now)\n\n## License\n\nMIT © [Travis Fischer](https://transitivebullsh.it)\n\nIf you found this project useful, please consider [sponsoring me](https://github.com/sponsors/transitive-bullshit) or \u003ca href=\"https://twitter.com/transitive_bs\"\u003efollowing me on twitter \u003cimg src=\"https://storage.googleapis.com/saasify-assets/twitter-logo.svg\" alt=\"twitter\" height=\"24px\" align=\"center\"\u003e\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftransitive-bullshit%2FOpenOpenAI","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftransitive-bullshit%2FOpenOpenAI","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftransitive-bullshit%2FOpenOpenAI/lists"}