{"id":13572567,"url":"https://github.com/OpenPipe/OpenPipe","last_synced_at":"2025-04-04T10:30:48.354Z","repository":{"id":176944844,"uuid":"655949187","full_name":"OpenPipe/OpenPipe","owner":"OpenPipe","description":"Turn expensive prompts into cheap fine-tuned models","archived":false,"fork":false,"pushed_at":"2024-05-25T00:18:13.000Z","size":12211,"stargazers_count":2557,"open_issues_count":5,"forks_count":136,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-03-28T08:07:43.573Z","etag":null,"topics":["ai","llm","llmops","prompt-engineering"],"latest_commit_sha":null,"homepage":"https://openpipe.ai","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenPipe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-20T00:55:24.000Z","updated_at":"2025-03-27T02:27:59.000Z","dependencies_parsed_at":"2023-11-25T22:20:24.545Z","dependency_job_id":"0a9c0463-27d7-4387-bb6a-1388122987be","html_url":"https://github.com/OpenPipe/OpenPipe","commit_stats":null,"previous_names":["corbt/prompt-lab","corbt/openpipe","openpipe/openpipe"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenPipe%2FOpenPipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenPipe%2FOpenPipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenPipe%2FOpenPipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenPipe%2FOpenPipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenPipe","download_url":"https://codeload.github.com/OpenPipe/OpenPipe/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247160240,"owners_count":20893796,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","llm","llmops","prompt-engineering"],"created_at":"2024-08-01T14:01:26.949Z","updated_at":"2025-04-04T10:30:48.311Z","avatar_url":"https://github.com/OpenPipe.png","language":"TypeScript","funding_links":[],"categories":["TypeScript","Repos"],"sub_categories":[],"readme":"## **Note:** we’ve temporarily stopped development on the open-source version of OpenPipe to integrate some proprietary third-party code. We hope to make the non-proprietary parts of the repository open again under an open core model once we have the bandwidth to do so!\n\n \u003cp align=\"center\"\u003e\n  \u003ca href=\"https://openpipe.ai\"\u003e\n    \u003cimg height=\"70\" src=\"https://github.com/openpipe/openpipe/assets/41524992/70af25fb-1f90-42d9-8a20-3606e3b5aaba\" alt=\"logo\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003ch1 align=\"center\"\u003e\n  OpenPipe\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ci\u003eOpen-source fine-tuning and model-hosting platform.\u003c/i\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"/LICENSE\"\u003e\u003cimg alt=\"License Apache-2.0\" src=\"https://img.shields.io/github/license/openpipe/openpipe?style=flat-square\"\u003e\u003c/a\u003e\n  \u003ca href='http://makeapullrequest.com'\u003e\u003cimg alt='PRs Welcome' src='https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square'/\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/openpipe/openpipe/graphs/commit-activity\"\u003e\u003cimg alt=\"GitHub commit activity\" src=\"https://img.shields.io/github/commit-activity/m/openpipe/openpipe?style=flat-square\"/\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/openpipe/openpipe/issues\"\u003e\u003cimg alt=\"GitHub closed issues\" src=\"https://img.shields.io/github/issues-closed/openpipe/openpipe?style=flat-square\"/\u003e\u003c/a\u003e\n \u003cimg src=\"https://img.shields.io/badge/Y%20Combinator-S23-orange?style=flat-square\" alt=\"Y Combinator S23\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://app.openpipe.ai/p/BRZFEx50Pf/request-logs\"\u003eDemo\u003c/a\u003e - \u003ca href=\"#running-locally\"\u003eRunning Locally\u003c/a\u003e - \u003ca href=\"https://docs.openpipe.ai\"\u003eDocs\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cbr\u003e\nUse powerful but expensive LLMs to fine-tune smaller and cheaper models suited to your exact needs. Query your past requests and evaluate models against one another. Switch between OpenAI and fine-tuned models with one line of code.\n\u003cbr\u003e\n\n## Features\n\n- Easy integration with OpenAI's SDK in both Python and TypeScript.\n  - [Python SDK](https://pypi.org/project/openpipe/)\n  - [TypeScript SDK](https://www.npmjs.com/package/openpipe)\n- OpenAI-compatible chat completions endpoint.\n- Fine-tune GPT 3.5, Mistral, and Llama 2 models. Host on-platform or download the weights.\n  - Model output is OpenAI-compatible.\n  - Switching from GPT 4 to a fine-tuned Mistral model only requires changing the model name.\n- Query logs using powerful built-in filters.\n- Import datasets in OpenAI-compatible JSONL files.\n- Prune large chunks of duplicate text like system prompts.\n- Compare output accuracy against base models like gpt-3.5-turbo.\n\n## Supported Base Models\n\n- [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)\n- [OpenPipe/mistral-ft-optimized-1227](https://huggingface.co/OpenPipe/mistral-ft-optimized-1227)\n- [meta-llama/Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B)\n- [meta-llama/Llama-3-70B](https://huggingface.co/meta-llama/Meta-Llama-3-70B)\n- [gpt-3.5-turbo-0613](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates)\n- [gpt-3.5-turbo-1106](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates)\n- [gpt-3.5-turbo-0125](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates)\n\n## Documentation\n\n- See [docs](https://docs.openpipe.ai/introduction)\n\n## Running Locally\n\n1. Install [Postgresql](https://www.postgresql.org/download/).\n2. Install [NodeJS 20](https://nodejs.org/en/download/current) (earlier versions will very likely work but aren't tested).\n3. Install `pnpm`: `npm i -g pnpm`\n4. Clone this repository: `git clone https://github.com/openpipe/openpipe`\n5. Install the dependencies: `cd openpipe \u0026\u0026 pnpm install`\n6. Create a `.env` file (`cd app \u0026\u0026 cp .env.example .env`) and enter your `OPENAI_API_KEY`.\n7. If you just installed postgres and wish to use the default `DATABASE_URL` run the following commands:\n\n```sh\npsql postgres\nCREATE ROLE postgres WITH LOGIN PASSWORD 'postgres';\nALTER ROLE postgres SUPERUSER;\n```\n\n8. Update `DATABASE_URL` if necessary to point to your Postgres instance and run `pnpm prisma migrate dev` in the `app` directory to create the database.\n9. Create a [GitHub OAuth App](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/creating-an-oauth-app), set the callback URL to `\u003cyour local instance\u003e/api/auth/callback/github`, e.g. `http://localhost:3000/api/auth/callback/github`.\n10. Update the `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET` values from the Github OAuth app (Note: a PR to make auth optional when running locally would be a great contribution!).\n11. To start the app run `pnpm dev` in the `app` directory.\n12. Navigate to [http://localhost:3000](http://localhost:3000)\n\n## Using Locally\n\n```sh\nimport os\nfrom openpipe import OpenAI\n\nclient = OpenAI(\n    api_key=\"Your API Key\",\n    openpipe={\n        \"api_key\": \"Your OpenPipe API Key\",\n        \"base_url\": \"http://localhost:3000/api/v1\", # Local OpenPipe instance\n    }\n)\n\ncompletion = client.chat.completions.create(\n    model=\"gpt-3.5-turbo\",\n    messages=[{\"role\": \"system\", \"content\": \"count to 10\"}],\n    openpipe={\n        \"tags\": {\"prompt_id\": \"counting\"},\n        \"log_request\": True\n    },\n)\n```\n\n## Testing Locally\n\n1. Copy your `.env` file to `.env.test`.\n2. Update the `DATABASE_URL` to have a different database name than your development one\n3. Run `DATABASE_URL=[your new datatase url] pnpm prisma migrate dev --skip-seed --skip-generate`\n4. Run `pnpm test`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenPipe%2FOpenPipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FOpenPipe%2FOpenPipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenPipe%2FOpenPipe/lists"}