{"id":18464903,"url":"https://github.com/neondatabase/db-per-tenant","last_synced_at":"2025-04-08T08:31:34.176Z","repository":{"id":253423789,"uuid":"842872629","full_name":"neondatabase/db-per-tenant","owner":"neondatabase","description":"Example chat-with-pdf app showing how to provision a dedicated database instance for each user. In this app, every database uses pgvector for similarity search. Powered by Neon","archived":false,"fork":false,"pushed_at":"2024-11-19T14:23:00.000Z","size":1327,"stargazers_count":44,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-03T03:53:40.532Z","etag":null,"topics":["ai","multitenancy","pgvector","postgres","postgresql","vector-database"],"latest_commit_sha":null,"homepage":"https://db-per-tenant.up.railway.app","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/neondatabase.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-15T09:27:59.000Z","updated_at":"2025-03-31T07:27:48.000Z","dependencies_parsed_at":"2024-08-16T16:43:54.440Z","dependency_job_id":"46124988-a865-440b-ae3e-09dd9f6e9ffd","html_url":"https://github.com/neondatabase/db-per-tenant","commit_stats":null,"previous_names":["neondatabase/ai-vector-db-per-tenant","neondatabase/db-per-tenant"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neondatabase%2Fdb-per-tenant","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neondatabase%2Fdb-per-tenant/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neondatabase%2Fdb-per-tenant/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neondatabase%2Fdb-per-tenant/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/neondatabase","download_url":"https://codeload.github.com/neondatabase/db-per-tenant/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247804486,"owners_count":20998993,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","multitenancy","pgvector","postgres","postgresql","vector-database"],"created_at":"2024-11-06T09:11:28.349Z","updated_at":"2025-04-08T08:31:34.168Z","avatar_url":"https://github.com/neondatabase.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"## AI app architecture: vector database per tenant\n\nThis repo contains an example of a scalable architecture for AI-powered applications. On the surface, it's an AI app where users can upload PDFs and chat with them. However, under the hood, each user gets a dedicated vector database instance (Postgres on [Neon](https://neon.tech/?ref=github) with pgvector).\n\nYou can check out the live version at https://db-per-tenant.up.railway.app/\n\n![Demo app](https://github.com/user-attachments/assets/d9dee48f-a6d6-4dd5-bb89-fa5d31ca26e3)\n\nThe app is built using the following technologies:\n\n- [Neon](https://neon.tech/ref=github) - Fully managed Postgres\n- [Remix](https://remix.run) - Full-stack React framework\n- [Remix Auth](https://github.com/sergiodxa/remix-auth) - Authentication\n- [Drizzle ORM](https://drizzle.team/) - TypeScript ORM\n- [Railway](https://railway.app) - Deployment Platform\n- [Vercel AI SDK](sdk.vercel.ai/) -  TypeScript toolkit for building AI-powered applications\n- [Cloudflare R2](https://www.cloudflare.com/developer-platform/r2/) - Object storage\n- [OpenAI](https://openai.com) with gpt-4o-mini - LLM\n- [Upstash](https://upstash.com) - Redis for rate limiting\n- [Langchain](https://js.langchain.com/v0.2/docs/introduction/) - Framework for developing applications powered by large language models (LLMs)\n\n## How it works\n\nRather than having all vector embeddings stored in a single Postgres database, you provide each tenant (a user, an organization, a workspace, or any other entity requiring isolation) with its own dedicated Postgres database instance where you can store and query its embeddings.\n\nDepending on your application, you will provision a vector database after a specific event (e.g., user signup, organization creation, or upgrade to paid tier). You will then track tenants and their associated vector databases in your application's main database. \n\nThis approach offers several benefits:\n1. Each tenant's data is stored in a separate, isolated database not shared with other tenants. This makes it possible for you to be compliant with data residency requirements (e.g., GDPR)\n2. Database resources can be allocated based on each tenant's requirements. \n3. A tenant with a large workload that can impact the database's performance won't affect other tenants; it would also be easier to manage.\n\nHere's the database architecture diagram of the demo app that's in this repo:\n\n![Architecture Diagram](https://github.com/user-attachments/assets/c788d581-1d0a-4201-842e-a20bd498e3db)\n\nThe main application's database consists of three tables: `documents`, `users`, and `vector_databases`.\n\n- The `documents` table stores information about files, including their titles, sizes, and timestamps, and is linked to users via a foreign key.\n- The `users` table maintains user profiles, including names, emails, and avatar URLs.\n- The `vector_databases` table tracks which vector database belongs to which user.\n\nThen, each vector database that gets provisioned has an `embeddings` table for storing document chunks for retrieval-augmented generation (RAG).\n\nFor this app, vector databases are provisioned when a user signs up. Once they upload a document, it gets chunked and stored in their dedicated vector database. Finally, once the user chats with their document, the vector similarity search runs against their database to retrieve the relevant information to answer their prompt.\n\n\u003cdetails\u003e\n  \u003csummary\u003eCode snippet example of provisioning a vector database\u003c/summary\u003e\n   \n   ![Provision Vector database for each signup](https://github.com/user-attachments/assets/01e31752-cddb-45c5-b595-92c3cb815a88)\n\n  ```ts\n  // Code from app/lib/auth.ts\n\n  authenticator.use(\n\tnew GoogleStrategy(\n\t\t{\n\t\t\tclientID: process.env.GOOGLE_CLIENT_ID,\n\t\t\tclientSecret: process.env.GOOGLE_CLIENT_SECRET,\n\t\t\tcallbackURL: process.env.GOOGLE_CALLBACK_URL,\n\t\t},\n\t\tasync ({ profile }) =\u003e {\n\t\t\tconst email = profile.emails[0].value;\n\n\t\t\ttry {\n\t\t\t\tconst userData = await db\n\t\t\t\t\t.select({\n\t\t\t\t\t\tuser: users,\n\t\t\t\t\t\tvectorDatabase: vectorDatabases,\n\t\t\t\t\t})\n\t\t\t\t\t.from(users)\n\t\t\t\t\t.leftJoin(vectorDatabases, eq(users.id, vectorDatabases.userId))\n\t\t\t\t\t.where(eq(users.email, email));\n\n\t\t\t\tif (\n\t\t\t\t\tuserData.length === 0 ||\n\t\t\t\t\t!userData[0].vectorDatabase ||\n\t\t\t\t\t!userData[0].user\n\t\t\t\t) {\n\t\t\t\t\tconst { data, error } = await neonApiClient.POST(\"/projects\", {\n\t\t\t\t\t\tbody: {\n\t\t\t\t\t\t\tproject: {},\n\t\t\t\t\t\t},\n\t\t\t\t\t});\n\n\t\t\t\t\tif (error) {\n\t\t\t\t\t\tthrow new Error(`Failed to create Neon project, ${error}`);\n\t\t\t\t\t}\n\n\t\t\t\t\tconst vectorDbId = data?.project.id;\n\n\t\t\t\t\tconst vectorDbConnectionUri = data.connection_uris[0]?.connection_uri;\n\n\t\t\t\t\tconst sql = postgres(vectorDbConnectionUri);\n\n\t\t\t\t\tawait sql`CREATE EXTENSION IF NOT EXISTS vector;`;\n\n\t\t\t\t\tawait migrate(drizzle(sql), { migrationsFolder: \"./drizzle\" });\n\n\t\t\t\t\tconst newUser = await db\n\t\t\t\t\t\t.insert(users)\n\t\t\t\t\t\t.values({\n\t\t\t\t\t\t\temail,\n\t\t\t\t\t\t\tname: profile.displayName,\n\t\t\t\t\t\t\tavatarUrl: profile.photos[0].value,\n\t\t\t\t\t\t\tuserId: generateId({ object: \"user\" }),\n\t\t\t\t\t\t})\n\t\t\t\t\t\t.onConflictDoNothing()\n\t\t\t\t\t\t.returning();\n\n\t\t\t\t\tawait db\n\t\t\t\t\t\t.insert(vectorDatabases)\n\t\t\t\t\t\t.values({\n\t\t\t\t\t\t\tvectorDbId,\n\t\t\t\t\t\t\tuserId: newUser[0].id,\n\t\t\t\t\t\t})\n\t\t\t\t\t\t.returning();\n\n\t\t\t\t\tconst result = {\n\t\t\t\t\t\t...newUser[0],\n\t\t\t\t\t\tvectorDbId,\n\t\t\t\t\t};\n\n\t\t\t\t\treturn result;\n\t\t\t\t}\n\n\t\t\t\treturn {\n\t\t\t\t\t...userData[0].user,\n\t\t\t\t\tvectorDbId: userData[0].vectorDatabase.vectorDbId,\n\t\t\t\t};\n\t\t\t} catch (error) {\n\t\t\t\tconsole.error(\"User creation error:\", error);\n\t\t\t\tthrow new Error(getErrorMessage(error));\n\t\t\t}\n\t\t},\n\t),\n);\n\n  ```\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n  \u003csummary\u003eCode snippet and diagram of RAG\u003c/summary\u003e\n\t\n![Vector database per tenant RAG](https://github.com/user-attachments/assets/43e0f872-6bab-4a06-8208-7871723f1fd0)\n\n  ```ts\n// Code from app/routes/api/document/chat\n// Get the user's messages and the document ID from the request body.\nconst {\n\t\tmessages,\n\t\tdocumentId,\n\t}: {\n\t\tmessages: Message[];\n\t\tdocumentId: string;\n\t} = await request.json();\n\n\tconst { content: prompt } = messages[messages.length - 1];\n\n\tconst { data, error } = await neonApiClient.GET(\n\t\t\"/projects/{project_id}/connection_uri\",\n\t\t{\n\t\t\tparams: {\n\t\t\t\tpath: {\n\t\t\t\t\tproject_id: user.vectorDbId,\n\t\t\t\t},\n\t\t\t\tquery: {\n\t\t\t\t\trole_name: \"neondb_owner\",\n\t\t\t\t\tdatabase_name: \"neondb\",\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t);\n\n\tif (error) {\n\t\treturn json({\n\t\t\terror: error,\n\t\t});\n\t}\n\n\tconst embeddings = new OpenAIEmbeddings({\n\t\tapiKey: process.env.OPENAI_API_KEY,\n\t\tdimensions: 1536,\n\t\tmodel: \"text-embedding-3-small\",\n\t});\n\n\tconst vectorStore = await NeonPostgres.initialize(embeddings, {\n\t\tconnectionString: data.uri,\n\t\ttableName: \"embeddings\",\n\t\tcolumns: {\n\t\t\tcontentColumnName: \"content\",\n\t\t\tmetadataColumnName: \"metadata\",\n\t\t\tvectorColumnName: \"embedding\",\n\t\t},\n\t});\n\n\tconst result = await vectorStore.similaritySearch(prompt, 2, {\n\t\tdocumentId,\n\t});\n\n\tconst model = new ChatOpenAI({\n\t\tapiKey: process.env.OPENAI_API_KEY,\n\t\tmodel: \"gpt-4o-mini\",\n\t\ttemperature: 0,\n\t});\n\n\tconst allMessages = messages.map((message) =\u003e\n\t\tmessage.role === \"user\"\n\t\t\t? new HumanMessage(message.content)\n\t\t\t: new AIMessage(message.content),\n\t);\n\n\tconst systemMessage = new SystemMessage(\n\t\t`You are a helpful assistant, here's some extra additional context that you can use to answer questions. Only use this information if it's relevant:\n\t\t\n\t\t${result.map((r) =\u003e r.pageContent).join(\" \")}`,\n\t);\n\n\tallMessages.push(systemMessage);\n\n\tconst stream = await model.stream(allMessages);\n\n\treturn LangChainAdapter.toDataStreamResponse(stream);\n  ```\n\u003c/details\u003e\n\n\nWhile this approach is beneficial, it can also be challenging to implement. You need to manage each database's lifecycle, including provisioning, scaling, and de-provisioning. Fortunately, Postgres on Neon is set up differently:\n\n1. Postgres on Neon can be provisioned via the in ~2 seconds, making provisioning a Postgres database for every tenant possible. You don't need to wait several minutes for the database to be ready.\n2. The database's compute can automatically scale up to meet an application's workload and can shut down when the database is unused.\n\nhttps://github.com/user-attachments/assets/96500fc3-3efa-4cfa-9339-81eb359ff105\n\n![Autoscaling on Neon](https://github.com/user-attachments/assets/7f093ead-d51b-46bc-a473-0df483d91c18)\n\nThis makes the proposed pattern of creating a database per tenant not only possible but also cost-effective.\n\n## Managing migrations\n\nWhen you have a database per tenant, you need to manage migrations for each database. This project uses [Drizzle](https://drizzle.team/):\n1. The schema is defined in `/app/lib/vector-db/schema.ts` using TypeScript.\n2. Migrations are then generated by running `bun run vector-db:generate`, and stored in `/app/lib/vector-db/migrations`.\n3. Finally, to migrate all databases, you can run `bun run vector-db:migrate`. This command will run a script that connects to each tenant's database and applies the migrations. \n\nIt's important to note that any schema changes you would like to introduce should be backward-compatible. Otherwise, you would need to handle schema migrations differently.\n\n## Conclusion\n\nWhile this pattern is useful in building AI applications, you can simply use it to provide each tenant with its own database. You could also use a database other than Postgres for your main application's database (e.g., MySQL, MongoDB, MSSQL server, etc.). \n\nIf you have any questions, feel free to reach out to in the [Neon Discord](https://neon.tech/discord) or contact the [Neon Sales team](https://neon.tech/contact-sales). We'd love to hear from you.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneondatabase%2Fdb-per-tenant","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneondatabase%2Fdb-per-tenant","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneondatabase%2Fdb-per-tenant/lists"}