{"id":27133194,"url":"https://github.com/serhii-chernenko/sqlite-id-uuid-performance-test","last_synced_at":"2026-05-05T23:32:13.433Z","repository":{"id":286351848,"uuid":"961108097","full_name":"serhii-chernenko/sqlite-id-uuid-performance-test","owner":"serhii-chernenko","description":"Load testing for Cloudflare D1 SQLite database with id column as a text primary key instead of auto-incremented integer","archived":false,"fork":false,"pushed_at":"2025-04-06T19:47:02.000Z","size":186,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-09T23:39:02.885Z","etag":null,"topics":["cloudflare","d1","drizzle","sqlite","ulid","uuid"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/serhii-chernenko.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-04-05T19:11:53.000Z","updated_at":"2025-04-06T19:47:05.000Z","dependencies_parsed_at":"2025-04-09T23:35:52.004Z","dependency_job_id":null,"html_url":"https://github.com/serhii-chernenko/sqlite-id-uuid-performance-test","commit_stats":null,"previous_names":["serhii-chernenko/sqlite-id-uuid-performance-test"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/serhii-chernenko/sqlite-id-uuid-performance-test","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serhii-chernenko%2Fsqlite-id-uuid-performance-test","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serhii-chernenko%2Fsqlite-id-uuid-performance-test/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serhii-chernenko%2Fsqlite-id-uuid-performance-test/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serhii-chernenko%2Fsqlite-id-uuid-performance-test/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/serhii-chernenko","download_url":"https://codeload.github.com/serhii-chernenko/sqlite-id-uuid-performance-test/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serhii-chernenko%2Fsqlite-id-uuid-performance-test/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32672603,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-05T11:29:49.557Z","status":"ssl_error","status_checked_at":"2026-05-05T11:29:48.587Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudflare","d1","drizzle","sqlite","ulid","uuid"],"created_at":"2025-04-07T23:19:51.103Z","updated_at":"2026-05-05T23:32:13.427Z","avatar_url":"https://github.com/serhii-chernenko.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Performance benchmarking for Cloudflare D1 SQLite database\n\nThe main goal is comparing executing time for the `users` (1kk records) and `posts` (10kk records) tables with the `id` column as `integer`, `uuid` (v4), and `ulid`.\n\n## Short comparing table\n\n### Request time\n\n|                      | integer | ulid  | uuidv4 |\n|----------------------|---------|-------|--------|\n| server response time | 472ms   | 878ms | 1769ms |\n\n### Seeding time\n\n|           | integer | ulid  | uuidv4 |\n|-----------|---------|-------|--------|\n| seed time | 24m     | 1h 6m | 1h 24m |\n\n### Database size\n\n|              | integer | ulid   | uuidv4 |\n|--------------|---------|--------|--------|\n| .sqlite size | 278MB   | 1,69GB | 1,96GB |\n\n## Investigation process\n\n### Prehistory\n\nAs a frontend developer, I mostly worked with MongoDB in my pet projects. Currently, I'm working on a bigger product. That's a reason why I decided to rethink my decisions regarding chosen infrastructure, including database. I'm migration to Cloudflare infra. Shotrly, because it has friendly DX, transparent pricing, wider free packages, services I need, etc.\n\nSo, the first step is migrating from usual MongoDB to Cloudflare D1 database based on SQLite. As a frontend developer I have no idea how to write good SQL queries. During the investigation, I found posts about Prisma and how it's slow. That's funny. My search brought me to Drizzle ORM. It has pretty nice documentations, it's free and open-sourced, it's faster than Prisma regarding provided benchmarks, I found many good feedbacks about it.\n\nWhen I started writing my first schemas, I started thinking about the architecture, security, and performance on the beggining, just because I usually do that. And first point, that I'm using Nuxt 3 application. I have a TypeScript interface provided by Drizzle from a table schema. This interface and data object type shared across server, API, and storefront components. It means, all the fields will be visible in the Network tab on request once it's fetched. I want to avoid cases when I get `/api/users/1` explicit `id` field due to aestetic and security reasons (yes, I know about session tokens).\n\nFrom the past while I was working with MongoDB, I remember that usually the `_id` field auto-generated by MongoDB uses `uuid` format. Not sure about the exact `uuid` version but I'm sure it's not just an auto-incremented `integer`. I was just just interested what if it can negatively affect a performance of the SQLite database.\n\n### Preparation for performance benchmarking\n\nI'm using Nuxt 3 application (with Nitro including), `nitro-dev-cloudflare`, wrangler, Miniflare for D1 local development, Drizzle ORM, Drizzle Kit, Drizzle Studio, and Drizzle Seed functionality.\n\nI decided to seed database with 1 million `users` and 10 millions `posts`. But found out, that D1 has pretty limited request row. In my case, I could see only around 10-15 users and posts, then I just got error\n```\ntoo many SQL variables at offset 425: SQLITE_ERROR\n```\n![image](https://github.com/user-attachments/assets/8f42822c-2a21-4bdc-9f09-0d20befd393b)\n\nSo, instead of using the `drizzle-seed` package, I just manually seeded the tables in a `for` loop. Code examples:\n- [Seed `int`](https://github.com/serhii-chernenko/sqlite-id-uuid-performance-test/blob/main/server/tasks/seed-int.ts)\n- [Seed `uuidv4`](https://github.com/serhii-chernenko/sqlite-id-uuid-performance-test/blob/main/server/tasks/seed-uuidv4.ts)\n- [Seed `ulid`](https://github.com/serhii-chernenko/sqlite-id-uuid-performance-test/blob/main/server/tasks/seed-ulid.ts)\n\nAlso, I had to increase RAM memory to 10GB (default is 2GB) for Node in my `~/.zshrc`:\n```bash\nexport NODE_OPTIONS=\"--max-old-space-size=10000\"\n```\n![image](https://github.com/user-attachments/assets/c0aaee48-915d-4dc6-aad4-cff7f01689c9)\n\n### Request example\n\nIt's pretty similar for all cases:\n![image](https://github.com/user-attachments/assets/1319e383-0998-4af8-ae6e-d05ed01a3dd5)\n\n### Integer\n\nSeeding result:\n![image](https://github.com/user-attachments/assets/11890dcc-0cd3-4185-9020-a1d63bd25647)\n\nResponse time is **~480ms**:\n![image](https://github.com/user-attachments/assets/493c210b-c418-4d9d-a029-31d63aede80b)\n\n### UUID v4\n\nSeeding result:\n![image](https://github.com/user-attachments/assets/90870717-644c-4e22-9802-ef39024e98f2)\n\nIt's failed almost in the end, that's a reason why I have 9,17M posts instead of 10M.\n\nResponse time is **~1789ms**:\n![image](https://github.com/user-attachments/assets/fb721c95-a89c-4762-998b-9216f6177927)\n\n### ULID\n\nIn comparison with UUID v4 (36 chars), ULID has less chars (26 chars), but it's lexicographically sortable, time-based. I guessed it has to be better for performance and database size if I want to keep `uuid` flow for the `id` column.\n\nSeeding result:\n![image](https://github.com/user-attachments/assets/b3e01dd5-a82d-450f-aa06-3b065d2c6935)\n\nResponse time is **~878ms**:\n![image](https://github.com/user-attachments/assets/237cea37-ec42-482b-b2a9-abe3020ea9b9)\n\n## Conclusions\n\n1. If you really care about perfomance and expect 1M+ records in tables on your product, I guess it'd be better to stay around auto-incremented `integer`. If not, it doesn't matter what you choose.\n2. If you are sure, you want to have `uuid` flow, choose `ulid` over `uuidv4`. Also, take a look at `uuidv7`. Feel free to use this repo, to test other approaches, including `uuidv7`.\n3. `drizzle-seed` needs adjustments for D1 for seeding more than 20 records.\n4. Drizzle Studio is pretty fast, clear and it has nice UX. Not a PhpMyAdmin if you are pretty old as me :D (I'm 29, LOL)\n\n## The best option\n\nAndrew Sherman (the creator of Drizzle ORM) suggested to me the best approach. I can create a custom type via Drizzle ORM that returns integer in the same way for the `id` column. So, just go forward with auto-incremented integer primary key. But! It can be encoded only on select (no need for insert) to hide the exact value when returned from API.\n\nScreenshot explanation:\n![image](https://github.com/user-attachments/assets/bc85fdb9-50d4-41c8-a9d5-74d8260e68d7)\n\nDatabase example:\n![image](https://github.com/user-attachments/assets/27e3ba13-afa3-4e25-a5b6-6ad90c3a5594)\n\nAPI response:\n![image](https://github.com/user-attachments/assets/09204c9a-6659-43ad-b7a3-ebb58bde7322)\n![image](https://github.com/user-attachments/assets/d25dc377-a17f-4d67-be9b-02d598c0a4c7)\n\nSo, it's equally fast as just using integer. In addition, I encoded it via the [`hashids`](https://www.npmjs.com/package/hashids) NPM package. It means, it could be decoded only when you know exact value of a decoding secret key. And it solves what I mean, it seems much better in the API response but still be the fastest option as needed due to the performance requirements.\n\nRelated commit:\nhttps://github.com/serhii-chernenko/sqlite-id-uuid-performance-test/commit/ea2d3a8ec2feb69e3ace49efc6acfe32e084aeaf\n\n## Just some videos\n\n### Integer\n\nhttps://github.com/user-attachments/assets/ee924e91-2c1d-45d5-92d4-5f3065a7ef6a\n\n### UUIDv4\n\nhttps://github.com/user-attachments/assets/d4031486-4335-4001-be04-092168d9732d\n\n### ULID\n\nhttps://github.com/user-attachments/assets/a0d4e036-17a6-4033-aa80-d33dca3bdb89\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fserhii-chernenko%2Fsqlite-id-uuid-performance-test","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fserhii-chernenko%2Fsqlite-id-uuid-performance-test","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fserhii-chernenko%2Fsqlite-id-uuid-performance-test/lists"}