{"id":20417471,"url":"https://github.com/tm9657/serverless-cloudflare-search","last_synced_at":"2025-04-12T17:24:05.038Z","repository":{"id":162358049,"uuid":"623499067","full_name":"TM9657/serverless-cloudflare-search","owner":"TM9657","description":"Using Cloudflare Worker + Queues + R2 Storage + Cache to implement a small scale to zero search system that is reasonably fast and cheap.","archived":false,"fork":false,"pushed_at":"2023-05-23T11:24:12.000Z","size":182,"stargazers_count":106,"open_issues_count":1,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-26T11:44:52.914Z","etag":null,"topics":["cloudflare","search","serverless"],"latest_commit_sha":null,"homepage":"https://tm9657.de","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TM9657.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-04T13:47:13.000Z","updated_at":"2025-03-18T03:51:42.000Z","dependencies_parsed_at":"2023-07-24T06:45:09.844Z","dependency_job_id":null,"html_url":"https://github.com/TM9657/serverless-cloudflare-search","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TM9657%2Fserverless-cloudflare-search","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TM9657%2Fserverless-cloudflare-search/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TM9657%2Fserverless-cloudflare-search/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TM9657%2Fserverless-cloudflare-search/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TM9657","download_url":"https://codeload.github.com/TM9657/serverless-cloudflare-search/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248602975,"owners_count":21131717,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudflare","search","serverless"],"created_at":"2024-11-15T06:26:25.614Z","updated_at":"2025-04-12T17:24:05.030Z","avatar_url":"https://github.com/TM9657.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"https://tm9657.de?ref=github\"\u003e\u003cp align=\"center\"\u003e\u003cimg width=250 src=\"https://cdn.tm9657.de/tm9657/images/serverless_search.png\" /\u003e\u003c/p\u003e\u003c/a\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://tm9657.de\"\u003e\u003cimg src=\"https://img.shields.io/badge/website-more_from_us-C0222C.svg?style=flat\u0026logo=PWA\"\u003e \u003c/a\u003e\n\t  \u003ca href=\"https://discord.ca9.io\"\u003e\u003cimg src=\"https://img.shields.io/discord/673169081704120334?label=discord\u0026style=flat\u0026color=5a66f6\u0026logo=Discord\"\u003e\u003c/a\u003e\n\t  \u003ca href=\"https://twitter.com/tm9657\"\u003e\u003cimg src=\"https://img.shields.io/badge/twitter-follow_us-1d9bf0.svg?style=flat\u0026logo=Twitter\"\u003e\u003c/a\u003e\n\t  \u003ca href=\"https://www.linkedin.com/company/tm9657/\"\u003e\u003cimg src=\"https://img.shields.io/badge/linkedin-connect-0a66c2.svg?style=flat\u0026logo=Linkedin\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://merch.ca9.io\"\u003e\u003cimg src=\"https://img.shields.io/badge/merch-support_us-red.svg?style=flat\u0026logo=Spreadshirt\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n# 🔍🌩️ Serverless Search on Cloudflare\nUsing Cloudflare Worker + Queues + R2 Storage + Cache to implement a small scale to zero search system that is reasonably fast and cheap.\nBenchmark welcome for performance measure :)\n\nEndpoints:\n- **search**   - public\n- **index**    - access restricted (see config)\n\nCached Index saved in R2. Cache read on search request. \nQueue -\u003e Writing Index (Batch size and concurrency 0)\n\n## Features\n- Generic Index support\n- Multiple Parallel indices per endpoint (infinite)\n- Good performance for smaller Datasets (up to 50k documents (I guess? Feel free to create a better benchmark!))\n\n## Setup\ncreate a .env file in your root with the following parameter: \n```\nCLOUDFLARE_AUTH_KEY=\nCLOUDFLARE_AUTH_EMAIL=\n```\n\n\u003e - `pnpm install` ➡️ populates your config with a strong secret\n\u003e - `pnpm run initialize` ➡️ creates the bucket and queue\n\u003e - `npx turbo build` ➡️ publishes your workers to cloudflare\n\n## \"Benchmark\"\nThis project is meant for smaller datasets (cheap serverless search).\nFor a movie dataset with **17920 documents** a search takes *800ms first time* (downloading the index from R2), after that we get a worker performance of *50-60ms per search*.\n\n## Todo\n- Alternative Flexsearch implementation (Problems with export / import and types)\n- Investigate Durable Object for faster initial response\n- Add serverless setup for AWS deployment\n- Add CLI tool\n\n**Provided by TM9657 GmbH with ❤️**\n### Check out some of our products:\n- [Kwirk.io](https://kwirk.io?ref=github) (Text Editor with AI integration, privacy focus and offline support)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftm9657%2Fserverless-cloudflare-search","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftm9657%2Fserverless-cloudflare-search","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftm9657%2Fserverless-cloudflare-search/lists"}