{"id":24991492,"url":"https://github.com/genmon/braggoscope-search","last_synced_at":"2025-04-12T02:05:04.256Z","repository":{"id":205820911,"uuid":"710824751","full_name":"genmon/braggoscope-search","owner":"genmon","description":"A search box for braggoscope.com, which is a static site","archived":false,"fork":false,"pushed_at":"2024-02-10T16:22:33.000Z","size":4196,"stargazers_count":10,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-12T02:04:59.030Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/genmon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-10-27T14:10:39.000Z","updated_at":"2024-09-18T07:51:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"c9222794-9c55-4c0b-be39-078f7048fb8a","html_url":"https://github.com/genmon/braggoscope-search","commit_stats":null,"previous_names":["genmon/braggoscope-search"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/genmon%2Fbraggoscope-search","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/genmon%2Fbraggoscope-search/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/genmon%2Fbraggoscope-search/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/genmon%2Fbraggoscope-search/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/genmon","download_url":"https://codeload.github.com/genmon/braggoscope-search/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248505862,"owners_count":21115354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-04T13:49:17.482Z","updated_at":"2025-04-12T02:05:04.224Z","avatar_url":"https://github.com/genmon.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# braggoscope-search\n\nA search engine for braggoscope.com, which is a static site.\n\n![](/assets/ai-search-in-braggoscope.gif)\n\n### What is this and how does it work?\n\n- This is a PartyKit server that implements a vector search engine\n- Indexing: it reads a JSON file of documents provided by braggoscope.com\n- Querying: it accepts a query over POST and returns a list of documents\n\nThe vector database is Vectorize from Cloudflare. The embedding model is part of Cloudflare Workers AI. Both the vector db and the embedding model are accessed via PartyKit.\n\nThis server also includes:\n\n- a basic admin UI to kick off indexing\n- a basic test UI to query the search engine\n\n### What is not included?\n\n- The static site braggoscope.com, and the JSON file of documents to index\n- The search UI itself, which is part of the website at braggoscope.com\n\n## Try it out\n\n- [Try the demo hosted on PartyKit](https://braggoscope-search.genmon.partykit.dev)\n- [Use the search on braggoscope.com](https://www.braggoscope.com) -- tap **Search** in the top nav\n\n## Usage\n\nThese instructions assume you have cloned the repo, installed packages, and signed into PartyKit (required for using certain features):\n\n```bash\nnpm install\nnpx partykit login\n```\n\n**1. Create the vector database**\n\nYou only need to do this once.\n\nVectorize is a Cloudflare-hosted vector database. We'll use one managed by PartyKit because then it's easier to use from the PartyKit server.\n\nCreate the vector index:\n\n```bash\nnpx partykit vectorize create braggoscope-index --preset @cf/baai/bge-base-en-v1.5\n```\n\nThis is made available in the PartyKit server with these lines in `partykit.json`:\n\n```jsonc\n{\n  // ...\n  \"vectorize\": {\n    \"searchIndex\": \"braggoscope-index\"\n  },\n  \"ai\": true\n}\n```\n\nNote the `\"ai\": true` which also makes Cloudflare's Workers AI available in the PartyKit server.\n\n**2. Build the index**\n\nThe vector database is empty. We need to add documents to it.\n\nbraggoscope.com is a static site for exploring episodes of BBC Radio 4's _In Our Time._ As part of the build process, it outputs a JSON file of documents to index.\n\nHave a look at episodes.json here: [www.braggoscope.com/episodes.json](https://www.braggoscope.com/episodes.json).\n\nThis document looks like:\n\n```jsonc\n[\n  {\n    \"id\": \"p0038x9h\",\n    \"title\": \"The Speed of Light\",\n    \"published\": \"2006-11-30\",\n    \"permalink\": \"/2006/11/30/the-speed-of-light.html\",\n    \"description\": \"Melvyn Bragg and guests discuss the speed of light. Scientists and thinkers ...\"\n  }\n  // ...\n]\n```\n\nWe want to create a vector embedding of the description, and store it against the other properties as metadata.\n\nWe have to kick off indexing manually, so we have some minimal security around it. We'll use an admin key to protect the endpoint. Store this in `.env`:\n\n```bash\necho \"BRAGGOSCOPE_SEARCH_ADMIN_KEY=foo-admin-key\\n\" \u003e .env\n```\n\nThen run the server locally and start indexing:\n\n```bash\nnpm run dev\n```\n\nThen visit [127.0.0.1:1999/admin?key=foo-admin-key](http://127.0.0.1:1999/admin?key=foo-admin-key) and click _\"Create Index\"_.\n\n**3. Test the search with the test UI**\n\nAfter indexing, visit [127.0.0.1:1999](http://127.0.0.1:1999) to test the search.\n\nTry the query \"greek myths\" and you should see a list of episodes related to the Greek myths.\n\n**4. Test the search API**\n\nThe search feature on braggoscope.com makes a POST request to the search API. You can test this with curl:\n\n```bash\ncurl \\\n--json '{\"query\": \"greek myths\"}' \\\nhttp://127.0.0.1:1999/parties/search/api\n```\n\nYou will see a JSON object of results being returned.\n\n**5. Deploy the server and test again.**\n\nDeploy the server to a public URL:\n\n```bash\nnpm run deploy\n```\n\nNote: do not use the usual `npx partykit deploy`! The `deploy` script we're using here also (1) ensures that the site is built first; and (2) sets the environment variables from `.env`.\n\nWait for the deploy to complete (you may have to wait a couple extra minutes for the domain to be provisioned) then build the index again:\n\nVisit [https://braggoscope-search.YOUR-PARTYKIT-USERNAME.partykit.dev/admin?key=foo-admin-key](https://braggoscope-search.YOUR-PARTYKIT-USERNAME.partykit.dev/admin?key=foo-admin-key)\n\n(Replace `YOUR-PARTYKIT-USERNAME` with your PartyKit username, and `foo-admin-key` with the admin key you set in `.env`.)\n\nAfter indexing, use the test UI and the search API to test the search again.\n\n**6. Integrate**\n\nView source at [www.braggoscope.com](https://www.braggoscope.com) to see how the search is integrated into the site. It's a simple form that makes a POST request to the search API, and displays the results.\n\nThere is also a process to trigger re-indexing whenever that site is updated.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgenmon%2Fbraggoscope-search","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgenmon%2Fbraggoscope-search","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgenmon%2Fbraggoscope-search/lists"}