{"id":13408223,"url":"https://github.com/tinysearch/tinysearch","last_synced_at":"2025-05-14T23:06:28.317Z","repository":{"id":37420077,"uuid":"119272629","full_name":"tinysearch/tinysearch","owner":"tinysearch","description":"🔍 Tiny, full-text search engine for static websites built with Rust and Wasm","archived":false,"fork":false,"pushed_at":"2023-10-18T18:49:08.000Z","size":787,"stargazers_count":2787,"open_issues_count":10,"forks_count":88,"subscribers_count":18,"default_branch":"master","last_synced_at":"2025-04-30T20:17:16.337Z","etag":null,"topics":["bloom-filter","elasticlunr","hacktoberfest","lunrjs","rust","search-engine","static-site","wasm"],"latest_commit_sha":null,"homepage":"https://endler.dev/2019/tinysearch","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinysearch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null},"funding":{"github":"mre"}},"created_at":"2018-01-28T15:54:10.000Z","updated_at":"2025-04-30T11:05:35.000Z","dependencies_parsed_at":"2024-01-02T23:13:26.699Z","dependency_job_id":"4651002c-1e02-4ebd-9a1a-f1690cedcd26","html_url":"https://github.com/tinysearch/tinysearch","commit_stats":{"total_commits":275,"total_committers":23,"mean_commits":"11.956521739130435","dds":0.36,"last_synced_commit":"10c060bb5a496b699be33ad3108e1cb36760d308"},"previous_names":["mre/tinysearch"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinysearch%2Ftinysearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinysearch%2Ftinysearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinysearch%2Ftinysearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinysearch%2Ftinysearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinysearch","download_url":"https://codeload.github.com/tinysearch/tinysearch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254243360,"owners_count":22038046,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bloom-filter","elasticlunr","hacktoberfest","lunrjs","rust","search-engine","static-site","wasm"],"created_at":"2024-07-30T20:00:51.516Z","updated_at":"2025-05-14T23:06:23.303Z","avatar_url":"https://github.com/tinysearch.png","language":"Rust","funding_links":["https://github.com/sponsors/mre"],"categories":["In Memory","HarmonyOS","Rust","Search","rust","Memory","wasm","hacktoberfest"],"sub_categories":["Windows Manager","Surveys"],"readme":"# tinysearch\n\n![Logo](logo.svg)\n\n![CI](https://github.com/mre/tinysearch/workflows/CI/badge.svg)\n\ntinysearch is a lightweight, fast, full-text search engine. It is designed for\nstatic websites.\n\ntinysearch is written in Rust, and then compiled to WebAssembly to run in a\nbrowser.\\\nIt can be used together with static site generators such as\n[Jekyll](https://jekyllrb.com/), [Hugo](https://gohugo.io/),\n[Zola](https://www.getzola.org/),\n[Cobalt](https://github.com/cobalt-org/cobalt.rs), or\n[Pelican](https://getpelican.com).\n\n![Demo](tinysearch.gif)\n\n## Is it tiny?\n\nThe test index file of my blog with around 40 posts creates a WASM payload of\n99kB (49kB gzipped, 40kB brotli).\\\nThat is smaller than the demo image above; so yes.\n\n## How it works\n\ntinysearch is a Rust/WASM port of the Python code from the article\n[\"Writing a full-text\nsearch engine using Bloom filters\"](https://www.stavros.io/posts/bloom-filter-search-engine/).\nIt can be seen as an alternative to [lunr.js](https://lunrjs.com/) and\n[elasticlunr](http://elasticlunr.com/), which are too heavy for smaller websites\nand load a lot of JavaScript.\n\nUnder the hood it uses a [Xor Filter](https://arxiv.org/abs/1912.08258) \u0026mdash;\na datastructure for fast approximation of set membership that is smaller than\nbloom and cuckoo filters. Each blog post gets converted into a filter that will\nthen be serialized to a binary blob using\n[bincode](https://github.com/bincode-org/bincode). Please note that the\nunderlying technologies are subject to change.\n\n## Limitations\n\n- Only finds entire words. As a consequence there are no search suggestions\n  (yet). This is a necessary tradeoff for reducing memory usage. A trie\n  datastructure was about 10x bigger than the xor filters. New research on\n  compact datastructures for prefix searches might lift this limitation in the\n  future.\n- Since we bundle all search indices for all articles into one static binary, we\n  recommend to only use it for small- to medium-size websites. Expect around 2\n  kB uncompressed per article (~1 kb compressed).\n\n## Installation\n\n[wasm-pack](https://rustwasm.github.io/wasm-pack/) is required to build the WASM\nmodule. Install it with\n\n```sh\ncargo install wasm-pack\n```\n\nTo optimize the JavaScript output, you'll also need\n[terser](https://github.com/terser/terser):\n\n```\nnpm install terser -g\n```\n\nIf you want to make the WebAssembly as small as possible, we recommend to\ninstall [binaryen](https://github.com/WebAssembly/binaryen) as well. On macOS\nyou can install it with [homebrew](https://brew.sh/):\n\n```sh\nbrew install binaryen\n```\n\nAlternatively, you can download the binary from the\n[release page](https://github.com/WebAssembly/binaryen/releases) or use your OS\npackage manager.\n\nAfter that, you can install tinysearch itself:\n\n```\ncargo install tinysearch\n```\n\n## Usage\n\nA JSON file, which contains the content to index, is required as an input.\nPlease take a look at the [example file](fixtures/index.json).\n\nℹ️ The `body` field in the JSON document is optional and can be skipped to just\nindex post titles.\n\nOnce you created the index, you can run\n\n```\ntinysearch fixtures/index.json\n```\n\nThis will create a WASM module and the JavaScript glue code to integrate it into\nyour website. You can open the `demo.html` from any webserver to see the result.\n\nFor example, Python has a built-in webserver that can be used for a quick test:\n\n```\npython3 -m http.server\n```\n\nthen browse to http://0.0.0.0:8000/demo.html to run the demo.\n\nYou can also take a look at the code examples for different static site\ngenerators [here](https://github.com/mre/tinysearch/tree/master/howto).\n\n## Advanced Usage\n\nFor advanced usage options, run\n\n```\ntinysearch --help\n```\n\nPlease check what's required to\n[host WebAssembly in production](https://rustwasm.github.io/book/reference/deploying-to-production.html)\n-- you will need to explicitly set gzip mime types.\n\n## Docker\n\nIf you don't have a full Rust setup available, you can also use our\nnightly-built Docker images.\n\nHere is how to quickly try tinysearch with Docker:\n\n```sh\n# Download a sample blog index from endler.dev\ncurl -O https://raw.githubusercontent.com/tinysearch/tinysearch/master/fixtures/index.json\n# Create the WASM output\ndocker run -v $PWD:/app tinysearch/cli --engine-version path=\\\"/engine\\\" --path /app/wasm_output /app/index.json\n```\n\nBy default, the most recent stable Alpine Rust image is used. To get nightly,\nrun\n\n```sh\ndocker build --build-arg RUST_IMAGE=rustlang/rust:nightly-alpine -t tinysearch/cli:nightly .\n```\n\n### Advanced Docker Build Args\n\n- `WASM_REPO`: Overwrite the wasm-pack repository\n- `WASM_BRANCH`: Overwrite the repository branch to use\n- `TINY_REPO`: Overwrite repository of tinysearch\n- `TINY_BRANCH`: Overwrite tinysearch branch\n\n## Github action\n\nTo integrate tinysearch in continuous deployment pipelines, a\n[github action](https://github.com/marketplace/actions/tinysearch-action) is\navailable.\n\n```yaml\n- name: Build tinysearch\n  uses: leonhfr/tinysearch-action@v1\n  with:\n    index: public/index.json\n    output_dir: public/wasm\n    output_types: |\n      wasm\n```\n\n## Users\n\nThe following websites use tinysearch:\n\n- [Matthias Endler's blog](https://endler.dev/2019/tinysearch/)\n- [OutOfCheeseError](https://out-of-cheese-error.netlify.app/)\n- [Museum of Warsaw Archdiocese](https://maw.art.pl/cyfrowemaw/)\n\nAre you using tinysearch, too? Add your site here!\n\n## Maintainers\n\n- Matthias Endler (@mre)\n- Jorge-Luis Betancourt (@jorgelbg)\n- Mad Mike (@fluential)\n\n## License\n\ntinysearch is licensed under either of\n\n- Apache License, Version 2.0, (LICENSE-APACHE or\n  http://www.apache.org/licenses/LICENSE-2.0)\n- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)\n\nat your option.\n\n[wasm-pack]: https://github.com/rustwasm/wasm-pack\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinysearch%2Ftinysearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinysearch%2Ftinysearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinysearch%2Ftinysearch/lists"}