{"id":32168394,"url":"https://github.com/dimamik/torus","last_synced_at":"2026-02-20T00:31:57.896Z","repository":{"id":272424719,"uuid":"916569283","full_name":"dimamik/torus","owner":"dimamik","description":"Torus bridges Ecto and PostgreSQL, simplifying building search queries.","archived":false,"fork":false,"pushed_at":"2026-02-04T20:58:57.000Z","size":222,"stargazers_count":239,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-02-04T23:42:44.398Z","etag":null,"topics":["elixir","postgresql","search"],"latest_commit_sha":null,"homepage":"https://hexdocs.pm/torus/Torus.html","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dimamik.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-14T11:00:57.000Z","updated_at":"2026-02-03T21:10:12.000Z","dependencies_parsed_at":"2025-01-14T11:16:07.634Z","dependency_job_id":"efdeee5b-5c90-45aa-bbb5-672ca9952410","html_url":"https://github.com/dimamik/torus","commit_stats":null,"previous_names":["dimamik/torus"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/dimamik/torus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimamik%2Ftorus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimamik%2Ftorus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimamik%2Ftorus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimamik%2Ftorus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dimamik","download_url":"https://codeload.github.com/dimamik/torus/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimamik%2Ftorus/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29637410,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T22:32:43.237Z","status":"ssl_error","status_checked_at":"2026-02-19T22:32:38.330Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","postgresql","search"],"created_at":"2025-10-21T15:53:54.999Z","updated_at":"2026-02-20T00:31:57.891Z","avatar_url":"https://github.com/dimamik.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Torus\n\n[![CI](https://github.com/dimamik/torus/actions/workflows/ci.yml/badge.svg)](https://github.com/dimamik/torus/actions/workflows/ci.yml)\n[![License](https://img.shields.io/hexpm/l/torus.svg)](https://github.com/dimamik/torus/blob/main/LICENSE)\n[![Version](https://img.shields.io/hexpm/v/torus.svg)](https://hex.pm/packages/torus)\n[![Hex Docs](https://img.shields.io/badge/documentation-gray.svg)](https://hexdocs.pm/torus)\n[![Live Demo](https://img.shields.io/badge/Live%20Demo-online-brightgreen?logo=bolt\u0026logoColor=white)](https://torus.dimamik.com)\n\n\u003c!-- MDOC --\u003e\n\nTorus is a plug-and-play Elixir library that seamlessly integrates PostgreSQL's search into Ecto, streamlining the construction of advanced search queries. See [live demo](https://torus.dimamik.com) for examples.\n\n## Usage\n\nThe package can be installed by adding `torus` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:torus, \"~\u003e 0.6\"}\n  ]\nend\n```\n\nThen, in any query, you can (for example) add a prefixed full-text search:\n\n```elixir\nimport Torus\n# ...\n\nPost\n# ... your complex query\n|\u003e Torus.full_text([p], [p.title, p.body], \"uncove hogwar\")\n|\u003e select([p], p.title)\n|\u003e Repo.all()\n[\"Uncovered hogwarts\"]\n```\n\nSee [`full_text/5`](https://hexdocs.pm/torus/Torus.html#full_text/5) for more details.\n\n## 7 types of search:\n\n1. **Pattern matching**: Searches for a specific pattern in a string.\n\n   ```elixir\n   iex\u003e insert_posts!([\"Wand\", \"Magic wand\", \"Owl\"])\n   ...\u003e Post\n   ...\u003e |\u003e Torus.similar_to([p], [p.title], \"(Wan|Ow)%\")\n   ...\u003e |\u003e select([p], p.title)\n   ...\u003e |\u003e Repo.all()\n   [\"Wand\", \"Owl\"]\n   ```\n\n   Use it for fast prefix-search when semantics of the data you search through live in its characters. For example phone number, invoice number, email, filename, etc.\n\n   See [`like/5`](https://hexdocs.pm/torus/Torus.html#like/5), [`ilike/5`](https://hexdocs.pm/torus/Torus.html#ilike/5), and [`similar_to/5`](https://hexdocs.pm/torus/Torus.html#similar_to/5) for more details.\n\n1. **Similarity:** Searches for records that closely match the input text using trigram distance.\n\n   ```elixir\n   insert_posts!([\"Hogwarts Secrets\", \"Quidditch Fever\", \"Hogwart’s Secret\"])\n\n   Post\n   |\u003e Torus.similarity([p], [p.title], \"hoggwarrds\")\n   |\u003e limit(2)\n   |\u003e select([p], p.title)\n   |\u003e Repo.all()\n   [\"Hogwarts Secrets\", \"Hogwart’s Secret\"]\n   ```\n\n   Use it for fuzzy matching and catching typos in short text fields, such as names or titles. Works best with short strings.\n\n   See [`similarity/5`](https://hexdocs.pm/torus/Torus.html#similarity/5) for more details.\n\n1. **Full text**: Uses term-document matrix vectors, enabling efficient querying and ranking based on term frequency. Supports prefix search and is great for large datasets to quickly return relevant results. See [PostgreSQL Full Text Search](https://www.postgresql.org/docs/current/textsearch.html) for internal implementation details.\n\n   ```elixir\n   insert_post!(title: \"Hogwarts Shocker\", body: \"A spell disrupts the Quidditch Cup.\")\n   insert_post!(title: \"Diagon Bombshell\", body: \"Secrets uncovered in the heart of Hogwarts.\")\n   insert_post!(title: \"Completely unrelated\", body: \"No magic here!\")\n\n   Post\n   |\u003e Torus.full_text([p], [p.title, p.body], \"uncov hogwar\")\n   |\u003e select([p], p.title)\n   |\u003e Repo.all()\n   [\"Diagon Bombshell\"]\n   ```\n\n   Use it when you don't care about spelling, the documents are long, you need multi-column search with weights, or if you need to order the results by rank.\n\n   See [`full_text/5`](https://hexdocs.pm/torus/Torus.html#full_text/5) for more details.\n\n1. **BM25 full text**: Modern [BM25](https://mbrenndoerfer.com/writing/bm25-search-algorithm-elasticsearch-implementation) ranking algorithm for superior relevance scoring using the [pg_textsearch](https://github.com/timescale/pg_textsearch) extension. BM25 generally provides better ranking than traditional built-in TF-IDF full text search and is optimized for top-k queries.\n\n   ```elixir\n   insert_post!(title: \"Potion Class Notes\", body: \"Wiggenweld potion heals wounds.\")\n   insert_post!(title: \"Complete Potion Encyclopedia\", body: \"Edurus potion grants protection. Focus potion improves concentration. Maxima potion amplifies spells. Thunderbrew potion creates explosions.\")\n   insert_post!(title: \"Combat Guide\", body: \"Use Wiggenweld potion to heal during goblin fights.\")\n\n   Post\n   |\u003e Torus.bm25([p], p.body, \"wiggenweld potion\")\n   |\u003e limit(2)\n   |\u003e select([p], p.title)\n   |\u003e Repo.all()\n   [\"Potion Class Notes\", \"Combat Guide\"]\n   ```\n\n   Use it when you need state-of-the-art relevance ranking for single-column search, especially with LIMIT clauses. Requires PostgreSQL 17+.\n\n   See [`bm25/5`](https://hexdocs.pm/torus/Torus.html#bm25/5) and the [BM25 Search Guide](https://dimamik.com/posts/bm25_search) for detailed setup instructions and examples.\n\n1. **Semantic Search**: Understands the contextual meaning of queries to match and retrieve related content utilizing natural language processing. Read more about semantic search in [Semantic search with Torus guide](/guides/semantic_search.md).\n\n   ```elixir\n   insert_post!(title: \"Hogwarts Shocker\", body: \"A spell disrupts the Quidditch Cup.\")\n   insert_post!(title: \"Diagon Bombshell\", body: \"Secrets uncovered in the heart of Hogwarts.\")\n   insert_post!(title: \"Completely unrelated\", body: \"No magic here!\")\n\n   embedding_vector = Torus.to_vector(\"A magic school in the UK\")\n\n   Post\n   |\u003e Torus.semantic([p], p.embedding, embedding_vector)\n   |\u003e select([p], p.title)\n   |\u003e Repo.all()\n   [\"Diagon Bombshell\"]\n   ```\n\n   Use it when you need to understand intent and handle synonyms.\n\n   See [`semantic/5`](https://hexdocs.pm/torus/Torus.html#semantic/5) for more details.\n\n1. **Hybrid Search**: Combines multiple search techniques (e.g., keyword and semantic) to leverage their strengths for more accurate results.\n\n   Will be added soon.\n\n1. **3rd Party Engines/Providers**: Utilizes external services or software specifically designed for optimized and scalable search capabilities, such as Elasticsearch or Algolia.\n\nYou can see all of the above search types in action on the [live demo page](https://torus.dimamik.com).\n\n## Optimizations and relevance\n\nTorus is designed to be as efficient and relevant as possible from the start. But handling large datasets and complex search queries tends to be tricky. The best way to combine these two to achieve the best result is to:\n\n1. Create a query that returns as relevant results as possible (by tweaking the options of search function). If there is any option missing - feel free to open an issue/contribute back with it, or implement it manually using fragments.\n2. Test its performance on real production data - maybe it's good enough already?\n3. If it's not:\n   - See optimization sections for your search type in [`Torus`](https://hexdocs.pm/torus/Torus.html) docs\n   - Inspect your query using [`Torus.QueryInspector.tap_substituted_sql/3`](https://hexdocs.pm/torus/Torus.QueryInspector.html#tap_substituted_sql/3) or [`Torus.QueryInspector.tap_explain_analyze/3`](https://hexdocs.pm/torus/Torus.QueryInspector.html#tap_explain_analyze/3)\n   - According to the above SQL - add indexes for the queried rows/vectors\n\n## Debugging your queries\n\nTorus offers a few helpers to debug, explain, and analyze your queries before using them on production. See [`Torus.QueryInspector`](https://hexdocs.pm/torus/Torus.QueryInspector.html) for more details.\n\n## Torus support\n\nFor now, Torus supports pattern match, similarity, full-text (TF-IDF and BM25), and semantic search, with plans to expand support further. These docs will be updated with more examples on which search type to choose and how to make them more performant (by adding indexes or using specific functions).\n\n\u003c!-- MDOC --\u003e\n\n## Future plans\n\n- [ ] Add support for highlighting search results. (Base off of a `ts_headline` function)\n- [ ] Extend similarity search to support [`fuzzystrmatch`](https://www.postgresql.org/docs/current/fuzzystrmatch.html) extension distance options.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdimamik%2Ftorus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdimamik%2Ftorus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdimamik%2Ftorus/lists"}