{"id":22152140,"url":"https://github.com/findinpath/elastic-percolator-primer","last_synced_at":"2025-03-24T13:25:15.230Z","repository":{"id":45159665,"uuid":"277479274","full_name":"findinpath/elastic-percolator-primer","owner":"findinpath","description":"Primer on the Elasticsearch percolator functionality","archived":false,"fork":false,"pushed_at":"2022-01-04T16:45:19.000Z","size":8,"stargazers_count":1,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-29T18:32:08.087Z","etag":null,"topics":["elasticsearch","percolator"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/findinpath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-06T08:00:29.000Z","updated_at":"2021-08-02T19:47:02.000Z","dependencies_parsed_at":"2022-09-24T20:01:15.643Z","dependency_job_id":null,"html_url":"https://github.com/findinpath/elastic-percolator-primer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/findinpath%2Felastic-percolator-primer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/findinpath%2Felastic-percolator-primer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/findinpath%2Felastic-percolator-primer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/findinpath%2Felastic-percolator-primer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/findinpath","download_url":"https://codeload.github.com/findinpath/elastic-percolator-primer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245277458,"owners_count":20589154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","percolator"],"created_at":"2024-12-02T00:47:24.613Z","updated_at":"2025-03-24T13:25:15.195Z","avatar_url":"https://github.com/findinpath.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"Elasticsearch Percolator Primer\n===============================\n\n\nThis simple project is more like a note to self to gather an understanding\non how Elasticsearch Percolator functionality works under the hood.\n\n\nThe _Elasticsearch Percolator_ functionlity can be found under the\n[percolator]((https://github.com/elastic/elasticsearch/tree/master/modules/percolator))\nmodule.\n\nThe basic idea behind the percolator is that (as denoted by the documentation\nof the [percolate](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-percolate-query.html)\nquery):\n \n\u003e It can be used to match queries stored in an index.\n\u003e The percolate query itself contains the document \n\u003e that will be used as query to match with the stored queries.\n\n\nInstead of naively executing all the queries stored in the index against\nthe percolated document, Elasticsearch will filter out the queries that don't\nmatch the document by doing a search  on top of the stored queries.\nOnly the queries that are matching the document (or the ones that can't be evaluated in bulk)\nwill be finally executed against a memory index that contains only the document to percolate.\n\nThe _percolate_ query is called _turning search upside down_ in the [Elasticsearch in action](https://livebook.manning.com/book/elasticsearch-in-action/appendix-e/)\nbook for the following reasons:\n\n- You index queries instead of documents. This registers the query in memory, so it can be quickly run later.\n- You send a document to Elasticsearch instead of a query. This is called percolating a document, basically indexing it into a small, in-memory index. Registered queries are run against the small index, so Elasticsearch finds out which queries match.\n- You get back a list of queries matching the document, instead of the other way around like a regular search.\n \n\n\n\nThe percolator analyses the queries and creates appropriate data structures in order to\nbe able to filter only the relevant search queries that need to be executed on the memory\nindex containing the percolated document: \n\n- for a  _term_ (e.g. : search for all the documents containing `elastic` ) query there will be used a `org.apache.lucene.search.TermScorer`\nthat will make use of the Lucene's inverted index functionality for retrieving the documents that contain the searched term\n- for a int field (e.g. : search all the real estate object with `4` rooms) an intersection will be made\nbetween the range of the percolated document (e.g. : `4` to `4`) and the [KD tree](https://www.youtube.com/watch?v=Z4dNLvno-EY) \ncontaining the ranges (e.g. : `0` TO `3` , `4` TO `*`, `2` TO `5`, etc.)\n\n  \n \nThis very simple project contains a series of tests in the [CandidateQueryTests.java](src/test/java/org/elasticsearch/percolator/CandidateQueryTests.java)\ntest class that showcase how the percolator actually works behind the scenes.\n ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffindinpath%2Felastic-percolator-primer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffindinpath%2Felastic-percolator-primer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffindinpath%2Felastic-percolator-primer/lists"}