{"id":19260434,"url":"https://github.com/mitre/quaerite","last_synced_at":"2026-03-05T01:31:14.448Z","repository":{"id":44858553,"uuid":"173306807","full_name":"mitre/quaerite","owner":"mitre","description":"Search relevance evaluation toolkit","archived":false,"fork":false,"pushed_at":"2022-01-21T23:23:57.000Z","size":4905,"stargazers_count":73,"open_issues_count":19,"forks_count":15,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-04-21T16:41:29.667Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mitre.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-01T13:30:14.000Z","updated_at":"2024-01-12T20:13:09.000Z","dependencies_parsed_at":"2022-08-23T17:40:45.842Z","dependency_job_id":null,"html_url":"https://github.com/mitre/quaerite","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/mitre/quaerite","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitre%2Fquaerite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitre%2Fquaerite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitre%2Fquaerite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitre%2Fquaerite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mitre","download_url":"https://codeload.github.com/mitre/quaerite/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitre%2Fquaerite/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30104277,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-05T01:06:53.091Z","status":"ssl_error","status_checked_at":"2026-03-05T01:02:35.679Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T19:21:03.323Z","updated_at":"2026-03-05T01:31:14.220Z","avatar_url":"https://github.com/mitre.png","language":"Java","funding_links":[],"categories":["Offline Search Relevance Testing","人工智能"],"sub_categories":["机器学习"],"readme":"\u003c!--\n  Licensed to the Apache Software Foundation (ASF) under one\n  or more contributor license agreements.  See the NOTICE file\n  distributed with this work for additional information\n  regarding copyright ownership.  The ASF licenses this file\n  to you under the Apache License, Version 2.0 (the\n  \"License\"); you may not use this file except in compliance\n  with the License.  You may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\n  Unless required by applicable law or agreed to in writing,\n  software distributed under the License is distributed on an\n  \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n  KIND, either express or implied.  See the License for the\n  specific language governing permissions and limitations\n  under the License.\n--\u003e\nNOTE: Active development of this project has moved to https://github.com/tballison/quaerite.  The namespaces in the new repo and on maven central have been converted from 'org.mitre' to 'org.tallison'. This repository is no longer actively maintained.\n===========================\n\nWelcome to _Quaerite_\n=================================================\n\nBackground and Goals\n--------------------\n\nThis project includes tools to help evaluate relevance\nranking.  This code has been tested with Solr 4.x, 7.x and 8.x,\nand ES 6.x and 7.x.\n\nThis project is not intended to compete with existing relevance\nevaluation tools, such as [Splainer](http://splainer.io/),\n[Quepid](https://quepid.com/), [Rated Ranking Evaluator](https://github.com/SeaseLtd/rated-ranking-evaluator/wiki/Maven-Plugin),\nor [Luigi's Box](https://www.luigisbox.com/).\nRather, this project was developed for use cases not currently \ncovered by open source software packages. The author encourages \ncollaboration among these projects.\n\n**NOTE: This project is under construction and is quite dynamic.  \nThere will be breaking changes before the first major release.**\n\nWhile the name of this project may change in the future, we selected\n_quaerite_ -- Latin imperative \"seek\", root of English \"query\" -- to\nallude not only to the challenges of creating queries, but also\nto the challenges of tuning search engines.  One may spend\na not insignificant amount of time tuning countless parameters.\nIn the end, we hope that _invenietis_ with slightly less effort\nthan without this project. For the pronunciation, see\n[this link](https://forvo.com/word/quaerite_et_invenietis/).\n\nSimilarities and Differences between the Genetic Algorithm (GA) in _Quaerite_ and Learning to Rank\n------------------------------------------ \nIn the research literature, the application of a GA or Genetic Programming (GP) is one method for learning to rank (see, e.g. \n[Andrew Trotman](https://www.academia.edu/282562/Learning_to_Rank) on GP).\n\nHowever, for integrators and developers who work in the Lucene ecosystem, \"Learning to Rank\" (LTR) connotes\na specific methodology/module initially added to Apache Solr by [Bloomberg](https://www.techatbloomberg.com/blog/bloomberg-integrated-learning-rank-apache-solr/) \nand then offered as a plugin for ElasticSearch by \n[Doug Turnbull and colleagues at OpenSource Connections, Wikimedia Foundation and Snagajob Engineering](https://elasticsearch-learning-to-rank.readthedocs.io/en/latest/).\nIn the following, I use LTR to refer to this Lucene-ecosystem-specific module and methodology.  \n\nIn _no_ way do I see this implementation of GA as a competitor to LTR; rather, it is another tool that \nmight help complement LTR and/or other tuning methodologies.\n\n### Similarities\n* All of the basic requirements for quality search must first be met -- analyzer chains must be well designed for the data, \n    the underlying data in the index should be accurate, well organized and well curated \n* There must be sufficient, high quality, accurate and representative ground truth judgments for training and testing\n* Machine learning can only do so much -- further tuning and/or adding new methods of enrichment may be required\n\n### Differences\n* In practice, LTR is designed to perform more costly calculations as a re-ranking step...that is, after the search \nengine has returned the best `n` documents, LTR is typically applied to carry out more costly calculations on \nthis smaller subset of documents to re-rank the results based on the models built offline.  The goal of \nthis implementation of GA (and the other tools in _Quaerite_) is to help tune the parameters used in the initial \nsearch system's ranking, _NOT_ as part of a secondary reranking.\n* Bloomberg, OpenSource Connections, Wikimedia and Snagajob have spent quite a bit of time and effort \ndeveloping and integrating these modules to make them easy to use.  This toolkit has been developed with \nfar fewer resources for use initially by one relevance engineer...there are areas for improvement.\n\n\nCurrent Status\n----------------\nAs of this writing, `Quaerite` allows for experimentation with the following parameters:\n`bf`, `bq`, `qf`, `pf`, `pf2`, `pf3`, `ps`, `ps2`, `ps3`, `q.op` (and `mm`), `solr url` \n(so that you can run experiments against \ndifferent cores and/or different versions of Solr),\n`customHandler` (so that you can compare different customized handlers), `tie`.\nFor ES, specifically, parameters include: `boost`, `fuzziness` and `multi_match_type` \n(e.g. `best_fields`, `most_fields`, `cross_fields` and `phrase`).\n\n\nGetting Started\n---------------\nSee the ```quaerite-examples``` module and its [README](https://github.com/mitre/quaerite/blob/master/quaerite-examples/README.md).\n\nReleases\n--------------\n* [1.0.0-ALPHA](https://github.com/mitre/quaerite/releases) March 22, 2019\n\n\nRoad Map\n----------\nHigh priorities\n* Add other features (e.g. ```bq```, ```bf```) as needed\n* See the issues on [the issue tracker](https://github.com/mitre/quaerite/issues)\n\nPlanned Releases\n* 1.0.0-beta1 May, 2019\n\nRelated Open Source Projects\n---------------------\n* https://quepid.com/\n* https://github.com/SeaseLtd/rated-ranking-evaluator\n* http://splainer.io/\n* https://github.com/DiceTechJobs/RelevancyTuning\n\nLicense (see also LICENSE.txt)\n------------------------------\n\nCopyright (c) 2019, The MITRE Corporation. All rights reserved.\n\nApproved for Public Release; Distribution Unlimited. Case Number 18-3138-7.\n\n\nLicensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.  See the NOTICE file distributed with this work for additional information regarding copyright ownership.  The ASF licenses this file to You under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License.  You may obtain a copy of the License at\n\n\u003chttp://www.apache.org/licenses/LICENSE-2.0\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitre%2Fquaerite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmitre%2Fquaerite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitre%2Fquaerite/lists"}