{"id":24985522,"url":"https://github.com/castorini/rank_llm","last_synced_at":"2025-04-08T19:21:10.687Z","repository":{"id":196728879,"uuid":"675847796","full_name":"castorini/rank_llm","owner":"castorini","description":"RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking.","archived":false,"fork":false,"pushed_at":"2025-03-28T15:57:10.000Z","size":33730,"stargazers_count":420,"open_issues_count":20,"forks_count":56,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-04-01T18:19:53.483Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://rankllm.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/castorini.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-07T21:49:33.000Z","updated_at":"2025-03-29T12:55:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"8c5ce601-65b6-40af-a9a9-07595abfbe57","html_url":"https://github.com/castorini/rank_llm","commit_stats":null,"previous_names":["castorini/rank_llm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/castorini%2Frank_llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/castorini%2Frank_llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/castorini%2Frank_llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/castorini%2Frank_llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/castorini","download_url":"https://codeload.github.com/castorini/rank_llm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247909203,"owners_count":21016479,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-04T10:52:53.004Z","updated_at":"2025-04-08T19:21:10.664Z","avatar_url":"https://github.com/castorini.png","language":"Python","funding_links":[],"categories":["文本匹配_文本检索_文本相似度","Python"],"sub_categories":["大语言对话模型及数据"],"readme":"# RankLLM\n\n[![PyPI](https://img.shields.io/pypi/v/rank-llm?color=brightgreen)](https://pypi.org/project/rank-llm/)\n[![Downloads](https://static.pepy.tech/personalized-badge/rank-llm?period=total\u0026units=international_system\u0026left_color=grey\u0026right_color=brightgreen\u0026left_text=downloads)](https://pepy.tech/project/rank-llm)\n[![Downloads](https://static.pepy.tech/personalized-badge/rank-llm?period=week\u0026units=international_system\u0026left_color=grey\u0026right_color=brightgreen\u0026left_text=downloads/week)](https://pepy.tech/project/rank-llm)\n[![Generic badge](https://img.shields.io/badge/arXiv-2309.15088-red.svg)](https://arxiv.org/abs/2309.15088)\n[![LICENSE](https://img.shields.io/badge/license-Apache-blue.svg?style=flat)](https://www.apache.org/licenses/LICENSE-2.0)\n\n\nWe offer a suite of rerankers - pointwise models like MonoT5, pairwise models like DuoT5 and listwise models with a focus on open source LLMs compatible with [vLLM](https://https://github.com/vllm-project/vllm), [SGLang](https://github.com/sgl-project/sglang), or [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM). We also support RankGPT and RankGemini variants, which are proprietary listwise rerankers. Addtionally, we support reranking with the first-token logits only to improve inference efficiency.  Some of the code in this repository is borrowed from [RankGPT](https://github.com/sunnweiwei/RankGPT), [PyGaggle](https://github.com/castorini/pygaggle), and [LiT5](https://github.com/castorini/LiT5)!\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"docs/images/rankllm_overview.png\" alt=\"RankLLM Overview\" style=\"width:95%;\"\u003e\n\u003c/p\u003e\n\n## Releases\ncurrent_version = 0.21.0\n\n## Content\n1. [Installation](#installation)\n2. [Quick Start](#quick-start)\n3. [End-to-end Run and 2CR](#end-to-end-run-and-2cr)\n4. [Model Zoo](#model-zoo)\n5. [Training](#training)\n6. [Community Contribution](#community-contribution)\n7. [References and Citations](#references)\n8. [Acknowledgments](#acknowledgments)\n\n\u003ca id=\"installation\"\u003e\u003c/a\u003e\n# 📟 Installation\n\n\u003e **⚠️ RankLLM is not compatible with macOS**, regardless of whether you are using an Intel-based Mac or Apple Silicon (M-series). We recommend using Linux or Windows instead.\n\n## ❗ JDK 21 Warning\n\nAs rank_llm relies on [Anserini](https://github.com/castorini/anserini), it is required that you have JDK 21 installed.\nPlease note that using JDK 11 is not supported and may lead to errors.\n\n## Create Conda Environment\n\n```bash\nconda create -n rankllm python=3.10\nconda activate rankllm\n```\n\n## Install Pytorch with CUDA (Windows/Linux)\n```bash\npip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121\n```\n\n## Install OpenJDK with Maven if you want to use the retriever\n```bash\nconda install -c conda-forge openjdk=21 maven -y\n```\n\n## Install Dependencies\n```bash\npip install -r requirements.txt\n```\n\n## Install SGLang, or TensorRT-LLM (Optional)\n\n### Install SGLang (Optional)\n\n```bash\npip install -e .[sglang]      # local installation for development\npip install rank-llm[sglang]  # or pip installation\n```\n\nRemember to install flashinfer to use `SGLang` backend.\n\n```bash\npip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/\n```\n\n### TensorRT-LLM\n\n```bash\npip install -e .[tensorrt-llm]      # local installation for development\npip install rank-llm[tensorrt-llm]  # or pip installation\n```\n\n\u003ca id=\"quick-start\"\u003e\u003c/a\u003e\n# ⏳ Quick Start\nThe following code snippet is a minimal walk through of retrieval, reranking, evalaution, and invocations analysis of top 100 retrieved documents for queries from `DL19`. In this example `BM25` is used as the retriever and `RankZephyr` as the reranker. Additional sample snippets are available to run under the `src/rank_llm/demo` directory.\n```python\nfrom pathlib import Path\n\nfrom rank_llm.analysis.response_analysis import ResponseAnalyzer\nfrom rank_llm.data import DataWriter\nfrom rank_llm.evaluation.trec_eval import EvalFunction\nfrom rank_llm.rerank import Reranker, get_openai_api_key\nfrom rank_llm.rerank.listwise import (\n    SafeOpenai,\n    VicunaReranker,\n    ZephyrReranker,\n)\nfrom rank_llm.retrieve.retriever import Retriever\nfrom rank_llm.retrieve.topics_dict import TOPICS\n\n# -------- Retrieval --------\n\n# By default BM25 is used for retrieval of top 100 candidates.\ndataset_name = \"dl19\"\nretrieved_results = Retriever.from_dataset_with_prebuilt_index(dataset_name)\n\n# Users can specify other retrieval methods and number of retrieved candidates.\n# retrieved_results = Retriever.from_dataset_with_prebuilt_index(\n#     dataset_name, RetrievalMethod.SPLADE_P_P_ENSEMBLE_DISTIL, k=50\n# )\n# ---------------------------\n\n# --------- Rerank ----------\n\n# Rank Zephyr model\nreranker = ZephyrReranker()\n\n# Rank Vicuna model\n# reranker = VicunaReranker()\n\n# RankGPT\n# model_coordinator = SafeOpenai(\"gpt-4o-mini\", 4096, keys=get_openai_api_key())\n# reranker = Reranker(model_coordinator)\n\nrerank_results = reranker.rerank_batch(requests=retrieved_results)\n# ---------------------------\n\n# ------- Evaluation --------\n\n# Evaluate retrieved results.\nndcg_10_retrieved = EvalFunction.from_results(retrieved_results, TOPICS[dataset_name])\nprint(ndcg_10_retrieved)\n\n# Evaluate rerank results.\nndcg_10_rerank = EvalFunction.from_results(rerank_results, TOPICS[dataset_name])\nprint(ndcg_10_rerank)\n\n# By default ndcg@10 is the eval metric, other value can be specified:\n# eval_args = [\"-c\", \"-m\", \"map_cut.100\", \"-l2\"]\n# map_100_rerank = EvalFunction.from_results(rerank_results, topics, eval_args)\n# print(map_100_rerank)\n\n# eval_args = [\"-c\", \"-m\", \"recall.20\"]\n# recall_20_rerank = EvalFunction.from_results(rerank_results, topics, eval_args)\n# print(recall_20_rerank)\n\n# ---------------------------\n\n# --- Analyze invocations ---\nanalyzer = ResponseAnalyzer.from_inline_results(rerank_results)\nerror_counts = analyzer.count_errors(verbose=True)\nprint(error_counts)\n# ---------------------------\n\n# ------ Save results -------\nwriter = DataWriter(rerank_results)\nPath(f\"demo_outputs/\").mkdir(parents=True, exist_ok=True)\nwriter.write_in_jsonl_format(f\"demo_outputs/rerank_results.jsonl\")\nwriter.write_in_trec_eval_format(f\"demo_outputs/rerank_results.txt\")\nwriter.write_inference_invocations_history(\n    f\"demo_outputs/inference_invocations_history.json\"\n)\n# ---------------------------\n```\n\n# End-to-end Run and 2CR\nIf you are interested in running retrieval and reranking end-to-end or reproducing the results from the [reference papers](#✨-references), `run_rank_llm.py` is a convinent wrapper script that combines these two steps.\n\nThe comperehensive list of our two-click reproduction commands are available on [MS MARCO V1](https://castorini.github.io/rank_llm/src/rank_llm/2cr/msmarco-v1-passage.html) and [MS MARCO V2](https://castorini.github.io/rank_llm/src/rank_llm/2cr/msmarco-v2-passage.html) webpages for DL19 and DL20 and DL21-23 datasets, respectively. Moving forward, we plan to cover more datasets and retrievers in our 2CR pages. The rest of this session provides some sample e2e runs. \n## RankZephyr\n\nWe can run the RankZephyr model with the following command:\n```bash\npython src/rank_llm/scripts/run_rank_llm.py  --model_path=castorini/rank_zephyr_7b_v1_full --top_k_candidates=100 --dataset=dl20 \\\n--retrieval_method=SPLADE++_EnsembleDistil_ONNX --prompt_mode=rank_GPT  --context_size=4096 --variable_passages\n```\n\nIncluding the `--sglang_batched` flag will allow you to run the model in batched mode using the `SGLang` library.\n\nIncluding the `--tensorrt_batched` flag will allow you to run the model in batched mode using the `TensorRT-LLM` library.\n\nIf you want to run multiple passes of the model, you can use the `--num_passes` flag.\n\n## RankGPT4-o\n\nWe can run the RankGPT4-o model with the following command:\n```bash\npython src/rank_llm/scripts/run_rank_llm.py  --model_path=gpt-4o --top_k_candidates=100 --dataset=dl20 \\\n  --retrieval_method=bm25 --prompt_mode=rank_GPT_APEER  --context_size=4096 --use_azure_openai\n```\nNote that the `--prompt_mode` is set to `rank_GPT_APEER` to use the LLM refined prompt from [APEER](https://arxiv.org/abs/2406.14449).\nThis can be changed to `rank_GPT` to use the original prompt.\n\n## LiT5\n\nWe can run the LiT5-Distill V2 model (which could rerank 100 documents in a single pass) with the following command:\n\n```bash\npython src/rank_llm/scripts/run_rank_llm.py  --model_path=castorini/LiT5-Distill-large-v2 --top_k_candidates=100 --dataset=dl19 \\\n    --retrieval_method=bm25 --prompt_mode=LiT5  --context_size=150 --batch_size=4 \\\n    --variable_passages --window_size=100\n```\n\nWe can run the LiT5-Distill original model (which works with a window size of 20) with the following command:\n\n```bash\npython src/rank_llm/scripts/run_rank_llm.py  --model_path=castorini/LiT5-Distill-large --top_k_candidates=100 --dataset=dl19 \\\n    --retrieval_method=bm25 --prompt_mode=LiT5  --context_size=150 --batch_size=32 \\\n    --variable_passages\n```\n\nWe can run the LiT5-Score model with the following command:\n\n```bash\npython src/rank_llm/scripts/run_rank_llm.py  --model_path=castorini/LiT5-Score-large --top_k_candidates=100 --dataset=dl19 \\\n    --retrieval_method=bm25 --prompt_mode=LiT5 --context_size=150 --batch_size=8 \\\n    --window_size=100 --variable_passages\n```\n\n## MonoT5\n\nThe following runs the 3B variant of MonoT5 trained for 10K steps:\n\n```bash\npython src/rank_llm/scripts/run_rank_llm.py --model_path=castorini/monot5-3b-msmarco-10k --top_k_candidates=1000 --dataset=dl19 \\\n    --retrieval_method=bm25 --prompt_mode=monot5 --context_size=512\n```\n\nNote that we usually rerank 1K candidates with MonoT5.\n\n## DuoT5\nThe following runs the #B variant of DuoT5 trained for 10K steps:\n```bash\npython src/rank_llm/scripts/run_rank_llm.py --model_path=castorini/duot5-3b-msmarco-10k --top_k_candidates=50 --dataset=dl19 \\\n    --retrieval_method=bm25 --prompt_mode=duot5\n```\n\nSince Duo's pairwise comparison has $O(n^2) runtime complexity, we recommend reranking top 50 candidates using DuoT5 models.\n\n## FirstMistral\n\nWe can run the FirstMistral model, reranking using the first-token logits only with the following command:\n\n```\npython src/rank_llm/scripts/run_rank_llm.py  --model_path=castorini/first_mistral --top_k_candidates=100 --dataset=dl20 --retrieval_method=SPLADE++_EnsembleDistil_ONNX --prompt_mode=rank_GPT  --context_size=4096 --variable_passages --use_logits --use_alpha --num_gpus 1\n```\n\nOmit `--use_logits` if you wish to perform traditional listwise reranking.\n\n## Gemini Flash 2.0\n\nFirst install genai:\n\n```bash\npip install -e .[genai]      # local installation for development\npip install rank-llm[genai]  # or pip installation\n```\n\nThen run the following command:\n\n```bash\npython src/rank_llm/scripts/run_rank_llm.py  --model_path=gemini-2.0-flash-001 --top_k_candidates=100 --dataset=dl20 \\\n    --retrieval_method=SPLADE++_EnsembleDistil_ONNX --prompt_mode=rank_GPT_APEER  --context_size=4096\n```\n\n\u003ca id=\"model-zoo\"\u003e\u003c/a\u003e\n# 🦙🐧 Model Zoo\n\nThe following is a table of the listwise models our repository was primarily built to handle (with the models hosted on HuggingFace):\n\n`vLLM`, `SGLang`, and `TensorRT-LLM` backends are only supported for `RankZephyr` and `RankVicuna` models.\n\n| Model Name        | Hugging Face Identifier/Link                            |\n|-------------------|---------------------------------------------|\n| RankZephyr 7B V1 - Full - BF16      | [castorini/rank_zephyr_7b_v1_full](https://huggingface.co/castorini/rank_zephyr_7b_v1_full)               |\n| RankVicuna 7B - V1      | [castorini/rank_vicuna_7b_v1](https://huggingface.co/castorini/rank_vicuna_7b_v1)               |\n| RankVicuna 7B - V1 - No Data Augmentation    | [castorini/rank_vicuna_7b_v1_noda](https://huggingface.co/castorini/rank_vicuna_7b_v1_noda)               |\n| RankVicuna 7B - V1 - FP16      | [castorini/rank_vicuna_7b_v1_fp16](https://huggingface.co/castorini/rank_vicuna_7b_v1_fp16)               |\n| RankVicuna 7B - V1 - No Data Augmentation - FP16   | [castorini/rank_vicuna_7b_v1_noda_fp16](https://huggingface.co/castorini/rank_vicuna_7b_v1_noda_fp16)               |\n\nWe also officially support the following rerankers built by our group:\n\n## LiT5 Suite\n\nThe following is a table specifically for our LiT5 suite of models hosted on HuggingFace:\n\n| Model Name            | 🤗 Hugging Face Identifier/Link                            |\n|-----------------------|---------------------------------------------|\n| LiT5 Distill base     | [castorini/LiT5-Distill-base](https://huggingface.co/castorini/LiT5-Distill-base)          |\n| LiT5 Distill large    | [castorini/LiT5-Distill-large](https://huggingface.co/castorini/LiT5-Distill-large)        |\n| LiT5 Distill xl       | [castorini/LiT5-Distill-xl](https://huggingface.co/castorini/LiT5-Distill-xl)              |\n| LiT5 Distill base v2  | [castorini/LiT5-Distill-base-v2](https://huggingface.co/castorini/LiT5-Distill-base-v2)    |\n| LiT5 Distill large v2 | [castorini/LiT5-Distill-large-v2](https://huggingface.co/castorini/LiT5-Distill-large-v2)  |\n| LiT5 Distill xl v2    | [castorini/LiT5-Distill-xl-v2](https://huggingface.co/castorini/LiT5-Distill-xl-v2)        |\n| LiT5 Score base       | [castorini/LiT5-Score-base](https://huggingface.co/castorini/LiT5-Score-base)              |\n| LiT5 Score large      | [castorini/LiT5-Score-large](https://huggingface.co/castorini/LiT5-Score-large)            |\n| LiT5 Score xl         | [castorini/LiT5-Score-xl](https://huggingface.co/castorini/LiT5-Score-xl)                  |\n\nNow you can run top-100 reranking with the v2 model in a single pass while maintaining efficiency!\n\n## MonoT5 Suite - Pointwise Rerankers\n\nThe following is a table specifically for our monoT5 suite of models hosted on HuggingFace:\n\n| Model Name                        | 🤗 Hugging Face Identifier/Link                            |\n|-----------------------------------|--------------------------------------------------------|\n| monoT5 Small MSMARCO 10K          | [castorini/monot5-small-msmarco-10k](https://huggingface.co/castorini/monot5-small-msmarco-10k)       |\n| monoT5 Small MSMARCO 100K         | [castorini/monot5-small-msmarco-100k](https://huggingface.co/castorini/monot5-small-msmarco-100k)     |\n| monoT5 Base MSMARCO               | [castorini/monot5-base-msmarco](https://huggingface.co/castorini/monot5-base-msmarco)                 |\n| monoT5 Base MSMARCO 10K           | [castorini/monot5-base-msmarco-10k](https://huggingface.co/castorini/monot5-base-msmarco-10k)         |\n| monoT5 Large MSMARCO 10K          | [castorini/monot5-large-msmarco-10k](https://huggingface.co/castorini/monot5-large-msmarco-10k)       |\n| monoT5 Large MSMARCO              | [castorini/monot5-large-msmarco](https://huggingface.co/castorini/monot5-large-msmarco)               |\n| monoT5 3B MSMARCO 10K             | [castorini/monot5-3b-msmarco-10k](https://huggingface.co/castorini/monot5-3b-msmarco-10k)             |\n| monoT5 3B MSMARCO                 | [castorini/monot5-3b-msmarco](https://huggingface.co/castorini/monot5-3b-msmarco)                     |\n| monoT5 Base Med MSMARCO           | [castorini/monot5-base-med-msmarco](https://huggingface.co/castorini/monot5-base-med-msmarco)         |\n| monoT5 3B Med MSMARCO             | [castorini/monot5-3b-med-msmarco](https://huggingface.co/castorini/monot5-3b-med-msmarco)             |\n\nWe recommend the Med models for biomedical retrieval. We also provide both 10K (generally better OOD effectiveness) and 100K checkpoints (better in-domain).\n# Training\nPlease check the `training` directory for finetuning open-source listwise rerankers.\n# Community Contribution\nIf you would like to contribute to the project, please refer to the [contribution guidelines](CONTRIBUTING.md).\n\n\u003ca id=references\u003e\u003c/a\u003e\n# ✨ References\n\nIf you use RankLLM, please cite the following relevant papers:\n\n[[2309.15088] RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models](https://arxiv.org/abs/2309.15088)\n\n\u003c!-- {% raw %} --\u003e\n```\n@ARTICLE{pradeep2023rankvicuna,\n  title   = {{RankVicuna}: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models},\n  author  = {Ronak Pradeep and Sahel Sharifymoghaddam and Jimmy Lin},\n  year    = {2023},\n  journal = {arXiv:2309.15088}\n}\n```\n\u003c!-- {% endraw %} --\u003e\n\n\n[[2312.02724] RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://arxiv.org/abs/2312.02724)\n\n\u003c!-- {% raw %} --\u003e\n```\n@ARTICLE{pradeep2023rankzephyr,\n  title   = {{RankZephyr}: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!},\n  author  = {Ronak Pradeep and Sahel Sharifymoghaddam and Jimmy Lin},\n  year    = {2023},\n  journal = {arXiv:2312.02724}\n}\n```\n\u003c!-- {% endraw %} --\u003e\n\nIf you use one of the LiT5 models please cite the following relevant paper:\n\n[[2312.16098] Scaling Down, LiTting Up: Efficient Zero-Shot Listwise Reranking with Seq2seq Encoder-Decoder Models](https://arxiv.org/abs/2312.16098)\n\n```\n@ARTICLE{tamber2023scaling,\n  title   = {Scaling Down, LiTting Up: Efficient Zero-Shot Listwise Reranking with Seq2seq Encoder-Decoder Models},\n  author  = {Manveer Singh Tamber and Ronak Pradeep and Jimmy Lin},\n  year    = {2023},\n  journal = {arXiv:2312.16098}\n}\n```\n\nIf you use one of the monoT5 models please cite the following relevant paper:\n\n[[2101.05667] The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models](https://arxiv.org/abs/2101.05667)\n\n```\n@ARTICLE{pradeep2021emd,\n  title = {The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models},\n  author = {Ronak Pradeep and Rodrigo Nogueira and Jimmy Lin},\n  year = {2021},\n  journal = {arXiv:2101.05667},\n}\n```\n\nIf you use the FirstMistral model, please consider citing:\n\n[[2411.05508] An Early FIRST Reproduction and Improvements to Single-Token Decoding for Fast Listwise Reranking](https://arxiv.org/abs/2411.05508)\n\n```\n@ARTICLE{chen2024firstrepro,\n  title   = title={An Early FIRST Reproduction and Improvements to Single-Token Decoding for Fast Listwise Reranking},\n  author  = {Zijian Chen and Ronak Pradeep and Jimmy Lin},\n  year    = {2024},\n  journal = {arXiv:2411.05508}\n}\n```\n\nIf you would like to cite the FIRST methodology, please consider citing:\n\n[[2406.15657] FIRST: Faster Improved Listwise Reranking with Single Token Decoding](https://arxiv.org/abs/2406.15657)\n\n```\n@ARTICLE{reddy2024first,\n  title   = {FIRST: Faster Improved Listwise Reranking with Single Token Decoding},\n  author  = {Reddy, Revanth Gangi and Doo, JaeHyeok and Xu, Yifei and Sultan, Md Arafat and Swain, Deevya and Sil, Avirup and Ji, Heng},\n  year    = {2024}\n  journal = {arXiv:2406.15657},\n}\n```\n\u003ca id=acknowledgments\u003e\u003c/a\u003e\n# 🙏 Acknowledgments\n\nThis research is supported in part by the Natural Sciences and Engineering Research Council (NSERC) of Canada.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcastorini%2Frank_llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcastorini%2Frank_llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcastorini%2Frank_llm/lists"}