{"id":43107154,"url":"https://github.com/hltcoe/routir","last_synced_at":"2026-01-31T18:11:00.184Z","repository":{"id":319938238,"uuid":"1073908775","full_name":"hltcoe/routir","owner":"hltcoe","description":"Fast and simple server to host state-of-the-art retrieval models","archived":false,"fork":false,"pushed_at":"2026-01-30T05:10:57.000Z","size":5550,"stargazers_count":10,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-30T21:34:10.604Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://hltcoe.github.io/routir/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hltcoe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-10T20:12:17.000Z","updated_at":"2026-01-30T05:10:10.000Z","dependencies_parsed_at":"2025-10-21T03:30:40.628Z","dependency_job_id":"6bfb5604-4e5e-4497-90ea-e78d208651b7","html_url":"https://github.com/hltcoe/routir","commit_stats":null,"previous_names":["hltcoe/routir"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/hltcoe/routir","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hltcoe%2Froutir","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hltcoe%2Froutir/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hltcoe%2Froutir/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hltcoe%2Froutir/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hltcoe","download_url":"https://codeload.github.com/hltcoe/routir/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hltcoe%2Froutir/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28949274,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-31T14:26:55.697Z","status":"ssl_error","status_checked_at":"2026-01-31T14:26:52.545Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-31T18:10:59.386Z","updated_at":"2026-01-31T18:11:00.176Z","avatar_url":"https://github.com/hltcoe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RoutIR: Fast Server for Hosting Retrieval Models for Retrieval-Augmented Generation\n\n\u003cdiv align=\"center\"\u003e\n\n[![PyPI version fury.io](https://badge.fury.io/py/routir.svg)](https://pypi.python.org/pypi/routir/)\n[![Made with Python](https://img.shields.io/badge/Python-\u003e=3.9-blue?logo=python\u0026logoColor=white)](https://python.org \"Go to Python homepage\")\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\n\u003c/div\u003e\n\nRoutIR is a Python package that provides a simple and efficient wrapper around arbitrary retrieval models, including first stage retrieval, reranking, query expansion, and result fusion, and provides efficient asynchronous query batching and serving. \n\n## Get Started\n\nYou can install routir in your environment through `pip` or `uv`. \n\n```bash\npip install routir\n```\n\nRoutIR comes with a number of extras to install only the dependencies for the models you would like to serve. \nThese extras include `dense`, `gpu`, `plaidx`, and `sparse`. \nYou can instal any combinations, such as \n\n```bash\npip install \"routir[dense,gpu]\"\n```\n\nTo start the service, simply provide the config file to the cli command `routir` to start. \nYou can also optionally specify the port through `--port` flag (default 8000). \n\n```bash\nroutir config.json --port 5000\n```\n\nYou can also use `uvx` to let `uv` creates a virtual environment on the fly for you:\n```bash\nuvx --with transformers --with torch routir config.json\n```\nUse `--with` to specify additional packages that you may need for serving the model. \nPlease refer to `uv` [documentation](https://docs.astral.sh/uv/) for more information. \n\n\n## Configuration\n\nThe configuration file four major blocks: `services`, `collections`, `server_imports`, and `file_imports`.\n- `services` and `collections` are list of object configuring each engine and each collection being served\n- `server_imports` is list of external RoutIR endpoints that you would like to mirror in this endpoint. This will allow the end users to construct retrieval pipelines using services hosted on other machines. This is particularly helpful in a distributed compute cluster.\n- `file_imports` is a list of custom Python scripts implemeting custom engines that RoutIR should load at initialization. More in the [Extension](#extension) section.\n\n\nFor example, if there are two other RoutIR instances running on `compute01:5000` and `compute02:5000` where each host `plaidx-neuclir` and `RankLllama`, you can import them as following. Users using this endpoint will be able to use both `plaidx-neuclir` and `RankLlama`. \nThe following is an example config. \n```json\n{\n    \"server_imports\": [\n        \"http://compute01:5000\",\n        \"http://compute02:5000\",\n    ],\n    \"file_imports\": [\n        \"./examples/rank1_extension.py\"\n    ],\n    \"services\": [\n        {\n            \"name\": \"qwen3-neuclir\",\n            \"engine\": \"Qwen3\", \n            \"cache\": 1024, \n            \"cache_ttl\": 1024000, \n            \"batch_size\": 32, \n            \"max_wait_time\": 0.05, \n            \"config\": {\n                \"index_path\": \"hfds:routir/neuclir-qwen3-8b-faiss-PQ2048x4fs\",\n                \"api_key\": \"YOUR_API_KEY_HERE OR AT OPENAI_API_KEY ENVIRONMENT VARIABLE\",\n                \"embedding_base_url\": \"https://api.fireworks.ai/inference/v1/\",\n                \"embedding_model_name\": \"accounts/fireworks/models/qwen3-embedding-8b\",\n                \"k_scale\": 5\n            }\n        },\n        {\n            \"name\": \"rank1\",\n            \"engine\": \"Rank1Engine\",\n            \"config\": {}\n        }\n    ],\n    \"collections\": [\n        {\n            \"name\": \"neuclir\",\n            \"doc_path\": \"./neuclir-doc.jsonl\"\n        }\n    ]\n}\n```\n\nIf you want to use Redis for caching, add `cache_redis_url` and `cache_redis_kwargs` to the `service` object. \nIf your Redis instance is password-protected (which you should), add `password` field to `cache_redis_kwargs`. \n\n## HTTP API\n\n1. Available services: GET `/avail`. An example output of the service initiated with the previous example config would be: \n```json\n{\n    \"content\": [\"neuclir\"],\n    \"score\": [\"Rank1\", \"RankLlama\"],\n    \"search\": [\"qwen3-neuclir\", \"plaidx-neuclir\"],\n    \"fuse\": [\"RRF\", \"ScoreFusion\"], \n    \"decompose_query\": []\n}\n```\n\n2. Search an index: POST `/search`. The following is an example request using `cURL`. \n```bash\ncurl -X POST http://localhost:5000/search \\\n-H \"Content-Type: application/json\" \\\n-d '{\"service\": \"qwen3-neuclir\", \"query\": \"my test queries\", \"limit\": 15}'\n```\nOutput: \n```json\n{\n  \"cached\": true,\n  \"processed\": true,\n  \"query\": \"my test queries\",\n  \"scores\": {\n    \"05a83946-dca2-4518-9bc3-3d394394d5e3\": 0.3807981014251709,\n    \"36faf9fc-3751-4047-bb1c-2bd90fa6f4d4\": 0.3675723671913147,\n    \"3a9ba832-f689-4204-8627-96abd73be65f\": 0.42572247982025146,\n    \"6a5b81f3-9154-4959-9e88-79edfcecb43f\": 0.3666379451751709,\n    \"6b086402-a00c-4fd8-8772-fade1f4b3198\": 0.3996303975582123,\n    \"76ec4dd1-fb6e-4a1e-b3e2-4b6214886e52\": 0.3723523020744324,\n    \"8c6e9e63-ea22-406e-a841-2dc645a3d2e2\": 0.4014992415904999,\n    \"90f2e4af-8a92-4869-9c73-013fead4876d\": 0.3644096851348877,\n    \"9dc749e8-f7a7-4c76-9883-03c7bc620d92\": 0.37544310092926025,\n    \"aa3542e0-0c62-4518-9a0a-07eaa5b1eb00\": 0.3768806755542755,\n    \"aeba1a4c-e02e-4d37-898c-68732c05b7d9\": 0.3764134645462036,\n    \"b564d3aa-983d-42a4-b5ba-e6d43e79c094\": 0.3760540783405304,\n    \"e46324a8-e9fb-442f-806d-1ed8f0efb2b0\": 0.37497588992118835,\n    \"f91c5cf9-020b-4019-a483-41aee141808c\": 0.3672129511833191,\n    \"fd6f8822-ddf4-4264-a449-5ecc7884c8ec\": 0.36940526962280273\n  },\n  \"service\": \"qwen3-neuclir\",\n  \"timestamp\": 1761023408.7890506\n}\n```\n\n3. Score/Rerank a list of text given a query: POST `/score`. \nThis allows you to score/rerank arbitrary pieces of text, such as document content, pasages in a document for context compression, \nor generated reponses for ranking answer relevancy. The following is an example request:\n```bash\ncurl -X POST http://localhost:5000/score \\\n-H \"Content-Type: application/json\" \\\n-d '{\n    \"service\": \"rank1\", \n    \"query\": \"what is routir\", \n    \"passages\": [\n        \"routir is a python package\", \n        \"sushi is the best food in the world\"\n    ]\n}'\n```\nOutput: \n```json\n{\n  \"cached\": false,\n  \"processed\": true,\n  \"query\": \"what is routir\",\n  \"scores\": [\n    0.9999997617631468,\n    7.889264466868659e-06\n  ],\n  \"service\": \"rank1\",\n  \"timestamp\": 1761026442.1780925\n}\n\n```\n\n4. Search with dynamic pipeline: POST `/pipeline`. \nThis allows the end users to construct an arbirary search pipeline with available engines on the fly. For example\n```bash\ncurl -X POST http://localhost:5000/pipeline \\\n-H \"Content-Type: application/json\" \\\n-d '{\n    \"pipeline\": \"{qwen3-neuclir, plaidx-neuclir}RRF%50 \u003e\u003e rank1\", \n    \"query\": \"which team is the world series champion in 2020?\",\n    \"collection\": \"neuclir\"\n}'\n```\nOutput: \n```json\n{\n  \"cached\": false,\n  \"collection\": \"neuclir\",\n  \"pipeline\": \"{qwen3-neuclir, plaidx-neuclir}RRF%50 \u003e\u003e rank1\",\n  \"processed\": true,\n  \"query\": \"which team is the world series champion in 2020?\",\n  \"scores\": {\n    \"027b3f6f-3dc6-4e69-86ae-2a98f8c4a881\": 0.999999712631481,\n    \"066e645a-a495-4622-bcc8-7a804f598bcf\": 5.4222202626709005e-06,\n    \"0ced1751-181a-4abb-8d64-37c362ede67c\": 0.9999986290429566,\n    \"1c0d1e33-ea2c-48f3-9422-6f81259095eb\": 0.9999996940976272,\n    \"27b429cc-b2a0-43cf-8b2b-883796486780\": 4.539786865487149e-05,\n    \"2d11d0a3-78de-4201-ad26-64a6ac4b148f\": 1.8925157266468097e-05,\n    \"302d1c1a-d620-4971-a44c-c1faead39494\": 1.6701429809483402e-05,\n    \"39d2608d-e0a5-4b52-bb0e-b04968e21a15\": 0.033085980653064666,\n    \"3c3c49f3-24b1-4dad-ac57-35b0565ab9b8\": 2.1444943303118133e-05,\n    \"6e65cae3-443d-4cdd-9efc-8dfb3e1fe0b1\": 6.962258739847376e-06,\n    \"7e4a4d57-9e73-4fb6-8ea7-584d0549c508\": 0.0052201256185966365,\n    \"7ecdc77d-ea8c-4d48-9235-c21df9086831\": 0.9999999397642365,\n    \"8660ca1b-ef5a-4c3a-a3e9-692e1e686f07\": 1.9947301971022554e-06,\n    \"88b3eff5-738a-4bcd-b31a-15d9f1b9e198\": 3.288748281343353e-06,\n    \"940bb6ff-f88a-40cd-88c1-2ae719d1dc74\": 0.99999980249468,\n    \"a3edc861-7cf5-4152-a32b-90961bd12b80\": 1.8925155010490798e-05,\n    \"c26f5a26-e732-4deb-80d6-b3ad6b249927\": 0.9999986290426297,\n    \"da57d712-c6a8-4fa3-8e68-7f21ea7d3167\": 0.9999996072138465,\n    \"ed231d01-05d9-4ed6-98d6-97b4e3e64aae\": 0.00317268301626477,\n    \"f3954f32-62e6-4cb3-9ef2-78fe3dcb8f7a\": 1.9947304348917116e-06\n  },\n  \"service\": \"rank1\",\n  \"timestamp\": 1761026586.5823486\n}\n```\n\n\n\n\n## Extension Examples\n\nWe provide several examples for integrating other IR toolkits with RoutIR. \nPlease refer to each example for details. \n\n\u003e [!WARNING]\n\u003e The Python script implementing the custom Engine needs to be imported through `file_imports` in the config. \n\u003e When using `uvx`, remember to put the essential packages at `--with`. \n\n1. PyTerrier\n```bash\npython ./examples/pyterrier_extension.py # to build the index\nuvx --with python-terrier routir ./examples/pyterrier_example_config.json --port 8000 # serve it at port 8000\n```\n\n2. Pyserini\n```bash\nuvx --with pyserini routir ./examples/pyserini_example_config.json --port 8000 # serve it at port 8000\n```\n\n3. Rank1\n```bash\nuvx --with mteb==1.39.0 --with vllm routir ./examples/rank1_example_config.json\n```\nThe specific mteb version is crucial for this example. \n\n\n## Other Helper Scripts \nHere is an example command to generate `.npy` files containing Qwen3 document embeddings from a `.jsonl` file with `id`, `title`, and `text` fields:\n\n```bash\npython -m routir.utils.qwen3_encode /path/to/docs.jsonl /output/path \\\n--id-field id --fields title text --docs-per-file 10000\n--batch-size 8 --model-name Qwen/Qwen3-Embedding-8B\n```\n\n\nTo provide reference for the FAISS index structure that RoutIR uses, you can refer to the \n`routir.utils.faiss_indexing` for details. \nHere is an example command to generate a FAISS index from a directory containing `.npy` files, each with `features` and `ids` fields (as generated by the above script):\n\n```bash\npython -m routir.utils.faiss_indexing \\\n./encoded_vectors/ ./faiss_index.PQ2048x4fs.IP/ \\\n--index_string \"PQ2048x4fs\" --use_gpu --sampling_rate 0.25\n```\n\n## Contribution\n\nWe welcome any feedback, feature requests and pull requests. Please raise issues on GitHub.\nFeel free to reach out to us through emails, ACM SIGIR Slack, or GitHub issues. \n\n## Attribution\nTBA\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhltcoe%2Froutir","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhltcoe%2Froutir","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhltcoe%2Froutir/lists"}