{"id":17873941,"url":"https://github.com/emingenc/llm_adaptive_router","last_synced_at":"2025-11-11T18:08:32.845Z","repository":{"id":257824069,"uuid":"871802571","full_name":"emingenc/llm_adaptive_router","owner":"emingenc","description":"llm_adaptive_router: Given the following query and candidate models, selects the most appropriate model","archived":false,"fork":false,"pushed_at":"2024-10-28T05:13:15.000Z","size":20,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-23T17:18:26.095Z","etag":null,"topics":["langchain","langgraph","language-model","llm","ollama","openai"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/emingenc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-13T01:25:15.000Z","updated_at":"2024-12-07T22:21:02.000Z","dependencies_parsed_at":"2025-03-21T22:31:36.665Z","dependency_job_id":"ccb807d0-7e33-48f7-8816-a8892e3ea4be","html_url":"https://github.com/emingenc/llm_adaptive_router","commit_stats":null,"previous_names":["emingenc/llm_adaptive_router"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/emingenc/llm_adaptive_router","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emingenc%2Fllm_adaptive_router","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emingenc%2Fllm_adaptive_router/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emingenc%2Fllm_adaptive_router/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emingenc%2Fllm_adaptive_router/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/emingenc","download_url":"https://codeload.github.com/emingenc/llm_adaptive_router/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emingenc%2Fllm_adaptive_router/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261162066,"owners_count":23118221,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["langchain","langgraph","language-model","llm","ollama","openai"],"created_at":"2024-10-28T11:06:08.937Z","updated_at":"2025-11-11T18:08:27.786Z","avatar_url":"https://github.com/emingenc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM Adaptive Router\n\nLLM Adaptive Router is a Python package that enables dynamic model selection based on query content. It uses efficient vector search for initial categorization and LLM-based fine-grained selection for complex cases. The router can adapt and learn from feedback, making it suitable for a wide range of applications.\n\n## Features\n\n- Dynamic model selection based on query content\n- Efficient vector search for initial categorization\n- LLM-based fine-grained selection for complex cases\n- Adaptive learning from feedback\n- Flexible configuration of routes and models\n- Easy integration with LangChain and various LLM providers\n\n## Installation\n\nYou can install LLM Adaptive Router using pip:\n\n```bash\npip3 install llm-adaptive-router\n```\n\n## Quick Start\n\nHere's a basic example of how to use LLM Adaptive Router:\n\n```python\nfrom llm_adaptive_router import AdaptiveRouter, RouteMetadata\nfrom langchain_chroma import Chroma\nfrom langchain_openai import OpenAIEmbeddings, ChatOpenAI\nfrom dotenv import load_dotenv\nload_dotenv()\n\ngpt_3_5_turbo = ChatOpenAI(model=\"gpt-3.5-turbo\")\nmini = ChatOpenAI(model=\"gpt-4o-mini\")\ngpt_4 = ChatOpenAI(model=\"gpt-4\")\n\nroutes = {\n    \"general\": RouteMetadata(\n        invoker=gpt_3_5_turbo,\n        capabilities=[\"general knowledge\"],\n        cost=0.002,\n        example_sentences=[\"What is the capital of France?\", \"Explain photosynthesis.\"]\n    ),\n    \"mini\": RouteMetadata(\n        invoker=mini,\n        capabilities=[\"general knowledge\"],\n        cost=0.002,\n        example_sentences=[\"What is the capital of France?\", \"Explain photosynthesis.\"]\n        \n    ),\n    \"math\": RouteMetadata(\n        invoker=gpt_4,\n        capabilities=[\"advanced math\", \"problem solving\"],\n        cost=0.01,\n        example_sentences=[\"Solve this differential equation.\", \"Prove the Pythagorean theorem.\"]\n    )\n}\n\nllm = ChatOpenAI(model=\"gpt-3.5-turbo\")\n\nrouter = AdaptiveRouter(\n    vectorstore=Chroma(embedding_function=OpenAIEmbeddings()),\n    llm=llm,\n    embeddings=OpenAIEmbeddings(),\n    routes=routes\n)\n\nquery = \"How are you\"\nquery2 = \"Write a Python function to hello world\"\nselected_model_route = router.route(query)\nselected_model_name = selected_model_route\nprint(selected_model_name)\ninvoker = selected_model_route.invoker\nresponse = invoker.invoke(query)\n\nprint(f\"Response: {response}\")\n```\n\n## Detailed Usage\n\n### Creating Route Metadata\n\nUse the `create_route_metadata` function to define routes:\n\n```python\nfrom llm_adaptive_router import create_route_metadata\n\nroute = create_route_metadata(\n    invoker=model_function,\n    capabilities=[\"capability1\", \"capability2\"],\n    cost=0.01,\n    example_sentences=[\"Example query 1\", \"Example query 2\"],\n    additional_info={\"key\": \"value\"}\n)\n```\n\n### Initializing the AdaptiveRouter\n\nCreate an instance of `AdaptiveRouter` with your configured routes:\n\n```python\nrouter = AdaptiveRouter(\n    vectorstore=your_vectorstore,\n    llm=your_llm,\n    embeddings=your_embeddings,\n    routes=your_routes\n)\n```\n\n### Routing Queries\n\nUse the `route` method to select the appropriate model for a query:\n\n```python\nselected_model_route = router.route(\"Your query here\")\nselected_model_name = selected_model_route.model\ninvoker = selected_model_route.invoker\nresponse = invoker.invoke(\"Your query here\")\n```\n\n### Adding Feedback\n\nImprove the router's performance by providing feedback:\n\n```python\nrouter.add_feedback(query, selected_model, performance_score)\n```\n\n### Advanced Features\n\n- Custom Vector Stores: LLM Adaptive Router supports various vector stores. You can use any vector store that implements the `VectorStore` interface from LangChain.\n- Dynamic Route Updates: You can add or remove routes dynamically:\n\n```python\nrouter.add_route(\"new_route\", new_route_metadata)\nrouter.remove_route(\"old_route\")\n```\n\n- Adjusting Router Behavior: Fine-tune the router's behavior:\n\n```python\nrouter.set_complexity_threshold(0.8)\nrouter.set_update_frequency(200)\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femingenc%2Fllm_adaptive_router","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femingenc%2Fllm_adaptive_router","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femingenc%2Fllm_adaptive_router/lists"}