{"id":51429998,"url":"https://github.com/unohee/freerouter","last_synced_at":"2026-07-05T03:03:07.955Z","repository":{"id":368352844,"uuid":"1284734097","full_name":"unohee/freerouter","owner":"unohee","description":"OpenAI-compatible proxy \u0026 library that auto-routes across OpenRouter's free models, with fallback on rate limits. Run it as a server or import it as a Python submodule.","archived":false,"fork":false,"pushed_at":"2026-06-30T06:51:49.000Z","size":34,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-30T08:24:21.022Z","etag":null,"topics":["api-gateway","fastapi","free-models","llm","openai","openrouter","proxy","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/unohee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-30T06:34:36.000Z","updated_at":"2026-06-30T06:51:52.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/unohee/freerouter","commit_stats":null,"previous_names":["unohee/freerouter"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/unohee/freerouter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unohee%2Ffreerouter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unohee%2Ffreerouter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unohee%2Ffreerouter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unohee%2Ffreerouter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/unohee","download_url":"https://codeload.github.com/unohee/freerouter/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unohee%2Ffreerouter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35141967,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-05T02:00:06.290Z","response_time":100,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-gateway","fastapi","free-models","llm","openai","openrouter","proxy","python"],"created_at":"2026-07-05T03:03:04.313Z","updated_at":"2026-07-05T03:03:07.928Z","avatar_url":"https://github.com/unohee.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# freerouter\n\n[![CI](https://github.com/unohee/freerouter/actions/workflows/ci.yml/badge.svg)](https://github.com/unohee/freerouter/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n\nAn OpenAI-compatible proxy that gathers **only OpenRouter's free model endpoints** and\nroutes/falls back across them automatically.\n\nPoint any existing OpenAI SDK/client at freerouter, and it picks a model from OpenRouter's\nfree pool (`pricing.prompt == 0 \u0026\u0026 pricing.completion == 0`), then **falls back to the next\nfree model automatically** on a rate limit (429) or a transient error.\n\n## How it works\n\n1. On startup, fetch OpenRouter `/api/v1/models`, **keep only free models**, and cache them\n   (TTL 600s by default).\n2. On `POST /v1/chat/completions`, the router builds a candidate order.\n   - A specific free `model` goes first; otherwise (`auto`/empty/paid/unknown) the free-pool\n     priority is used (`:free`-suffixed first, then larger context).\n   - A model that just returned 429 is pushed back during its cooldown (60s by default).\n3. Try candidates in order, falling back on 429/402/404/5xx. If all fail, return 502.\n4. The response header `X-freerouter-Model` carries the model actually used.\n\n## Free models\n\nThe routable pool is OpenRouter's free, chat-capable models. The list below is generated from\nthe live catalog — run `python scripts/update_models.py` to refresh it locally, and CI refreshes\nit automatically once a week.\n\n\u003c!-- FREE_MODELS:START --\u003e\n_26 free, chat-capable models (prompt + completion priced at $0). Auto-generated by `scripts/update_models.py`._\n\n| Model ID | Context |\n|----------|---------|\n| `qwen/qwen3-coder:free` | 1,048,576 |\n| `nvidia/nemotron-3-ultra-550b-a55b:free` | 1,000,000 |\n| `nvidia/nemotron-3-super-120b-a12b:free` | 1,000,000 |\n| `poolside/laguna-xs.2:free` | 262,144 |\n| `poolside/laguna-m.1:free` | 262,144 |\n| `google/gemma-4-26b-a4b-it:free` | 262,144 |\n| `google/gemma-4-31b-it:free` | 262,144 |\n| `qwen/qwen3-next-80b-a3b-instruct:free` | 262,144 |\n| `cohere/north-mini-code:free` | 256,000 |\n| `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free` | 256,000 |\n| `nvidia/nemotron-3-nano-30b-a3b:free` | 256,000 |\n| `openai/gpt-oss-120b:free` | 131,072 |\n| `openai/gpt-oss-20b:free` | 131,072 |\n| `meta-llama/llama-3.3-70b-instruct:free` | 131,072 |\n| `meta-llama/llama-3.2-3b-instruct:free` | 131,072 |\n| `nousresearch/hermes-3-llama-3.1-405b:free` | 131,072 |\n| `nvidia/nemotron-3.5-content-safety:free` | 128,000 |\n| `nvidia/nemotron-nano-12b-v2-vl:free` | 128,000 |\n| `nvidia/nemotron-nano-9b-v2:free` | 128,000 |\n| `liquid/lfm-2.5-1.2b-thinking:free` | 32,768 |\n| `liquid/lfm-2.5-1.2b-instruct:free` | 32,768 |\n| `cognitivecomputations/dolphin-mistral-24b-venice-edition:free` | 32,768 |\n| `openrouter/owl-alpha` | 1,048,756 |\n| `google/lyria-3-pro-preview` | 1,048,576 |\n| `google/lyria-3-clip-preview` | 1,048,576 |\n| `openrouter/free` | 200,000 |\n\u003c!-- FREE_MODELS:END --\u003e\n\n## Install\n\n```bash\npython -m venv .venv \u0026\u0026 source .venv/bin/activate\npip install -e \".[dev]\"\ncp .env.example .env   # fill in OPENROUTER_API_KEY\n```\n\n## Run\n\n```bash\nfreerouter            # or: python -m freerouter\n# defaults to http://127.0.0.1:8000\n```\n\n## Usage (with the OpenAI SDK as-is)\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://127.0.0.1:8000/v1\", api_key=\"unused\")\n\nresp = client.chat.completions.create(\n    model=\"auto\",  # freerouter picks from the free pool automatically\n    messages=[{\"role\": \"user\", \"content\": \"hello\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\ncurl:\n\n```bash\ncurl http://127.0.0.1:8000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"auto\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}'\n```\n\n## Use as a library (no server)\n\nYou can embed freerouter into another agent program **as a submodule** without running an\nHTTP server. `FreeRouterClient` fetches the free-model list over REST (TTL-cached) and routes\nwith fallback in-process.\n\n```python\nimport asyncio\nfrom freerouter import FreeRouterClient\n\nasync def main():\n    async with FreeRouterClient() as fr:           # uses OPENROUTER_API_KEY from .env\n        free = await fr.models()                    # routable free models\n        data = await fr.chat(\n            [{\"role\": \"user\", \"content\": \"hello\"}],\n            model=\"auto\",                           # auto = pick from the free pool + fallback\n            max_tokens=64,\n        )\n        print(data[\"model\"], data[\"choices\"][0][\"message\"][\"content\"])\n\nasyncio.run(main())\n```\n\nIn synchronous code (outside an event loop), use the sync wrapper:\n\n```python\nfrom freerouter import FreeRouterClient\n\nfr = FreeRouterClient(api_key=\"sk-or-...\")          # passing the key directly also works\ndata = fr.chat_sync([{\"role\": \"user\", \"content\": \"hi\"}], model=\"auto\")\n```\n\n- **Share an httpx client**: if the agent already uses an `httpx.AsyncClient`, inject it —\n  `FreeRouterClient(http_client=my_client)`. When injected, the caller owns its lifecycle.\n- **Share state**: inject `registry`/`router` so several clients share the free-list cache and\n  cooldowns.\n- **Streaming**: `async for chunk in fr.stream_raw(payload)` — raw SSE bytes.\n- A non-retryable 4xx (e.g. a bad request) raises `httpx.HTTPStatusError`; all-candidates-failed\n  raises `FreeRouterError`.\n\n\u003e The same routing/fallback core is shared by the proxy server (`proxy.py`) and the library.\n\n## Endpoints\n\n| Method | Path | Description |\n|--------|------|-------------|\n| POST | `/v1/chat/completions` | OpenAI-compatible. Supports streaming (`\"stream\": true`). |\n| GET | `/v1/models` | Exposes only routable free models. |\n| GET | `/health` | Health check. |\n\n## Configuration\n\nControlled via `.env` or environment variables. See `src/freerouter/config.py` for the full list.\n\n| Key | Default | Description |\n|-----|---------|-------------|\n| `OPENROUTER_API_KEY` | (required) | OpenRouter key |\n| `MODEL_REFRESH_TTL` | 600 | free-list cache TTL (seconds) |\n| `MAX_ATTEMPTS` | 8 | max number of models to try as fallbacks |\n| `COOLDOWN_SECONDS` | 60 | how long to skip a model after a 429 (seconds) |\n\n## Tests\n\n```bash\npytest\n```\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funohee%2Ffreerouter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funohee%2Ffreerouter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funohee%2Ffreerouter/lists"}