{"id":50393880,"url":"https://github.com/tashfeenahmed/freellmapi","last_synced_at":"2026-06-16T11:01:09.347Z","repository":{"id":352884963,"uuid":"1217040185","full_name":"tashfeenahmed/freellmapi","owner":"tashfeenahmed","description":"OpenAI-compatible proxy that stacks the free tiers of 16 LLM providers (~1.7B tokens/month) behind one /v1 endpoint — plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.","archived":false,"fork":false,"pushed_at":"2026-06-15T22:37:55.000Z","size":3045,"stargazers_count":10449,"open_issues_count":36,"forks_count":1665,"subscribers_count":45,"default_branch":"main","last_synced_at":"2026-06-16T00:16:49.516Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://freellmapi.co","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tashfeenahmed.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-21T13:38:59.000Z","updated_at":"2026-06-15T23:37:19.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tashfeenahmed/freellmapi","commit_stats":null,"previous_names":["tashfeenahmed/freellmapi"],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/tashfeenahmed/freellmapi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tashfeenahmed%2Ffreellmapi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tashfeenahmed%2Ffreellmapi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tashfeenahmed%2Ffreellmapi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tashfeenahmed%2Ffreellmapi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tashfeenahmed","download_url":"https://codeload.github.com/tashfeenahmed/freellmapi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tashfeenahmed%2Ffreellmapi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34402663,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-16T02:00:06.860Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-30T20:00:23.489Z","updated_at":"2026-06-16T11:01:09.339Z","avatar_url":"https://github.com/tashfeenahmed.png","language":"TypeScript","funding_links":[],"categories":["🤖 AI \u0026 Machine Learning","3. Inference Engines \u0026 Serving","🛡️ AI Gateways, Proxies \u0026 Agent Security","TypeScript"],"sub_categories":["2026-07-12 — Weekly Update"],"readme":"\u003cdiv align=\"center\"\u003e\n\n# FreeLLMAPI\n\n**One OpenAI-compatible endpoint. Sixteen free LLM providers. ~1.7B tokens per month.**\n\nAggregate the free tiers from Google, Groq, Cerebras, NVIDIA, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare, HuggingFace, Z.ai (Zhipu), Ollama, Kilo, Pollinations, LLM7, OVH AI Endpoints, and OpenCode Zen — plus any custom OpenAI-compatible endpoint (llama.cpp, LM Studio, vLLM, local Ollama) — behind a single `/v1/chat/completions` endpoint. Keys are stored encrypted. A router picks the best available model for each request, falls over to the next provider when one is rate-limited, and tracks per-key usage so you stay under every free-tier cap.\n\n[![CI](https://github.com/tashfeenahmed/freellmapi/actions/workflows/ci.yml/badge.svg)](https://github.com/tashfeenahmed/freellmapi/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](./LICENSE)\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](#contributing)\n[![Docker image](https://img.shields.io/badge/ghcr.io-freellmapi-2496ED?logo=docker\u0026logoColor=white)](https://github.com/tashfeenahmed/freellmapi/pkgs/container/freellmapi)\n\n**[freellmapi.co](https://freellmapi.co)** — browse the live model catalog\n\n![Fallback chain with per-provider token budget](repo-assets/fallback-chain.png)\n\n\u003c/div\u003e\n\n---\n\n## Contents\n\n- [Why this exists](#why-this-exists)\n- [Supported providers](#supported-providers)\n- [Features](#features)\n- [Not yet supported](#not-yet-supported)\n- [Quick start](#quick-start)\n- [Docker](#docker)\n- [Desktop app](#desktop-app)\n- [Premium (live catalog)](#premium-live-catalog)\n- [Using the API](#using-the-api)\n- [Screenshots](#screenshots)\n- [How it works](#how-it-works)\n- [Context Handoff](#context-handoff)\n- [Limitations](#limitations)\n- [Contributing](#contributing)\n- [Terms of Service review](#terms-of-service-review)\n- [Disclaimer](#disclaimer)\n\n## Why this exists\n\nEvery serious AI lab now offers a free tier — a few million tokens a month, a few thousand requests a day. On its own each tier is a toy. Stacked together, they add up to roughly **1.7 billion tokens per month** of working inference capacity, across 100+ models from small-and-fast to reasonably capable.\n\nThe problem is that stacking them by hand is painful: seventeen different SDKs, seventeen different rate limits, seventeen places a request can fail. FreeLLMAPI collapses that into one OpenAI-compatible endpoint. Point any OpenAI client library at your local server, and it routes transparently across whichever providers you've added keys for.\n\n## Supported providers\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd align=\"center\" width=\"180\"\u003e\u003ca href=\"https://ai.google.dev\"\u003e\u003cb\u003eGoogle\u003c/b\u003e\u003cbr/\u003eGemini 2.5 Flash · 3.x previews\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\" width=\"180\"\u003e\u003ca href=\"https://groq.com\"\u003e\u003cb\u003eGroq\u003c/b\u003e\u003cbr/\u003eLlama 3.3, Llama 4, GPT-OSS, Qwen3\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\" width=\"180\"\u003e\u003ca href=\"https://cerebras.ai\"\u003e\u003cb\u003eCerebras\u003c/b\u003e\u003cbr/\u003eQwen3 235B\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\" width=\"180\"\u003e\u003ca href=\"https://opencode.ai/zen\"\u003e\u003cb\u003eOpenCode Zen\u003c/b\u003e\u003cbr/\u003eDeepSeek V4 Flash · Nemotron (promo)\u003c/a\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://mistral.ai\"\u003e\u003cb\u003eMistral\u003c/b\u003e\u003cbr/\u003eLarge 3 · Medium 3.5 · Codestral · Devstral\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://openrouter.ai\"\u003e\u003cb\u003eOpenRouter\u003c/b\u003e\u003cbr/\u003e21 free-tier models\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/marketplace/models\"\u003e\u003cb\u003eGitHub Models\u003c/b\u003e\u003cbr/\u003eGPT-4.1 · GPT-4o\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://developers.cloudflare.com/workers-ai\"\u003e\u003cb\u003eCloudflare\u003c/b\u003e\u003cbr/\u003eKimi K2 · GLM-4.7 · GPT-OSS · Granite 4\u003c/a\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://cohere.com\"\u003e\u003cb\u003eCohere\u003c/b\u003e\u003cbr/\u003eCommand R+ · Command-A (trial)\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://docs.z.ai\"\u003e\u003cb\u003eZ.ai (Zhipu)\u003c/b\u003e\u003cbr/\u003eGLM-4.5 · GLM-4.7 Flash\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://build.nvidia.com\"\u003e\u003cb\u003eNVIDIA\u003c/b\u003e\u003cbr/\u003eNIM · 40 RPM free (eval-only ToS)\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://huggingface.co/docs/inference-providers\"\u003e\u003cb\u003eHuggingFace\u003c/b\u003e\u003cbr/\u003eRouter → DeepSeek V4 · Kimi K2.6 · Qwen3\u003c/a\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://ollama.com\"\u003e\u003cb\u003eOllama Cloud\u003c/b\u003e\u003cbr/\u003eGLM-4.7 · Kimi K2 · gpt-oss · Qwen3\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://kilo.ai\"\u003e\u003cb\u003eKilo Gateway\u003c/b\u003e\u003cbr/\u003e:free routes (anon ok)\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://pollinations.ai\"\u003e\u003cb\u003ePollinations\u003c/b\u003e\u003cbr/\u003eGPT-OSS 20B (anon ok)\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://llm7.io\"\u003e\u003cb\u003eLLM7\u003c/b\u003e\u003cbr/\u003eGPT-OSS · Llama 3.1 · GLM (anon ok)\u003c/a\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://endpoints.ai.cloud.ovh.net\"\u003e\u003cb\u003eOVH AI Endpoints\u003c/b\u003e\u003cbr/\u003eQwen3.5 397B · GPT-OSS · Llama 3.3 (anon ok)\u003c/a\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\nPlus a **custom** provider — point at any OpenAI-compatible endpoint (llama.cpp, LM Studio, vLLM, a local Ollama, or a remote gateway) from the Keys page.\n\n## Features\n\n- **OpenAI-compatible** — `POST /v1/chat/completions` and `GET /v1/models` work with the official OpenAI SDKs and any OpenAI-compatible client (LangChain, LlamaIndex, Continue, Hermes, etc.). Just change `base_url`.\n- **Responses API** — `POST /v1/responses` (the wire format current Codex CLI versions require) is implemented as a translating shim over the same router, with full streaming events and tool calls.\n- **Streaming and non-streaming** — Server-Sent Events for `stream: true`, JSON response otherwise. Every provider adapter implements both.\n- **Tool calling** — OpenAI-style `tools` / `tool_choice` requests are passed through, and assistant `tool_calls` + `tool` role follow-up messages round-trip across providers.\n- **Embeddings** — `/v1/embeddings` with family-based routing: failover only ever happens between providers serving the *same* model (vectors from different models are incompatible), never across models. See [Embeddings](#embeddings).\n- **Automatic fallover** — If the chosen provider returns a 429, 5xx, or times out, the router skips it, puts the key on a short cooldown, and retries on the next model in your fallback chain (up to 20 attempts).\n- **Per-key rate tracking** — RPM, RPD, TPM, and TPD counters per `(platform, model, key)` so the router always picks a key that's under its caps.\n- **Sticky sessions** — Multi-turn conversations keep talking to the same model for 30 minutes to avoid the hallucination spike that comes from mid-conversation model switches.\n- **Encrypted key storage** — API keys are encrypted with AES-256-GCM before hitting SQLite; decryption happens in-memory just before a request.\n- **Unified API key** — Clients authenticate to your proxy with a single `freellmapi-…` bearer token. You never expose upstream provider keys to your apps.\n- **Dashboard login** — The admin UI and all `/api/*` routes are gated behind an email + password account (scrypt-hashed, session-token auth), set on first run. The `/v1` proxy keeps its own unified-key auth for apps.\n- **Health checks** — Periodic probes mark keys as `healthy`, `rate_limited`, `invalid`, or `error` so the router skips dead ones automatically.\n- **Admin dashboard** — React + Vite UI to manage keys, reorder the fallback chain, inspect analytics, and run prompts in a playground. Dark mode included.\n- **Analytics** — Per-request logging with latency, token counts, success rate, and per-provider breakdowns.\n- **Context handoff on model switch** — Optional. When a session falls over to a different model, injects one compact system message so the new model knows it is continuing an existing task. Disabled by default; enable with `FREELLMAPI_CONTEXT_HANDOFF=on_model_switch`. See [Context Handoff](#context-handoff).\n- **Runs anywhere Node 20+ runs** — Windows, macOS, Linux servers, or a small ARM SBC (Raspberry Pi included). ~40 MB RSS at idle behind PM2 / systemd / whatever supervisor you prefer.\n\n## Not yet supported\n\nThe scope is deliberately narrow. If a feature isn't on this list and isn't below, assume it isn't there yet.\n\n- **Image generation** (`/v1/images/*`)\n- **Audio / speech** (`/v1/audio/*`)\n- **Legacy completions** (`/v1/completions`) — only the chat endpoint is implemented\n- **Moderation** (`/v1/moderations`)\n- **`n \u003e 1`** (multiple completions per request)\n- **Per-user billing / multi-tenant auth** — single-user by design\n\nPRs that add any of these are very welcome. See [Contributing](#contributing).\n\n## Quick start\n\n**One-liner** (Docker required — sets up `~/freellmapi`, generates an encryption key, pulls the image, and starts the container):\n\n```bash\ncurl -fsSL https://freellmapi.co/install.sh | bash\n```\n\nPrefer to read before you pipe to bash? [The script is here](https://freellmapi.co/install.sh). Re-running it is safe: your `.env` (and encryption key) is preserved and the container updates to `:latest`. Override the defaults with `FREELLMAPI_DIR`, `PORT`, or `HOST_BIND` env vars.\n\n**Or manually with Docker Compose.** It runs the API and dashboard together on port 3001 and persists SQLite in a named volume.\n\n**Prerequisites:** Docker, Docker Compose, OpenSSL.\n\n```bash\ngit clone https://github.com/tashfeenahmed/freellmapi.git\ncd freellmapi\n\n# Generate an encryption key for at-rest key storage\nENCRYPTION_KEY=\"$(openssl rand -hex 32)\"\nprintf \"ENCRYPTION_KEY=%s\\nPORT=3001\\n\" \"$ENCRYPTION_KEY\" \u003e .env\n\ndocker compose up -d\n```\n\nOpen http://localhost:3001, add your provider keys on the **Keys** page, reorder the **Fallback Chain** to taste, and grab your unified API key from the **Keys** page header. That unified key is what you point your OpenAI SDK at.\n\n\u003e **Reaching it from another machine?** By default the container is published only on `127.0.0.1`, so `http://\u003cserver-ip\u003e:3001` won't load from another device (the page just hangs). To expose it on your LAN — e.g. a Raspberry Pi at `http://192.168.1.x:3001` — start it with `HOST_BIND=0.0.0.0`:\n\u003e\n\u003e ```bash\n\u003e HOST_BIND=0.0.0.0 docker compose up -d\n\u003e ```\n\u003e\n\u003e Only do this on a trusted network: the proxy is single-user and guarded only by the unified API key.\n\n### Local development\n\n**Prerequisites:** Node.js 20+, npm.\n\n```bash\ngit clone https://github.com/tashfeenahmed/freellmapi.git\ncd freellmapi\nnpm install\ncp .env.example .env\nENCRYPTION_KEY=\"$(node -e 'console.log(require(\"crypto\").randomBytes(32).toString(\"hex\"))')\"\nprintf \"ENCRYPTION_KEY=%s\\nPORT=3001\\n\" \"$ENCRYPTION_KEY\" \u003e .env\nnpm run dev\n```\n\n`ENCRYPTION_KEY` is required for startup. The server only falls back to a\ndatabase-stored development key when `DEV_MODE=true` and `NODE_ENV` is not\n`production`; do not use that fallback with real provider keys.\n\nRequest analytics are retained for 90 days or 100000 request rows by default,\nwhichever limit prunes first. Set `REQUEST_ANALYTICS_RETENTION_DAYS=0` or\n`REQUEST_ANALYTICS_MAX_ROWS=0` in `.env` to disable either retention limit.\n\nOpen http://localhost:5173 (the Vite dev UI), add your provider keys on the **Keys** page, reorder the **Fallback Chain** to taste, and grab your unified API key from the **Keys** page header. That unified key is what you point your OpenAI SDK at.\n\n\u003e **Reaching the dev UI from another device on your LAN?** Use `npm run dev:lan` — it passes `--host` through to Vite, which then prints a `Network: http://\u003cyour-ip\u003e:5173` URL you can open from a phone or another machine. (Plain `npm run dev -- --host` does *not* work here: the root `dev` script is a `concurrently` wrapper, so the flag never reaches Vite.) API calls go through Vite's dev proxy, so no extra server config is needed.\n\nFor a production build without Docker:\n\n```bash\nnpm run build\nnode server/dist/index.js     # server + dashboard both served on :3001\n```\n\n## Docker\n\nFreeLLMAPI publishes a single production image that contains the Express server and the built React dashboard:\n\n```bash\ndocker pull ghcr.io/tashfeenahmed/freellmapi:latest   # or pin a release, e.g. :v1.2.3\n```\n\nThe image is multi-arch (`linux/amd64` + `linux/arm64`, so it runs on a Raspberry Pi). Published tags: `latest` (default branch), `v*.*.*` (git release tags), and `sha-\u003ccommit\u003e`.\n\nThe included `docker-compose.yml` is the recommended install path:\n\n```bash\ndocker compose up -d\ndocker compose logs -f freellmapi\n```\n\nBy default the container's port is bound to `127.0.0.1` (localhost only). To reach the dashboard/API from another machine on your network, publish it on all interfaces with `HOST_BIND=0.0.0.0 docker compose up -d` — only on a trusted LAN, since the proxy is single-user.\n\nSQLite data is stored in the `freellmapi-data` volume at `/app/server/data`. Keep the same `.env` `ENCRYPTION_KEY` and volume when upgrading, because provider keys are encrypted at rest.\n\nMore Docker operations and examples live in [docker/README.md](./docker/README.md).\n\n## Desktop app\n\nA native menu-bar app lives in [`desktop/`](./desktop): the entire router +\ndashboard running locally from your tray, with a glass popover showing live\nrequest stats.\n\n![FreeLLMAPI desktop app](repo-assets/desktop.png)\n\n**[Download the macOS app from Releases](https://github.com/tashfeenahmed/freellmapi/releases/latest)**, or build it from this repo in a few minutes:\n\n```bash\nnpm install\nnpm run desktop:dist        # macOS: desktop/dist-electron/FreeLLMAPI-…-arm64.dmg\nnpm run desktop:dist:win    # Windows installer\n```\n\n\u003e **Windows:** the build config is in place but not tested yet — if you try it,\n\u003e a quick report (working or not) in an issue would be much appreciated.\n\n## Premium (live catalog)\n\nThe router keeps its model catalog fresh on its own: it pulls a signed catalog\nfrom [freellmapi.co](https://freellmapi.co) twice a day and applies new models,\nquota changes, and provider quirk fixes to your local DB (your own enable/disable\nchoices and custom providers are never touched; every download is verified\nagainst a pinned Ed25519 key before it is applied).\n\n- **Free** installs follow a **monthly snapshot** — zero cost, forever.\n- **[Premium](https://freellmapi.co/#pricing)** ($19/yr or $49 lifetime) follows\n  the **live feed**, refreshed every 2-3 days, so new free models are in your\n  router the moment they exist. One key covers all your devices; activate it in\n  the dashboard under **Premium**. Cancel or manage billing self-serve at\n  [freellmapi.co/manage](https://freellmapi.co/manage).\n\nThe catalog server never sees your prompts, completions, or provider keys — the\nrouter stays fully self-hosted either way.\n\nLocally built apps launch without Gatekeeper/SmartScreen warnings — no code\nsigning involved. Full instructions in [desktop/README.md](./desktop/README.md).\n\n## Using the API\n\nAny OpenAI-compatible client works. Examples:\n\n**Python**\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http://localhost:3001/v1\",\n    api_key=\"freellmapi-your-unified-key\",\n)\n\nresp = client.chat.completions.create(\n    model=\"auto\",  # let the router pick; or specify e.g. \"gemini-2.5-flash\"\n    messages=[{\"role\": \"user\", \"content\": \"Summarise the fall of Rome in one sentence.\"}],\n)\nprint(resp.choices[0].message.content)\nprint(\"Routed via:\", resp.headers.get(\"x-routed-via\"))\n```\n\n**curl**\n\n```bash\ncurl http://localhost:3001/v1/chat/completions \\\n  -H \"Authorization: Bearer freellmapi-your-unified-key\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"auto\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"hi\"}]\n  }'\n```\n\n**Streaming**\n\n```python\nstream = client.chat.completions.create(\n    model=\"auto\",\n    messages=[{\"role\": \"user\", \"content\": \"Stream me a haiku about SQLite.\"}],\n    stream=True,\n)\nfor chunk in stream:\n    print(chunk.choices[0].delta.content or \"\", end=\"\", flush=True)\n```\n\n**Tool calling**\n\nPass OpenAI-style `tools` and `tool_choice`; the assistant response round-trips back through the proxy exactly like the OpenAI API. Multi-step flows (assistant `tool_calls` → `tool` role follow-up → final answer) work across every provider the router can reach.\n\n```python\ntools = [{\n    \"type\": \"function\",\n    \"function\": {\n        \"name\": \"get_weather\",\n        \"description\": \"Get current weather for a city.\",\n        \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\"city\": {\"type\": \"string\"}},\n            \"required\": [\"city\"],\n        },\n    },\n}]\n\n# 1. Model asks for a tool call\nfirst = client.chat.completions.create(\n    model=\"auto\",\n    messages=[{\"role\": \"user\", \"content\": \"What's the weather in Karachi?\"}],\n    tools=tools,\n    tool_choice=\"required\",\n)\ncall = first.choices[0].message.tool_calls[0]\n\n# 2. You execute the tool, feed the result back\nfinal = client.chat.completions.create(\n    model=\"auto\",\n    messages=[\n        {\"role\": \"user\", \"content\": \"What's the weather in Karachi?\"},\n        first.choices[0].message,\n        {\"role\": \"tool\", \"tool_call_id\": call.id, \"content\": '{\"temp_c\": 32, \"cond\": \"sunny\"}'},\n    ],\n    tools=tools,\n)\nprint(final.choices[0].message.content)\n```\n\n**Gemini Google Search grounding**\n\nGoogle's models can ground their answers in live Google Search results. Since the OpenAI wire format has no way to express that, request a tool named `google_search` and the Google provider translates it into Gemini's native grounding tool. It can be sent on its own or alongside your normal function tools.\n\n```python\nresp = client.chat.completions.create(\n    model=\"gemini-2.5-flash\",  # pin a Google model so the request routes there\n    messages=[{\"role\": \"user\", \"content\": \"Who won the F1 race this weekend?\"}],\n    tools=[{\"type\": \"function\", \"function\": {\"name\": \"google_search\", \"parameters\": {}}}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Vision / image input**\n\nSend images with the standard OpenAI `image_url` content blocks (base64 `data:` URLs or `http(s)` URLs). When a request contains an image, the router restricts itself to **vision-capable models** and ignores text-only ones. Vision models are tagged with a **Vision** badge on the Fallback Chain page; the current set includes Gemini (2.5 / 3.x), Llama 4 Scout/Maverick (Groq, NVIDIA), GLM-4.6V Flash (Z.ai), Nemotron Nano 12B VL (OpenRouter), and GitHub's GPT-4o / GPT-4.1.\n\n```python\nresp = client.chat.completions.create(\n    model=\"auto\",  # auto-routes to a vision model\n    messages=[{\n        \"role\": \"user\",\n        \"content\": [\n            {\"type\": \"text\", \"text\": \"What's in this image?\"},\n            {\"type\": \"image_url\", \"image_url\": {\"url\": \"data:image/png;base64,\u003c...\u003e\"}},\n        ],\n    }],\n)\nprint(resp.choices[0].message.content)\n```\n\nIf no vision-capable model is enabled in your Fallback Chain, an image request returns a clear `422` (`code: \"no_vision_model\"`) rather than silently dropping the image. (Image input on `/v1/responses` isn't supported yet — use `/v1/chat/completions`.)\n\nWorks with `stream=True` as well — you'll get `delta.tool_calls` chunks followed by a `finish_reason: \"tool_calls\"` close. Under the hood, OpenAI-compatible providers (Groq, Cerebras, Mistral, OpenRouter, GitHub Models, HuggingFace, Cloudflare, Cohere compat) get the request passed through; Gemini requests get translated into Google's `functionDeclarations` / `functionResponse` shape and the response is translated back.\n\nEvery response carries an `X-Routed-Via: \u003cplatform\u003e/\u003cmodel\u003e` header so you can see which provider actually served each call. If a request fell over between providers, you'll also see `X-Fallback-Attempts: N`.\n\n### Embeddings\n\n`/v1/embeddings` is OpenAI-compatible, with one deliberate difference from chat routing: **failover never crosses models.** Vectors from different models live in incompatible spaces — silently switching models would corrupt any vector store built on top of the proxy. So embeddings route by **family** (one model identity + dimension), and failover only walks the providers serving that same family.\n\n```python\nresp = client.embeddings.create(\n    model=\"auto\",          # default family; or a family name like \"bge-m3\"\n    input=[\"the quick brown fox\", \"pack my box with five dozen liquor jugs\"],\n)\nprint(len(resp.data), \"vectors of\", len(resp.data[0].embedding), \"dims\")\n```\n\n```bash\ncurl http://localhost:3001/v1/embeddings \\\n  -H \"Authorization: Bearer freellmapi-your-unified-key\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\": \"auto\", \"input\": \"hello world\"}'\n```\n\n`model` accepts `auto` (the configured default family), a family name, or a provider-specific model id (which resolves to its family). Available families:\n\n| Family (`model`) | Dims | Providers (failover order) |\n| --- | --- | --- |\n| `gemini-embedding-001` *(default)* | 3072 | Google |\n| `text-embedding-3-large` | 3072 | GitHub Models |\n| `text-embedding-3-small` | 1536 | GitHub Models |\n| `embed-v4.0` | 1536 | Cohere |\n| `bge-m3` | 1024 | Cloudflare → Hugging Face |\n| `qwen3-embedding-0.6b` | 1024 | Cloudflare |\n| `nv-embedqa-e5-v5` | 1024 | NVIDIA |\n| `llama-nemotron-embed-1b-v2` | 2048 | NVIDIA |\n| `llama-nemotron-embed-vl-1b-v2` | 2048 | NVIDIA → OpenRouter |\n| `embeddinggemma-300m` | 768 | Cloudflare |\n\nThe default family, per-provider toggles, and priorities live on the dashboard's **Models → Embeddings** page. Pick your family once and stick with it for a given vector store — that's the whole point of the family model.\n\n## Screenshots\n\n### Keys\n\nManage provider credentials and grab the unified API key your apps connect with. Each key shows a status dot and when it was last health-checked.\n\n![Keys page](repo-assets/keys.png)\n\n### Playground\n\nSend a chat completion through the router and see which provider served it, with the model ID and latency printed right on the message.\n\n![Playground page](repo-assets/playground.png)\n\n### Analytics\n\nRequest volume, success rate, tokens in and out, average latency, and per-provider breakdowns over 24h / 7d / 30d windows.\n\n![Analytics page](repo-assets/analytics.png)\n\n## How it works\n\n```\n┌──────────────────┐   Bearer freellmapi-…   ┌─────────────────────────┐\n│  OpenAI SDK /    │ ──────────────────────▶ │  Express proxy (:3001)  │\n│  curl / any      │ ◀────────────────────── │  /v1/chat/completions   │\n│  OpenAI client   │      streamed tokens    └────────────┬────────────┘\n└──────────────────┘                                      │\n                                                          ▼\n                             ┌────────────────────────────────────────────────┐\n                             │  Router                                        │\n                             │   1. Pick highest-priority model that          │\n                             │      (a) has a healthy key and                 │\n                             │      (b) is under all its rate limits.         │\n                             │   2. Decrypt key, call provider SDK.           │\n                             │   3. On 429/5xx → cooldown + retry next model. │\n                             └────────────────────────────────────────────────┘\n                                          │\n   ┌──────────────┬────────────┬──────────┴─────────┬─────────────┬──────────┐\n   ▼              ▼            ▼                    ▼             ▼          ▼\n Google         Groq        Cerebras           OpenRouter        HF       …10 more\n```\n\n- **Router** (`server/src/services/router.ts`) — picks a model per request.\n- **Rate-limit ledger** (`server/src/services/ratelimit.ts`) — in-memory RPM/RPD/TPM/TPD counters backed by SQLite, with cooldowns on 429s.\n- **Provider adapters** (`server/src/providers/*.ts`) — one file per provider, implementing the `Provider` base class: `chatCompletion()` and `streamChatCompletion()`.\n- **Health service** (`server/src/services/health.ts`) — periodic probe keeps key status fresh.\n- **Dashboard** (`client/`) — React + Vite + shadcn/ui admin surface.\n- **Storage** — SQLite (`better-sqlite3`) with AES-256-GCM envelope encryption for keys.\n\n## Context Handoff\n\nWhen FreeLLMAPI falls over to a different model mid-conversation (quota, rate limit, cooldown), the new model has no idea it is picking up someone else's task. **Context handoff** adds a single compact `system` message to the outbound request that tells the new model exactly that:\n\n```\nFreeLLMAPI context handoff:\nYou are taking over an ongoing conversation from another model (groq:llama-3 → google:gemini-flash).\nContinue the user's task using the conversation context already provided in this request.\nDo not restart the task, re-ask already answered setup questions, or discard prior tool results.\nRespect the user's latest message as the highest-priority instruction.\n\nRecent session summary:\nUser: …\nAssistant: …\n```\n\n**Enable it in `.env`:**\n\n```env\nFREELLMAPI_CONTEXT_HANDOFF=on_model_switch\n```\n\n**How it works:**\n\n- Messages per session are stored in memory (TTL: 3 hours).\n- Only injected when the selected model changes for a given session key.\n- Not injected on the first request, on same-model continuations, or if a handoff message is already present.\n- Session key: `X-Session-Id` header if present, otherwise SHA-1 of the first user message (same as sticky sessions).\n- Storage is in-memory only. Nothing is written to disk or logged.\n\n\u003e **Important:** Context Handoff improves continuity for conversations routed through FreeLLMAPI. It cannot recover provider-internal hidden state or messages that were never sent to the proxy.\n\n## Limitations\n\nStacking free tiers has real trade-offs. Be honest with yourself about them:\n\n- **No frontier models.** The free-tier catalog tops out around Llama 3.3 70B, GLM-4.5, Qwen 3 Coder, and Gemini 2.5 Pro. You will not get GPT-5 or Claude Opus class reasoning through this. For hard problems, pay for a real API.\n- **Intelligence degrades as the day progresses.** Your top-ranked models (usually Gemini 2.5 Pro, GPT-4o via GitHub Models) have the lowest daily caps. Once they hit their limits, the router falls down your priority chain to smaller/weaker models. Expect the effective intelligence of the endpoint to drop in the late hours of each day — then reset at UTC midnight.\n- **Latency is highly variable.** Cerebras and Groq are extremely fast; others are not. You get whichever one is available.\n- **Free tiers can change without notice.** Providers regularly tighten, loosen, or remove free tiers. When that happens you'll see 429s or auth errors until you update the catalog. Re-seed scripts live in `server/src/scripts/`.\n- **No SLA, by definition.** If you need reliability, use a paid provider with a contract.\n- **Local-first.** There's no multi-tenant auth. Run this for yourself; don't expose it to the internet.\n\n## Contributing\n\nContributors very welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for the dev loop, PR expectations, and the policy on AI/LLM-assisted contributions (short version: welcome, same quality bar as any other PR). Good first PRs:\n\n- **Add a provider** — copy `server/src/providers/openai-compat.ts` as a template, wire it into `server/src/providers/index.ts`, seed its models in `server/src/db/index.ts`, add a test in `server/src/__tests__/providers/`.\n- **Add an endpoint** — images, moderations, audio. The provider base class can grow new methods; adapters declare which they support.\n- **Improve the router** — cost-aware routing (cheapest-healthy-fastest tradeoffs), better latency-weighted priority, regional pinning.\n- **Dashboard polish** — charts on the Analytics page, key rotation UX, batch import of keys from `.env`.\n- **Docs** — more examples, client library snippets for Go/Rust/etc., a deployment recipe for Docker or Fly.\n\n**Development loop:**\n\n```bash\nnpm install\nnpm run dev      # server on :3001, dashboard on :5173, both with HMR\nnpm test         # server vitest; also runs client tests if the workspace adds them\nnpm run build    # compile server and dashboard\n```\n\nPRs should include a test, keep the existing test suite green, and match the `.editorconfig` / tsconfig defaults already in the repo. Issues and discussions are open.\n\n### Contributors\n\n\u003ca href=\"https://github.com/moaaz12-web\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/moaaz12-web.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@moaaz12-web\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/lukasulc\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/lukasulc.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@lukasulc\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/VinhPhamAI\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/VinhPhamAI.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@VinhPhamAI\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/deadc\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/deadc.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@deadc\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/zhangyu1324\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/zhangyu1324.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@zhangyu1324\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/Tazrif-Raim\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/Tazrif-Raim.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@Tazrif-Raim\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/hodlmybeer69-bit\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/hodlmybeer69-bit.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@hodlmybeer69-bit\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/phoenixikkifullstack\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/phoenixikkifullstack.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@phoenixikkifullstack\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/jtbrennan-git\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/jtbrennan-git.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@jtbrennan-git\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/praveenkumarpranjal\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/praveenkumarpranjal.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@praveenkumarpranjal\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/nordbyte\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/nordbyte.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@nordbyte\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/mybropro\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/mybropro.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@mybropro\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/danscMax\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/danscMax.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@danscMax\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/jhash\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/jhash.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@jhash\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/JammyJames1234\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/JammyJames1234.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@JammyJames1234\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/Sumit4codes\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/Sumit4codes.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@Sumit4codes\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/meliani\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/meliani.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@meliani\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/thedavidweng\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/thedavidweng.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@thedavidweng\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/bharvey42\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/bharvey42.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@bharvey42\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/yuvrxj-afk\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/yuvrxj-afk.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@yuvrxj-afk\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/Tushar49\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/Tushar49.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@Tushar49\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/nicyoong\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/nicyoong.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@nicyoong\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/Aldo-f\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/Aldo-f.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@Aldo-f\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/Tazrif-Raim\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/Tazrif-Raim.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@Tazrif-Raim\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/m1nuzz\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/m1nuzz.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@m1nuzz\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/LoneRifle\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/LoneRifle.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@LoneRifle\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/ita333\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/ita333.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@ita333\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/barbotkonv\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/barbotkonv.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@barbotkonv\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/Naster17\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/Naster17.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@Naster17\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/StealthTensor\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/StealthTensor.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@StealthTensor\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/EmranAhmed\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/EmranAhmed.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@EmranAhmed\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/itsfuad\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/itsfuad.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@itsfuad\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/RobinHoodO\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/RobinHoodO.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@RobinHoodO\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/hmm183\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/hmm183.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@hmm183\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/duemilionidieuro-bot\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/duemilionidieuro-bot.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@duemilionidieuro-bot\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/hjhhoni\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/hjhhoni.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@hjhhoni\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/immanuelsavio\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/immanuelsavio.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@immanuelsavio\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/Slyker\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/Slyker.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@Slyker\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/wells1013\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/wells1013.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@wells1013\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/evgkrsk\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/evgkrsk.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@evgkrsk\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/aaronjmars\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/aaronjmars.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@aaronjmars\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/Robs87\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/Robs87.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@Robs87\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/dashitongzhi\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/dashitongzhi.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@dashitongzhi\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/QingJ01\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/QingJ01.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@QingJ01\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/3215\"\u003e\u003cimg src=\"https://images.weserv.nl/?url=github.com/3215.png\u0026w=60\u0026h=60\u0026fit=cover\u0026mask=circle\" width=\"60\" alt=\"@3215\" /\u003e\u003c/a\u003e\n\n## Terms of Service review\n\nA self-hosted, single-user, personal-use setup was re-reviewed against each provider's ToS (May 2026). Summary:\n\n| Provider | Verdict | Notes |\n|---|---|---|\n| Google Gemini | ⚠️ Caution | March 2026 ToS narrows scope to *\"professional or business purposes, not for consumer use\"* — a self-hosted developer proxy is still defensible, but the clause is new. |\n| Groq | ✅ Likely OK | GroqCloud Services Agreement permits Customer Application integration. |\n| Cerebras | ✅ Likely OK | Permitted; explicitly forbids selling/transferring API keys. |\n| Mistral | ✅ Likely OK | APIs allowed for personal/internal business use. |\n| OpenRouter | ✅ Likely OK | April 2026 ToS sharpens the no-resale / no-competing-service clause; private single-user proxy still fine. |\n| Cloudflare Workers AI | ⚠️ Ambiguous | No anti-proxy clause; covered by general Self-Serve Subscription Agreement. |\n| NVIDIA NIM | ⚠️ Caution | Trial ToS §1.2 / §1.4: *\"evaluation only, not production.\"* Free access is a recurring 40 RPM rate limit (the 2025 credit system was discontinued), but the evaluation-only scope stands. |\n| GitHub Models | ⚠️ Caution | Free tier explicitly scoped to *\"experimentation\"* and *\"prototyping.\"* |\n| Cohere | ❌ Avoid | Terms §14 still forbids *\"personal, family or household purposes.\"* |\n| Zhipu (open.bigmodel.cn) | ✅ Likely OK | Personal/non-commercial research carve-out still in the platform docs. |\n| Z.ai (api.z.ai) | ⚠️ Caution | New row — Singapore entity (distinct from Zhipu CN). §III.3(l) anti-traffic-redirect clause could plausibly be read against a proxy; no explicit personal-use carve-out. |\n| Ollama Cloud | ✅ Likely OK | New row — Free plan permits cloud-model access (1 concurrent, 5-hour session caps). No anti-proxy / anti-resale clauses found. *(Integration tracked in #14.)* |\n| OVH AI Endpoints | ✅ Likely OK | New row (June 2026) — anonymous access is officially documented (2 req/min per IP per model). OVH reserves the right to introduce token/consumption caps. |\n\nRules of thumb that keep most providers happy: **one account per provider**, **no reselling**, **no sharing your endpoint with other humans**, **don't hammer a free tier as a paid production backend**. This is informational, not legal advice — read each provider's ToS and make your own call.\n\nRemoved since the April 2026 review: Hugging Face, Moonshot, and MiniMax direct integrations were dropped from the catalog (HF — tool-call format issues; Moonshot — moved to paid only; MiniMax — superseded by the OpenRouter `minimax/minimax-m2.5:free` route).\n\n## Disclaimer\n\n**This project is for personal experimentation and learning, not production.** Free tiers exist so developers can prototype against them; they aren't a stable, supported inference substrate and shouldn't be treated as one. If you build something real on top of FreeLLMAPI, swap in a paid API before you ship. Your relationship with each upstream provider is governed by the terms you accepted when you created your account — those terms still apply when the traffic is proxied through this project, and you're responsible for complying with them.\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/chart?repos=tashfeenahmed/freellmapi\u0026type=date\u0026legend=top-left)](https://www.star-history.com/?repos=tashfeenahmed%2Ffreellmapi\u0026type=date\u0026legend=top-left)\n\n## License\n\n[MIT](./LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftashfeenahmed%2Ffreellmapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftashfeenahmed%2Ffreellmapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftashfeenahmed%2Ffreellmapi/lists"}