{"id":51227885,"url":"https://github.com/xberg-io/liter-llm","last_synced_at":"2026-07-05T09:00:40.500Z","repository":{"id":347292817,"uuid":"1191384809","full_name":"xberg-io/liter-llm","owner":"xberg-io","description":"Universal LLM API client — 142+ providers, 11 native language bindings, powered by Rust core","archived":false,"fork":false,"pushed_at":"2026-07-02T16:45:33.000Z","size":444692,"stargazers_count":222,"open_issues_count":1,"forks_count":15,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-07-02T17:32:01.322Z","etag":null,"topics":["anthropic","api-client","llm","machine-learning","openai","polyglot","python","rust","streaming","typescript"],"latest_commit_sha":null,"homepage":"https://docs.liter-llm.xberg.io","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xberg-io.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-25T07:30:37.000Z","updated_at":"2026-07-02T15:41:17.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/xberg-io/liter-llm","commit_stats":null,"previous_names":["kreuzberg-dev/liter-llm","xberg-io/liter-llm"],"tags_count":188,"template":false,"template_full_name":null,"purl":"pkg:github/xberg-io/liter-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xberg-io%2Fliter-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xberg-io%2Fliter-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xberg-io%2Fliter-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xberg-io%2Fliter-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xberg-io","download_url":"https://codeload.github.com/xberg-io/liter-llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xberg-io%2Fliter-llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35062689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-02T02:00:06.368Z","response_time":173,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anthropic","api-client","llm","machine-learning","openai","polyglot","python","rust","streaming","typescript"],"created_at":"2026-06-28T13:00:36.014Z","updated_at":"2026-07-05T09:00:40.494Z","avatar_url":"https://github.com/xberg-io.png","language":"Rust","funding_links":[],"categories":["llm","Libraries","Rust"],"sub_categories":["Artificial Intelligence"],"readme":"\u003cp align=\"center\"\u003e\n  \u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://cdn.jsdelivr.net/gh/xberg-io/assets@v1/banner/readme-banner-dark.svg\"\u003e\n    \u003cimg alt=\"Xberg\" width=\"420\" src=\"https://cdn.jsdelivr.net/gh/xberg-io/assets@v1/banner/readme-banner-light.svg\"\u003e\n  \u003c/picture\u003e\n\u003c/p\u003e\n\n# liter-llm\n\n\u003cdiv align=\"center\" style=\"display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0\"\u003e\n  \u003c!-- Built with --\u003e\n  \u003ca href=\"https://github.com/xberg-io/alef\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Bindings-alef%20%D7%90-007ec6\" alt=\"Bindings\" /\u003e\n  \u003c/a\u003e\n  \u003c!-- Language Bindings --\u003e\n  \u003ca href=\"https://crates.io/crates/liter-llm\"\u003e\n    \u003cimg src=\"https://img.shields.io/crates/v/liter-llm?label=Rust\u0026color=007ec6\" alt=\"Rust\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/liter-llm/\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/v/liter-llm?label=Python\u0026color=007ec6\" alt=\"Python\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.npmjs.com/package/@xberg-io/liter-llm\"\u003e\n    \u003cimg src=\"https://img.shields.io/npm/v/@xberg-io/liter-llm?label=Node.js\u0026color=007ec6\" alt=\"Node.js\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.npmjs.com/package/@xberg-io/liter-llm-wasm\"\u003e\n    \u003cimg src=\"https://img.shields.io/npm/v/@xberg-io/liter-llm-wasm?label=WASM\u0026color=007ec6\" alt=\"WASM\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://central.sonatype.com/artifact/io.xberg.literllm/liter-llm\"\u003e\n    \u003cimg src=\"https://img.shields.io/maven-central/v/io.xberg.literllm/liter-llm?label=Java\u0026color=007ec6\" alt=\"Java\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/xberg-io/liter-llm/tree/main/packages/go\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/v/tag/xberg-io/liter-llm?label=Go\u0026color=007ec6\" alt=\"Go\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.nuget.org/packages/LiterLlm\"\u003e\n    \u003cimg src=\"https://img.shields.io/nuget/v/LiterLlm?label=C%23\u0026color=007ec6\" alt=\"C#\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://packagist.org/packages/xberg-io/liter-llm\"\u003e\n    \u003cimg src=\"https://img.shields.io/packagist/v/xberg-io/liter-llm?label=PHP\u0026color=007ec6\" alt=\"PHP\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://rubygems.org/gems/liter_llm\"\u003e\n    \u003cimg src=\"https://img.shields.io/gem/v/liter_llm?label=Ruby\u0026color=007ec6\" alt=\"Ruby\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://hex.pm/packages/liter_llm\"\u003e\n    \u003cimg src=\"https://img.shields.io/hexpm/v/liter_llm?label=Elixir\u0026color=007ec6\" alt=\"Elixir\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/xberg-io/liter-llm/pkgs/container/liter-llm\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Docker-007ec6?logo=docker\u0026logoColor=white\" alt=\"Docker\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/xberg-io/homebrew-tap/blob/main/Formula/liter-llm.rb\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Homebrew-007ec6?logo=homebrew\u0026logoColor=white\" alt=\"Homebrew\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/xberg-io/liter-llm/tree/main/crates/liter-llm-ffi\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/C-FFI-007ec6\" alt=\"C FFI\" /\u003e\n  \u003c/a\u003e\n\n  \u003c!-- Project Info --\u003e\n  \u003ca href=\"https://github.com/xberg-io/liter-llm/blob/main/LICENSE\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/License-MIT-007ec6\" alt=\"License\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://docs.liter-llm.xberg.io\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Docs-liter--llm-007ec6\" alt=\"Docs\" /\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\" style=\"display: flex; flex-wrap: wrap; gap: 12px; justify-content: center; margin: 28px 0 24px\"\u003e\n  \u003ca href=\"https://discord.gg/xt9WY3GnKR\"\u003e\n    \u003cimg\n      height=\"22\"\n      src=\"https://img.shields.io/badge/Discord-Chat-007ec6?logo=discord\u0026logoColor=white\"\n      alt=\"Join Discord\"\n    /\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n**A lighter, faster, safer universal LLM API client** — one Rust core, 14 native language bindings, 143 providers.\n\n## What and Why?\n\nliter-llm is a universal LLM API client compiled from the ground up in Rust: one core, 14 native language bindings, and 143 providers. No interpreter, no transitive dependency tree, no supply-chain surface area — and a drop-in OpenAI-compatible proxy plus an MCP server in a single 35 MB binary.\n\n- **Compiled Rust core** — no `pip install` supply chain, no `.pth` auto-execution hooks, no runtime dependency tree to compromise.\n- **Secrets stay secret** — API keys are wrapped in [`secrecy::SecretString`](https://docs.rs/secrecy/): zeroed on drop, redacted in logs, never serialized.\n- **Polyglot from day one** — Python, TypeScript, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly, all thin wrappers over the same Rust core, plus a C/FFI surface for everything else.\n- **Observability built in** — production-grade [OpenTelemetry](https://opentelemetry.io/) with GenAI semantic conventions, not an afterthought callback system.\n- **Composable middleware** — rate limiting, caching, cost tracking, health checks, and fallback as [Tower](https://docs.rs/tower/) layers you stack like building blocks.\n\nWe credit [litellm](https://github.com/BerriAI/litellm) for proving the category; our provider registry was bootstrapped from theirs. See [ATTRIBUTIONS.md](ATTRIBUTIONS.md).\n\n### Features\n\n| Feature | Description |\n| ------- | ----------- |\n| **143 providers** | OpenAI, Anthropic, Google, AWS Bedrock, Groq, Mistral, Together, Fireworks, DeepSeek, Cohere, and 130+ more — compiled at build time |\n| **14 native bindings** | Rust, Python, Node.js, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WebAssembly — plus a shared C/FFI surface |\n| **First-class streaming** | SSE and AWS EventStream binary protocol with zero-copy buffers |\n| **Proxy \u0026 MCP server** | Drop-in OpenAI-compatible proxy (22 endpoints) and MCP tool server in a 35 MB Docker image |\n| **Tower middleware** | Rate limiting, caching (40+ OpenDAL backends), cost tracking, budget enforcement, health checks, and fallback — all composable |\n| **Observability** | OpenTelemetry with GenAI semantic conventions, cost-tracking spans, and HTTP-level tracing |\n| **Tool calling** | Parallel tools, structured outputs, and JSON-schema validation |\n| **Search \u0026 OCR** | Web search across 12 providers, document OCR across 4 |\n| **TOML configuration** | `liter-llm.toml` auto-discovery, custom providers, cache backends, and middleware config |\n| **Local LLM support** | Ollama, LM Studio, vLLM, llama.cpp, LocalAI, and llamafile via OpenAI-compatible APIs |\n\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://github.com/xberg-io/liter-llm/stargazers\"\u003e\n    \u003cimg src=\"docs/assets/star.gif\" alt=\"Star liter-llm on GitHub\" width=\"640\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\u003cstrong\u003e⭐ Star this repo to show your support — it helps others discover liter-llm.\u003c/strong\u003e\u003c/p\u003e\n\n## Quick Start\n\n### Language Packages\n\n\u003cdetails open\u003e\n\u003csummary\u003e\u003cstrong\u003ePython\u003c/strong\u003e\u003c/summary\u003e\n\n```sh\npip install liter-llm\n```\n\nSee [Python README](packages/python/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eNode.js\u003c/strong\u003e\u003c/summary\u003e\n\n```sh\npnpm add @xberg-io/liter-llm\n```\n\nSee [Node.js README](crates/liter-llm-node/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eRust\u003c/strong\u003e\u003c/summary\u003e\n\n```sh\ncargo add liter-llm\n```\n\nSee [Rust crate](crates/liter-llm) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eGo\u003c/strong\u003e\u003c/summary\u003e\n\n```sh\ngo get github.com/xberg-io/liter-llm/packages/go\n```\n\nSee [Go README](packages/go/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eJava\u003c/strong\u003e\u003c/summary\u003e\n\nAvailable on Maven Central as `io.xberg.literllm:liter-llm`. See [Java README](packages/java/README.md) for the dependency snippet and current version.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eRuby\u003c/strong\u003e\u003c/summary\u003e\n\n```sh\ngem install liter_llm\n```\n\nSee [Ruby README](packages/ruby/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003ePHP\u003c/strong\u003e\u003c/summary\u003e\n\n```sh\ncomposer require xberg-io/liter-llm\n```\n\nSee [PHP README](packages/php/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eC#\u003c/strong\u003e\u003c/summary\u003e\n\n```sh\ndotnet add package LiterLlm\n```\n\nSee [.NET README](packages/csharp/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eElixir\u003c/strong\u003e\u003c/summary\u003e\n\nAdd `{:liter_llm, \"~\u003e 1.6\"}` to your `mix.exs` dependencies. See [Elixir README](packages/elixir/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eDart / Flutter\u003c/strong\u003e\u003c/summary\u003e\n\n```sh\ndart pub add liter_llm\n```\n\nSee [Dart README](packages/dart/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eSwift\u003c/strong\u003e\u003c/summary\u003e\n\nAdd via Swift Package Manager. See [Swift README](packages/swift/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eKotlin (Android)\u003c/strong\u003e\u003c/summary\u003e\n\nAvailable on Maven Central as `io.xberg.literllm:liter-llm-android`. See [Kotlin README](packages/kotlin-android/README.md) for the dependency snippet and current version.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eZig\u003c/strong\u003e\u003c/summary\u003e\n\nSee [Zig README](packages/zig/README.md) for installation and usage.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eWebAssembly\u003c/strong\u003e\u003c/summary\u003e\n\n```sh\npnpm add @xberg-io/liter-llm-wasm\n```\n\nSee [WebAssembly README](crates/liter-llm-wasm/README.md) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eC/C++ (FFI)\u003c/strong\u003e\u003c/summary\u003e\n\nBuild from source as part of this workspace. See [FFI crate](crates/liter-llm-ffi) for full documentation.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eCLI, Proxy \u0026 MCP Server\u003c/strong\u003e\u003c/summary\u003e\n\nThe `liter-llm` CLI ships both the OpenAI-compatible proxy and the MCP tool server. Install it any of these ways:\n\n```sh\nbrew install xberg-io/tap/liter-llm\ncargo install liter-llm-cli                 # from crates.io\nnpx @xberg-io/liter-llm-cli --help         # npm (self-installs the binary)\ndocker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/xberg-io/liter-llm\n```\n\nThen run the proxy or the MCP server:\n\n```sh\nliter-llm api --config liter-llm-proxy.toml   # OpenAI-compatible proxy (22 endpoints)\nliter-llm mcp --transport stdio               # MCP tool server (stdio)\nliter-llm mcp --transport http --port 3001    # MCP tool server (Streamable HTTP)\n```\n\nSee the [MCP server guide](https://docs.liter-llm.xberg.io/server/mcp-server/) and the [proxy guide](https://docs.liter-llm.xberg.io/server/proxy-server/) for transports, routing, virtual keys, and budgets. To use the MCP server inside a coding agent, install the **liter-llm plugin** (below) — it auto-registers the server, no manual config required.\n\n\u003c/details\u003e\n\n### AI Coding Assistants\n\nInstall the liter-llm plugin from the [`xberg-io/plugins`](https://github.com/xberg-io/plugins) marketplace. It ships the liter-llm agent skills (chat, streaming, tools, embeddings across 143 providers) and works with every major coding agent — expand your harness below.\n\n\u003cdetails open\u003e\n\u003csummary\u003e\u003cstrong\u003eClaude Code\u003c/strong\u003e\u003c/summary\u003e\n\n```text\n/plugin marketplace add xberg-io/plugins\n/plugin install liter-llm@xberg\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eCodex CLI\u003c/strong\u003e\u003c/summary\u003e\n\n```text\n/plugins add https://github.com/xberg-io/plugins\n```\n\nThen search for `liter-llm` and select **Install Plugin**.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eCursor\u003c/strong\u003e\u003c/summary\u003e\n\nSettings → Plugins → Add from URL → `https://github.com/xberg-io/plugins`, then select **liter-llm**.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eGemini CLI\u003c/strong\u003e\u003c/summary\u003e\n\n```text\ngemini extensions install https://github.com/xberg-io/plugins\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eFactory Droid\u003c/strong\u003e\u003c/summary\u003e\n\n```text\ndroid plugin marketplace add https://github.com/xberg-io/plugins\ndroid plugin install liter-llm@xberg\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eGitHub Copilot CLI\u003c/strong\u003e\u003c/summary\u003e\n\n```text\ncopilot plugin marketplace add https://github.com/xberg-io/plugins\ncopilot plugin install liter-llm@xberg\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eopencode\u003c/strong\u003e\u003c/summary\u003e\n\nNot yet published as an opencode package. Install via any harness above (self-hosted marketplace); opencode support is tracked in [`xberg-io/plugins`](https://github.com/xberg-io/plugins).\n\n\u003c/details\u003e\n\n## Documentation\n\nFull guides, the unified `chat()` API for every binding, multimodal I/O, the proxy/gateway, and the complete provider list live at **[docs.liter-llm.xberg.io](https://docs.liter-llm.xberg.io)**.\n\n## Contributing\n\nContributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\nJoin our [Discord community](https://discord.gg/xt9WY3GnKR) for questions and discussion.\n\n## Part of Xberg.io\n\n- [Xberg](https://github.com/xberg-io/xberg) — document intelligence: text, tables, metadata from 91+ formats with optional OCR.\n- [Xberg Enterprise](https://github.com/xberg-io/xberg-enterprise) — managed extraction API with SDKs, dashboards, and observability.\n- [crawlberg](https://github.com/xberg-io/crawlberg) — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.\n- [html-to-markdown](https://github.com/xberg-io/html-to-markdown) — fast, lossless HTML→Markdown engine.\n- [liter-llm](https://github.com/xberg-io/liter-llm) — universal LLM API client with native bindings for 14 languages and 143 providers.\n- [tree-sitter-language-pack](https://github.com/xberg-io/tree-sitter-language-pack) — tree-sitter grammars and code-intelligence primitives.\n- [alef](https://github.com/xberg-io/alef) — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.\n\n## License\n\nMIT — see [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxberg-io%2Fliter-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxberg-io%2Fliter-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxberg-io%2Fliter-llm/lists"}