https://github.com/BerriAI/litellm
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
https://github.com/BerriAI/litellm
ai-gateway anthropic azure-openai bedrock gateway langchain llm llm-gateway llmops openai openai-proxy vertex-ai
Last synced: about 1 year ago
JSON representation
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
- Host: GitHub
- URL: https://github.com/BerriAI/litellm
- Owner: BerriAI
- License: other
- Created: 2023-07-27T00:09:52.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-03-18T01:36:04.000Z (about 1 year ago)
- Last Synced: 2025-03-18T01:45:34.347Z (about 1 year ago)
- Topics: ai-gateway, anthropic, azure-openai, bedrock, gateway, langchain, llm, llm-gateway, llmops, openai, openai-proxy, vertex-ai
- Language: Python
- Homepage: https://docs.litellm.ai/docs/
- Size: 353 MB
- Stars: 19,229
- Watchers: 106
- Forks: 2,382
- Open Issues: 1,437
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Security: security.md
Awesome Lists containing this project
- awesome-prompts - LiteLLM
- awesome-mlops - LiteLLM
- my-awesome - BerriAI/litellm - gateway,anthropic,azure-openai,bedrock,gateway,langchain,litellm,llm,llm-gateway,llmops,mcp-gateway,openai,openai-proxy,vertex-ai pushed_at:2026-05 star:48.9k fork:8.5k Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM] (Python)
- awesome-llm-tools - LiteLLM
- awesome-llm-services - LiteLLM
- awesome-llm-json - LiteLLM
- awesome-production-llm - LiteLLM
- StarryDivineSky - BerriAI/litellm
- awesome-open-source-ai-tools - BerriAI/litellm - Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Co... (Coding & Development)
- awesome-mistral - LiteLLM
- awesome-ml-python-packages - litellm
- Awesome-RAG-Production - LiteLLM - host / Cloud | 100+ providers | Redis / S3 / Disk | Built-in | Multi-provider routing, OpenAI-compatible proxy | (LLM Gateways & Routing / Tools)
- awesome-ai - **LiteLLM**
- awesome-LLM-resources - LiteLLM
- awesome-harness-engineering - LiteLLM - compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider resilience (automatic failover on 429/500 errors), budget guardrails, or the ability to swap models without touching orchestration code.  (Design Primitives / Task Runners & Orchestration)
- awesome-x-ops - LiteLLM - compatible LLM gateway with routing, budgets, logging, and provider abstraction. (LLM and Agent Observability)
- awesome_ai_agents - LiteLLM - LiteLLM has added support for the OpenAI Assistants API, enabling seamless integration of stateful operations and automatic RAG pipelines into existing chatbots [github](https://github.com/BerriAI/litellm) (Learning / Repositories)
- awesome-agents - LiteLLM - Unified interface to call 100+ LLM APIs using the OpenAI format with load balancing and spend tracking. (SDKs and Libraries)
- awesome-safety-critical-ai - `BerriAI/litellm`
- Ultimate-AI-Resources - LiteLLM
- awesome-generative-ai-data-scientist - LiteLLM
- awesome-harness-engineering - BerriAI/litellm
- AiTreasureBox - BerriAI/litellm - 11-03_30593_4](https://img.shields.io/github/stars/BerriAI/litellm.svg)|Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)| (Repos)
- awesome-ai-coding-tools - LiteLLM
- awesome-ai - litellm - Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq] (Uncategorized / Uncategorized)
- awesome-local-ai - LiteLLM - Call all LLM APIs using the OpenAI format. (Developer tools)
- awesome-data-analysis - LiteLLM - Unified interface to call all LLM APIs (OpenAI, Anthropic, Cohere, etc.) with consistent output formatting. (🚀 MLOps / Tools)
- awesome-ai-agents - BerriAI/litellm - LiteLLM is a Python SDK and proxy server that provides a unified OpenAI-compatible interface to call over 100 large language model APIs from multiple providers, featuring consistent output, retry logic, budget controls, and extensive observability integrations. (Agent Integration & Deployment Tools / Stateful Serverless Frameworks)
- awesome-production-machine-learning - LiteLLM - LiteLLM is a Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq. (Deployment and Serving)
- Awesome-RAG - litellm - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]. | [](https://github.com/BerriAI/litellm/stargazers) | (Model Serving / UI/Interface Tutorials)
- awesome - BerriAI/litellm - Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. \[Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM] (Python)
- Awesome-LLMOps - LiteLLM - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq].    (Inference / LLM Router)
- awesome - BerriAI/litellm - Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM] (<a name="Python"></a>Python)
- awesome-ai-eval - **LiteLLM** - Unified API for 100+ LLM providers with cost tracking, fallbacks, and load balancing. (Platforms / Open Source Platforms)
- awesome-llm-tools - LiteLLM - compatible proxy for 100+ LLM providers | Provider routing | Cloud + Local | (8. Inference Engines / Server / Production)
- awesome-rainmana - BerriAI/litellm - Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropi (Python)
- awesome-cloud-native - litellm - Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]. (API Gateways & Proxies)
- awesome-agent-cortex - LiteLLM - LLM gateway and proxy with logging, cost tracking, and routing controls. (Agent Observability and Testing / Benchmark Reality Check (real-world tool use))
- awesome-llm - LiteLLM - **最流行代理**。用统一的 OpenAI 格式调用 100+ 种 LLM API，支持负载均衡和成本追踪。 (LLM 部署与推理 (Deployment & Inference) / 推理网关 (Inference Gateways))
- awesome-agent-collaborate-tools - LiteLLM - Unified API for 100+ LLMs. Use the same code to call OpenAI, Anthropic, Gemini, Groq, Ollama. Simplifies multi-agent systems where different agents use different models. (Developer Tools & SDKs)
- awesome-rag - LiteLLM - Python SDK for 100+ LLM APIs in OpenAI format (Libraries and SDKs / Cost Calculators)
- stars - BerriAI/litellm
- awesome-ai - LiteLLM
- awesome-ai-devtools - LiteLLM Gateway - in proxy that unifies many LLM providers behind an OpenAI-compatible API, with logging and rate limiting. | API · CLI · Self-hosted | [Docs](https://docs.litellm.ai) / [Repo](https://github.com/BerriAI/litellm) | (Browse The Shelves / Local LLM developer tools)
- awesome - BerriAI/litellm - Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. \[Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM] (Python)
- jimsghstars - BerriAI/litellm - Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq] (Python)
- awesome-github-projects - litellm - Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM] ⭐48,961 `Python` 🔥 (🤖 AI & Machine Learning)
- awesome-ai-coding-agent-tools - LiteLLM - Unified LLM API wrapper exposing 100+ models in OpenAI-compatible format with proxy server. (Agent SDKs & Frameworks / Structured Output & Multi-Provider)
- awesome-llm-gateways - BerriAI/litellm - compatible routing quickly. | 100+ provider support, cost tracking, load balancing, logging, guardrails, active ecosystem. | Python proxy can be less attractive for very low-latency edge gateway use cases; license should be reviewed before commercial embedding. | (Quick Comparison)
- awesome-opensource-ai - LiteLLM - AI Gateway to call 100+ LLM APIs in OpenAI format with unified cost tracking, guardrails, load balancing, and logging.  (8. MLOps / LLMOps & Production)
- awesome-ai - Lite LLM
- awesome-rust-ai-libraries - LiteLLM - A library that provides OpenAI API-compatible proxy support and added support for the OpenAI Assistants API, enabling seamless integration of stateful operations and automatic RAG pipelines into existing chatbots. ([Read more](/details/litellm.md)) `Proxy` `Openai Compatible` `Multi Model` (LLM Tools)
- awesome-llm-cost - LiteLLM - The reference open source LLM proxy. 100+ providers behind one OpenAI compatible API, with budgets, virtual keys, fallbacks, and spend logging. (Routing and Model Selection / Gateways and proxies)