{"id":13645227,"url":"https://github.com/berriai/litellm","last_synced_at":"2026-06-11T03:02:52.035Z","repository":{"id":184080397,"uuid":"671269505","full_name":"BerriAI/litellm","owner":"BerriAI","description":"Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]","archived":false,"fork":false,"pushed_at":"2026-01-22T09:48:40.000Z","size":728845,"stargazers_count":34284,"open_issues_count":1147,"forks_count":5430,"subscribers_count":155,"default_branch":"main","last_synced_at":"2026-01-22T10:25:41.589Z","etag":null,"topics":["ai-gateway","anthropic","azure-openai","bedrock","gateway","langchain","litellm","llm","llm-gateway","llmops","mcp-gateway","openai","openai-proxy","vertex-ai"],"latest_commit_sha":null,"homepage":"https://docs.litellm.ai/docs/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BerriAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"security.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":"https://buy.stripe.com/9AQ03Kd3P91o0Q8bIS"}},"created_at":"2023-07-27T00:09:52.000Z","updated_at":"2026-01-22T10:14:25.000Z","dependencies_parsed_at":"2023-09-27T05:32:48.439Z","dependency_job_id":"c88025ff-c055-4896-9963-01c5e148b4c4","html_url":"https://github.com/BerriAI/litellm","commit_stats":{"total_commits":16614,"total_committers":375,"mean_commits":44.304,"dds":"0.44113398338750454","last_synced_commit":"55139b8fd6c9a249460871eae2fd64ec4c4ff279"},"previous_names":["berriai/litellm"],"tags_count":1211,"template":false,"template_full_name":null,"purl":"pkg:github/BerriAI/litellm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BerriAI%2Flitellm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BerriAI%2Flitellm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BerriAI%2Flitellm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BerriAI%2Flitellm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BerriAI","download_url":"https://codeload.github.com/BerriAI/litellm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BerriAI%2Flitellm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28744419,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-25T02:46:29.005Z","status":"ssl_error","status_checked_at":"2026-01-25T02:44:29.968Z","response_time":113,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-gateway","anthropic","azure-openai","bedrock","gateway","langchain","litellm","llm","llm-gateway","llmops","mcp-gateway","openai","openai-proxy","vertex-ai"],"created_at":"2024-08-02T01:02:31.586Z","updated_at":"2026-01-25T05:01:49.810Z","avatar_url":"https://github.com/BerriAI.png","language":"Python","funding_links":["https://buy.stripe.com/9AQ03Kd3P91o0Q8bIS"],"categories":["Langchain"],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n šŸš… LiteLLM\n \u003c/h1\u003e\n \u003cp align=\"center\"\u003e\n \u003cp align=\"center\"\u003e\n \u003ca href=\"https://render.com/deploy?repo=https://github.com/BerriAI/litellm\" target=\"_blank\" rel=\"nofollow\"\u003e\u003cimg src=\"https://render.com/images/deploy-to-render-button.svg\" alt=\"Deploy to Render\"\u003e\u003c/a\u003e\n \u003ca href=\"https://railway.app/template/HLP0Ub?referralCode=jch2ME\"\u003e\n \u003cimg src=\"https://railway.app/button.svg\" alt=\"Deploy on Railway\"\u003e\n \u003c/a\u003e\n \u003c/p\u003e\n \u003cp align=\"center\"\u003eCall all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, Groq etc.]\n \u003cbr\u003e\n \u003c/p\u003e\n\u003ch4 align=\"center\"\u003e\u003ca href=\"https://docs.litellm.ai/docs/simple_proxy\" target=\"_blank\"\u003eLiteLLM Proxy Server (LLM Gateway)\u003c/a\u003e | \u003ca href=\"https://docs.litellm.ai/docs/hosted\" target=\"_blank\"\u003e Hosted Proxy (Preview)\u003c/a\u003e | \u003ca href=\"https://docs.litellm.ai/docs/enterprise\"target=\"_blank\"\u003eEnterprise Tier\u003c/a\u003e\u003c/h4\u003e\n\u003ch4 align=\"center\"\u003e\n \u003ca href=\"https://pypi.org/project/litellm/\" target=\"_blank\"\u003e\n \u003cimg src=\"https://img.shields.io/pypi/v/litellm.svg\" alt=\"PyPI Version\"\u003e\n \u003c/a\u003e\n \u003ca href=\"https://dl.circleci.com/status-badge/redirect/gh/BerriAI/litellm/tree/main\" target=\"_blank\"\u003e\n \u003cimg src=\"https://dl.circleci.com/status-badge/img/gh/BerriAI/litellm/tree/main.svg?style=svg\" alt=\"CircleCI\"\u003e\n \u003c/a\u003e\n \u003ca href=\"https://www.ycombinator.com/companies/berriai\"\u003e\n \u003cimg src=\"https://img.shields.io/badge/Y%20Combinator-W23-orange?style=flat-square\" alt=\"Y Combinator W23\"\u003e\n \u003c/a\u003e\n \u003ca href=\"https://wa.link/huol9n\"\u003e\n \u003cimg src=\"https://img.shields.io/static/v1?label=Chat%20on\u0026message=WhatsApp\u0026color=success\u0026logo=WhatsApp\u0026style=flat-square\" alt=\"Whatsapp\"\u003e\n \u003c/a\u003e\n \u003ca href=\"https://discord.gg/wuPM9dRgDw\"\u003e\n \u003cimg src=\"https://img.shields.io/static/v1?label=Chat%20on\u0026message=Discord\u0026color=blue\u0026logo=Discord\u0026style=flat-square\" alt=\"Discord\"\u003e\n \u003c/a\u003e\n\u003c/h4\u003e\n\nLiteLLM manages:\n\n- Translate inputs to provider's `completion`, `embedding`, and `image_generation` endpoints\n- [Consistent output](https://docs.litellm.ai/docs/completion/output), text responses will always be available at `['choices'][0]['message']['content']`\n- Retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - [Router](https://docs.litellm.ai/docs/routing)\n- Set Budgets \u0026 Rate limits per project, api key, model [LiteLLM Proxy Server (LLM Gateway)](https://docs.litellm.ai/docs/simple_proxy)\n\n[**Jump to LiteLLM Proxy (LLM Gateway) Docs**](https://github.com/BerriAI/litellm?tab=readme-ov-file#openai-proxy---docs) \u003cbr\u003e\n[**Jump to Supported LLM Providers**](https://github.com/BerriAI/litellm?tab=readme-ov-file#supported-providers-docs)\n\nšŸšØ **Stable Release:** Use docker images with the `-stable` tag. These have undergone 12 hour load tests, before being published. [More information about the release cycle here](https://docs.litellm.ai/docs/proxy/release_cycle)\n\nSupport for more providers. Missing a provider or LLM Platform, raise a [feature request](https://github.com/BerriAI/litellm/issues/new?assignees=\u0026labels=enhancement\u0026projects=\u0026template=feature_request.yml\u0026title=%5BFeature%5D%3A+).\n\n# Usage ([**Docs**](https://docs.litellm.ai/docs/))\n\n\u003e [!IMPORTANT]\n\u003e LiteLLM v1.0.0 now requires `openai\u003e=1.0.0`. Migration guide [here](https://docs.litellm.ai/docs/migration) \n\u003e LiteLLM v1.40.14+ now requires `pydantic\u003e=2.0.0`. No changes required.\n\n\u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/BerriAI/litellm/blob/main/cookbook/liteLLM_Getting_Started.ipynb\"\u003e\n \u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\n\u003c/a\u003e\n\n```shell\npip install litellm\n```\n\n```python\nfrom litellm import completion\nimport os\n\n## set ENV variables\nos.environ[\"OPENAI_API_KEY\"] = \"your-openai-key\"\nos.environ[\"ANTHROPIC_API_KEY\"] = \"your-anthropic-key\"\n\nmessages = [{ \"content\": \"Hello, how are you?\",\"role\": \"user\"}]\n\n# openai call\nresponse = completion(model=\"openai/gpt-4o\", messages=messages)\n\n# anthropic call\nresponse = completion(model=\"anthropic/claude-3-sonnet-20240229\", messages=messages)\nprint(response)\n```\n\n### Response (OpenAI Format)\n\n```json\n{\n \"id\": \"chatcmpl-565d891b-a42e-4c39-8d14-82a1f5208885\",\n \"created\": 1734366691,\n \"model\": \"claude-3-sonnet-20240229\",\n \"object\": \"chat.completion\",\n \"system_fingerprint\": null,\n \"choices\": [\n {\n \"finish_reason\": \"stop\",\n \"index\": 0,\n \"message\": {\n \"content\": \"Hello! As an AI language model, I don't have feelings, but I'm operating properly and ready to assist you with any questions or tasks you may have. How can I help you today?\",\n \"role\": \"assistant\",\n \"tool_calls\": null,\n \"function_call\": null\n }\n }\n ],\n \"usage\": {\n \"completion_tokens\": 43,\n \"prompt_tokens\": 13,\n \"total_tokens\": 56,\n \"completion_tokens_details\": null,\n \"prompt_tokens_details\": {\n \"audio_tokens\": null,\n \"cached_tokens\": 0\n },\n \"cache_creation_input_tokens\": 0,\n \"cache_read_input_tokens\": 0\n }\n}\n```\n\nCall any model supported by a provider, with `model=\u003cprovider_name\u003e/\u003cmodel_name\u003e`. There might be provider-specific details here, so refer to [provider docs for more information](https://docs.litellm.ai/docs/providers)\n\n## Async ([Docs](https://docs.litellm.ai/docs/completion/stream#async-completion))\n\n```python\nfrom litellm import acompletion\nimport asyncio\n\nasync def test_get_response():\n user_message = \"Hello, how are you?\"\n messages = [{\"content\": user_message, \"role\": \"user\"}]\n response = await acompletion(model=\"openai/gpt-4o\", messages=messages)\n return response\n\nresponse = asyncio.run(test_get_response())\nprint(response)\n```\n\n## Streaming ([Docs](https://docs.litellm.ai/docs/completion/stream))\n\nliteLLM supports streaming the model response back, pass `stream=True` to get a streaming iterator in response. \nStreaming is supported for all models (Bedrock, Huggingface, TogetherAI, Azure, OpenAI, etc.)\n\n```python\nfrom litellm import completion\nresponse = completion(model=\"openai/gpt-4o\", messages=messages, stream=True)\nfor part in response:\n print(part.choices[0].delta.content or \"\")\n\n# claude 2\nresponse = completion('anthropic/claude-3-sonnet-20240229', messages, stream=True)\nfor part in response:\n print(part)\n```\n\n### Response chunk (OpenAI Format)\n\n```json\n{\n \"id\": \"chatcmpl-2be06597-eb60-4c70-9ec5-8cd2ab1b4697\",\n \"created\": 1734366925,\n \"model\": \"claude-3-sonnet-20240229\",\n \"object\": \"chat.completion.chunk\",\n \"system_fingerprint\": null,\n \"choices\": [\n {\n \"finish_reason\": null,\n \"index\": 0,\n \"delta\": {\n \"content\": \"Hello\",\n \"role\": \"assistant\",\n \"function_call\": null,\n \"tool_calls\": null,\n \"audio\": null\n },\n \"logprobs\": null\n }\n ]\n}\n```\n\n## Logging Observability ([Docs](https://docs.litellm.ai/docs/observability/callbacks))\n\nLiteLLM exposes pre defined callbacks to send data to Lunary, MLflow, Langfuse, DynamoDB, s3 Buckets, Helicone, Promptlayer, Traceloop, Athina, Slack\n\n```python\nfrom litellm import completion\n\n## set env variables for logging tools (when using MLflow, no API key set up is required)\nos.environ[\"LUNARY_PUBLIC_KEY\"] = \"your-lunary-public-key\"\nos.environ[\"HELICONE_API_KEY\"] = \"your-helicone-auth-key\"\nos.environ[\"LANGFUSE_PUBLIC_KEY\"] = \"\"\nos.environ[\"LANGFUSE_SECRET_KEY\"] = \"\"\nos.environ[\"ATHINA_API_KEY\"] = \"your-athina-api-key\"\n\nos.environ[\"OPENAI_API_KEY\"] = \"your-openai-key\"\n\n# set callbacks\nlitellm.success_callback = [\"lunary\", \"mlflow\", \"langfuse\", \"athina\", \"helicone\"] # log input/output to lunary, langfuse, supabase, athina, helicone etc\n\n#openai call\nresponse = completion(model=\"openai/gpt-4o\", messages=[{\"role\": \"user\", \"content\": \"Hi šŸ‘‹ - i'm openai\"}])\n```\n\n# LiteLLM Proxy Server (LLM Gateway) - ([Docs](https://docs.litellm.ai/docs/simple_proxy))\n\nTrack spend + Load Balance across multiple projects\n\n[Hosted Proxy (Preview)](https://docs.litellm.ai/docs/hosted)\n\nThe proxy provides:\n\n1. [Hooks for auth](https://docs.litellm.ai/docs/proxy/virtual_keys#custom-auth)\n2. [Hooks for logging](https://docs.litellm.ai/docs/proxy/logging#step-1---create-your-custom-litellm-callback-class)\n3. [Cost tracking](https://docs.litellm.ai/docs/proxy/virtual_keys#tracking-spend)\n4. [Rate Limiting](https://docs.litellm.ai/docs/proxy/users#set-rate-limits)\n\n## šŸ“– Proxy Endpoints - [Swagger Docs](https://litellm-api.up.railway.app/)\n\n\n## Quick Start Proxy - CLI\n\n```shell\npip install 'litellm[proxy]'\n```\n\n### Step 1: Start litellm proxy\n\n```shell\n$ litellm --model huggingface/bigcode/starcoder\n\n#INFO: Proxy running on http://0.0.0.0:4000\n```\n\n### Step 2: Make ChatCompletions Request to Proxy\n\n\n\u003e [!IMPORTANT]\n\u003e šŸ’” [Use LiteLLM Proxy with Langchain (Python, JS), OpenAI SDK (Python, JS) Anthropic SDK, Mistral SDK, LlamaIndex, Instructor, Curl](https://docs.litellm.ai/docs/proxy/user_keys) \n\n```python\nimport openai # openai v1.0.0+\nclient = openai.OpenAI(api_key=\"anything\",base_url=\"http://0.0.0.0:4000\") # set proxy to base_url\n# request sent to model set on litellm proxy, `litellm --model`\nresponse = client.chat.completions.create(model=\"gpt-3.5-turbo\", messages = [\n {\n \"role\": \"user\",\n \"content\": \"this is a test request, write a short poem\"\n }\n])\n\nprint(response)\n```\n\n## Proxy Key Management ([Docs](https://docs.litellm.ai/docs/proxy/virtual_keys))\n\nConnect the proxy with a Postgres DB to create proxy keys\n\n```bash\n# Get the code\ngit clone https://github.com/BerriAI/litellm\n\n# Go to folder\ncd litellm\n\n# Add the master key - you can change this after setup\necho 'LITELLM_MASTER_KEY=\"sk-1234\"' \u003e .env\n\n# Add the litellm salt key - you cannot change this after adding a model\n# It is used to encrypt / decrypt your LLM API Key credentials\n# We recommend - https://1password.com/password-generator/ \n# password generator to get a random hash for litellm salt key\necho 'LITELLM_SALT_KEY=\"sk-1234\"' \u003e .env\n\nsource .env\n\n# Start\ndocker-compose up\n```\n\n\nUI on `/ui` on your proxy server\n![ui_3](https://github.com/BerriAI/litellm/assets/29436595/47c97d5e-b9be-4839-b28c-43d7f4f10033)\n\nSet budgets and rate limits across multiple projects\n`POST /key/generate`\n\n### Request\n\n```shell\ncurl 'http://0.0.0.0:4000/key/generate' \\\n--header 'Authorization: Bearer sk-1234' \\\n--header 'Content-Type: application/json' \\\n--data-raw '{\"models\": [\"gpt-3.5-turbo\", \"gpt-4\", \"claude-2\"], \"duration\": \"20m\",\"metadata\": {\"user\": \"ishaan@berri.ai\", \"team\": \"core-infra\"}}'\n```\n\n### Expected Response\n\n```shell\n{\n \"key\": \"sk-kdEXbIqZRwEeEiHwdg7sFA\", # Bearer token\n \"expires\": \"2023-11-19T01:38:25.838000+00:00\" # datetime object\n}\n```\n\n## Supported Providers ([Docs](https://docs.litellm.ai/docs/providers))\n\n| Provider | [Completion](https://docs.litellm.ai/docs/#basic-usage) | [Streaming](https://docs.litellm.ai/docs/completion/stream#streaming-responses) | [Async Completion](https://docs.litellm.ai/docs/completion/stream#async-completion) | [Async Streaming](https://docs.litellm.ai/docs/completion/stream#async-streaming) | [Async Embedding](https://docs.litellm.ai/docs/embedding/supported_embedding) | [Async Image Generation](https://docs.litellm.ai/docs/image_generation) |\n|-------------------------------------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------|-------------------------------------------------------------------------|\n| [openai](https://docs.litellm.ai/docs/providers/openai) | āœ… | āœ… | āœ… | āœ… | āœ… | āœ… |\n| [azure](https://docs.litellm.ai/docs/providers/azure) | āœ… | āœ… | āœ… | āœ… | āœ… | āœ… |\n| [AI/ML API](https://docs.litellm.ai/docs/providers/aiml) | āœ… | āœ… | āœ… | āœ… | āœ… | āœ… |\n| [aws - sagemaker](https://docs.litellm.ai/docs/providers/aws_sagemaker) | āœ… | āœ… | āœ… | āœ… | āœ… | |\n| [aws - bedrock](https://docs.litellm.ai/docs/providers/bedrock) | āœ… | āœ… | āœ… | āœ… | āœ… | |\n| [google - vertex_ai](https://docs.litellm.ai/docs/providers/vertex) | āœ… | āœ… | āœ… | āœ… | āœ… | āœ… |\n| [google - palm](https://docs.litellm.ai/docs/providers/palm) | āœ… | āœ… | āœ… | āœ… | | |\n| [google AI Studio - gemini](https://docs.litellm.ai/docs/providers/gemini) | āœ… | āœ… | āœ… | āœ… | | |\n| [mistral ai api](https://docs.litellm.ai/docs/providers/mistral) | āœ… | āœ… | āœ… | āœ… | āœ… | |\n| [cloudflare AI Workers](https://docs.litellm.ai/docs/providers/cloudflare_workers) | āœ… | āœ… | āœ… | āœ… | | |\n| [cohere](https://docs.litellm.ai/docs/providers/cohere) | āœ… | āœ… | āœ… | āœ… | āœ… | |\n| [anthropic](https://docs.litellm.ai/docs/providers/anthropic) | āœ… | āœ… | āœ… | āœ… | | |\n| [empower](https://docs.litellm.ai/docs/providers/empower) | āœ… | āœ… | āœ… | āœ… |\n| [huggingface](https://docs.litellm.ai/docs/providers/huggingface) | āœ… | āœ… | āœ… | āœ… | āœ… | |\n| [replicate](https://docs.litellm.ai/docs/providers/replicate) | āœ… | āœ… | āœ… | āœ… | | |\n| [together_ai](https://docs.litellm.ai/docs/providers/togetherai) | āœ… | āœ… | āœ… | āœ… | | |\n| [openrouter](https://docs.litellm.ai/docs/providers/openrouter) | āœ… | āœ… | āœ… | āœ… | | |\n| [ai21](https://docs.litellm.ai/docs/providers/ai21) | āœ… | āœ… | āœ… | āœ… | | |\n| [baseten](https://docs.litellm.ai/docs/providers/baseten) | āœ… | āœ… | āœ… | āœ… | | |\n| [vllm](https://docs.litellm.ai/docs/providers/vllm) | āœ… | āœ… | āœ… | āœ… | | |\n| [nlp_cloud](https://docs.litellm.ai/docs/providers/nlp_cloud) | āœ… | āœ… | āœ… | āœ… | | |\n| [aleph alpha](https://docs.litellm.ai/docs/providers/aleph_alpha) | āœ… | āœ… | āœ… | āœ… | | |\n| [petals](https://docs.litellm.ai/docs/providers/petals) | āœ… | āœ… | āœ… | āœ… | | |\n| [ollama](https://docs.litellm.ai/docs/providers/ollama) | āœ… | āœ… | āœ… | āœ… | āœ… | |\n| [deepinfra](https://docs.litellm.ai/docs/providers/deepinfra) | āœ… | āœ… | āœ… | āœ… | | |\n| [perplexity-ai](https://docs.litellm.ai/docs/providers/perplexity) | āœ… | āœ… | āœ… | āœ… | | |\n| [Groq AI](https://docs.litellm.ai/docs/providers/groq) | āœ… | āœ… | āœ… | āœ… | | |\n| [Deepseek](https://docs.litellm.ai/docs/providers/deepseek) | āœ… | āœ… | āœ… | āœ… | | |\n| [anyscale](https://docs.litellm.ai/docs/providers/anyscale) | āœ… | āœ… | āœ… | āœ… | | |\n| [IBM - watsonx.ai](https://docs.litellm.ai/docs/providers/watsonx) | āœ… | āœ… | āœ… | āœ… | āœ… | |\n| [voyage ai](https://docs.litellm.ai/docs/providers/voyage) | | | | | āœ… | |\n| [xinference [Xorbits Inference]](https://docs.litellm.ai/docs/providers/xinference) | | | | | āœ… | |\n| [FriendliAI](https://docs.litellm.ai/docs/providers/friendliai) | āœ… | āœ… | āœ… | āœ… | | |\n| [Galadriel](https://docs.litellm.ai/docs/providers/galadriel) | āœ… | āœ… | āœ… | āœ… | | |\n\n[**Read the Docs**](https://docs.litellm.ai/docs/)\n\n## Contributing\n\nTo contribute: Clone the repo locally -\u003e Make a change -\u003e Submit a PR with the change.\n\nHere's how to modify the repo locally:\n\nStep 1: Clone the repo\n\n```\ngit clone https://github.com/BerriAI/litellm.git\n```\n\nStep 2: Install dependencies:\n\n```\npip install -r requirements.txt\n```\n\nStep 3: Test your change:\n\na. Add a pytest test within `tests/litellm/`\n\nThis folder follows the same directory structure as `litellm/`.\n\nIf a corresponding test file does not exist, create one.\n\nb. Run the test\n\n```\ncd tests/litellm # pwd: Documents/litellm/litellm/tests/litellm\npytest /path/to/test_file.py\n```\n\nStep 4: Submit a PR with your changes! šŸš€\n\n- push your fork to your GitHub repo\n- submit a PR from there\n\n### Building LiteLLM Docker Image \n\nFollow these instructions if you want to build / run the LiteLLM Docker Image yourself.\n\nStep 1: Clone the repo\n\n```\ngit clone https://github.com/BerriAI/litellm.git\n```\n\nStep 2: Build the Docker Image\n\nBuild using Dockerfile.non_root\n```\ndocker build -f docker/Dockerfile.non_root -t litellm_test_image .\n```\n\nStep 3: Run the Docker Image\n\nMake sure config.yaml is present in the root directory. This is your litellm proxy config file.\n```\ndocker run \\\n -v $(pwd)/proxy_config.yaml:/app/config.yaml \\\n -e DATABASE_URL=\"postgresql://xxxxxxxx\" \\\n -e LITELLM_MASTER_KEY=\"sk-1234\" \\\n -p 4000:4000 \\\n litellm_test_image \\\n --config /app/config.yaml --detailed_debug\n```\n\n# Enterprise\nFor companies that need better security, user management and professional support\n\n[Talk to founders](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat)\n\nThis covers: \n- āœ… **Features under the [LiteLLM Commercial License](https://docs.litellm.ai/docs/proxy/enterprise):**\n- āœ… **Feature Prioritization**\n- āœ… **Custom Integrations**\n- āœ… **Professional Support - Dedicated discord + slack**\n- āœ… **Custom SLAs**\n- āœ… **Secure access with Single Sign-On**\n\n# Code Quality / Linting\n\nLiteLLM follows the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html).\n\nWe run: \n- Ruff for [formatting and linting checks](https://github.com/BerriAI/litellm/blob/e19bb55e3b4c6a858b6e364302ebbf6633a51de5/.circleci/config.yml#L320)\n- Mypy + Pyright for typing [1](https://github.com/BerriAI/litellm/blob/e19bb55e3b4c6a858b6e364302ebbf6633a51de5/.circleci/config.yml#L90), [2](https://github.com/BerriAI/litellm/blob/e19bb55e3b4c6a858b6e364302ebbf6633a51de5/.pre-commit-config.yaml#L4)\n- Black for [formatting](https://github.com/BerriAI/litellm/blob/e19bb55e3b4c6a858b6e364302ebbf6633a51de5/.circleci/config.yml#L79)\n- isort for [import sorting](https://github.com/BerriAI/litellm/blob/e19bb55e3b4c6a858b6e364302ebbf6633a51de5/.pre-commit-config.yaml#L10)\n\n\nIf you have suggestions on how to improve the code quality feel free to open an issue or a PR.\n\n\n# Support / talk with founders\n\n- [Schedule Demo šŸ‘‹](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version)\n- [Community Discord šŸ’­](https://discord.gg/wuPM9dRgDw)\n- Our numbers šŸ“ž +1 (770) 8783-106 / ā€­+1 (412) 618-6238ā€¬\n- Our emails āœ‰ļø ishaan@berri.ai / krrish@berri.ai\n\n# Why did we build this\n\n- **Need for simplicity**: Our code started to get extremely complicated managing \u0026 translating calls between Azure, OpenAI and Cohere.\n\n# Contributors\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\n\u003c!-- markdownlint-restore --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n\n\u003ca href=\"https://github.com/BerriAI/litellm/graphs/contributors\"\u003e\n \u003cimg src=\"https://contrib.rocks/image?repo=BerriAI/litellm\" /\u003e\n\u003c/a\u003e\n\n\n## Run in Developer mode\n### Services\n1. Setup .env file in root\n2. Run dependant services `docker-compose up db prometheus`\n\n### Backend\n1. (In root) create virtual environment `python -m venv .venv`\n2. Activate virtual environment `source .venv/bin/activate`\n3. Install dependencies `pip install -e \".[all]\"`\n4. Start proxy backend `uvicorn litellm.proxy.proxy_server:app --host localhost --port 4000 --reload`\n\n### Frontend\n1. Navigate to `ui/litellm-dashboard`\n2. Install dependencies `npm install`\n3. Run `npm run dev` to start the dashboard\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fberriai%2Flitellm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fberriai%2Flitellm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fberriai%2Flitellm/lists"}