{"id":29803413,"url":"https://github.com/lightning-ai/litai","last_synced_at":"2025-07-30T13:02:48.222Z","repository":{"id":306617451,"uuid":"1025097068","full_name":"Lightning-AI/litAI","owner":"Lightning-AI","description":"The easiest way to use any AI model. Build chatbots, agents, and AI apps that just work - with auto-retries, fallback, and unified billing built in.","archived":false,"fork":false,"pushed_at":"2025-07-26T17:16:30.000Z","size":90,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-26T20:46:58.659Z","etag":null,"topics":["agents","ai","ai-gateway","api","chatbot","llm","llm-gateway","llm-inference","openai","openai-proxy"],"latest_commit_sha":null,"homepage":"https://lightning.ai/docs/litai/home","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Lightning-AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-23T18:04:31.000Z","updated_at":"2025-07-26T17:16:33.000Z","dependencies_parsed_at":"2025-07-26T20:47:02.824Z","dependency_job_id":"39334d63-fc45-4990-a671-ebeaa77b1698","html_url":"https://github.com/Lightning-AI/litAI","commit_stats":null,"previous_names":["lightning-ai/litai"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/Lightning-AI/litAI","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FlitAI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FlitAI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FlitAI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FlitAI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Lightning-AI","download_url":"https://codeload.github.com/Lightning-AI/litAI/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2FlitAI/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267505099,"owners_count":24098346,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agents","ai","ai-gateway","api","chatbot","llm","llm-gateway","llm-inference","openai","openai-proxy"],"created_at":"2025-07-28T11:08:04.622Z","updated_at":"2025-07-28T11:08:05.208Z","avatar_url":"https://github.com/Lightning-AI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align='center'\u003e\n\n\u003ch2\u003e\n  Chat with any AI model with one line of Python.\n  \u003cbr/\u003e\n  Build agents, chatbots, and apps that just work with no downtime.\n\u003c/h2\u003e    \n\n\u003cimg alt=\"Lightning\" src=\"https://github.com/user-attachments/assets/0d0b40a7-d7b9-4b59-a0b6-51ba865e5211\" width=\"800px\" style=\"max-width: 100%;\"\u003e\n\n\u0026#160;\n\n\u003c/div\u003e\n\nLitAI is the easiest way to chat with any model (ChatGPT, Anthropic, etc) in one line of Python. LitAI handles retries, fallback, billing, and logging - so you can build agents, chatbots, or apps without managing flaky APIs or writing wrapper code.\n\n\u0026#160;\n\n\u003cdiv align='center'\u003e\n\u003cpre\u003e\n✅ Use any AI model (OpenAI, etc.) ✅ 20+ public models  ✅ Bring your model API keys\n✅ Unified usage dashboard         ✅ No subscription    ✅ Auto retries and fallback\n✅ Deploy dedicated models on-prem ✅ Start instantly    ✅ No MLOps glue code       \n\u003c/pre\u003e\n\u003c/div\u003e  \n\n\u003cdiv align='center'\u003e\n\n[![PyPI Downloads](https://static.pepy.tech/badge/litai)](https://pepy.tech/projects/litai)\n[![Discord](https://img.shields.io/discord/1077906959069626439?label=Get%20help%20on%20Discord)](https://discord.gg/WajDThKAur)\n![cpu-tests](https://github.com/Lightning-AI/litai/actions/workflows/ci-testing.yml/badge.svg)\n[![codecov](https://codecov.io/gh/Lightning-AI/litai/graph/badge.svg?token=SmzX8mnKlA)](https://codecov.io/gh/Lightning-AI/litai)\n[![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/Lightning-AI/litai/blob/main/LICENSE)\n\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#quick-start\"\u003eQuick start\u003c/a\u003e •\n  \u003ca href=\"#key-features\"\u003eFeatures\u003c/a\u003e •\n  \u003ca href=\"#examples\"\u003eExamples\u003c/a\u003e •\n  \u003ca href=\"#performance\"\u003ePerformance\u003c/a\u003e •\n  \u003ca href=\"#faq\"\u003eFAQ\u003c/a\u003e •\n  \u003ca href=\"https://lightning.ai/docs/litai\"\u003eDocs\u003c/a\u003e\n\u003c/p\u003e\n\n______________________________________________________________________\n\n# Quick Start\n\nInstall LitAI via pip ([more options](https://lightning.ai/docs/litai/home/install)):\n\n```bash\npip install litai\n```\nAdd AI to any Python program in 3 lines:   \n\n```python\nfrom litai import LLM\n\nllm = LLM(model=\"openai/gpt-4\")\nanswer = llm.chat(\"who are you?\")\nprint(answer)\n\n# I'm an AI by OpenAI\n```\n\u003cbr/\u003e\n\n# Examples\nWhat we ***love*** about LitAI is that you can build agents, chatbots and apps in plain Python - no heavy frameworks or magic. Agents can be just simple Python programs with a few decisions made by a model.   \n\n### Agent\nHere's a simple agent that tells you the latest news\n\n```python\nimport re, requests\nfrom litai import LLM\n\nllm = LLM(model=\"openai/gpt-4o\")\n\nwebsite_url = \"https://text.npr.org/\"\nwebsite_text = re.sub(r'\u003c[^\u003e]+\u003e', ' ', requests.get(website_url).text)\n\nresponse = llm.chat(f\"Based on this, what is the latest: {website_text}\")\nprint(response)\n```\n\n### Agentic if statement\nWe believe the best way to build agents is with normal Python programs and simple **“agentic if statements.”** \nThat way, 90% of the logic stays deterministic, and the model only steps in when needed. No complex abstractions, no framework magic - just code you can trust and debug.\n\n```python\nfrom litai import LLM\n\nllm = LLM(model=\"openai/gpt-3.5-turbo\")\n\nproduct_review = \"This TV has terrible picture quality and the sound cuts out constantly.\"\nresponse = llm.chat(f\"Is this review good or bad? Reply only with 'good' or 'bad': {product_review}\").strip().lower()\n\nif response == \"good\":\n    print(\"good review\")\nelse:\n    print(\"bad review\")\n```\n\n\u003cbr/\u003e\n\n# Key features\nTrack usage and spending in your [Lightning AI](https://lightning.ai/) dashboard. Model calls are paid for with Lightning AI credits.\n\n\u003cdiv align='center'\u003e\n\u003cpre\u003e\n✅ No subscription     ✅ 15 free credits (~37M tokens)      ✅ Pay as you go for more credits\n\u003c/pre\u003e\n\u003c/div\u003e  \n\n\n\u003cdiv align='center'\u003e\n\u003cimg alt=\"Lightning\" src=\"https://github.com/user-attachments/assets/b1e7049c-c7b0-42f3-a43c-c1e156929f50\" width=\"800px\" style=\"max-width: 100%;\"\u003e\n\u003c/div\u003e\n\n\u003cbr/\u003e\n\n✅ [Use over 20+ models (ChatGPT, Claude, etc...)](https://lightning.ai/)    \n✅ [Monitor all usage in one place](https://lightning.ai/model-apis)    \n✅ [Async support](https://lightning.ai/docs/litai/features/async-litai/)     \n✅ [Auto retries on failure](https://lightning.ai/docs/litai/features/fallback-retry/)    \n✅ [Auto model switch on failure](https://lightning.ai/docs/litai/features/fallback-retry/)    \n✅ [Switch models](https://lightning.ai/docs/litai/features/models/)    \n✅ [Multi-turn conversation logs](https://lightning.ai/docs/litai/features/multi-turn-conversation/)    \n✅ [Streaming](https://lightning.ai/docs/litai/features/streaming/)    \n✅ Bring your own model (connect your API keys, coming soon...)    \n✅ Chat logs (coming soon...)    \n\n\u003cbr/\u003e\n\n# Advanced features\n\n### Auto fallbacks and retries\n\nModel APIs can flake or can have outages. LitAI automatically retries in case of failures. After multiple failures it can automatically fallback to other models in case the provider is down.\n\n```python\nfrom litai import LLM\n\nllm = LLM(\n    model=\"openai/gpt-4\",\n    fallback_models=[\"google/gemini-2.5-flash\", \"anthropic/claude-3-5-sonnet-20240620\"],\n    max_retries=4,\n)\n\nprint(llm.chat(\"What is a fun fact about space?\"))\n```\n\n\u003cdetails\u003e\n  \u003csummary\u003eStreaming\u003c/summary\u003e\n\n\u003cbr/\u003e\n\nReal-time chat applications benefit from showing words as they generate which gives the illusion of faster speed to the user.  Streaming\nis the mechanism that allows you to do this.\n\n```python\nfrom litai import LLM\n\nllm = LLM(model=\"openai/gpt-4\")\nfor chunk in llm.chat(\"hello\", stream=True):\n    print(chunk, end=\"\", flush=True)\n````\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eUse your own client (like OpenAI)\u003c/summary\u003e\n\n\u003cbr/\u003e\n\nFor those who already have their own SDK to call LLMs (like the OpenAI sdk), you can still use LitAI via the `https://lightning.ai/api/v1` endpoint,\nwhich will track usage, billing, etc...\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n  base_url=\"https://lightning.ai/api/v1\",\n  api_key=\"LIGHTNING_API_KEY\",\n)\n\ncompletion = client.chat.completions.create(\n  model=\"openai/gpt-4o\",\n  messages=[\n    {\n      \"role\": \"user\",\n      \"content\": \"What is a fun fact about space?\"\n    }\n  ]\n)\n\nprint(completion.choices[0].message.content)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eConcurrency with async\u003c/summary\u003e\n\n\u003cbr/\u003e\n\nAdvanced Python programs that process multiple requests at once rely on \"async\" to do this. LitAI can work with async libraries without blocking calls. This is especially useful in high-throughput applications like chatbots, APIs, or agent loops.   \n\nTo enable async behavior, set `enable_async=True` when initializing the `LLM` class. Then use `await llm.chat(...)` inside an `async` function.\n\n```python\nimport asyncio\nfrom litai import LLM\n\nasync def main():\n    llm = LLM(model=\"openai/gpt-4\", teamspace=\"lightning-ai/litai\", enable_async=True)\n    print(await llm.chat(\"who are you?\"))\n\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n  \u003csummary\u003eMulti-turn conversations\u003c/summary\u003e\n\n\u003cbr/\u003e\n\nModels only know the message that was sent to them. To enable them to respond with memory of all the messages sent to it so far, track the related\nmessage under the same conversation.  This is useful for assistants, summarizers, or research tools that need multi-turn chat history.\n\nEach conversation is identified by a unique name. LitAI stores conversation history separately for each name.\n\n```python\nfrom litai import LLM\n\nllm = LLM(model=\"openai/gpt-4\")\n\n# Continue a conversation across multiple turns\nllm.chat(\"What is Lightning AI?\", conversation=\"intro\")\nllm.chat(\"What can it do?\", conversation=\"intro\")\n\nprint(llm.get_history(\"intro\"))  # View all messages from the 'intro' thread\nllm.reset_conversation(\"intro\")  # Clear conversation history\n```\n\nCreate multiple named conversations for different tasks.\n\n```python\nfrom litai import LLM\n\nllm = LLM(model=\"openai/gpt-4\")\n\nllm.chat(\"Summarize this text\", conversation=\"summarizer\")\nllm.chat(\"What's a RAG pipeline?\", conversation=\"research\")\n\nprint(llm.list_conversations())\n```\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n  \u003csummary\u003eSwitch models on each call\u003c/summary\u003e\n\n\u003cbr/\u003e\n\nIn certain applications you may want to call ChatGPT in one message and Anthropic in another so you can use the best model for each task. \nLitAI lets you dynamically switch models at request time.\n\nSet a default model when initializing `LLM` and override it with the `model` parameter only when needed.\n\n```python\nfrom litai import LLM\n\nllm = LLM(model=\"openai/gpt-4\")\n\n# Uses the default model (openai/gpt-4)\nprint(llm.chat(\"Who created you?\"))\n# \u003e\u003e I am a large language model, trained by OpenAI.\n\n# Override the default model for this request\nprint(llm.chat(\"Who created you?\", model=\"google/gemini-2.5-flash\"))\n# \u003e\u003e I am a large language model, trained by Google.\n\n# Uses the default model again\nprint(llm.chat(\"Who created you?\"))\n# \u003e\u003e I am a large language model, trained by OpenAI.\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eMultiple models, same conversation\u003c/summary\u003e\n\n\u003cbr/\u003e\n\nOne application of LitAI is to reduce costs of chats by using separate models for the same conversation. For example, use a cheap model to answer\nthe first question and a more expensive model for something that requires more intelligence.\n\n```python\nfrom litai import LLM\n\nllm = LLM(model=\"openai/gpt-4\")\n\n# use a cheap model for this question\nllm.chat(\"Is this a number or word: '5'\", model=\"google/gemini-2.5-flash\", conversation=\"story\")\n\n# go back to the expensive model\nllm.chat(\"Create a story about that number like Lord of the Rings\", conversation=\"story\")\n\nprint(llm.get_history(\"story\"))  # View all messages from the 'story' thread\n```\n\n\u003c/details\u003e\n\n\u003cbr/\u003e\n\n# Performance\nLitAI does smart routing across a global network of servers. As a result, it only adds 25ms of overhead for an API call.   \n\n\u003cbr/\u003e\n\n# FAQ\n\n\u003cdetails\u003e\n  \u003csummary\u003eDo I need a subscription to use LitAI? (Nope) \u003c/summary\u003e\n   \nNope. You can start instantly without a subscription. LitAI is pay-as-you-go and lets you use your own model API keys (like OpenAI, Anthropic, etc.).\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eDo I need an OpenAI account?  (Nope)\u003c/summary\u003e\n\nNope. You get access to all models and all model providers without a subscription.   \n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eWhat happens if a model API fails or goes down? \u003c/summary\u003e\n\nLitAI automatically retries the same model and can fall back to other models you specify. You’ll get the best chance of getting a response, even during outages.\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eCan I bring my own API keys for OpenAI, Anthropic, etc.? (Yes)\u003c/summary\u003e\n\nYes. You can plug in your own keys to any OpenAI compatible API \n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eCan I connect private models? (Yes)\u003c/summary\u003e\n\nYes. You can connect any endpoint that supports the OpenAI spec.   \n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eCan you deploy a dedicated, private model like Llama for me? (Yes)\u003c/summary\u003e\n\nYes. We can deploy dedicated models on any cloud (Lambda, AWS, etc).\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eCan you deploy models on-prem? (Yes)\u003c/summary\u003e\n\nYes. We can deploy on any dedicated VPC on the cloud or your own physical data center.\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eDo deployed models support Kubernetes? (Yes)\u003c/summary\u003e\n\nYes. We can use the Lightning AI orchestrator custom built for AI or Kubernetes, whatever you want!\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eHow do I pay for the model APIs?\u003c/summary\u003e\n\nBuy Lightning AI credits on Lightning to pay for the APIs.\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eDo you add fees?\u003c/summary\u003e\n\nAt this moment we don't add fees on top of the API calls, but that might change in the future.\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eAre you SOC2, HIPAA compliant? (Yes)\u003c/summary\u003e\n\nLitAI is built by Lightning AI. Our enterprise AI platform powers teams all the way from Fortune 100 to startups. Our platform is fully SOC2, HIPAA compliant.   \n\u003c/details\u003e\n\n\u003cbr/\u003e\n\n# Community\nLitAI is a [community project accepting contributions](https://lightning.ai/docs/litai/community) - Let's make the world's most advanced AI routing engine.\n\n💬 [Get help on Discord](https://discord.com/invite/XncpTy7DSt)    \n📋 [License: Apache 2.0](https://github.com/Lightning-AI/litAI/blob/main/LICENSE)     \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightning-ai%2Flitai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flightning-ai%2Flitai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightning-ai%2Flitai/lists"}