{"id":24020158,"url":"https://github.com/viddexa/autollm","last_synced_at":"2026-01-06T17:18:21.440Z","repository":{"id":204127201,"uuid":"694565231","full_name":"viddexa/autollm","owner":"viddexa","description":"Ship RAG based LLM web apps in seconds.","archived":true,"fork":false,"pushed_at":"2024-01-29T08:07:50.000Z","size":263,"stargazers_count":996,"open_issues_count":3,"forks_count":99,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-09-12T23:35:40.767Z","etag":null,"topics":["anthropic","bedrock","cohere","fastapi","gradio","langchain","large-language-models","llama-index","llama2","llm","openai","palm","pypi","python","retrieval-augmented-generation","vector-database","vertex-ai"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/viddexa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-21T08:53:25.000Z","updated_at":"2025-09-11T19:36:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"1f634735-f535-4b8b-8685-ec2fdc55d6db","html_url":"https://github.com/viddexa/autollm","commit_stats":null,"previous_names":["safevideo/autollm","viddexa/autollm"],"tags_count":33,"template":false,"template_full_name":null,"purl":"pkg:github/viddexa/autollm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viddexa%2Fautollm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viddexa%2Fautollm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viddexa%2Fautollm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viddexa%2Fautollm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/viddexa","download_url":"https://codeload.github.com/viddexa/autollm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viddexa%2Fautollm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274920110,"owners_count":25373954,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-13T02:00:10.085Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anthropic","bedrock","cohere","fastapi","gradio","langchain","large-language-models","llama-index","llama2","llm","openai","palm","pypi","python","retrieval-augmented-generation","vector-database","vertex-ai"],"created_at":"2025-01-08T12:00:47.866Z","updated_at":"2026-01-06T17:18:21.391Z","avatar_url":"https://github.com/viddexa.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话"],"sub_categories":["大语言对话模型及数据"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cp\u003e\n    \u003ca align=\"center\" href=\"\" target=\"_blank\"\u003e\n      \u003cimg\n        width=\"1280\"\n        src=\"https://user-images.githubusercontent.com/44926076/278813325-e545319a-4652-43a7-b02b-45ec877bcfdc.png\"\n      \u003e\n    \u003c/a\u003e\n  \u003c/p\u003e\n\n\u003cbr\u003e\n\n[questions](https://github.com/safevideo/autollm/discussions/categories/q-a) | [feature requests](https://github.com/safevideo/autollm/discussions/categories/feature-requests)\n\n\u003cbr\u003e\n\n[![version](https://badge.fury.io/py/autollm.svg)](https://badge.fury.io/py/autollm)\n\u003ca href=\"https://pepy.tech/project/autollm\"\u003e\u003cimg src=\"https://pepy.tech/badge/autollm\" alt=\"total autollm downloads\"\u003e\u003c/a\u003e\n[![license](https://img.shields.io/pypi/l/autollm)](LICENSE)\n[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/safevideo/autollm/blob/main/examples/quickstart.ipynb)\n\n\u003c/div\u003e\n\n## 🤔 why autollm?\n\n**Simplify. Unify. Amplify.**\n\n| Feature                          | AutoLLM | LangChain | LlamaIndex | LiteLLM |\n| -------------------------------- | :-----: | :-------: | :--------: | :-----: |\n| **100+ LLMs**                    |    ✅    |     ✅     |     ✅      |    ✅    |\n| **Unified API**                  |    ✅    |     ❌     |     ❌      |    ✅    |\n| **20+ Vector Databases**         |    ✅    |     ✅     |     ✅      |    ❌    |\n| **Cost Calculation (100+ LLMs)** |    ✅    |     ❌     |     ❌      |    ✅    |\n| **1-Line RAG LLM Engine**        |    ✅    |     ❌     |     ❌      |    ❌    |\n| **1-Line FastAPI**               |    ✅    |     ❌     |     ❌      |    ❌    |\n\n______________________________________________________________________\n\n## 📦 installation\n\neasily install **autollm** package with pip in [**Python\u003e=3.8**](https://www.python.org/downloads/) environment.\n\n```bash\npip install autollm\n```\n\nfor built-in data readers (github, pdf, docx, ipynb, epub, mbox, websites..), install with:\n\n```bash\npip install autollm[readers]\n```\n\n______________________________________________________________________\n\n## 🎯 quickstart\n\n### tutorials\n\n- **video tutorials**:\n\n  - [Make RAG with API Just in a few Seconds!!!](https://www.youtube.com/watch?v=sgKpBMGC6M0\u0026list=LL\u0026index=2)\n  - [Autollm automagically create llm apps in seconds](https://www.youtube.com/watch?v=HZik9309wQc\u0026list=LL\u0026index=3)\n  - [AutoLLM: Create RAG Based LLM Web Apps in SECONDS!](https://www.youtube.com/watch?v=kPaiZe_qD34)\n  - [AutoLLM: Ship RAG based LLM Apps and API in Seconds](https://www.youtube.com/watch?v=iTGbwD-sSxM)\n  - [🚀 AutoLLM: Unlock the Power of 100+ Language Models! Step-by-Step Tutorial](https://www.youtube.com/watch?v=Dh9pIWO1VJw)\n\n- **blog posts**:\n\n  - [Introduction to AutoLLM](https://abrahimzaman360.medium.com/introduction-to-autollm-c8cd31be2a5f)\n\n- **colab notebooks**:\n\n  - quickstart: [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/safevideo/autollm/blob/main/examples/quickstart.ipynb)\n\n### create a query engine in seconds\n\n```python\n\u003e\u003e\u003e from autollm import AutoQueryEngine, read_files_as_documents\n\n\u003e\u003e\u003e documents = read_files_as_documents(input_dir=\"path/to/documents\")\n\u003e\u003e\u003e query_engine = AutoQueryEngine.from_defaults(documents)\n\n\u003e\u003e\u003e response = query_engine.query(\n...     \"Why did SafeVideo AI develop this project?\"\n... )\n\n\u003e\u003e\u003e response.response\n\"Because they wanted to deploy rag based llm apis in no time!\"\n```\n\n\u003cdetails\u003e\n    \u003csummary\u003e👉 advanced usage \u003c/summary\u003e\n\n```python\n\u003e\u003e\u003e from autollm import AutoQueryEngine\n\n\u003e\u003e\u003e query_engine = AutoQueryEngine.from_defaults(\n...     documents=documents,\n...     llm_model=\"gpt-3.5-turbo\",\n...     llm_max_tokens=\"256\",\n...     llm_temperature=\"0.1\",\n...     system_prompt='...',\n...     query_wrapper_prompt='...',\n...     enable_cost_calculator=True,\n...     embed_model=\"huggingface/BAAI/bge-large-zh\",\n...     chunk_size=512,\n...     chunk_overlap=64,\n...     context_window=4096,\n...     similarity_top_k=3,\n...     response_mode=\"compact\",\n...     structured_answer_filtering=False,\n...     vector_store_type=\"LanceDBVectorStore\",\n...     lancedb_uri=\"./lancedb\",\n...     lancedb_table_name=\"vectors\",\n...     exist_ok=True,\n...     overwrite_existing=False,\n... )\n\n\u003e\u003e\u003e response = query_engine.query(\"Who is SafeVideo AI?\")\n\n\u003e\u003e\u003e print(response.response)\n\"A startup that provides self hosted AI API's for companies!\"\n```\n\n\u003c/details\u003e\n\n### convert it to a FastAPI app in 1-line\n\n```python\n\u003e\u003e\u003e import uvicorn\n\n\u003e\u003e\u003e from autollm import AutoFastAPI\n\n\u003e\u003e\u003e app = AutoFastAPI.from_query_engine(query_engine)\n\n\u003e\u003e\u003e uvicorn.run(app, host=\"0.0.0.0\", port=8000)\nINFO:    Started server process [12345]\nINFO:    Waiting for application startup.\nINFO:    Application startup complete.\nINFO:    Uvicorn running on http://http://0.0.0.0:8000/\n```\n\n\u003cdetails\u003e\n    \u003csummary\u003e👉 advanced usage \u003c/summary\u003e\n\n```python\n\u003e\u003e\u003e from autollm import AutoFastAPI\n\n\u003e\u003e\u003e app = AutoFastAPI.from_query_engine(\n...      query_engine,\n...      api_title='...',\n...      api_description='...',\n...      api_version='...',\n...      api_term_of_service='...',\n    )\n\n\u003e\u003e\u003e uvicorn.run(app, host=\"0.0.0.0\", port=8000)\nINFO:    Started server process [12345]\nINFO:    Waiting for application startup.\nINFO:    Application startup complete.\nINFO:    Uvicorn running on http://http://0.0.0.0:8000/\n```\n\n\u003c/details\u003e\n\n______________________________________________________________________\n\n## 🌟 features\n\n### supports [100+ LLMs](https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json)\n\n```python\n\u003e\u003e\u003e from autollm import AutoQueryEngine\n\n\u003e\u003e\u003e os.environ[\"HUGGINGFACE_API_KEY\"] = \"huggingface_api_key\"\n\n\u003e\u003e\u003e llm_model = \"huggingface/WizardLM/WizardCoder-Python-34B-V1.0\"\n\u003e\u003e\u003e llm_api_base = \"https://my-endpoint.huggingface.cloud\"\n\n\u003e\u003e\u003e AutoQueryEngine.from_defaults(\n...     documents='...',\n...     llm_model=llm_model,\n...     llm_api_base=llm_api_base,\n... )\n```\n\n\u003cdetails\u003e\n    \u003csummary\u003e👉 more llms:\u003c/summary\u003e\n\n- huggingface - ollama example:\n\n  ```python\n  \u003e\u003e\u003e from autollm import AutoQueryEngine\n\n  \u003e\u003e\u003e llm_model = \"ollama/llama2\"\n  \u003e\u003e\u003e llm_api_base = \"http://localhost:11434\"\n\n  \u003e\u003e\u003e AutoQueryEngine.from_defaults(\n  ...     documents='...',\n  ...     llm_model=llm_model,\n  ...     llm_api_base=llm_api_base,\n  ... )\n  ```\n\n- microsoft azure - openai example:\n\n  ```python\n  \u003e\u003e\u003e from autollm import AutoQueryEngine\n\n  \u003e\u003e\u003e os.environ[\"AZURE_API_KEY\"] = \"\"\n  \u003e\u003e\u003e os.environ[\"AZURE_API_BASE\"] = \"\"\n  \u003e\u003e\u003e os.environ[\"AZURE_API_VERSION\"] = \"\"\n\n  \u003e\u003e\u003e llm_model = \"azure/\u003cyour_deployment_name\u003e\")\n\n  \u003e\u003e\u003e AutoQueryEngine.from_defaults(\n  ...     documents='...',\n  ...     llm_model=llm_model\n  ... )\n  ```\n\n- google - vertexai example:\n\n  ```python\n  \u003e\u003e\u003e from autollm import AutoQueryEngine\n\n  \u003e\u003e\u003e os.environ[\"VERTEXAI_PROJECT\"] = \"hardy-device-38811\"  # Your Project ID`\n  \u003e\u003e\u003e os.environ[\"VERTEXAI_LOCATION\"] = \"us-central1\"  # Your Location\n\n  \u003e\u003e\u003e llm_model = \"text-bison@001\"\n\n  \u003e\u003e\u003e AutoQueryEngine.from_defaults(\n  ...     documents='...',\n  ...     llm_model=llm_model\n  ... )\n  ```\n\n- aws bedrock - claude v2 example:\n\n  ```python\n  \u003e\u003e\u003e from autollm import AutoQueryEngine\n\n  \u003e\u003e\u003e os.environ[\"AWS_ACCESS_KEY_ID\"] = \"\"\n  \u003e\u003e\u003e os.environ[\"AWS_SECRET_ACCESS_KEY\"] = \"\"\n  \u003e\u003e\u003e os.environ[\"AWS_REGION_NAME\"] = \"\"\n\n  \u003e\u003e\u003e llm_model = \"anthropic.claude-v2\"\n\n  \u003e\u003e\u003e AutoQueryEngine.from_defaults(\n  ...     documents='...',\n  ...     llm_model=llm_model\n  ... )\n  ```\n\n\u003c/details\u003e\n\n### supports [20+ VectorDBs](https://docs.llamaindex.ai/en/stable/module_guides/storing/vector_stores.html#vector-store-options-feature-support)\n\n🌟**Pro Tip**: `autollm` defaults to `lancedb` as the vector store:\nit's setup-free, serverless, and 100x more cost-effective!\n\n\u003cdetails\u003e\n    \u003csummary\u003e👉 more vectordbs:\u003c/summary\u003e\n\n- QdrantVectorStore example:\n  ```python\n  \u003e\u003e\u003e from autollm import AutoQueryEngine\n  \u003e\u003e\u003e import qdrant_client\n\n  \u003e\u003e\u003e vector_store_type = \"QdrantVectorStore\"\n  \u003e\u003e\u003e client = qdrant_client.QdrantClient(\n  ...     url=\"http://\u003chost\u003e:\u003cport\u003e\",\n  ...     api_key=\"\u003cqdrant-api-key\u003e\"\n  ... )\n  \u003e\u003e\u003e collection_name = \"quickstart\"\n\n  \u003e\u003e\u003e AutoQueryEngine.from_defaults(\n  ...     documents='...',\n  ...     vector_store_type=vector_store_type,\n  ...     client=client,\n  ...     collection_name=collection_name,\n  ... )\n  ```\n\n\u003c/details\u003e\n\n### automated cost calculation for [100+ LLMs](https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json)\n\n```python\n\u003e\u003e\u003e from autollm import AutoServiceContext\n\n\u003e\u003e\u003e service_context = AutoServiceContext(enable_cost_calculation=True)\n\n# Example verbose output after query\nEmbedding Token Usage: 7\nLLM Prompt Token Usage: 1482\nLLM Completion Token Usage: 47\nLLM Total Token Cost: $0.002317\n```\n\n### create FastAPI App in 1-Line\n\n\u003cdetails\u003e\n    \u003csummary\u003e👉 example\u003c/summary\u003e\n\n```python\n\u003e\u003e\u003e from autollm import AutoFastAPI\n\n\u003e\u003e\u003e app = AutoFastAPI.from_config(config_path, env_path)\n```\n\nHere, `config` and `env` should be replaced by your configuration and environment file paths.\n\nAfter creating your FastAPI app, run the following command in your terminal to get it up and running:\n\n```bash\nuvicorn main:app\n```\n\n\u003c/details\u003e\n\n______________________________________________________________________\n\n## 🔄 migration from llama-index\n\nswitching from Llama-Index? We've got you covered.\n\n\u003cdetails\u003e\n    \u003csummary\u003e👉 easy migration \u003c/summary\u003e\n\n```python\n\u003e\u003e\u003e from llama_index import StorageContext, ServiceContext, VectorStoreIndex\n\u003e\u003e\u003e from llama_index.vectorstores import LanceDBVectorStore\n\n\u003e\u003e\u003e from autollm import AutoQueryEngine\n\n\u003e\u003e\u003e vector_store = LanceDBVectorStore(uri=\"./.lancedb\")\n\u003e\u003e\u003e storage_context = StorageContext.from_defaults(vector_store=vector_store)\n\u003e\u003e\u003e service_context = ServiceContext.from_defaults()\n\u003e\u003e\u003e index = VectorStoreIndex.from_documents(\n        documents=documents,\n        storage_context=storage_contex,\n        service_context=service_context,\n    )\n\n\u003e\u003e\u003e query_engine = AutoQueryEngine.from_instances(index)\n```\n\n\u003c/details\u003e\n\n## ❓ FAQ\n\n**Q: Can I use this for commercial projects?**\n\nA: Yes, AutoLLM is licensed under GNU Affero General Public License (AGPL 3.0),\nwhich allows for commercial use under certain conditions. [Contact](#contact) us for more information.\n\n______________________________________________________________________\n\n## roadmap\n\nour roadmap outlines upcoming features and integrations to make autollm the most extensible and powerful base package for large language model applications.\n\n- [ ] **1-line [Gradio](https://www.gradio.app/) app creation and deployment**\n\n- [ ] **Budget based email notification**\n\n- [ ] **Automated LLM evaluation**\n\n- [ ] **Add more quickstart apps on pdf-chat, documentation-chat, academic-paper-analysis, patent-analysis and more!**\n\n______________________________________________________________________\n\n## 📜 license\n\nautollm is available under the [GNU Affero General Public License (AGPL 3.0)](LICENSE).\n\n______________________________________________________________________\n\n## 📞 contact\n\nfor more information, support, or questions, please contact:\n\n- **Email**: [support@safevideo.ai](mailto:support@safevideo.ai)\n- **Website**: [SafeVideo](https://safevideo.ai/)\n- **LinkedIn**: [SafeVideo AI](https://www.linkedin.com/company/safevideo/)\n\n______________________________________________________________________\n\n## 🏆 contributing\n\n**love autollm? star the repo or contribute and help us make it even better!** see our [contributing guidelines](CONTRIBUTING.md) for more information.\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/safevideo/autollm/graphs/contributors\"\u003e\n      \u003cimg src=\"https://contrib.rocks/image?repo=safevideo/autollm\" /\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n______________________________________________________________________\n\n\u003cdiv align=\"center\"\u003e\n  \u003cb\u003efollow us for more!\u003c/b\u003e\n  \u003cbr\u003e\n  \u003ca href=\"https://www.linkedin.com/company/safevideo/\"\u003e\n      \u003cimg\n        src=\"https://user-images.githubusercontent.com/44926076/278822352-30e06f9b-1915-4aed-8081-6796432daa7a.png\"\n        height=\"32px\"\n      /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://huggingface.co/safevideo\"\u003e\n      \u003cimg\n        src=\"https://user-images.githubusercontent.com/34196005/278877706-ed074c9c-0938-48a1-98e8-39a322faf01d.png\"\n        height=\"32px\"\n      /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://twitter.com/safevideo_ai\"\u003e\n      \u003cimg\n        src=\"https://user-images.githubusercontent.com/34196005/278877049-141925a9-aa1b-4730-829e-74f6d08ee8ca.png\"\n        height=\"32px\"\n      /\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fviddexa%2Fautollm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fviddexa%2Fautollm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fviddexa%2Fautollm/lists"}