https://github.com/bentoml/OpenLLM

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint, locally and in the cloud.
https://github.com/bentoml/OpenLLM

ai bentoml falcon fine-tuning llama llama2 llm llm-inference llm-ops llm-serving llmops mistral ml mlops model-inference mpt open-source-llm openllm stablelm vicuna

Last synced: about 1 month ago
JSON representation

Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint, locally and in the cloud.

• Host: GitHub
• URL: https://github.com/bentoml/OpenLLM
• Owner: bentoml
• License: apache-2.0
• Created: 2023-04-19T00:27:52.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-04-13T01:36:10.000Z (5 months ago)
• Last Synced: 2024-04-14T06:06:05.896Z (5 months ago)
• Topics: ai, bentoml, falcon, fine-tuning, llama, llama2, llm, llm-inference, llm-ops, llm-serving, llmops, mistral, ml, mlops, model-inference, mpt, open-source-llm, openllm, stablelm, vicuna
• Language: Python
• Homepage: https://bentoml.com
• Size: 38.3 MB
• Stars: 8,657
• Watchers: 51
• Forks: 541
• Open Issues: 83
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE.md
  • Code of conduct: .github/CODE_OF_CONDUCT.md
  • Citation: CITATION.cff
  • Codeowners: .github/CODEOWNERS
  • Security: .github/SECURITY.md

Awesome Lists containing this project

• awesome-local-llms - OpenLLM - source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud. | 9,108 | 581 | 56 | 26 | 118 | Apache License 2.0 | 0 days, 20 hrs, 10 mins | (Open-Source Local LLM Projects)
• Awesome-LLM-Productization - OpenLLM from BentoML - an open-source platform designed to facilitate the deployment and operation of large language models (LLMs) in real-world applications. (Models and Tools / LLM Deployment)
• Self-Hosting-Guide - OpenLLM - tune, serve, deploy, and monitor any LLMs with ease. (Tools for Self-Hosting / LLMs)
• awesome-langchain-zh - OpenLLM
• awesome - bentoml/OpenLLM - Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud. (Python)
• awesome-llm-list - OpenLLM
• awesome-llmops - OpenLLM - tune, serve, deploy, and monitor any LLMs with ease. |  | (Large Scale Deployment / ML Platforms)
• AiTreasureBox - bentoml/OpenLLM - 09-08_9697_1](https://img.shields.io/github/stars/bentoml/OpenLLM.svg)|An open platform for operating large language models (LLMs) in production. Fine-tune, serve, deploy, and monitor any LLMs with ease.| (Repos)
• awesome-langchain - OpenLLM - tune, serve, deploy, and monitor any LLMs with ease using OpenLLM.  (Other LLM Frameworks / Videos Playlists)
• StarryDivineSky - bentoml/OpenLLM
• Awesome-LLM - OpenLLM - Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at [BentoML](https://bentoml.com/) for LLMs-based applications. (LLM Deployment)
• awesome-production-machine-learning - OpenLLM - OpenLLM allows developers to run any open-source LLMs (Llama 3.1, Qwen2, Phi3 and more) or custom models as OpenAI-compatible APIs with a single command. (Model Serving and Monitoring)
• awesome-homelab - OpenLLM - source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud. | (Apps / AI)
• Awesome-LLM-RAG-Application - OpenLLM