https://github.com/ModelTC/lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
https://github.com/ModelTC/lightllm

deep-learning gpt llama llm model-serving nlp openai-triton

Last synced: about 1 month ago
JSON representation

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

• Host: GitHub
• URL: https://github.com/ModelTC/lightllm
• Owner: ModelTC
• License: apache-2.0
• Created: 2023-07-22T08:11:15.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-06-20T06:19:34.000Z (3 months ago)
• Last Synced: 2024-06-20T09:27:57.769Z (3 months ago)
• Topics: deep-learning, gpt, llama, llm, model-serving, nlp, openai-triton
• Language: Python
• Homepage:
• Size: 1.53 MB
• Stars: 1,967
• Watchers: 20
• Forks: 172
• Open Issues: 55
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-azure-openai-llm - TokenAttention
• awesome-production-machine-learning - LightLLM - LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance. (Model Serving and Monitoring)
• StarryDivineSky - ModelTC/lightllm
• Awesome-LLM-Inference - **LightLLM**