https://github.com/InternLM/lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
https://github.com/InternLM/lmdeploy

codellama cuda-kernels deepspeed fastertransformer internlm llama llama2 llama3 llm llm-inference turbomind

Last synced: about 2 months ago
JSON representation

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

• Host: GitHub
• URL: https://github.com/InternLM/lmdeploy
• Owner: InternLM
• License: apache-2.0
• Created: 2023-06-15T12:38:06.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-10-18T09:16:49.000Z (about 2 months ago)
• Last Synced: 2024-10-19T02:44:44.785Z (about 2 months ago)
• Topics: codellama, cuda-kernels, deepspeed, fastertransformer, internlm, llama, llama2, llama3, llm, llm-inference, turbomind
• Language: Python
• Homepage: https://lmdeploy.readthedocs.io/en/latest/
• Size: 5.37 MB
• Stars: 4,455
• Watchers: 37
• Forks: 403
• Open Issues: 320
• Metadata Files:
  • Readme: README.md
  • Contributing: .github/CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-local-llms - lmdeploy
• awesome-local-ai - LmDeploy - LMDeploy is a toolkit for compressing, deploying, and serving LLMs. | Pytorch / Turbomind | Both | ❌ | Python/C++ | Text-Gen | (Inference Engine)
• StarryDivineSky - InternLM/lmdeploy
• Awesome-LLM - LMDeploy - A high-throughput and low-latency inference and serving framework for LLMs and VLs (LLM Deployment)
• awesome-LLM-resourses - LMDeploy
• awesome-ai-papers - [LMDeploy - ai/Mooncake)\]\[[inference](https://github.com/xorbitsai/inference)\]\[[LitServe](https://github.com/Lightning-AI/LitServe)\] (NLP / 3. Pretraining)
• awesome-ai-papers - [LMDeploy - ai/Mooncake)\]\[[inference](https://github.com/xorbitsai/inference)\]\[[LitServe](https://github.com/Lightning-AI/LitServe)\] (NLP / 3. Pretraining)
• alan_awesome_llm - LMDeploy
• alan_awesome_llm - LMDeploy