https://github.com/MooreThreads/vllm_musa

A high-throughput and memory-efficient inference and serving engine for LLMs
https://github.com/MooreThreads/vllm_musa

Last synced: about 2 months ago
JSON representation

A high-throughput and memory-efficient inference and serving engine for LLMs

• Host: GitHub
• URL: https://github.com/MooreThreads/vllm_musa
• Owner: MooreThreads
• License: other
• Fork: true (vllm-project/vllm)
• Created: 2024-09-29T13:53:39.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-10-28T03:33:47.000Z (4 months ago)
• Last Synced: 2024-10-28T06:56:36.019Z (4 months ago)
• Language: Python
• Homepage: https://docs.vllm.ai
• Size: 8.01 MB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0

Awesome Lists containing this project

• awesome-llm-and-aigc - MooreThreads/vllm_musa - throughput and memory-efficient inference and serving engine for LLMs. [docs.vllm.ai](https://docs.vllm.ai/) (Summary)