https://github.com/mit-han-lab/TinyChatEngine

TinyChatEngine: On-Device LLM Inference Library
https://github.com/mit-han-lab/TinyChatEngine

arm c cpp cuda-programming deep-learning edge-computing large-language-models on-device-ai quantization x86-64

Last synced: about 1 month ago
JSON representation

TinyChatEngine: On-Device LLM Inference Library

• Host: GitHub
• URL: https://github.com/mit-han-lab/TinyChatEngine
• Owner: mit-han-lab
• License: mit
• Created: 2023-05-24T09:29:41.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-07-04T04:20:45.000Z (2 months ago)
• Last Synced: 2024-07-29T00:37:07.414Z (about 2 months ago)
• Topics: arm, c, cpp, cuda-programming, deep-learning, edge-computing, large-language-models, on-device-ai, quantization, x86-64
• Language: C++
• Homepage: https://mit-han-lab.github.io/TinyChatEngine/
• Size: 83.3 MB
• Stars: 649
• Watchers: 12
• Forks: 62
• Open Issues: 31
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• ai-game-devtools - TinyChatEngine - Device LLM Inference Library. | | | Tool | (Project List / <span id="tool">Tool (AI LLM)</span>)
• Awesome-LLM-Compression - [Code