https://github.com/autogptq/autogptq

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
https://github.com/autogptq/autogptq

deep-learning inference large-language-models llms nlp pytorch quantization transformer transformers

Last synced: about 2 hours ago
JSON representation

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

• Host: GitHub
• URL: https://github.com/autogptq/autogptq
• Owner: AutoGPTQ
• License: mit
• Created: 2023-04-13T02:18:11.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-08-19T02:41:00.000Z (about 1 month ago)
• Last Synced: 2024-09-04T09:09:00.216Z (21 days ago)
• Topics: deep-learning, inference, large-language-models, llms, nlp, pytorch, quantization, transformer, transformers
• Language: Python
• Homepage:
• Size: 7.98 MB
• Stars: 4,292
• Watchers: 32
• Forks: 459
• Open Issues: 250
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project