https://github.com/IST-DASLab/gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

Last synced: 08 Apr 2025

https://github.com/IST-DASLab/marlin

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

4bit kernel llm quantization

Last synced: 30 Aug 2025

https://github.com/IST-DASLab/PanzaMail

Last synced: 05 Apr 2025

https://github.com/IST-DASLab/qmoe

Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".

Last synced: 12 Apr 2025