Projects in Awesome Lists tagged with int4

A curated list of projects in awesome lists tagged with int4 .

• Sort

  • Recently synced
  • Stars

https://github.com/intel/neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

auto-tuning awq fp4 gptq int4 int8 knowledge-distillation large-language-models low-precision mxformat post-training-quantization pruning quantization quantization-aware-training smoothquant sparsegpt sparsity

Last synced: 12 May 2025

https://github.com/tpoisonooo/how-to-optimize-gemm

row-major matmul optimization

arm64 armv7 cuda cuda-kernel gemm-optimization int4 ptx vulkan

Last synced: 04 Apr 2025

https://github.com/intel/auto-round

Advanced Quantization Algorithm for LLMs/VLMs.

awq gptq int4 neural-compressor quantization rounding

Last synced: 19 Apr 2025

https://github.com/intel/neural-speed

An innovative library for efficient LLM inference via low-bit quantization

cpu fp4 fp8 gaudi2 gpu int1 int2 int3 int4 int5 int6 int7 int8 llamacpp llm-fine-tuning llm-inference low-bit mxformat nf4 sparsity

Last synced: 10 Feb 2025