Projects in Awesome Lists tagged with sparsity

https://intel.github.io/neural-compressor/

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

auto-tuning awq fp4 gptq int4 int8 knowledge-distillation large-language-models low-precision mxformat post-training-quantization pruning quantization quantization-aware-training smoothquant sparsegpt sparsity

Last synced: 09 Dec 2025

https://github.com/intel/neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

auto-tuning awq fp4 gptq int4 int8 knowledge-distillation large-language-models low-precision mxformat post-training-quantization pruning quantization quantization-aware-training smoothquant sparsegpt sparsity

Last synced: 12 May 2025

https://github.com/neuralmagic/sparseml

Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

automl computer-vision-algorithms deep-learning-algorithms deep-learning-library deep-learning-models image-classification keras nlp object-detection onnx pruning pruning-algorithms pytorch smaller-models sparsification sparsification-recipes sparsity tensorflow transfer-learning

Last synced: 14 May 2025

https://github.com/pytorch/ao

PyTorch native quantization and sparsity for training and inference

brrr cuda dtypes float8 inference llama mx offloading optimizer pytorch quantization sparsity training transformer

Last synced: 12 May 2025

https://github.com/paddlepaddle/paddleslim

PaddleSlim is an open-source library for deep model compression and architecture search.

bert compression detection distillation ernie nas pruning quantization segmentation sparsity tensorrt transformer yolov5 yolov6 yolov7

Last synced: 14 May 2025

https://github.com/PaddlePaddle/PaddleSlim

PaddleSlim is an open-source library for deep model compression and architecture search.

bert compression detection distillation ernie nas pruning quantization segmentation sparsity tensorrt transformer yolov5 yolov6 yolov7

Last synced: 20 Mar 2025

https://github.com/tensorflow/model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

compression deep-learning keras machine-learning ml model-compression optimization pruning quantization quantized-networks quantized-neural-networks quantized-training sparsity tensorflow

Last synced: 12 May 2025

https://github.com/vllm-project/llm-compressor

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

compression quantization sparsity

Last synced: 14 May 2025

https://github.com/openvinotoolkit/nncf

Neural Network Compression Framework for enhanced OpenVINO™ inference

bert classification compression deep-learning genai llm mixed-precision-training nlp object-detection onnx openvino pruning pytorch quantization quantization-aware-training semantic-segmentation sparsity tensorflow transformers

Last synced: 08 Apr 2026

https://github.com/eric-mingjie/network-slimming

Network Slimming (Pytorch) (ICCV 2017)

channel-pruning convolutional-neural-networks deep-learning pytorch sparsity

Last synced: 13 Apr 2025

https://github.com/Bobo-y/flexible-yolov5

More readable and flexible yolov5 with more backbone(gcn, resnet, shufflenet, moblienet, efficientnet, hrnet, swin-transformer, etc) and (cbam，dcn and so on), and tensorrt

backbone cbam dcnv2 gcn hrnet moblienet neck object-detection ptq pytorch qat resnet shufflenet sparsity swin-transformer tensorrt triton-server yolov3 yolov5

Last synced: 20 Apr 2025

https://github.com/fminference/h2o

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

gpt-3 heavy-hitters high-throughput kv-cache large-language-models sparsity

Last synced: 05 Apr 2025

https://github.com/FMInference/H2O

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

gpt-3 heavy-hitters high-throughput kv-cache large-language-models sparsity

Last synced: 09 May 2025

https://github.com/wenwei202/caffe

Caffe for Sparse and Low-rank Deep Neural Networks

acceleration caffe compression deep-neural-networks low-rank-approximation sparse-convolution sparsity

Last synced: 15 Mar 2025

https://github.com/intel/neural-speed

An innovative library for efficient LLM inference via low-bit quantization

cpu fp4 fp8 gaudi2 gpu int1 int2 int3 int4 int5 int6 int7 int8 llamacpp llm-fine-tuning llm-inference low-bit mxformat nf4 sparsity

Last synced: 25 Oct 2025

https://github.com/bwohlberg/sporco

Sparse Optimisation Research Code

admm convolutional-dictionary-learning convolutional-sparse-coding cuda dictionary-learning fista optimization optimization-algorithms plug-and-play-priors python robust-pca sparse-coding sparse-representations sparsity total-variation total-variation-minimization

Last synced: 15 May 2025

https://github.com/jack-willturner/deep-compression

Learning both Weights and Connections for Efficient Neural Networks https://arxiv.org/abs/1506.02626

deep-learning pruning pytorch sparsity

Last synced: 03 Apr 2025

https://github.com/lucaslie/torchprune

A research library for pytorch-based neural network pruning, compression, and more.

compression coresets deep-learning filter-pruning generalization-ability machine-learning neural-architecture-search neural-networks pruning pruning-algorithms pytorch sparsification sparsity tensor-decomposition tinyml weight-pruning

Last synced: 08 Jan 2026

https://github.com/fasterdecoding/teal

llm llm-inference sparsity

Last synced: 06 Apr 2025

https://github.com/nvidia-ai-iot/clip-distillation

Zero-label image classification via OpenCLIP knowledge distillation

clip distillation inference jetson knowledge nvidia qat sparsity tensorrt

Last synced: 13 Oct 2025

https://github.com/opensparsellms/llama-moe-v2

🚀 LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training

attention fine-tuning instruction-tuning llama llama3 mixture-of-experts moe sft sparsity

Last synced: 11 Aug 2025

https://github.com/satabios/sconce

E2E AutoML Model Compression Package

deployment pruning pytorch quantization sparsity torch

Last synced: 18 Feb 2026

https://github.com/adrhill/sparseconnectivitytracer.jl

Fast operator-overloading Jacobian & Hessian sparsity detection.

autodiff hessian jacobian julia sparsity

Last synced: 21 Apr 2025

https://github.com/vita-group/sparsity-win-robust-generalization

[ICLR 2022] "Sparsity Winning Twice: Better Robust Generalization from More Efficient Training" by Tianlong Chen*, Zhenyu Zhang*, Pengjun Wang*, Santosh Balachandra*, Haoyu Ma*, Zehao Wang, Zhangyang Wang

dynamic-sparse-training generalization lottery-ticket-hypothesis pruning robust-generalization robust-overfitting sparsity

Last synced: 29 Oct 2025

https://github.com/sebastianament/compressedsensing.jl

Contains a wide-ranging collection of compressed sensing and feature selection algorithms. Examples include matching pursuit algorithms, forward and backward stepwise regression, sparse Bayesian learning, and basis pursuit.

basis-pursuit compressed-sensing feature-selection julia matching-pursuit sparse-bayesian-learning sparse-linear-systems sparse-regression sparsity stepwise-regression subset-selection

Last synced: 30 Jul 2025

https://github.com/astorfi/attention-guided-sparsity

Attention-Based Guided Structured Sparsity of Deep Neural Networks

attention-mechanism convolutional-neural-networks deep-learning sparsity

Last synced: 30 Apr 2025

https://github.com/vita-group/smc-bench

[ICLR 2023] "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!" Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen, Tianjin Huang, AJAY KUMAR JAISWAL, Zhangyang Wang

benchmark deep-learning dynamic-sparse-training pruning sparse-neural-networks sparsity

Last synced: 19 Apr 2025

https://github.com/vita-group/tost

[ICML2022] Training Your Sparse Neural Network Better with Any Mask. Ajay Jaiswal, Haoyu Ma, Tianlong Chen, ying Ding, and Zhangyang Wang

lottery-tickets sparse-training sparsity

Last synced: 19 Apr 2025

https://github.com/vita-group/backdoor-lth

[CVPR 2022] "Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free" by Tianlong Chen*, Zhenyu Zhang*, Yihua Zhang*, Shiyu Chang, Sijia Liu, and Zhangyang Wang

backdoor-attacks linear-mode-connectivity lottery-ticket-hypothesis reverse-engineering sparsity trojan

Last synced: 19 Apr 2025

https://github.com/wenbihan/strollr2d_icassp2017

Image Denoising Codes using STROLLR learning, the Matlab implementation of the paper in ICASSP2017

image-denoising joint-models lowrankdenoising self-similarity sparsity state-of-the-art transform-learning unsupervised-learning

Last synced: 25 Oct 2025

https://github.com/openmendel/mendeliht.jl

Iterative hard thresholding for l0 penalized regression

genetic-algorithms glm gwas julia model-selection multivariate multivariate-regression regression sparse-linear-solver sparsity

Last synced: 12 Apr 2025

https://github.com/ryantd/veloce

WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.

data-parallelism deep-learning distributed distributed-computing heterogeneity model-parallelism parameter-server pytorch ray sparsity

Last synced: 10 Apr 2025

https://github.com/huangcongqing/model-compression-optimization

model compression and optimization for deployment for Pytorch, including knowledge distillation, quantization and pruning.(知识蒸馏，量化，剪枝)

knowledge-distillation model-compression nas pruning pytorch quantization quantized-networks sparsity sparsity-optimization

Last synced: 05 May 2025

https://github.com/vita-group/linearity-grafting

[ICML 2022] "Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness" by Tianlong Chen*, Huan Zhang*, Zhenyu Zhang, Shiyu Chang, Sijia Liu, Pin-Yu Chen, Zhangyang Wang

certifiable-robustness certification linearity linearity-grafting neuron-pruning sparsity

Last synced: 11 Sep 2025

https://github.com/soran-ghaderi/efficientcof

A new efficient subspace and K-Means clustering based method to improve Collaborative Filtering

algorithm clustering collaborative-filtering high-dimensionality machine-learning movielens neighbor-users nusccf pearson python recommender-system similarity sparsity subspace subspace-clustering tree-structure

Last synced: 07 Sep 2025

https://github.com/vita-group/double-win-lth

[ICML 2022] "Data-Efficient Double-Win Lottery Tickets from Robust Pre-training" by Tianlong Chen, Zhenyu Zhang, Sijia Liu, Yang Zhang, Shiyu Chang, Zhangyang Wang

adversarial-robustness data-efficient generalization lottery-ticket-hypothesis pretraining robust-pretraining sparsity transfer-learning

Last synced: 19 Apr 2025

https://github.com/zib-iol/sms

Code to reproduce the experiments of the ICLR24-paper: "Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging"

averaging deep-learning neural-network optimization pruning pytorch sparsity

Last synced: 15 Apr 2025

https://github.com/zib-iol/bimp

Code to reproduce the experiments of ICLR2023-paper: How I Learned to Stop Worrying and Love Retraining

deep-learning learning-rate-scheduling neural-network optimization pruning pytorch sparsity

Last synced: 07 May 2025

https://github.com/vita-group/dataefficientlth

[NeurIPS 2022] "Sparse Winning Tickets are Data-Efficient Image Recognizers" by Mukund Varma T, Xuxi Chen, Zhenyu Zhang, Tianlong Chen, Subhashini Venugopalan, Zhangyang Wang

data-efficient-learning sparsity

Last synced: 29 Jul 2025

https://github.com/d-becking/neurips-2019-micronet-challenge

NeurIPS 2019 MicroNet Challenge

compound-scaling ec2t efficient-inference efficient-neural-networks entropy-coding micronet-challenge neural-architecture-search neural-network pytorch quantization quantization-algorithms sparsity sparsity-optimization

Last synced: 17 Mar 2026

https://github.com/cypriengille/supervised-autoencoder

A supervised autoencoder with structured sparsity for efficient and informed clinical prognosis.

autoencoder feature-selection interpretability metabolomics metabolomics-database prognostic-score sparsity supervised-learning

Last synced: 12 Apr 2025

https://github.com/deeplite/activ-sparse

Official PyTorch training code of Accelerating Deep Neural Networks via Semi-Structured Activation Sparsity (ICCV2023-RCV)

deep-neural-networks efficient-deep-learning efficient-inference low-latency raspberry-pi sparsity tinyml

Last synced: 09 Jul 2025

https://github.com/mmxgn/smooth-convex-kl-nmf

Repository holding various implementation of specific NMF methods for speaker diarization

nmf nonnegative-matrix-factorization smoothness sparsity speaker-diarization

Last synced: 06 Apr 2025

https://github.com/vita-group/quantumsea

[QCE 2023]"QuantumSEA: In-Time Sparse Exploration for Noise Adaptive Quantum Circuits" Tianlong Chen, Zhenyu Zhang, Hanrui Wang, Jiaqi Gu, Zirui Li, David Z Pan, Frederic T Chong, Song Han, Zhangyang Wang

quantum-chemistry quantum-computing sparsity vqe

Last synced: 29 Jul 2025

https://github.com/owensgroup/sparsify.me

A simple C++14 and CUDA-based header-only library with tools for sparse-machine learning.

deep-neural-networks deeplearning sparse-matrix sparsification sparsifying-transform sparsity

Last synced: 15 Jun 2025

https://github.com/sanjaradylov/sparse-cheml

Molecular-property prediction with sparsity

computational-chemistry elastic-net group-lasso molecular-property-prediction sparse-group-lasso sparsity

Last synced: 22 Feb 2025

https://github.com/sparsity-xyz/nova-app-template

nova app template for developers to start with

aws enclave enclaver nitro nova sparsity tee

Last synced: 08 Apr 2026

https://github.com/sparsity-xyz/sparsity-nova-examples

Reference apps for building verifiable AWS Nitro Enclave applications on Sparsity Nova

aws computation computing enclave enclaver nitro nova sparsity tee trust trustless verifiable

Last synced: 08 Apr 2026

https://github.com/paigejo/lk-inla

This repository is out of date. See instead: https://github.com/paigejo/ELK

computational-statistics gaussian-processes inla non-gaussian sparsity spatial-data-analysis spatial-statistics

Last synced: 17 Apr 2026

https://github.com/niteshchawla/movie-recommender-system

To create a Recommender System to show personalized movie recommendations based on ratings given by a user and other users similar to them in order to improve user experience.

collaborative-filtering correlation-matrix cosine-similarity exploratory-data-analysis feature-engineering knearest-neighbor-algorithm mape matrix-factorization pca-analysis pearson-correlation recommender-system rmse sparsity tsne-visualization visualization