Projects in Awesome Lists tagged with knowledge-distillation

https://github.com/paddlepaddle/paddleclas

A treasure chest for visual classification and recognition powered by PaddlePaddle

autoaugment cutmix deit fastdeploy gridmask image-classification image-recognition image-retrieval-system knowledge-distillation pretrained-models product-recognition randaugment repvgg swin-transformer

Last synced: 13 May 2025

https://github.com/PaddlePaddle/PaddleClas

A treasure chest for visual classification and recognition powered by PaddlePaddle

autoaugment cutmix deit fastdeploy gridmask image-classification image-recognition image-retrieval-system knowledge-distillation pretrained-models product-recognition randaugment repvgg swin-transformer

Last synced: 20 Mar 2025

https://github.com/huawei-noah/pretrained-language-model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

knowledge-distillation large-scale-distributed model-compression pretrained-models quantization

Last synced: 14 May 2025

https://github.com/huawei-noah/Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

knowledge-distillation large-scale-distributed model-compression pretrained-models quantization

Last synced: 16 Mar 2025

https://github.com/jingyi0000/vlm_survey

Collection of AWESOME vision-language models for vision tasks

clip computer-vision deep-learning knowledge-distillation multi-modal-model survey transfer-learning vision-language-model

Last synced: 19 Feb 2026

https://intel.github.io/neural-compressor/

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

auto-tuning awq fp4 gptq int4 int8 knowledge-distillation large-language-models low-precision mxformat post-training-quantization pruning quantization quantization-aware-training smoothquant sparsegpt sparsity

Last synced: 09 Dec 2025

https://github.com/IDEA-Research/DWPose

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

controlnet knowledge-distillation pose-estimation stable-diffusion-webui

Last synced: 22 Jul 2025

https://github.com/idea-research/dwpose

"Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023, CV4Metaverse Workshop)

controlnet knowledge-distillation pose-estimation stable-diffusion-webui

Last synced: 14 Apr 2025

https://github.com/intel/neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

auto-tuning awq fp4 gptq int4 int8 knowledge-distillation large-language-models low-precision mxformat post-training-quantization pruning quantization quantization-aware-training smoothquant sparsegpt sparsity

Last synced: 12 May 2025

https://github.com/alibaba/easynlp

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

bert deep-learning fewshot-learning knowledge-distillation knowledge-pretraining machine-learning nlp pretrained-models pytorch text-classification text-image-retrieval text-to-image-synthesis transfer-learning transformers

Last synced: 29 Apr 2025

https://github.com/alibaba/EasyNLP

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

bert deep-learning fewshot-learning knowledge-distillation knowledge-pretraining machine-learning nlp pretrained-models pytorch text-classification text-image-retrieval text-to-image-synthesis transfer-learning transformers

Last synced: 18 Mar 2025

https://github.com/haitongli/knowledge-distillation-pytorch

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

cifar10 computer-vision dark-knowledge deep-neural-networks knowledge-distillation model-compression pytorch

Last synced: 15 May 2025

https://github.com/microsoft/cream

This is a collection of our NAS and Vision Transformer work.

automl efficiency knowledge-distillation nas rpe vision-transformer vit-compression

Last synced: 15 May 2025

https://github.com/microsoft/Cream

This is a collection of our NAS and Vision Transformer work.

automl efficiency knowledge-distillation nas rpe vision-transformer vit-compression

Last synced: 05 Apr 2025

https://github.com/open-mmlab/mmrazor

OpenMMLab Model Compression Toolbox and Benchmark.

autoslim classification darts detection knowledge-distillation nas pruning pytorch quantization segmentation spos

Last synced: 14 May 2025

https://github.com/yoshitomo-matsubara/torchdistill

A coding-free framework built on PyTorch for reproducible deep learning studies. PyTorch Ecosystem. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.

amazon-sagemaker-lab cifar10 cifar100 coco colab-notebook glue google-colab image-classification imagenet knowledge-distillation natural-language-processing nlp object-detection pascal-voc pytorch pytorch-ecosystem semantic-segmentation transformer

Last synced: 14 May 2025

https://github.com/microsoft/NeuronBlocks

NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego

artificial-intelligence deep-learning dnn knowledge-distillation model-compression natural-language-processing pytorch qna question-answering sequence-labeling text-classification text-matching

Last synced: 07 Apr 2025

https://github.com/szagoruyko/attention-transfer

Improving Convolutional Networks via Attention Transfer (ICLR 2017)

attention deep-learning knowledge-distillation pytorch

Last synced: 16 May 2025

https://github.com/lxztju/pytorch_classification

利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码

cnn densenet deployment flask image-classification knn knowledge-distillation label-smoothing pytorch random-forest resnet resnext svm

Last synced: 16 May 2025

https://github.com/huawei-noah/efficient-computing

Efficient computing methods developed by Huawei Noah's Ark Lab

binary-neural-networks knowledge-distillation model-compression pruning quantization self-supervised

Last synced: 14 May 2025

https://github.com/huawei-noah/Efficient-Computing

Efficient computing methods developed by Huawei Noah's Ark Lab

binary-neural-networks knowledge-distillation model-compression pruning quantization self-supervised

Last synced: 20 Mar 2025

https://github.com/MingSun-Tse/EfficientDNNs

Collection of recent methods on (deep) neural network compression and acceleration.

deep-learning deep-neural-networks efficient-deep-learning knowledge-distillation model-compression network-pruning

Last synced: 27 Apr 2025

https://github.com/MingSun-Tse/Efficient-Deep-Learning

Collection of recent methods on (deep) neural network compression and acceleration.

deep-learning deep-neural-networks efficient-deep-learning knowledge-distillation model-compression network-pruning

Last synced: 16 Mar 2025

https://github.com/Tebmer/Awesome-Knowledge-Distillation-of-LLMs

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.

alignment compression data-augmentation data-synthesis feedback instruction-following kd knowledge-distillation large-language-model llm multi-modal self-distillation self-training supervised-finetuning survey

Last synced: 12 Apr 2025

https://github.com/alibaba/EasyTransfer

EasyTransfer is designed to make the development of transfer learning in NLP applications easier.

bert knowledge-distillation nlp-applications transfer-learning

Last synced: 09 May 2025

https://github.com/alibaba/easytransfer

EasyTransfer is designed to make the development of transfer learning in NLP applications easier.

bert knowledge-distillation nlp-applications transfer-learning

Last synced: 14 Oct 2025

https://github.com/lhyfst/knowledge-distillation-papers

knowledge distillation papers

dark-knowledge knowledge-distillation model-compression paper reading-list

Last synced: 05 May 2025

https://github.com/aimagelab/mammoth

An Extendible (General) Continual Learning Framework based on Pytorch - official codebase of Dark Experience for General Continual Learning

continual-learning dark-experience-replay deep-learning der experience-replay knowledge-distillation neurips2020 pytorch

Last synced: 14 May 2025

https://github.com/megvii-research/mdistiller

The official implementation of [CVPR2022] Decoupled Knowledge Distillation https://arxiv.org/abs/2203.08679 and [ICCV2023] DOT: A Distillation-Oriented Trainer https://openaccess.thecvf.com/content/ICCV2023/papers/Zhao_DOT_A_Distillation-Oriented_Trainer_ICCV_2023_paper.pdf

cifar coco computer-vision cvpr2022 deep-learning iccv2023 imagenet knowledge-distillation pytorch

Last synced: 01 Feb 2026

https://github.com/sforaidl/kd_lib

A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.

algorithm-implementations benchmarking data-science deep-learning-library knowledge-distillation machine-learning model-compression pruning pytorch quantization

Last synced: 16 May 2025

https://github.com/segmind/distill-sd

Segmind Distilled diffusion

distillation inference knowledge-distillation stable-diffusion

Last synced: 28 Mar 2025

https://github.com/Picsart-AI-Research/MI-GAN

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

image-inpainting knowledge-distillation mi-gan mobile-inpainting object-removal re-parameterization

Last synced: 03 Apr 2025

https://github.com/picsart-ai-research/mi-gan

[ICCV 2023] MI-GAN: A Simple Baseline for Image Inpainting on Mobile Devices

image-inpainting knowledge-distillation mi-gan mobile-inpainting object-removal re-parameterization

Last synced: 15 May 2025

https://github.com/chaitjo/efficient-gnns

Code and resources on scalable and efficient Graph Neural Networks

contrastive-learning efficient-deep-learning efficient-neural-networks gnn graph-convolutional-networks graph-neural-networks graph-representation-learning knowledge-distillation scalable-deep-learning

Last synced: 05 Apr 2025

https://github.com/yanx27/2dpass

2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds (ECCV 2022) :fire:

deep-learning knowledge-distillation lidar-point-cloud nuscenes pytorch semantic-segmentation semantickitti

Last synced: 05 Apr 2025

https://github.com/yanx27/2DPASS

2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds (ECCV 2022) :fire:

deep-learning knowledge-distillation lidar-point-cloud nuscenes pytorch semantic-segmentation semantickitti

Last synced: 20 Mar 2025

https://github.com/clovaai/overhaul-distillation

Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)

iccv2019 knowledge-distillation knowledge-transfer network-compression teacher-student

Last synced: 06 Apr 2025

https://github.com/yzhao062/SUOD

(MLSys' 21) An Acceleration System for Large-scare Unsupervised Heterogeneous Outlier Detection (Anomaly Detection)

anomaly-detection data-mining distributed-systems knowledge-distillation machine-learning machine-learning-algorithms machine-learning-library outlier-detection python

Last synced: 18 Jul 2025

https://github.com/yzhao062/suod

(MLSys' 21) An Acceleration System for Large-scare Unsupervised Heterogeneous Outlier Detection (Anomaly Detection)

anomaly-detection data-mining distributed-systems knowledge-distillation machine-learning machine-learning-algorithms machine-learning-library outlier-detection python

Last synced: 12 Apr 2025

https://github.com/HikariTJU/LD

Localization Distillation for Object Detection (CVPR 2022, TPAMI 2023)

deep-learning detection knowledge-distillation ld object-detection pytorch

Last synced: 05 Apr 2025

https://github.com/datawhalechina/llm-deploy

大模型/LLM推理和部署理论与实践

knowledge-distillation llm llm-deploy lora pruning quantization

Last synced: 30 Jan 2026

https://github.com/yuanyuanli85/Fast_Human_Pose_Estimation_Pytorch

Pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419

knowledge-distillation pose-estimation pytorch stacked-hourglass-networks

Last synced: 04 Apr 2025

https://github.com/Vexa-ai/vexa

meeting notetaker and knowledge chat for teams and individuals

ai-chat google-meet knowledge-distillation knowledge-management meeting-assistant meeting-minutes meeting-notes ms-teams ms-teams-app notetaker rag zoom

Last synced: 02 May 2025

https://github.com/czg1225/SlimSAM

[NeurIPS 2024] SlimSAM: 0.1% Data Makes Segment Anything Slim

knowledge-distillation model-compression model-pruning segment-anything-model

Last synced: 24 Jul 2025

https://github.com/vita-group/slak

[ICLR 2023] "More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity"; [ICML 2023] "Are Large Kernels Better Teachers than Transformers for ConvNets?"

51x51 convnet deep-learning dynamic-sparsity knowledge-distillation large-kernels object-detection pytorch segmentation

Last synced: 09 Apr 2025

https://github.com/VITA-Group/SLaK

[ICLR 2023] "More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity"; [ICML 2023] "Are Large Kernels Better Teachers than Transformers for ConvNets?"

51x51 convnet deep-learning dynamic-sparsity knowledge-distillation large-kernels object-detection pytorch segmentation

Last synced: 20 Mar 2025

https://github.com/sseung0703/kd_methods_with_tf

Knowledge distillation methods implemented with Tensorflow (now there are 11 (+1) methods, and will be added more.)

knowledge-distillation tensorflow tensorflow-examples

Last synced: 09 Apr 2025

https://github.com/ltkong218/IFRNet

IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation (CVPR 2022)

deep-learning deep-networks frame-interpolation knowledge-distillation optical-flow pytorch real-time slow-motion video view-synthesis

Last synced: 06 Mar 2025

https://github.com/FasterAI-Labs/fasterai

FasterAI: Prune and Distill your models with FastAI and PyTorch

compression fastai knowledge-distillation pruning pytorch

Last synced: 04 May 2025

https://github.com/SsisyphusTao/Object-Detection-Knowledge-Distillation

An Object Detection Knowledge Distillation framework powered by pytorch, now having SSD and yolov5.

deep-learning knowledge-distillation object-detection pytorch ssd yolov5

Last synced: 21 Apr 2025

https://github.com/lirongwu/kdga

Code for NeurIPS 2022 paper "Knowledge Distillation Improves Graph Structure Augmentation for Graph Neural Networks"

graph-augmentation graph-neural-networks graph-structure-learning knowledge-distillation

Last synced: 22 Feb 2026

https://github.com/zh320/realtime-semantic-segmentation-pytorch

PyTorch implementation of over 30 realtime semantic segmentations models, e.g. BiSeNetv1, BiSeNetv2, CGNet, ContextNet, DABNet, DDRNet, EDANet, ENet, ERFNet, ESPNet, ESPNetv2, FastSCNN, ICNet, LEDNet, LinkNet, PP-LiteSeg, SegNet, ShelfNet, STDC, SwiftNet, and support knowledge distillation, distributed training, Optuna etc.

cityscapes distributed-training enet knowledge-distillation optuna pytorch real-time semantic-segmentation

Last synced: 04 Apr 2025

https://github.com/miccunifi/SEARLE

[ICCV 2023] - Zero-shot Composed Image Retrieval with Textual Inversion

circo cirr clip composed-image-retrieval fashion-iq knowledge-distillation multimodal-learning pytorch textual-inversion

Last synced: 03 Apr 2025

https://github.com/ai4ce/disconet

[NeurIPS2021] Learning Distilled Collaboration Graph for Multi-Agent Perception

3d-object-detection 3d-scene-understanding autonomous-driving collaborative-learning communication-networks computer-vision deep-learning graph graph-learning knowledge-distillation multi-agent-learning multi-agent-perception multi-agent-system point-cloud-processing v2v v2x-communication

Last synced: 24 Feb 2026

https://github.com/ai4ce/DiscoNet

[NeurIPS2021] Learning Distilled Collaboration Graph for Multi-Agent Perception

3d-object-detection 3d-scene-understanding autonomous-driving collaborative-learning communication-networks computer-vision deep-learning graph graph-learning knowledge-distillation multi-agent-learning multi-agent-perception multi-agent-system point-cloud-processing v2v v2x-communication

Last synced: 20 Mar 2025

https://github.com/hnuzhy/SSDA-YOLO

Codes for my paper "SSDA-YOLO: Semi-supervised Domain Adaptive YOLO for Cross-Domain Object Detection"

domain-adaptation image-style-transfer knowledge-distillation object-detection transfer-learning yolov5

Last synced: 20 Mar 2025

https://datawhalechina.github.io/awesome-compression/

模型压缩的小白入门教程

compression kd knowledge-distillation model-compression model-pruning model-quantization neural-architecture-search prune quantization tinyml

Last synced: 24 Sep 2025

https://datawhalechina.github.io/llm-deploy/

大模型/LLM推理和部署理论与实践

knowledge-distillation llm llm-deploy lora pruning quantization

Last synced: 24 Sep 2025

https://github.com/VICO-UoE/URL

Universal Representation Learning from Multiple Domains for Few-shot Classification - ICCV 2021, Cross-domain Few-shot Learning with Task-specific Adapters - CVPR 2022

few-shot-learning knowledge-distillation multi-domain-learning multi-domain-meta-training

Last synced: 08 May 2025

https://github.com/DRSY/MoTIS

[NAACL 2022]Mobile Text-to-Image search powered by multimodal semantic representation models(e.g., OpenAI's CLIP)

ai clip cross-modal image-search ios-swift k-means k-means-clustering knn knowledge-distillation lsh naacl random-projection retrieval semantic-search vector-search

Last synced: 08 May 2025

https://github.com/Qrange-group/SUR-adapter

ACM MM'23 (oral), SUR-adapter for pre-trained diffusion models can acquire the powerful semantic understanding and reasoning capabilities from large language models to build a high-quality textual semantic representation for text-to-image generation.

adapter diffusion-models image-generation knowledge-distillation large-language-models pytorch

Last synced: 28 Mar 2025

https://github.com/microsoft/simxns

SimXNS is a research project for information retrieval. This repo contains official implementations by MSRA NLC team.

decision-time-planning dense-retrieval information-retrieval knowledge-distillation natural-language-processing negative-sampling pretraining-for-ir

Last synced: 16 Oct 2025

https://github.com/zhanghengdev/MutualGuide

Localize to Classify and Classify to Localize: Mutual Guidance in Object Detection

deep-learning knowledge-distillation object-detection pytorch tensorrt

Last synced: 16 Apr 2025

https://github.com/sseung0703/knowledge_distillation_via_tf2.0

The codes for recent knowledge distillation algorithms and benchmark results via TF2.0 low-level API

benchmark knowledge-distillation tensorflow2 tensorflow2-models

Last synced: 15 Apr 2025

https://github.com/juliagusak/model-compression-and-acceleration-progress

Repository to track the progress in model compression and acceleration

acceleration architecture-search compression knowledge-distillation low-rank neural-network pruning sparsification tensor-decomposition

Last synced: 26 Feb 2026

https://github.com/bhheo/ab_distillation

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons (AAAI 2019)

knowledge-distillation knowledge-transfer network-compression teacher-student-learning transfer-learning

Last synced: 30 Apr 2025

https://github.com/Sharpiless/yolov5-knowledge-distillation

yolov5目标检测模型的知识蒸馏（基于响应的蒸馏）

knowledge-distillation object-detection yolo yolov5

Last synced: 20 Apr 2025

https://github.com/archsyscall/aquvitae

Knowledge Distillation Toolkit

deep-learning knowledge-distillation light-weight machine-learning model-compression pytorch tensorflow

Last synced: 02 Apr 2026

https://github.com/snap-research/graphless-neural-networks

[ICLR 2022] Code for Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation (GLNN)

deep-learning distillation efficient-inference gnn graph-algorithm graph-neural-networks knowledge-distillation pytorch scalability

Last synced: 06 Apr 2025

https://github.com/sayakpaul/funmatch-distillation

TF2 implementation of knowledge distillation using the "function matching" hypothesis from https://arxiv.org/abs/2106.05237.

bit-resnet image-classification keras knowledge-distillation tensorflow tpu transfer-learning vision

Last synced: 29 Jul 2025

https://github.com/d-li14/dhm

[CVPR 2020] Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives

cvpr2020 deeply-supervised imagenet knowledge-distillation pretrained-models pytorch

Last synced: 21 Jul 2025

https://github.com/ahmdtaha/knowledge_evolution

(CVPR-Oral 2021) PyTorch implementation of Knowledge Evolution approach and Split-Nets

classification computer-vision deep-learning deep-neural-networks few-shot-learning knowledge-distillation knowledge-evolution machine-learning python3 pytorch slim-networks

Last synced: 24 Jul 2025

https://github.com/vita-group/nasty-teacher

[ICLR 2021 Spotlight Oral] "Undistillable: Making A Nasty Teacher That CANNOT teach students", Haoyu Ma, Tianlong Chen, Ting-Kuei Hu, Chenyu You, Xiaohui Xie, Zhangyang Wang

copyright knowledge-distillation privacy-protection teacher-student undistillable

Last synced: 10 Aug 2025

https://github.com/qcraftai/distill-bev

DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillation (ICCV 2023)

3d-object-detection autonomous-driving bev cross-modal distillation knowledge-distillation lidar multi-camera multi-modal nuscenes point-cloud self-driving

Last synced: 20 Mar 2025

https://github.com/naver/dune

Code repository for "DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers"

computer-vision foundation-model image-encoder knowledge-distillation vision-transformer

Last synced: 04 Apr 2026

https://github.com/aimagelab/vkd

PyTorch code for ECCV 2020 paper: "Robust Re-Identification by Multiple Views Knowledge Distillation"

deep-learning eccv-2020 knowledge-distillation re-id

Last synced: 06 Nov 2025

https://github.com/BUPT-GAMMA/CPF

The official code of WWW2021 paper: Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework

graph-neural-networks knowledge-distillation

Last synced: 21 Jul 2025

https://github.com/bupt-gamma/cpf

The official code of WWW2021 paper: Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework

graph-neural-networks knowledge-distillation

Last synced: 10 Apr 2025

https://github.com/aimagelab/VKD

PyTorch code for ECCV 2020 paper: "Robust Re-Identification by Multiple Views Knowledge Distillation"

deep-learning eccv-2020 knowledge-distillation re-id

Last synced: 16 Apr 2025

https://github.com/RunpeiDong/PointDistiller

[CVPR 2023] PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection

3d-object-detection knowledge-distillation point-cloud

Last synced: 20 Mar 2025

https://github.com/chenhongyiyang/pgd

[ECCV 2022] Prediction-Guided Distillation for Dense Object Detection

knowledge-distillation object-detection

Last synced: 09 Feb 2026

https://github.com/bhheo/bss_distillation

Knowledge Distillation with Adversarial Samples Supporting Decision Boundary (AAAI 2019)

adversarial-attacks adversarial-samples image-classification knowledge-distillation network-compression teacher-student-learning

Last synced: 30 Apr 2025

https://github.com/yochengliu/mlic-kd-wsd

Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection (ACM MM 2018)

computer-vision deep-learning knowledge-distillation multi-label-classification weakly-supervised-detection

Last synced: 12 Jun 2025

https://github.com/megengine/icd

This is the official implementation of the paper "Instance-conditional Knowledge Distillation for Object Detection", based on MegEngine and Pytorch.

knowledge-distillation megengine object-detection pytorch

Last synced: 12 Apr 2025

https://github.com/sayakpaul/knowledge-distillation-in-keras

Demonstrates knowledge distillation for image-based models in Keras.

computer-vision keras knowledge-distillation model-optimization tensorflow

Last synced: 11 Jul 2025

https://github.com/sseung0703/sskd_svd

deep-learning knowledge-distillation

Last synced: 30 Apr 2025

https://github.com/sseung0703/zero-shot_knowledge_distillation

Zero-Shot Knowledge Distillation in Deep Networks in ICML2019

knowledge-distillation tensorflow zero-shot-learning

Last synced: 30 Apr 2025

https://github.com/jiequancui/dkl

Decoupled Kullback-Leibler Divergence Loss (DKL), NeurIPS 2024 / Generalized Kullback-Leibler Divergence Loss (GKL)

adversarial-robustness clip image-classification imagenet knowledge-distillation kullback-leibler-divergence llava long-tailed-learning neurips2024 semi-supervised-learning

Last synced: 14 Oct 2025

https://github.com/d-li14/san

[ECCV 2020] Scale Adaptive Network: Learning to Learn Parameterized Classification Networks for Scalable Input Images

classification eccv2020 efficient-neural-networks imagenet knowledge-distillation meta-learning mobilenetv2 pytorch resnet scale-invariance

Last synced: 10 Apr 2025

https://github.com/tjddus9597/LabelRelaxation-CVPR21

Official PyTorch Implementation of Embedding Transfer with Label Relaxation for Improved Metric Learning, CVPR 2021

cvpr2021 deep-metric-learning embedding-trasnfer image-retrieval knowledge-distillation pytorch

Last synced: 08 May 2025

https://github.com/canqin001/efficient_graph_similarity_computation

[NeurIPS-2021] Slow Learning and Fast Inference: Efficient Graph Similarity Computation via Knowledge Distillation

graph knowledge-distillation regression

Last synced: 31 Oct 2025

https://github.com/mit-han-lab/neurips-micronet

[JMLR'20] NeurIPS 2019 MicroNet Challenge Efficient Language Modeling, Champion

efficient-model knowledge-distillation language-modeling natural-language-processing pruning quantization

Last synced: 07 Jul 2025

https://github.com/IAAR-Shanghai/ICSFSurvey

A comprehensive survey on Internal Consistency and Self-Feedback in Large Language Models.

attention-head chain-of-thought data-augmentation decoding hallucination internal-consistency knowledge-distillation large-language-model large-language-models preference-learning reasoning self-consistency self-correct self-correction self-feedback self-improvement self-refine

Last synced: 31 Mar 2025

https://github.com/iaar-shanghai/icsfsurvey

A comprehensive survey on Internal Consistency and Self-Feedback in Large Language Models.

attention-head chain-of-thought data-augmentation decoding hallucination internal-consistency knowledge-distillation large-language-model large-language-models preference-learning reasoning self-consistency self-correct self-correction self-feedback self-improvement self-refine

Last synced: 06 Apr 2025

https://github.com/yoshitomo-matsubara/head-network-distillation

[IEEE Access] "Head Network Distillation: Splitting Distilled Deep Neural Networks for Resource-constrained Edge Computing Systems" and [ACM MobiCom HotEdgeVideo 2019] "Distilled Split Deep Neural Networks for Edge-assisted Real-time Systems"

image-classification imagenet knowledge-distillation pytorch split-computing

Last synced: 14 Apr 2025