https://github.com/piyush-555/gaussiandistillation
Data-free knowledge distillation using Gaussian noise (NeurIPS paper)
https://github.com/piyush-555/gaussiandistillation
knowledge-distillation neurips
Last synced: about 1 month ago
JSON representation
Data-free knowledge distillation using Gaussian noise (NeurIPS paper)
- Host: GitHub
- URL: https://github.com/piyush-555/gaussiandistillation
- Owner: Piyush-555
- License: mit
- Created: 2022-09-23T09:34:50.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-03-24T07:32:37.000Z (about 2 years ago)
- Last Synced: 2025-03-26T13:45:47.430Z (about 2 months ago)
- Topics: knowledge-distillation, neurips
- Language: Python
- Homepage:
- Size: 301 KB
- Stars: 15
- Watchers: 2
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# GaussianDistillation
### Running CIFAR10 experiment
```python
# Train teacher
python utils/train_teacher.py CIFAR10 100
# Distill students
python distill_gaussian.py CIFAR10 200 200 --compare True --model_student resnet18
python distill_gaussian.py CIFAR10 200 200 --compare True --model_student mobilenetv2
```
### Citation
```
@inproceedings{NEURIPS2022_1f96b24d,
author = {Raikwar, Piyush and Mishra, Deepak},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
pages = {4902--4912},
publisher = {Curran Associates, Inc.},
title = {Discovering and Overcoming Limitations of Noise-engineered Data-free Knowledge Distillation},
url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/1f96b24df4b06f5d68389845a9a13ed9-Paper-Conference.pdf},
volume = {35},
year = {2022}
}
```