{"id":17968742,"url":"https://github.com/vainf/diff-pruning","last_synced_at":"2025-06-30T07:35:28.636Z","repository":{"id":167071583,"uuid":"642385454","full_name":"VainF/Diff-Pruning","owner":"VainF","description":"[NeurIPS 2023] Structural Pruning for Diffusion Models","archived":false,"fork":false,"pushed_at":"2024-07-08T15:48:17.000Z","size":26401,"stargazers_count":185,"open_issues_count":12,"forks_count":12,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-11T02:05:49.163Z","etag":null,"topics":["diffusion-models","efficient-deep-learning","model-compression","network-pruning","pytorch","structured-pruning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VainF.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-18T12:57:52.000Z","updated_at":"2025-04-09T03:29:39.000Z","dependencies_parsed_at":"2023-12-19T20:55:16.828Z","dependency_job_id":"ba55f759-6d51-4d84-aa01-bd6180459841","html_url":"https://github.com/VainF/Diff-Pruning","commit_stats":null,"previous_names":["vainf/diff-pruning"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/VainF/Diff-Pruning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VainF%2FDiff-Pruning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VainF%2FDiff-Pruning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VainF%2FDiff-Pruning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VainF%2FDiff-Pruning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VainF","download_url":"https://codeload.github.com/VainF/Diff-Pruning/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VainF%2FDiff-Pruning/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262731858,"owners_count":23355438,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diffusion-models","efficient-deep-learning","model-compression","network-pruning","pytorch","structured-pruning"],"created_at":"2024-10-29T14:41:25.208Z","updated_at":"2025-06-30T07:35:28.588Z","avatar_url":"https://github.com/VainF.png","language":"Python","readme":"# Diff-Pruning: Structural Pruning for Diffusion Models\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/framework.png\" width=\"80%\"\u003e\u003c/img\u003e\n\u003c/div\u003e\n\n## Update\nCheck our latest work [DeepCache](https://horseee.github.io/Diffusion_DeepCache/), a **training-free and almost loessless** method for diffusion model acceleration. It can be viewed as a special pruning technique that dynamically drops deep layers and only runs shallow ones during inference.\n\n\n## Introduction\n\u003e **Structural Pruning for Diffusion Models** [[arxiv]](https://arxiv.org/abs/2305.10924)  \n\u003e *[Gongfan Fang](https://fangggf.github.io/), [Xinyin Ma](https://horseee.github.io/), [Xinchao Wang](https://sites.google.com/site/sitexinchaowang/)*    \n\u003e *National University of Singapore*\n\nThis work presents *Diff-Pruning*, an efficient structrual pruning method for diffusion models. Our empirical assessment highlights two primary features:\n1) ``Efficiency``: It enables approximately a 50% reduction in FLOPs at a mere 10% to 20% of the original training expenditure; \n2) ``Consistency``: The pruned diffusion models inherently preserve generative behavior congruent with the pre-trained ones.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/LSUN.png\" width=\"80%\"\u003e\u003c/img\u003e\n\u003c/div\u003e\n\n### Supported Methods\n- [x] Magnitude Pruning\n- [x] Random Pruning\n- [x] Taylor Pruning\n- [x] Diff-Pruning (A taylor-based method proposed in our paper)   \n\n### TODO List\n- [ ] Support more diffusion models from Diffusers\n- [ ] Upload checkpoints of pruned models\n- [ ] Training scripts for CelebA-HQ, LSUN Church \u0026 LSUN Bedroom\n- [ ] Align the performance with the [DDIM Repo](https://github.com/ermongroup/ddim). \n\n## Our Exp Code (Unorganized)\n\n### Pruning with DDIM codebase\nThis example shows how to prune a DDPM model pre-trained on CIFAR-10 using the [DDIM codebase](https://github.com/ermongroup/ddim). Since that [Huggingface Diffusers](https://github.com/huggingface/diffusers) do not support [``skip_type='quad'``](https://github.com/ermongroup/ddim/issues/3) in DDIM, you may get slightly worse FID scores with Diffusers for both pre-trained models (FID=4.5) and pruned models (FID=5.6). We are working on this to implement the quad strategy for Diffusers. For reproducibility, we provide our original **but unorganized** exp code for the paper in [ddpm_exp](ddpm_exp). \n\n```bash\ncd ddpm_exp\n# Prune \u0026 Finetune\nbash scripts/simple_cifar_our.sh 0.05 # the pre-trained model and data will be automatically prepared\n# Sampling\nbash scripts/sample_cifar_ddpm_pruning.sh run/finetune_simple_v2/cifar10_ours_T=0.05.pth/logs/post_training/ckpt_100000.pth run/sample\n```\n\nFor FID, please refer to [this section](https://github.com/VainF/Diff-Pruning#4-fid-score).  \n\nOutput:\n```\nFound 49984 files.\n100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 391/391 [00:49\u003c00:00,  7.97it/s]\nFID:  5.242662673752534\n```\n\n### Pruning with LDM codebase\n\nPlease check [ldm_exp/run.sh](ldm_exp/run.sh) for an example of pruning a pre-trained LDM model on ImageNet. This codebase is still unorganized. We will clean it up in the future.\n\n## Pruning with Huggingface Diffusers\n\nThe following pipeline prunes a pre-trained DDPM on CIFAR-10 with [Huggingface Diffusers](https://github.com/huggingface/diffusers).\n\n### 0. Requirements, Data and Pretrained Model\n\n* Requirements\n```bash\npip install -r requirements.txt\n```\n \n* Data\n  \nDownload and extract CIFAR-10 images to *data/cifar10_images* for training and evaluation.\n```bash\npython tools/extract_cifar10.py --output data\n```\n* Pretrained Models\n  \nThe following script will download an official DDPM model and convert it to the format of Huggingface Diffusers. You can find the converted model at *pretrained/ddpm_ema_cifar10*. It is an EMA version of [google/ddpm-cifar10-32](https://huggingface.co/google/ddpm-cifar10-32)\n```bash\nbash tools/convert_cifar10_ddpm_ema.sh\n```\n\n(Optional) You can also download a pre-converted model using wget\n```bash\nwget https://github.com/VainF/Diff-Pruning/releases/download/v0.0.1/ddpm_ema_cifar10.zip\n```\n\n### 1. Pruning\nCreate a pruned model at *run/pruned/ddpm_cifar10_pruned*\n```bash\nbash scripts/prune_ddpm_cifar10.sh 0.3  # pruning ratio = 30\\%\n```\n\n### 2. Finetuning (Post-Training)\nFinetune the model and save it at *run/finetuned/ddpm_cifar10_pruned_post_training*\n```bash\nbash scripts/finetune_ddpm_cifar10.sh\n```\n\n### 3. Sampling\n**Pruned:** Sample and save images to *run/sample/ddpm_cifar10_pruned*\n```bash\nbash scripts/sample_ddpm_cifar10_pruned.sh\n```\n\n**Pretrained:** Sample and save images to *run/sample/ddpm_cifar10_pretrained*\n```bash\nbash scripts/sample_ddpm_cifar10_pretrained.sh\n```\n\n### 4. FID Score\nThis script was modified from https://github.com/mseitzer/pytorch-fid. \n\n```bash\n# pre-compute the stats of CIFAR-10 dataset\npython fid_score.py --save-stats data/cifar10_images run/fid_stats_cifar10.npz --device cuda:0 --batch-size 256\n```\n\n```bash\n# Compute the FID score of sampled images\npython fid_score.py run/sample/ddpm_cifar10_pruned run/fid_stats_cifar10.npz --device cuda:0 --batch-size 256\n```\n\n### 5. (Optional) Distributed Training and Sampling with Accelerate\nThis project supports distributed training and sampling. \n```bash\npython -m torch.distributed.launch --nproc_per_node=8 --master_port 22222 --use_env \u003cddpm_sample.py|ddpm_train.py\u003e ...\n```\nA multi-processing example can be found at [scripts/sample_ddpm_cifar10_pretrained_distributed.sh](scripts/sample_ddpm_cifar10_pretrained_distributed.sh).\n\n\n## Prune Pre-trained DPMs from [HuggingFace Diffusers](https://huggingface.co/models?library=diffusers)\n\n### :rocket: [Denoising Diffusion Probabilistic Models (DDPMs)](https://arxiv.org/abs/2006.11239)\nExample: [google/ddpm-ema-bedroom-256](https://huggingface.co/google/ddpm-ema-bedroom-256)\n```bash\npython ddpm_prune.py \\\n--dataset \"\u003cpath/to/imagefoler\u003e\" \\  \n--model_path google/ddpm-ema-bedroom-256 \\\n--save_path run/pruned/ddpm_ema_bedroom_256_pruned \\\n--pruning_ratio 0.05 \\\n--pruner \"\u003crandom|magnitude|reinit|taylor|diff-pruning\u003e\" \\\n--batch_size 4 \\\n--thr 0.05 \\\n--device cuda:0 \\\n```\nThe ``dataset`` and ``thr`` arguments only work for taylor \u0026 diff-pruning.\n\n\n### :rocket: [Latent Diffusion Models (LDMs)](https://arxiv.org/abs/2112.10752)\nExample: [CompVis/ldm-celebahq-256](https://huggingface.co/CompVis/ldm-celebahq-256)\n```bash\npython ldm_prune.py \\\n--model_path CompVis/ldm-celebahq-256 \\\n--save_path run/pruned/ldm_celeba_pruned \\\n--pruning_ratio 0.05 \\\n--pruner \"\u003crandom|magnitude|reinit\u003e\" \\\n--device cuda:0 \\\n--batch_size 4 \\\n```\n\n## Results\n\n* **DDPM on Cifar-10, CelebA and LSUN**\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/exp.png\" width=\"75%\"\u003e\u003c/img\u003e\n\u003cimg src=\"https://github.com/VainF/Diff-Pruning/assets/18592211/39b3a7ad-2abb-4934-9ee0-07724029660b\" width=\"75%\"\u003e\u003c/img\u003e\n\u003c/div\u003e\n\n* **Conditional LDM on ImageNet-1K 256**\n\nWe also have some results on Conditional LDM for ImageNet-1K 256x256, where we finetune a pruned LDM for only 4 epochs. Will release the training script soon.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/VainF/Diff-Pruning/assets/18592211/31dbf489-2ca2-4625-ba54-5a5ff4e4a626\" width=\"75%\"\u003e\u003c/img\u003e\n\u003cimg src=\"https://github.com/VainF/Diff-Pruning/assets/18592211/20d546c5-9012-4ba9-80b2-96ed29da7d07\" width=\"85%\"\u003e\u003c/img\u003e\n\u003c/div\u003e\n\n\n## Acknowledgement\n\nThis project is heavily based on [Diffusers](https://github.com/huggingface/diffusers), [Torch-Pruning](https://github.com/VainF/Torch-Pruning), [pytorch-fid](https://github.com/mseitzer/pytorch-fid). Our experiments were conducted on [ddim](https://github.com/ermongroup/ddim) and [LDM](https://github.com/CompVis/latent-diffusion).\n\n## Citation\nIf you find this work helpful, please cite:\n```\n@inproceedings{fang2023structural,\n  title={Structural pruning for diffusion models},\n  author={Gongfan Fang and Xinyin Ma and Xinchao Wang},\n  booktitle={Advances in Neural Information Processing Systems},\n  year={2023},\n}\n```\n\n```\n@inproceedings{fang2023depgraph,\n  title={Depgraph: Towards any structural pruning},\n  author={Fang, Gongfan and Ma, Xinyin and Song, Mingli and Mi, Michael Bi and Wang, Xinchao},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={16091--16101},\n  year={2023}\n}\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvainf%2Fdiff-pruning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvainf%2Fdiff-pruning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvainf%2Fdiff-pruning/lists"}