{"id":17714309,"url":"https://github.com/mingyuanzhou/SiD","last_synced_at":"2025-03-13T22:32:21.964Z","repository":{"id":243353576,"uuid":"789990425","full_name":"mingyuanzhou/SiD","owner":"mingyuanzhou","description":"PyTorch code and model checkpoints for Score identity Distillation (SiD) and its adversarial version (SiDA) ","archived":false,"fork":false,"pushed_at":"2025-03-12T15:08:43.000Z","size":114970,"stargazers_count":104,"open_issues_count":0,"forks_count":6,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-12T15:37:50.284Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mingyuanzhou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-22T04:02:31.000Z","updated_at":"2025-03-12T15:08:47.000Z","dependencies_parsed_at":"2024-12-24T00:26:36.274Z","dependency_job_id":"838d51fd-de3b-48b3-af50-8de9f90d4911","html_url":"https://github.com/mingyuanzhou/SiD","commit_stats":null,"previous_names":["mingyuanzhou/sid"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mingyuanzhou%2FSiD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mingyuanzhou%2FSiD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mingyuanzhou%2FSiD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mingyuanzhou%2FSiD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mingyuanzhou","download_url":"https://codeload.github.com/mingyuanzhou/SiD/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243494536,"owners_count":20299831,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-25T11:02:21.094Z","updated_at":"2025-03-13T22:32:21.956Z","avatar_url":"https://github.com/mingyuanzhou.png","language":"Python","funding_links":[],"categories":["Accelerate"],"sub_categories":[],"readme":"# Distilling Pretrained Diffusion-Based Generative Models with SiD\n\nThis repository now contains **two distinct branches**:\n\n1. **Main Branch:** Focused on Score identity Distillation (SiD) as described in our ICML 2024 paper.\n2. **SiDA Branch:** Focused on Adversarial Score identity Distillation (SiDA), with additional techniques for distilling EDM2 models.\n\nFor more details on the SiDA branch, visit: [https://github.com/mingyuanzhou/SiD/tree/sida](https://github.com/mingyuanzhou/SiD/tree/sida).\n\n---\n\n## About This Repository\n\nThis repository contains the code necessary to replicate the findings of our ICML 2024 paper titled **\"Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation,\"** available at [arXiv:2404.04057](https://arxiv.org/abs/2404.04057). The technique, Score identity Distillation (SiD), is used to distill pretrained EDM diffusion models.\n\n### Branch Overview\n\n- **Main Branch:**\n  - Includes the implementation of Score identity Distillation (SiD) as described in the ICML 2024 paper.\n\n- **SiDA Branch:**\n  - Provides the latest advancements in Adversarial Score identity Distillation (SiDA), including techniques for distilling EDM2 models and leveraging adversarial learning.\n\n\n## Citations \nIf you find our work useful or incorporate our findings in your own research, please consider citing our paper:\n```bibtex\n@inproceedings{zhou2024score,\n  title={Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation},\n  author={Mingyuan Zhou and Huangjie Zheng and Zhendong Wang and Mingzhang Yin and Hai Huang},\n  booktitle={International Conference on Machine Learning},\n  year={2024}\n}\n```\nWe also have a follow-up paper, available at https://arxiv.org/abs/2406.01561, that extends our SiD methodology to distill Stable Diffusion models for one-step text-to-image generation:\n```bibtex\n@article{zhou2024long,\ntitle={Long and Short Guidance in Score identity Distillation for One-Step Text-to-Image Generation},\nauthor={Mingyuan Zhou and Zhendong Wang and Huangjie Zheng and Hai Huang},\njournal={ArXiv 2406.01561},\nurl={https://arxiv.org/abs/2406.01561},\nyear={2024}\n}\n```\n\n\n## State-of-the-art Performance\nSiD operates as a data-free distillation method but still demonstrates superior performance compared to the teacher EDM model across most datasets, with the notable exception of ImageNet 64x64. It outperforms all previous diffusion distillation approaches—whether one-step or few-step, data-free or training data-dependent—in terms of generation quality. This achievement sets new standards for efficiency and effectiveness in diffusion distillation.\n\nIt achieves the following Fréchet Inception Distances (FID):\n\n| Dataset              | FID   |\n|----------------------|-------|\n| CIFAR10 Unconditional| 1.923 |\n| CIFAR10 Conditional  | 1.710 |\n| ImageNet 64x64       | 1.524 |\n| FFHQ 64x64           | 1.550 |\n| AFHQ-v2 64x64        | 1.628 |\n\n\n## Prerequisites\n\nBefore you begin, ensure you have met the following requirements:\n* You have installed the latest version of [Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html).\n* You have a `Windows/Linux/Mac` machine.\n\n## Installation\n\nTo install the necessary packages and set up the environment, follow these steps:\n\n### Clone the Repository\n\nFirst, clone the repository to your local machine:\n\n```bash\ngit clone https://github.com/mingyuanzhou/SiD.git\ncd SiD\n```\n\n### Create the Conda Environment\n\nTo create the Conda environment with all the required dependencies, run:\n\n```bash\nconda env create -f environment.yaml\n```\n\nThis command will read the `environment.yaml` file in the repository, which contains all the necessary package information.\n\n### Activate the Environment\n\nAfter creating the environment, you can activate it by running:\n\n```bash\nconda activate sid\n```\n\n### Prepare the Datasets\n\nFollow the instructions detailed in the [EDM codebase](https://github.com/NVlabs/edm/tree/main?tab=readme-ov-file#preparing-datasets) to prepare the training datasets. Once prepared, place them into the `/data/datasets/` folder:\n\n- `cifar10-32x32.zip`\n- `imagenet-64x64.zip`\n- `ffhq-64x64.zip`\n- `afhqv2-64x64.zip`\n\n**Note:** Although a training dataset is not necessary for distilling the pretrained EDM model, it is used in our code to calculate evaluation metrics such as FID and Inception Score. Optionally, you can create a dummy dataset and either disable the evaluation code if you wish to run the SID distillation code without these metrics, or provide an npz file of the training dataset if you need to compute these metrics.\n\n## Usage\n\n\n### Training\nAfter activating the environment, you can run the scripts or use the modules provided in the repository. Example:\n\n```bash\nsh run_sid.sh 'cifar10-uncond'\n```\n\nAdjust the --batch-gpu parameter according to your GPU memory limitations. The default setting for cifar10-uncond consumes less than 10 GB of memory per GPU.\n\n### Generation\n\nGenerate example images:\n\n#### Generate images and save them as out/*.png and out.npz\n\n- Using a single GPU\n```bash\npython generate_onestep.py --outdir=image_experiment/sid-train-runs/out --seeds=0-63 --batch=64 --network=\u003cnetwork_path\u003e\n```\n- Using multiple GPU\n```bash\ntorchrun --standalone --nproc_per_node=2 generate_onestep.py --outdir=image_experiment/sid-train-runs/out --seeds=0-999 --batch=64 --network=\u003cnetwork_path\u003e\n```\n\n### Evaluations\n\nFor ImageNet, there are two different versions of the training data, each associated with its own set of reference statistics. To ensure apples-to-apples comparisons between EDM and its distilled generators with other diffusion models, `imagenet-64x64.npz` should be used for computing FID (Fréchet Inception Distance). Conversely, for computing Precision and Recall, `VIRTUAL_imagenet64_labeled.npz` should be utilized.\n\n`imagenet-64x64.npz` is available at [NVIDIA](https://nvlabs-fi-cdn.nvidia.com/edm/fid-refs/imagenet-64x64.npz).\n\n`VIRTUAL_imagenet64_labeled.npz` is available at [OpenAI](https://openaipublic.blob.core.windows.net/diffusion/jul-2021/ref_batches/imagenet/64/VIRTUAL_imagenet64_labeled.npz).\n\n#### Use `sid_generator.py` to generate and save 50,000 images, and compute FID using the saved images\n\n##### Use a single GPU\n```bash \npython sid_generate.py --outdir=image_experiment/out --seeds=0-49999 --batch=128 --network='https://huggingface.co/UT-Austin-PML/SiD/resolve/main/cifar10-uncond/alpha1.2/network-snapshot-1.200000-403968.pkl' --ref=https://nvlabs-fi-cdn.nvidia.com/edm/fid-refs/cifar10-32x32.npz \n```\n\n##### Use four GPUs\n```bash \ntorchrun --standalone --nproc_per_node=4 sid_generate.py --outdir=out --seeds=0-49999 --batch=128 --network='https://huggingface.co/UT-Austin-PML/SiD/resolve/main/imagenet64/alpha1.2/network-snapshot-1.200000-939176.pkl' --ref=https://nvlabs-fi-cdn.nvidia.com/edm/fid-refs/imagenet-64x64.npz\n```\n\n\n#### Use `sid_metrics.py` to perform 10 random trials, each trial computes the metrics using 50,000 randomly generated images\n\n##### Compute FID and/or IS \n\n```bash\ntorchrun --standalone --nproc_per_node=4  sid_metrics.py  --cond=False --metrics='fid50k_full,is50k' --network='https://huggingface.co/UT-Austin-PML/SiD/resolve/main/cifar10-uncond/alpha1.2/network-snapshot-1.200000-403968.pkl' --data='/data/datasets/cifar10-32x32.zip'\n```\n\n\n```bash\ntorchrun --standalone --nproc_per_node=4  sid_metrics.py  --cond=True --metrics='fid50k_full' --network='https://huggingface.co/UT-Austin-PML/SiD/resolve/main/imagenet64/alpha1.2/network-snapshot-1.200000-939176.pkl' --data='/data/datasets/imagenet-64x64.zip' --data_stat='https://nvlabs-fi-cdn.nvidia.com/edm/fid-refs/imagenet-64x64.npz' \n```\n\n\n##### Compute Precision and Recall for ImageNet\n\n```bash\ntorchrun --standalone --nproc_per_node=4  sid_metrics.py  --cond=True --metrics='pr50k3_full' --network='https://huggingface.co/UT-Austin-PML/SiD/resolve/main/imagenet64/alpha1.2/network-snapshot-1.200000-939176.pkl' --data='/data/datasets/imagenet-64x64.zip' --data_stat='https://openaipublic.blob.core.windows.net/diffusion/jul-2021/ref_batches/imagenet/64/VIRTUAL_imagenet64_labeled.npz'\n```\n\n\n## Checkpoints of one-step generators produced by SiD\n\nThe one-step generators produced by SiD are provided in [huggingface/UT-Austin-PML/SiD](https://huggingface.co/UT-Austin-PML/SiD/tree/main)\n\n\n### Acknowledgements\n\nWe extend our gratitude to the authors of the **EDM paper** for sharing their code, which served as the foundational framework for developing SiD. The repository can be found here: [NVlabs/edm](https://github.com/NVlabs/edm).\n\nAdditionally, we are thankful to the authors of the **Diff Instruct paper** for making their code available. Their contributions have been instrumental in integrating the evaluation pipeline into our training iterations. Their repository is accessible here: [pkulwj1994/diff_instruct](https://github.com/pkulwj1994/diff_instruct).\n\n\n\n### Code Contributions\n- **Mingyuan Zhou**: Led the project and wrote the majority of the code.\n- **Huangjie Zheng, Zhendong Wang, Hai Huang**: Worked closely with Mingyuan Zhou, co-developing essential components and writing various subfunctions.\n\n\n## Contributing to the Project\n\nTo contribute to this project, follow these steps:\n\n1. Fork this repository.\n2. Create a new branch: `git checkout -b \u003cbranch_name\u003e`.\n3. Make your changes and commit them: `git commit -m '\u003ccommit_message\u003e'`\n4. Push to the original branch: `git push origin \u003cproject_name\u003e/\u003clocation\u003e`\n5. Create the pull request.\n\nAlternatively, see the GitHub documentation on [creating a pull request](https://help.github.com/articles/creating-a-pull-request/).\n\n## Contact\n\nIf you want to contact me you can reach me at `mingyuanzhou@gmail.com`.\n\n## License\n\nThis project uses the following license: [Apache-2.0 license](README.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmingyuanzhou%2FSiD","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmingyuanzhou%2FSiD","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmingyuanzhou%2FSiD/lists"}