{"id":16271972,"url":"https://github.com/horseee/deepcache","last_synced_at":"2025-05-15T10:07:06.338Z","repository":{"id":210537496,"uuid":"726013572","full_name":"horseee/DeepCache","owner":"horseee","description":"[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free","archived":false,"fork":false,"pushed_at":"2024-06-27T05:47:35.000Z","size":107478,"stargazers_count":890,"open_issues_count":45,"forks_count":43,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-05-13T12:18:06.655Z","etag":null,"topics":["diffusion-models","efficient-inference","model-compression","stable-diffusion","training-free"],"latest_commit_sha":null,"homepage":"https://horseee.github.io/Diffusion_DeepCache/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/horseee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-01T10:54:04.000Z","updated_at":"2025-05-12T00:53:41.000Z","dependencies_parsed_at":"2024-06-27T06:55:13.574Z","dependency_job_id":null,"html_url":"https://github.com/horseee/DeepCache","commit_stats":null,"previous_names":["horseee/deepcache"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horseee%2FDeepCache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horseee%2FDeepCache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horseee%2FDeepCache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horseee%2FDeepCache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/horseee","download_url":"https://codeload.github.com/horseee/DeepCache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254319720,"owners_count":22051073,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diffusion-models","efficient-inference","model-compression","stable-diffusion","training-free"],"created_at":"2024-10-10T18:15:32.381Z","updated_at":"2025-05-15T10:07:01.252Z","avatar_url":"https://github.com/horseee.png","language":"Python","readme":"# DeepCache: Accelerating Diffusion Models for Free\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/horseee/Diffusion_DeepCache/blob/master/static/images/example_compress.gif\" width=\"100%\" \u003e\u003c/img\u003e\n  \u003cbr\u003e\n  \u003cem\u003e\n      (Results on Stable Diffusion v1.5. Left: 50 PLMS steps. Right: 2.3x acceleration upon 50 PLMS steps) \n  \u003c/em\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\n\u003e **DeepCache: Accelerating Diffusion Models for Free**   \n\u003e [Xinyin Ma](https://horseee.github.io/), [Gongfan Fang](https://fangggf.github.io/), [Xinchao Wang](https://sites.google.com/site/sitexinchaowang/)   \n\u003e [Learning and Vision Lab](http://lv-nus.org/), National University of Singapore  \n\u003e 🥯[[Arxiv]](https://arxiv.org/abs/2312.00858)🎄[[Project Page]](https://horseee.github.io/Diffusion_DeepCache/)\n\n### Why DeepCache  \n* 🚀 Training-free and almost lossless\n* 🚀 Support Stable Diffusion, Stable Diffusion XL, Stable Video Diffusion, Stable Diffusion Pipeline / XL Pipeline for Inpainting, Stable Diffusion Img2Img Pipeline, DDPM\n* 🚀 Compatible with sampling algorithms like DDIM and PLMS\n\n### Updates\n* **June 27, 2024**: 🔥Our new work [AsyncDiff](https://github.com/czg1225/AsyncDiff) enables parallel inference of diffusion models on multiple GPUs. Check our [paper](https://arxiv.org/abs/2406.06911) and [code](https://github.com/czg1225/AsyncDiff)!\n* **June 5, 2024**: 🔥Our new work, [Learning-to-Cache](https://github.com/horseee/learning-to-cache/tree/main), an improved version of DeepCache on DiT. Code and checkpoints are released.\n* **January 5, 2024**: 💥A doc page for DeepCache has been added in [Diffusers](https://github.com/huggingface/diffusers)! Check [here](https://huggingface.co/docs/diffusers/main/en/optimization/deepcache) for more information. Many thanks to the Diffusers team!\n* **December 26, 2023**: 🔥Update a plug-and-play implementation of DeepCache, no longer requiring any modifications of the diffuser's code! Check [here](https://github.com/horseee/DeepCache#usage) for the detailed usage! Big thanks to @[yuanshi9815](https://github.com/Yuanshi9815) for contributing the code!\n* **December 25, 2023**: A demo is available via Colab [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1COM9tfGvHSJ8Zn4tScZCGgt8nxAEOlAz?usp=sharing)\n* **December 21, 2023**: Release the code for **Stable Video Diffusion** and **Text2Video-Zero**. In the figure below, the upper row shows the original videos generated by SVD-XT, and the lower row is accelerated by DeepCache. For Text2Video-Zero, the results can be found [here](https://github.com/horseee/DeepCache/blob/master/assets/Text2Video-Zero.gif)\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/svd.gif\" width=\"90%\" \u003e\u003c/img\u003e\n  \u003cbr\u003e\n  \u003cem\u003e\n      (1.7x acceleration of SVD-XT) \n  \u003c/em\u003e\n\u003c/div\u003e\n\n\n* **December 20, 2023**: Release the code for **DDPM**. See [here](https://github.com/horseee/DeepCache/tree/master/DeepCache#experiment-code-for-ddpm) for the experimental code and instructions.\n\n* **December 6, 2023**: Release the code for **Stable Diffusion XL**. The results of the `stabilityai/stable-diffusion-xl-base-1.0` are shown in the below figure, with the same prompts from the first figure.\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"assets/sdxl.png\" width=\"90%\" \u003e\u003c/img\u003e\n  \u003cbr\u003e\n  \u003cem\u003e\n      (2.6x acceleration of Stable Diffusion XL) \n  \u003c/em\u003e\n\u003c/div\u003e\n\n\n### Introduction\nWe introduce **DeepCache**, a novel **training-free and almost lossless** paradigm that accelerates diffusion models from the perspective of model architecture. Utilizing the property of the U-Net, we reuse the high-level features while updating the low-level features in a very cheap way. DeepCache accelerates Stable Diffusion v1.5 by 2.3x with only a 0.05 decline in CLIP Score, and LDM-4-G(ImageNet) by 4.1x with a 0.22 decrease in FID.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg width=\"50%\" alt=\"image\" src=\"https://github.com/horseee/DeepCache/assets/18592211/9ce3930c-c84c-4af8-8c6a-b6803a5a7b1d\"\u003e\n\u003c/div\u003e\n\n## Quick Start\n\n### Install\n```bash\npip install DeepCache\n```\n\n### Usage\n\n```python\nimport torch\n\n# Loading the original pipeline\nfrom diffusers import StableDiffusionPipeline\npipe = StableDiffusionPipeline.from_pretrained('runwayml/stable-diffusion-v1-5', torch_dtype=torch.float16).to(\"cuda:0\")\n\n# Import the DeepCacheSDHelper\nfrom DeepCache import DeepCacheSDHelper\nhelper = DeepCacheSDHelper(pipe=pipe)\nhelper.set_params(\n    cache_interval=3,\n    cache_branch_id=0,\n)\nhelper.enable()\n\n# Generate Image\ndeepcache_image = pipe(\n        prompt,\n        output_type='pt'\n).images[0]\nhelper.disable()\n```\nWe here take the Stable Diffusion pipeline as an example. You can replace pipe with any variants of the Stable Diffusion pipeline, including choices like SDXL, SVD, and more. You can find examples in the [script](https://github.com/horseee/DeepCache#a-general-script-for-sd). The argument `cache_branch_id` specifies the selected skip branch. For the skip branches that are deeper, the model will engage them only during the caching steps, and exclude them during the retrieval steps. The argument `cache_interval` represents the interval for updating the cache.\n\n### A general script for SD\n```bash\npython main.py --model_type sdxl #Support [sdxl, sd1.5, sd2.1, svd, sd-inpaint, sdxl-inpaint, sd-img2img]\n```\n\n## Experimental code for DeepCache\nThe above implementation does not require changes to the `forward` or `__call__` functions in the Diffusers pipeline, and is, therefore, more general. The following section is the experimental code that can be used to reproduce the results in the paper. It is implemented one by one for different model structures and pipelines, and thus, may not work properly due to the update of diffusers.\n\n### Setup\n```\npip install diffusers==0.24.0 transformers\n```\n\n### Stable Diffusion XL\n```bash\npython stable_diffusion_xl.py --model stabilityai/stable-diffusion-xl-base-1.0\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eOutput:\u003c/summary\u003e\n\n```bash\nLoading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:01\u003c00:00,  6.62it/s]\n2023-12-06 01:44:28,578 - INFO - Running baseline...\n100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:17\u003c00:00,  2.93it/s]\n2023-12-06 01:44:46,095 - INFO - Baseline: 17.52 seconds\nLoading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00\u003c00:00,  8.06it/s]\n2023-12-06 01:45:02,865 - INFO - Running DeepCache...\n100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:06\u003c00:00,  8.01it/s]\n2023-12-06 01:45:09,573 - INFO - DeepCache: 6.71 seconds\n2023-12-06 01:45:10,678 - INFO - Saved to output.png. Done!\n```\n\n\u003c/details\u003e\n\nYou can add `--refine` at the end of the command to activate the refiner model for SDXL.\n\n### Stable Diffusion v1.5\n```bash\npython stable_diffusion.py --model runwayml/stable-diffusion-v1-5\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eOutput:\u003c/summary\u003e\n\n```bash\n2023-12-03 16:18:13,636 - INFO - Loaded safety_checker as StableDiffusionSafetyChecker from `safety_checker` subfolder of runwayml/stable-diffusion-v1-5.\n2023-12-03 16:18:13,699 - INFO - Loaded vae as AutoencoderKL from `vae` subfolder of runwayml/stable-diffusion-v1-5.\nLoading pipeline components...: 100%|██████████████████████████████████████████████████████████████████| 7/7 [00:01\u003c00:00,  5.88it/s]\n2023-12-03 16:18:22,837 - INFO - Running baseline...\n100%|████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:03\u003c00:00, 15.33it/s]\n2023-12-03 16:18:26,174 - INFO - Baseline: 3.34 seconds\n2023-12-03 16:18:26,174 - INFO - Running DeepCache...\n100%|████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:01\u003c00:00, 34.06it/s]\n2023-12-03 16:18:27,718 - INFO - DeepCache: 1.54 seconds\n2023-12-03 16:18:27,935 - INFO - Saved to output.png. Done!\n```\n\n\u003c/details\u003e\n\n### Stable Diffusion v2.1\n\n```bash\npython stable_diffusion.py --model stabilityai/stable-diffusion-2-1\n```\n\n\u003cdetails\u003e\n  \u003csummary\u003eOutput:\u003c/summary\u003e\n  \n```bash\n2023-12-03 16:21:17,858 - INFO - Loaded feature_extractor as CLIPImageProcessor from `feature_extractor` subfolder of stabilityai/stable-diffusion-2-1.\n2023-12-03 16:21:17,864 - INFO - Loaded scheduler as DDIMScheduler from `scheduler` subfolder of stabilityai/stable-diffusion-2-1.\nLoading pipeline components...: 100%|██████████████████████████████████████████████████████████████████| 6/6 [00:01\u003c00:00,  5.35it/s]\n2023-12-03 16:21:49,770 - INFO - Running baseline...\n100%|████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:14\u003c00:00,  3.42it/s]\n2023-12-03 16:22:04,551 - INFO - Baseline: 14.78 seconds\n2023-12-03 16:22:04,551 - INFO - Running DeepCache...\n100%|████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:08\u003c00:00,  6.10it/s]\n2023-12-03 16:22:12,911 - INFO - DeepCache: 8.36 seconds\n2023-12-03 16:22:13,417 - INFO - Saved to output.png. Done!\n```\n\n\u003c/details\u003e\n\nCurrently, our code supports the models that can be loaded by [StableDiffusionPipeline](https://huggingface.co/docs/diffusers/v0.24.0/en/api/pipelines/stable_diffusion/text2img#diffusers.StableDiffusionPipeline). You can specify the model name by the argument `--model`, which by default, is `runwayml/stable-diffusion-v1-5`. \n\n### Stable Video Diffusion \n```bash\npython stable_video_diffusion.py\n```\n\n\u003cdetails\u003e\n  \u003csummary\u003eOutput:\u003c/summary\u003e\n  \n```bash\nLoading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00\u003c00:00,  8.36it/s]\n2023-12-21 04:56:47,329 - INFO - Running baseline...\n100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [01:27\u003c00:00,  3.49s/it]\n2023-12-21 04:58:26,121 - INFO - Origin: 98.66 seconds\nLoading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00\u003c00:00, 10.59it/s]\n2023-12-21 04:58:27,202 - INFO - Running DeepCache...\n100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:49\u003c00:00,  1.96s/it]\n2023-12-21 04:59:26,607 - INFO - DeepCache: 59.31 seconds\n```\n\n\u003c/details\u003e\n\n### DDPM and LDM\nPlease check [here](https://github.com/horseee/DeepCache/tree/master/experiments) for the experimental code of DDPM and LDM.\n\n\n## Visualization\n\nImages in the upper line are the baselines and the images in the lower line are accelerated by DeepCache. \n\n### Stable Diffusion v1.5 (2.15x Acceleration)\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/horseee/Diffusion_DeepCache/blob/master/static/images/sd_examples_high_res.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n### LDM-4-G for ImageNet (6.96x Acceleration)\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/horseee/Diffusion_DeepCache/blob/master/static/images/imagenet.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n### DDPM for LSUN Church \u0026 Bedroom (1.48x Acceleration)\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/horseee/Diffusion_DeepCache/blob/master/static/images/bedroom.png\" width=\"100%\"\u003e\n  \u003cimg src=\"https://github.com/horseee/Diffusion_DeepCache/blob/master/static/images/church.png\" width=\"100%\"\u003e\n\u003c/div\u003e\n\n## Quantitative Results\n\n### ImageNet\n\u003cdiv align=\"center\"\u003e\n\u003cimg width=\"80%\" alt=\"image\" src=\"https://github.com/horseee/DeepCache/assets/18592211/151d7639-2501-45cf-8de5-2af3bb5a354b\"\u003e\n\u003c/div\u003e\n\n\n### Stable Diffusion v1.5\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg width=\"80%\" alt=\"image\" src=\"https://github.com/horseee/DeepCache/assets/18592211/e9bd7a8e-07c8-4296-95a2-12d008995807\"\u003e\n\u003c/div\u003e\n\nMore results can be found in our [paper](https://arxiv.org/abs/2312.00858)\n\n## Other Implementations and Plugins\nWe sincerely thank the authors listed below who implemented DeepCache in plugins or other contexts. \n\n* Diffusers: https://huggingface.co/docs/diffusers/main/en/optimization/deepcache\n* OneDiff Integration: https://github.com/siliconflow/onediff?tab=readme-ov-file#easy-to-use by @[SiliconFlow](https://github.com/siliconflow). OneDiff also has implementations for DeepCache on SVD, check [this](https://github.com/siliconflow/onediff/pull/438) for details.\n* Comfyui: https://gist.github.com/laksjdjf/435c512bc19636e9c9af4ee7bea9eb86 by @[laksjdjf](https://github.com/laksjdjf)\n* Colab \u0026 Gradio: https://github.com/camenduru/DeepCache-colab by @[camenduru](https://github.com/camenduru/DeepCache-colab)\n* WebUI: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14210 by @[aria1th](https://github.com/aria1th)\n* SD.Next: https://github.com/vladmandic/automatic by @[vladmandic](https://github.com/vladmandic)\n* MLX_deepcache for running SD on M1 Mac: https://github.com/Barry0121/mlx_deepcache by @[Barry Xue](https://github.com/Barry0121)\n\nWe warmly welcome contributions from everyone. Please feel free to reach out to us.\n\n\n## Bibtex\n```\n@inproceedings{ma2023deepcache,\n  title={DeepCache: Accelerating Diffusion Models for Free},\n  author={Ma, Xinyin and Fang, Gongfan and Wang, Xinchao},\n  booktitle={The IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  year={2024}\n}\n```\n\n\n\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorseee%2Fdeepcache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhorseee%2Fdeepcache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorseee%2Fdeepcache/lists"}