{"id":24483723,"url":"https://github.com/shineiarakawa/patch-based-ddpm","last_synced_at":"2025-07-16T01:04:22.798Z","repository":{"id":242771307,"uuid":"810281989","full_name":"ShineiArakawa/patch-based-ddpm","owner":"ShineiArakawa","description":"Official PyTorch implementation of the CVPRW 2023 paper","archived":false,"fork":false,"pushed_at":"2025-05-22T10:49:56.000Z","size":713,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-22T12:36:19.372Z","etag":null,"topics":["ddpm","denoising-diffusion-models","diffusion-model","memory-efficient","patch-based"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ShineiArakawa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-04T11:50:54.000Z","updated_at":"2025-05-22T10:49:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"ab59a1ff-1353-402e-b1b9-09c0832c3e0f","html_url":"https://github.com/ShineiArakawa/patch-based-ddpm","commit_stats":null,"previous_names":["shineiarakawa/patch-based-ddpm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ShineiArakawa/patch-based-ddpm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShineiArakawa%2Fpatch-based-ddpm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShineiArakawa%2Fpatch-based-ddpm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShineiArakawa%2Fpatch-based-ddpm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShineiArakawa%2Fpatch-based-ddpm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ShineiArakawa","download_url":"https://codeload.github.com/ShineiArakawa/patch-based-ddpm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShineiArakawa%2Fpatch-based-ddpm/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265472473,"owners_count":23771886,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ddpm","denoising-diffusion-models","diffusion-model","memory-efficient","patch-based"],"created_at":"2025-01-21T13:12:13.429Z","updated_at":"2025-07-16T01:04:22.788Z","avatar_url":"https://github.com/ShineiArakawa.png","language":"Python","readme":"# Patch-based DDPM\u003cbr\u003e\u003csub\u003eOfficial PyTorch implementation of the CVPRW 2023 paper\u003c/sub\u003e\n\n![Teaser image](./assets/teaser.jpg)\n\n**Memory Efficient Diffusion Probabilistic Models via Patch-based Generation**\u003cbr\u003e\n[Shinei Arakawa](https://shineiarakawa.github.io/). Hideki Tsunashima. Daichi Horita. [Keitaro Tanaka](https://sites.google.com/view/keitarotanaka/). Shigeo Morishima\u003cbr\u003e\nhttps://generative-vision.github.io/workshop-CVPR-23/data/9.pdf\u003cbr\u003e\n\nAbstract: *Diffusion probabilistic models have been successful in generating high-quality and diverse images. However, traditional models, whose input and output are high-resolution images, suffer from excessive memory requirements, making them less practical for edge devices. Previous approaches for generative adversarial networks proposed a patch-based method that uses positional encoding and global content information. Nevertheless, designing a patch-based approach for diffusion probabilistic models is non-trivial. In this paper, we resent a diffusion probabilistic model that generates images on a patch-by-patch basis. We propose two conditioning methods for a patch-based generation. First, we propose position-wise conditioning using one-hot representation to ensure patches are in proper positions. Second, we propose Global Content Conditioning (GCC) to ensure patches have coherent content when concatenated together. We evaluate our model qualitatively and quantitatively on CelebA and LSUN bedroom datasets and demonstrate a moderate trade-off between maximum memory consumption and generated image quality. Specifically, when an entire image is divided into 2 x 2 patches, our proposed approach can reduce the maximum memory consumption by half while maintaining comparable image quality.*\n\n## Setup\n### Requirements\n- Python 3.10 (or later)\n- Pipenv\n\n### Install dependencies\nWe provide `Pipfile` to setup all dependencies through `pipenv` library. Make sure `pipenv` is already installed and then run the following command to install libraries and launch a new shell in the project root directory. This might take a few minites.\n```bash\npipenv install\npipenv shell\n```\n## Pre-trained models\nWe provide the following pre-trained weights. You can quickly run sampling by specifying the path to your downloaded pre-trained weights.\n\n- ~~2 x 2 patch division on CelebA~~\n- 4 x 4 patch division on CelebA: [here](https://drive.google.com/file/d/14ir8mTkjlUOkjnDjknCFD8Z14_Xi7Wn2/view?usp=sharing)\n- ~~8 x 8 patch division on CelebA~~\n- ~~2 x 2 patch division on LSUN bedroom~~\n- ~~4 x 4 patch division on LSUN bedroom~~\n- ~~8 x 8 patch division on LSUN bedroom~~\n\n## Getting Started\nWe use `accelerate` library to manage multi-GPU configurations. Before moving on to the next section, make sure to configure `accelerate` settings by the following command.\n```bash\naccelerate config\n```\n\n### Sampling\nRun the following command to sample images from the pre-trained models.\n```bash\naccelerate launch -m patch_based_ddpm.sample \\\n    --config ${CONFIG_FILE} \\\n    --ckpt ${CHECKPOINT} \\\n    --out-dir ${OUT_DIR} \\\n    --n-samples ${NUM_SAMPLES} \\\n    --batch-size ${BATCH_SIZE}\n```\n\nFor example, if you are running a 4x4 divided patch-based DDPM, run the command like this.\n```bash\naccelerate launch -m patch_based_ddpm.sample \\\n    --config configs/config_patch_divider=4.json \\\n    --ckpt logs/celeba128/checkpoints/model-last.pt \\\n    --out-dir sampled/celeba128 \\\n    --n-samples 16 \\\n    --batch-size 8\n```\n\n### Training\nRun the following command to train the models. During training, `wandb` automatically records all the training stats. Please make sure to log in to `wandb` with your account beforehand.\n```bash\naccelerate launch -m patch_based_ddpm.train \\\n    --config ${CONFIG_FILE}\n```\n\nFor example, if you are running a 4x4 divided patch-based DDPM, run the command like this.\n```bash\naccelerate launch -m patch_based_ddpm.train \\\n    --config configs/config_patch_divider=4.json\n```\n\nTraining with 12.8 million images takes approximately 20 hours using four NVIDIA RTX A6000 GPUs.\n\n## Citation\n```\n@misc{arakawa2023memory,\n      title={Memory Efficient Diffusion Probabilistic Models via Patch-based Generation}, \n      author={Shinei Arakawa and Hideki Tsunashima and Daichi Horita and Keitaro Tanaka and Shigeo Morishima},\n      year={2023},\n      eprint={2304.07087},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\n## Acknowledgements\nWe thank Asst. Prof. Qi Feng for valuable feedbacks as well as English proofreading and Yoshiki Kubotani for feedbacks regarding the graphical design.\nThis research is supported by the Japan Society for the Promotion of Science KAKENHI Grant Number 21H05054.\nThis implementation is heavily based on the repo [denoising-diffusion-pytorch](https://github.com/lucidrains/denoising-diffusion-pytorch).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshineiarakawa%2Fpatch-based-ddpm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshineiarakawa%2Fpatch-based-ddpm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshineiarakawa%2Fpatch-based-ddpm/lists"}