{"id":13737891,"url":"https://github.com/wgcban/ddpm-cd","last_synced_at":"2025-04-07T05:15:47.111Z","repository":{"id":38685324,"uuid":"483710165","full_name":"wgcban/ddpm-cd","owner":"wgcban","description":"Remote Sensing Change Detection using Denoising Diffusion Probabilistic Models","archived":false,"fork":false,"pushed_at":"2024-04-17T10:01:14.000Z","size":21610,"stargazers_count":253,"open_issues_count":7,"forks_count":30,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-08-04T03:11:15.945Z","etag":null,"topics":["change-detection","climate-change","diffusion-models","remote-sensing","segmentation"],"latest_commit_sha":null,"homepage":"https://www.wgcban.com/research#h.ar24vwqlm021","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wgcban.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-20T15:27:08.000Z","updated_at":"2024-07-28T17:39:10.000Z","dependencies_parsed_at":"2023-01-21T22:48:04.055Z","dependency_job_id":"46409b8c-c99b-44e4-bffc-352a49606e47","html_url":"https://github.com/wgcban/ddpm-cd","commit_stats":{"total_commits":271,"total_committers":11,"mean_commits":"24.636363636363637","dds":0.4649446494464945,"last_synced_commit":"f10eeb46cedb48b3ce6b5f79fb3660b551aa9a3a"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wgcban%2Fddpm-cd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wgcban%2Fddpm-cd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wgcban%2Fddpm-cd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wgcban%2Fddpm-cd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wgcban","download_url":"https://codeload.github.com/wgcban/ddpm-cd/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247595335,"owners_count":20963943,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["change-detection","climate-change","diffusion-models","remote-sensing","segmentation"],"created_at":"2024-08-03T03:02:04.778Z","updated_at":"2025-04-07T05:15:47.040Z","avatar_url":"https://github.com/wgcban.png","language":"Python","funding_links":[],"categories":["Python","Multispectral"],"sub_categories":["Deep Learning"],"readme":"# DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection (WACV'25)\n(Previosely: DDPM-CD: Remote Sensing Change Detection using Denoising Diffusion Probabilistic Models)\n\n\u003e [Wele Gedara Chaminda Bandara](https://www.wgcban.com), [Nithin Gopalakrishnan Nair](https://nithin-gk.github.io), [Vishal M. Patel](https://engineering.jhu.edu/faculty/vishal-patel/)\n\n\u003e Offical Pytorch implementation of DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection / Remote Sensing Change Detection using Denoising Diffusion Probabilistic Models\n\n\u003e [Latest Version of the Paper](https://arxiv.org/abs/2206.11892)\n\n### Updates:\n- 🎉 DDPM-CD has been accepted at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2025.\n- :exclamation: Paper-v3: We have (completely) revised the wrting of our paper. Please refer to [v3 on arxiv](https://arxiv.org/abs/2206.11892v3). \n\n## 1. Motivation \u0026 Contribution\n![image-20210228153142126](./imgs/samples.jpeg)\n\n\u003e Images sampled from the DDPM model pre-trained on off-the-shelf remote sensing images.\n\n- Images generated from the pre-trained diffusion model trained on off-the-shelf remote sensing images.\n- The generated images exhibit common objects typically observed in real remote sensing imagery, including buildings, trees, roads, vegetation, water surfaces, etc.\n- This showcases the remarkable capability of diffusion models to grasp essential semantics from the training dataset.\n- Although our primary focus isn't image synthesis, we explore the effectiveness of DDPM as a feature extractor for change detection.\n\n## 2. Method\n![image-20210228153142126](./imgs/method.jpg)\n\n\u003e We fine-tune a lightweight change classifier utilizing the feature representations produced by the pre-trained DDPM alongside change labels\n\n## 3. Usage\n### 3.1 Requirements\nBefore using this repository, make sure you have the following prerequisites installed:\n\n- [Anaconda](https://www.anaconda.com/download/)\n- [PyTorch](https://pytorch.org)\n\nYou can install PyTorch with the following [command](https://pytorch.org/get-started/locally/) (in Linux OS):\n```bash\nconda install pytorch torchvision pytorch-cuda=11.8 -c pytorch -c nvidia\n```\n\n### 3.2 Installation\n\nTo get started, clone this repository:\n```bash\ngit clone https://github.com/wgcban/ddpm-cd.git\n```\n\nNext, create the [conda](https://docs.conda.io/projects/conda/en/stable/) environment named `ddpm-cd` by executing the following command:\n```bash\nconda env create -f environment.yml\n```\n\nThen activate the environment:\n```bash\nconda activate ddpm-cd\n```\n\nDownload the datasets and place them in the `dataset` folder. **-\u003e[See Section 5.1 for download links.](https://github.com/wgcban/ddpm-cd/tree/master#51-download-the-change-detection-datasets)**\n\nIf you wish to only test, download the pre-trained DDPM and fine-tuned DDPM-CD models and place them in the `experiments` folder. **-\u003e[See Section 7 for links.](https://github.com/wgcban/ddpm-cd#7-links-to-download-pre-trained-models)**\n\nAll the train-val-test statistics will be automatically upload to [`wandb`](https://wandb.ai/home), and please refer [`wandb-quick-start`](https://wandb.ai/quickstart?utm_source=app-resource-center\u0026utm_medium=app\u0026utm_term=quickstart) documentation if you are not familiar with using `wandb`. \n\n## 4. Pre-training DDPM\n### 4.1 Collect off-the-shelf remote sensing data to train diffusion model\n\nDump all the remote sensing data sampled from Google Earth Engine and any other publically available remote sensing images to dataset folder or create a [simlink](https://www.geeksforgeeks.org/python-os-symlink-method/). \n\n### 4.2 Pre-train/resume (unconditional) DDPM\n\nWe use `ddpm_train.json` to setup the configurations. Update the dataset `name` and `dataroot` in the json file. Then run the following command to start training the diffusion model. The results and log files will be save to ``experiments`` folder. Also, we upload all the metrics to [wandb](https://wandb.ai/home).\n\n```python\npython ddpm_train.py --config config/ddpm_train.json -enable_wandb -log_eval\n```\n\nIn case, if you want to resume the training from previosely saved point, provide the path to saved model in ``path/resume_state``, else keep it as `null`.\n\n### 4.3 Sampling from the pre-trained DDPM\nIf you want generate samples from the pre-trained DDPM, first update the path to trained diffusion model in [`path`][`resume_state`]. Then run the following command.\n```python\npython ddpm_train.py --config config/ddpm_sampling.json --phase val\n```\nThe generated images will be saved in `experiments`.\n\n## 5. Fine-tuning for change detection\n### 5.1 Download the change detection datasets\nDownload the change detection datasets from the following links. Place them inside your `datasets` folder.\n\n- [`LEVIR-CD`](https://www.dropbox.com/s/18fb5jo0npu5evm/LEVIR-CD256.zip?dl=0)\n- [`WHU-CD`](https://www.dropbox.com/s/r76a00jcxp5d3hl/WHU-CD-256.zip?dl=0)\n- [`DSIFN-CD`](https://www.dropbox.com/s/1lr4m70x8jdkdr0/DSIFN-CD-256.zip?dl=0)\n- [`CDD`](https://www.dropbox.com/s/ls9fq5u61k8wxwk/CDD.zip?dl=0)\n\n\nThen, update the paths to those folders here [`datasets`][`train`][`dataroot`], [`datasets`][`val`][`dataroot`], [`datasets`][`test`][`dataroot`] in `levir.json`, `whu.json`, `dsifn.json`, and `cdd.json`.\n\n### 5.2 Provide the path to pre-trained diffusion model\nUdate the path to pre-trained diffusion model weights (`*_gen.pth` and `*_opt.pth`) here [`path`][`resume_state`] in `levir.json`, `whu.json`, `dsifn.json`, and `cdd.json`..\n\n### 5.3 Indicate time-steps used for feature extraction\nIndicate the time-steps using to extract feature representations in [`model_cd`][`t`]. As shown in the ablation section of the paper, our best model is obtained with time-steps: {50,100,400}. However, time-steps of {50,100} works well too.\n\n### 5.4 Start fine-tuning for change detection\nRun the following code to start the training.\n- Training on LEVIR-CD:\n    ```python\n    python ddpm_cd.py --config config/levir.json -enable_wandb -log_eval\n    ```\n- Training on WHU-CD:\n    ```python\n    python ddpm_cd.py --config config/whu.json -enable_wandb -log_eval\n    ```\n- Training on DSIFN-CD:\n    ```python\n    python ddpm_cd.py --config config/dsifn.json -enable_wandb -log_eval\n    ```\n- Training on CDD:\n    ```python\n    python ddpm_cd.py --config config/cdd.json -enable_wandb -log_eval\n    ```\n\nThe results will be saved in `experiments` and also upload to `wandb`.\n\n## 6. Testing\nTo obtain the predictions and performance metrics (IoU, F1, and OA), first provide the path to pre-trained diffusion model here [`path`][`resume_state`] and path to trained change detection model (the best model) here [`path_cd`][`resume_state`] in `levir_test.json`, `whu_test.json`, `dsifn_test.json`, and `cdd_test.json`. Also make sure you specify the time steps used in fine-tuning here: [`model_cd`][`t`].\n\nRun the following code to start the training.\n- Test on LEVIR-CD:\n    ```python\n    python ddpm_cd.py --config config/levir_test.json --phase test -enable_wandb -log_eval\n    ```\n- Test on WHU-CD:\n    ```python\n    python ddpm_cd.py --config config/whu_test.json --phase test -enable_wandb -log_eval\n    ```\n- Test on DSIFN-CD:\n    ```python\n    python ddpm_cd.py --config config/dsifn_test.json --phase test -enable_wandb -log_eval\n    ```\n- Test on CDD:\n    ```python\n    python ddpm_cd.py --config config/cdd_test.json --phase test -enable_wandb -log_eval\n    ```\n\nPredictions will be saved in `experiments` and performance metrics will be uploaded to wandb.\n\n## 7. Links to download pre-trained models\n### 7.1 Pre-trianed DDPM\nPre-trained diffusion model can be download from: [`Dropbox`](https://www.dropbox.com/sh/z6k5ixlhkpwgzt5/AAApBOGEUhHa4qZon0MxUfmua?dl=0)\n\n### 7.2 Fine-tuned DDPM-CD models\nFine-tunes chande detection networks can be download from following links:\n- \"t\": [50, 100]\n    - LEVIR-CD [`Dropbox-cd-levir-50-100`](https://www.dropbox.com/sh/ie9rapb1j2zgvb7/AAALkpLS-tvngTb4HXqAcbbTa?dl=0)\n    - WHU-CD [`Dropbox-cd-whu-50-100`](https://www.dropbox.com/sh/9idrobnmhufo1e7/AABRf38iq-wE7plKZZmwFywva?dl=0)\n    - DSIFN-CD [`Dropbox-cd-dsifn-50-100`](https://www.dropbox.com/sh/001czxn335bul5g/AACRaR-nqQNNHEge6iSH_z-6a?dl=0)\n    - CDD-CD [`Dropbox-cd-cdd-50-100`](https://www.dropbox.com/sh/62wsy9cl8xizx2h/AAB5Dmu-PuOVAfIBugGqlsd8a?dl=0)\n\n- \"t\": [50, 100, 400] (*Best Model*)\n    - LEVIR-CD [`Dropbox-cd-levir-50-100-400`](https://www.dropbox.com/sh/sx0aopz230lbuwc/AADKpwP30OHvtYub9FYTyk53a?dl=0)\n    - WHU-CD [`Dropbox-cd-whu-50-100-400`](https://www.dropbox.com/sh/l8iuzb2tudb3yrk/AAA7aZwb5eM12SamCXPh7R-Ra?dl=0)\n    - DSIFN-CD [`Dropbox-cd-dsifn-50-100-400`](https://www.dropbox.com/sh/ekj7kwsohhnjico/AADuz0vBtxCCrYgdgOCG3LX5a?dl=0)\n    - CDD-CD [`Dropbox-cd-cdd-50-100-400`](https://www.dropbox.com/sh/a8dj1i8pnexd5yu/AADnmBGT4VdGY8aZMo7enfS7a?dl=0)\n\n- \"t\": [50, 100, 400, 650]\n    - LEVIR-CD [`Dropbox-cd-levir-50-100-400-650`](https://www.dropbox.com/sh/sx0aopz230lbuwc/AADKpwP30OHvtYub9FYTyk53a?dl=0)\n    - WHU-CD [`Dropbox-cd-whu-50-100-400-650`](https://www.dropbox.com/sh/l8iuzb2tudb3yrk/AAA7aZwb5eM12SamCXPh7R-Ra?dl=0)\n    - DSIFN-CD [`Dropbox-cd-dsifn-50-100-400-650`](https://www.dropbox.com/sh/ekj7kwsohhnjico/AADuz0vBtxCCrYgdgOCG3LX5a?dl=0)\n    - CDD-CD [`Dropbox-cd-cdd-50-100-400-650`](https://www.dropbox.com/sh/a8dj1i8pnexd5yu/AADnmBGT4VdGY8aZMo7enfS7a?dl=0)\n \n ### 7.2 Downloading from GoogleDrive/GitHub\n If you face a problem when downloading from the DropBox try one of the following options:\n - [GoogleDrive] All pre-trained models in **GooleDrive**: [GoogleDrive-pretrianed-models](https://drive.google.com/file/d/1RXWtGdSNCBEAf7nr61uNyZP6HwNl_Zyi/view?usp=sharing)\n - [GitHub] [Pre-trained-models in GitHub](https://github.com/wgcban/ddpm-cd/releases/tag/initial_release)\n \n\n### 7.3 Train/Val Reports on `wandb`\n- [`LEVIR-CD-Train-Val-Reports-Wandb`](https://wandb.ai/wgcban/ddpm-RS-CDHead/reports/Change-Detection-Results-on-LEVIR-CD-Dataset--VmlldzoyMDE5MzIz?accessToken=3hubg8q23d3527klbojjdhklo8h66k5k1acrly6jtoxd7du35vwyci9dwv8urmin)\n- [`WHU-CD-Train-Val-reports-Wandb`](https://wandb.ai/wgcban/ddpm-RS-CDHead/reports/Change-Detection-on-WHU-CD-Dataset--VmlldzoyMDE5NDA0?accessToken=5d8a9q6g008ct94lx5171knen1dd9xpptzohe92ic65rx3wflkciq1rhbp4bozca)\n- [`DSIFN-CD-Train-Val-Reports-Wandb`](https://wandb.ai/wgcban/ddpm-RS-CDHead/reports/Change-Detection-on-DSIFN-CD-Dataset--VmlldzoyMDE5NDMy?accessToken=hfef99pxr03pi4zxmcw3jkpo2na1sd1c5t7stai2vl76908fnh3wnrhcy4mfoaae)\n- [`CDD-CD-Train-Val-Reports-Wandb`](https://wandb.ai/wgcban/ddpm-RS-CDHead/reports/Change-Detection-on-CDD-Dataset--VmlldzoyMDE5NDQw?accessToken=l4omatpi7jng6mw32hp7oh0wkqet8jne3wkqrb6hxigpjluv4yy9yzdir62ics9y)\n\n### 7.4 Test results on `wandb`\n- [`LEVIR-WHU-DSIFN-CDD-Test-Results`](https://wandb.ai/wgcban/ddpm-RS-CDHead/reports/Change-Detection-Performance-on-Test-sets-of-LEVIR-CD-WHU-CD-DSIFN-CD-and-CDD--VmlldzoyMDE5NDg5?accessToken=6eikgovmk7ct25ar00eggsuslh8bzdz9e8215qn5xa0omqe5uo5u1jf4lh2liajx)\n\n## 8. Results\n### 8.1 Quantitative\n![image-20210228153142126](./imgs/results.png)\n\n\u003e The average quantitative change detection results on the LEVIR-CD, WHU-CD, DSIFN-CD, and CDD test- sets. “-” indicates not reported or not available to us. (IN1k) indicates pre-training process is initialized with the ImageNet pre-trained weights. IN1k, IBSD, and GE refers to ImageNet1k, Inria Building Segmentation Dataset, and Google Earth.\n\n\n### 8.2 Qualitative\n- LEVIR-CD\n\n    ![image-20210228153142126](./imgs/levir.png)\n\n    \u003e (a) Pre-change image, (b) Post-change image, (c) FC-EF, (d) FC-Siam-diff, (e) FC-Siam- conc, (f) DT-SCN, (g) BIT, (h) ChangeFormer, (i) DDPM-CD (ours), and (j) Ground-truth. Note: true positives (change class) are indicated in white, true negatives (no-change class) are indicated in black, and false positives plus false negatives indicates in red.\n\n- WHU-CD\n\n    ![image-20210228153142126](./imgs/whu.png)\n\n    \u003e (a) Pre-change image, (b) Post-change image, (c) FC-EF, (d) FC-Siam-diff, (e) FC-Siam- conc, (f) DT-SCN, (g) BIT, (h) ChangeFormer, (i) DDPM-CD (ours), and (j) Ground-truth. Note: true positives (change class) are indicated in white, true negatives (no-change class) are indicated in black, and false positives plus false negatives indicates in red\n\n- DSIFN-CD\n\n    ![image-20210228153142126](./imgs/dsifn.png)\n\n    \u003e (a) Pre-change image, (b) Post-change image, (c) FC-EF, (d) FC-Siam-diff, (e) FC-Siam- conc, (f) DT-SCN, (g) BIT, (h) ChangeFormer, (i) DDPM-CD (ours), and (j) Ground-truth. Note: true positives (change class) are indicated in white, true negatives (no-change class) are indicated in black, and false positives plus false negatives indicates in red\n\n- CDD\n\n    ![image-20210228153142126](./imgs/cdd.png)\n\n    \u003e (a) Pre-change image, (b) Post-change image, (c) FC-EF, (d) FC-Siam-diff, (e) FC-Siam- conc, (f) DT-SCN, (g) BIT, (h) ChangeFormer, (i) DDPM-CD (ours), and (j) Ground-truth. Note: true positives (change class) are indicated in white, true negatives (no-change class) are indicated in black, and false positives plus false negatives indicates in red\n\n\n\n## 9. Citation\n```bibtex\n@misc{bandara2024ddpmcdv2,\n    title = {Remote Sensing Change Detection (Segmentation) using Denoising Diffusion Probabilistic Models},\n    author = {Bandara, Wele Gedara Chaminda and Nair, Nithin Gopalakrishnan and Patel, Vishal M.},\n    year = {2022},\n    eprint={2206.11892},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV},\n    doi = {10.48550/ARXIV.2206.11892},\n}\n```\n```bibtex\n@misc{bandara2024ddpmcdv3,\n    title={DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection}, \n    author={Wele Gedara Chaminda Bandara and Nithin Gopalakrishnan Nair and Vishal M. Patel},\n    year={2024},\n    eprint={2206.11892},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV},\n    doi = {10.48550/ARXIV.2206.11892},\n}\n```\n\n## 10. References\n- The code of diffusion model is from [`here`](https://github.com/Janspiry/Image-Super-Resolution-via-Iterative-Refinement).\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwgcban%2Fddpm-cd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwgcban%2Fddpm-cd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwgcban%2Fddpm-cd/lists"}