{"id":18850934,"url":"https://github.com/igitugraz/weatherdiffusion","last_synced_at":"2025-04-06T16:11:20.122Z","repository":{"id":50495187,"uuid":"518738546","full_name":"IGITUGraz/WeatherDiffusion","owner":"IGITUGraz","description":"Code for \"Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models\" [TPAMI 2023]","archived":false,"fork":false,"pushed_at":"2023-02-23T14:38:50.000Z","size":40,"stargazers_count":375,"open_issues_count":16,"forks_count":33,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-03-30T15:12:24.305Z","etag":null,"topics":["dehazing","denoising-diffusion-models","deraining","desnowing","diffusion-model","image-restoration"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IGITUGraz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-28T07:06:33.000Z","updated_at":"2025-03-24T08:09:50.000Z","dependencies_parsed_at":"2024-11-08T03:33:53.026Z","dependency_job_id":"1c5b948b-9e68-4505-b890-5518204ede25","html_url":"https://github.com/IGITUGraz/WeatherDiffusion","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IGITUGraz%2FWeatherDiffusion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IGITUGraz%2FWeatherDiffusion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IGITUGraz%2FWeatherDiffusion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IGITUGraz%2FWeatherDiffusion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IGITUGraz","download_url":"https://codeload.github.com/IGITUGraz/WeatherDiffusion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247509221,"owners_count":20950232,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dehazing","denoising-diffusion-models","deraining","desnowing","diffusion-model","image-restoration"],"created_at":"2024-11-08T03:32:40.284Z","updated_at":"2025-04-06T16:11:20.097Z","avatar_url":"https://github.com/IGITUGraz.png","language":"Python","readme":"# Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models\n\nThis is the code repository of the following [paper](https://arxiv.org/pdf/2207.14626.pdf) to train and perform inference with patch-based diffusion models for image restoration under adverse weather conditions.\n\n\"Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models\"\\\n\u003cem\u003eOzan Özdenizci, Robert Legenstein\u003c/em\u003e\\\nIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023.\\\nhttps://doi.org/10.1109/TPAMI.2023.3238179\n\n## Datasets\n\nWe perform experiments for image desnowing on [Snow100K](https://sites.google.com/view/yunfuliu/desnownet), combined image deraining and dehazing on [Outdoor-Rain](https://github.com/liruoteng/HeavyRainRemoval), and raindrop removal on\nthe [RainDrop](https://github.com/rui1996/DeRaindrop) datasets. To train multi-weather restoration, we used the AllWeather training set from [TransWeather](https://github.com/jeya-maria-jose/TransWeather), which is composed of subsets of training images from these three benchmarks.\n\n\n## Saved Model Weights\n\nWe share a pre-trained diffusive **multi-weather** restoration model [WeatherDiff\u003csub\u003e64\u003c/sub\u003e](https://igi-web.tugraz.at/download/OzdenizciLegensteinTPAMI2023/WeatherDiff64.pth.tar) with the network configuration in `configs/allweather.yml`.\nTo evaluate WeatherDiff\u003csub\u003e64\u003c/sub\u003e using the pre-trained model checkpoint with the current version of the repository:\n```bash\npython eval_diffusion.py --config \"allweather.yml\" --resume 'WeatherDiff64.pth.tar' --test_set 'raindrop' --sampling_timesteps 25 --grid_r 16\npython eval_diffusion.py --config \"allweather.yml\" --resume 'WeatherDiff64.pth.tar' --test_set 'rainfog' --sampling_timesteps 25 --grid_r 16\npython eval_diffusion.py --config \"allweather.yml\" --resume 'WeatherDiff64.pth.tar' --test_set 'snow' --sampling_timesteps 25 --grid_r 16\n```\n\nA smaller value for `grid_r` will yield slightly better results and higher image quality:\n```bash\npython eval_diffusion.py --config \"allweather.yml\" --resume 'WeatherDiff64.pth.tar' --test_set 'raindrop' --sampling_timesteps 25 --grid_r 4\npython eval_diffusion.py --config \"allweather.yml\" --resume 'WeatherDiff64.pth.tar' --test_set 'rainfog' --sampling_timesteps 25 --grid_r 4\npython eval_diffusion.py --config \"allweather.yml\" --resume 'WeatherDiff64.pth.tar' --test_set 'snow' --sampling_timesteps 25 --grid_r 4\n```\n\nWe also share our pre-trained diffusive multi-weather restoration model [WeatherDiff\u003csub\u003e128\u003c/sub\u003e](https://igi-web.tugraz.at/download/OzdenizciLegensteinTPAMI2023/WeatherDiff128.pth.tar) with the network configuration in `configs/allweather128.yml`.\n\nCheck out below for some visualizations of our patch-based diffusive image restoration approach.\n\n## Image Desnowing\n\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eInput Condition\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eRestoration Process\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eOutput\u003c/td\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/181769278-2ab420b3-6e81-4e9d-9d41-3c1bbbae6d7e.png\" alt=\"snow11\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351181-9528c4cb-218d-4b06-8c4c-210219ace8bc.gif\" alt=\"snow12\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/181769282-242711d5-e809-45c3-ab89-3e8fabbe1e97.png\" alt=\"snow13\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/181769267-24c7541f-670a-484c-8e44-12c5e95f1e58.png\" alt=\"snow21\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351179-b0183145-ce70-4ded-87eb-077a22c9112a.gif\" alt=\"snow22\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/181769271-08328a77-5452-4bfe-93fd-eccfcc3099c5.png\" alt=\"snow23\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n  \n## Image Deraining \\\u0026 Dehazing\n\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eInput Condition\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eRestoration Process\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eOutput\u003c/td\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/181770508-490da62c-2f73-4d4f-9a97-45c8f5f5ff66.png\" alt=\"rh11\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351163-7913703b-977f-4117-95ce-2e88397be6be.gif\" alt=\"rh12\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/181770509-24266aa7-e177-455a-bbce-6d43e71acb77.png\" alt=\"rh13\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351171-fd874818-d797-409a-9988-28824091417f.png\" alt=\"rh21\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351167-94807242-a5ba-473e-8503-11f9c294b9bf.gif\" alt=\"rh22\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351176-a9f49787-e7ed-45bc-b9ac-d6585a81bd09.png\" alt=\"rh23\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Raindrop Removal\n\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eInput Condition\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eRestoration Process\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003cb\u003eOutput\u003c/td\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/181769984-0072cb4e-c5fc-472a-8c57-58eace811521.png\" alt=\"rd11\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351153-785519aa-3df2-4141-89f3-c8837345eeb3.gif\" alt=\"rd12\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/181769987-54b1ba62-e023-4a97-9d9d-32a644037109.png\" alt=\"rd13\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351159-e9953ae1-652a-4bdd-a254-6ba823e5444d.png\" alt=\"rd21\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351158-5ccb3215-5d52-4cda-8dcf-825629fb9f1c.gif\" alt=\"rd22\"\u003e\u003c/td\u003e\n    \u003ctd\u003e \u003cimg src=\"https://user-images.githubusercontent.com/30931390/182351162-1251b9bb-da71-4d1a-9a11-a4ee5729e1b2.png\" alt=\"rd23\"\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\n## Reference\nIf you use this code or models in your research and find it helpful, please cite the following paper:\n```\n@article{ozdenizci2023,\n  title={Restoring vision in adverse weather conditions with patch-based denoising diffusion models},\n  author={Ozan \\\"{O}zdenizci and Robert Legenstein},\n  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, \n  pages={1-12},\n  year={2023},\n  doi={10.1109/TPAMI.2023.3238179}\n}\n```\n\n## Acknowledgments\n\nAuthors of this work are affiliated with Graz University of Technology, Institute of Theoretical Computer Science, and Silicon Austria Labs, TU Graz - SAL Dependable Embedded Systems Lab, Graz, Austria. This work has been supported by the \"University SAL Labs\" initiative of Silicon Austria Labs (SAL) and its Austrian partner universities for applied fundamental research for electronic based systems.\n\nParts of this code repository is based on the following works:\n\n* https://github.com/ermongroup/ddim\n* https://github.com/bahjat-kawar/ddrm\n* https://github.com/JingyunLiang/SwinIR\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Figitugraz%2Fweatherdiffusion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Figitugraz%2Fweatherdiffusion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Figitugraz%2Fweatherdiffusion/lists"}