{"id":18573557,"url":"https://github.com/prs-eth/diffusion-super-resolution","last_synced_at":"2025-04-07T06:05:36.293Z","repository":{"id":148963030,"uuid":"616333632","full_name":"prs-eth/Diffusion-Super-Resolution","owner":"prs-eth","description":"[CVPR 2023] Guided Depth Super-Resolution by Deep Anisotropic Diffusion","archived":false,"fork":false,"pushed_at":"2024-12-24T13:00:28.000Z","size":608,"stargazers_count":100,"open_issues_count":1,"forks_count":10,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-31T05:03:00.426Z","etag":null,"topics":["cvpr2023","depth","super-resolution"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prs-eth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-20T07:19:45.000Z","updated_at":"2025-03-13T16:21:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"dada83a9-6a07-4c6e-bf51-c29075ee2c66","html_url":"https://github.com/prs-eth/Diffusion-Super-Resolution","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FDiffusion-Super-Resolution","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FDiffusion-Super-Resolution/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FDiffusion-Super-Resolution/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FDiffusion-Super-Resolution/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prs-eth","download_url":"https://codeload.github.com/prs-eth/Diffusion-Super-Resolution/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247601447,"owners_count":20964864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cvpr2023","depth","super-resolution"],"created_at":"2024-11-06T23:10:51.052Z","updated_at":"2025-04-07T06:05:36.249Z","avatar_url":"https://github.com/prs-eth.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\n\u003ch2 align=\"center\"\u003e Guided Depth Super-Resolution by Deep Anisotropic Diffusion [CVPR2023] \u003c/h2\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://nandometzger.github.io/\"\u003e\u003cstrong\u003eNando Metzger\u003c/strong\u003e\u003c/a\u003e*,\n    \u003ca href=\"https://rcdaudt.github.io/\"\u003e\u003cstrong\u003eRodrigo Caye Daudt\u003c/strong\u003e\u003c/a\u003e*,\n    \u003ca href=\"https://igp.ethz.ch/personen/person-detail.html?persid=143986\"\u003e\u003cstrong\u003eKonrad Schindler\u003c/strong\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n{metzgern, rodrigo.cayedaudt, schindler}@ethz.ch, Photogrammetry and Remote Sensing, ETH Zürich\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n*Equal Contribution\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n[\u003ca href=\"https://arxiv.org/abs/2211.11592\"\u003e\u003cstrong\u003ePaper\u003c/strong\u003e\u003c/a\u003e]\n[\u003ca href=\"https://www.youtube.com/watch?v=7RgXJz_3kcg\"\u003e\u003cstrong\u003eVideo\u003c/strong\u003e\u003c/a\u003e]\n[\u003ca href=\"https://rcdaudt.github.io/dada/\"\u003e\u003cstrong\u003eProject Page\u003c/strong\u003e\u003c/a\u003e]\n[\u003ca href=\"https://openaccess.thecvf.com/content/CVPR2023/html/Metzger_Guided_Depth_Super-Resolution_by_Deep_Anisotropic_Diffusion_CVPR_2023_paper.html\"\u003e\u003cstrong\u003eProceeding\u003c/strong\u003e\u003c/a\u003e]\n[\u003ca href=\"https://drive.google.com/file/d/1xHmnH5F0ckmtB7OLWYtQ6oPF7oo79m8l/view?usp=sharing\"\u003e\u003cstrong\u003ePoster\u003c/strong\u003e\u003c/a\u003e]\n\u003c/p\u003e\n  \nPerforming super-resolution of a depth image using the guidance from an RGB image is a problem that concerns several fields, such as robotics, medical imaging, and remote sensing. While deep learning methods have achieved good results in this problem, recent work highlighted the value of combining modern methods with more formal frameworks. In this work we propose a novel approach which combines guided anisotropic diffusion with a deep convolutional network and advances the state of the art for guided depth super-resolution. The edge transferring/enhancing properties of the diffusion are boosted by the contextual reasoning capabilities of modern networks, and a strict adjustment step guarantees perfect adherence to the source image. We achieve unprecedented results in three commonly used benchmarks for guided depth super resolution. The performance gain compared to other methods is the largest at larger scales, such as x32 scaling. Code for the proposed method will be made available to promote reproducibility of our results.\n\n![Teaser](images/teaser_new.png)\n\n\n## 🛠️ Setup\n\n### 🐍💓 Dependencies\nWe recommend creating a new conda environment with all required dependencies by running\n```bash\nconda env create -f environment.yml\nconda activate DADA-SR\n```\n\n### 💾🦌 Data\nTo reproduce our results, create a data directory (e.g. `./datafolder`) with the three datasets:\n* **Middlebury**: Download the 2005-2014 scenes (full size, two-view) from [[here]](https://vision.middlebury.edu/stereo/data/) and place the extracted scenes in `./datafolder/Middlebury/\u003cyear\u003e/\u003cscene\u003e`. For the 2005 dataset, make sure to only put the scenes for which ground truth is available. The data splits are defined in code.\n* **NYUv2**: Download the labeled dataset from [[here]](https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html) and place the `nyu_depth_v2_labeled.mat` in `./datafolder/NYU Depth v2`, the split file is already provided in this repository at `data/split_idc_nyuv2.json`.\n* **DIML**: Download the indoor data sample from [[here]](https://dimlrgbd.github.io) and extract it into `./data/DIML/{train,test}` respectively. Then run `python scripts/create_diml_npy.py ./datafolder/DIML` to create numpy binary files for faster data loading.\n\nYour folder structure should look like this:\n```\ndatafolder\n└───Middlebury\n│   └───2005\n│   │   └───Art\n│   │   |   |   disp1.png\n│   │   |   |   disp5.png\n│   │   |   |   dmin.text\n│   │   |   └───Illum1\n│   │   |   └───Illum2\n│   │   |   └───Illum3\n│   │   │   ...\n│   └───...\n│   \n└───NYUv2\n│       nyu_depth_v2_labeled.mat\n│\n└───DIML\n│   └───train\n│   │   └───HR\n│   │   └───LR\n│   └───test\n│   │   └───HR\n│   │   └───LR\n│   └───npy\n│       \u003c...will be generated by create_diml_npy.py...\u003e\n\n```\n\n### 🚩💾 Checkpoints\nOur pretrained model checkpoints can be downloaded [here](https://drive.google.com/file/d/1pfJ94vkDl6OQQ4QYItRxEbEjPhalrs_5/view?usp=share_link).\n\n## 🏋️ Training\n\nRun the training script via\n```bash\npython run_train.py --dataset Middlebury --data-dir ./datafolder/ --save-dir ./save_dir/ --wandb --num-epochs 4500 --scaling 8 --val-every-n-epochs 10 --lr-step 100 --in-memory\npython run_train.py --dataset NYUv2 --data-dir ./datafolder/ --save-dir ./save_dir/ --wandb --num-epochs 550 --scaling 8 --val-every-n-epochs 4 --lr-step 10 --in-memory\npython run_train.py --dataset DIML --data-dir ./datafolder/ --save-dir ./save_dir/ --wandb --num-epochs 300 --scaling 8 --val-every-n-epochs 2 --lr-step 6 --in-memory\n```\nor use the prepared setups in ```.vscode/launch.json```. \n\nNote, that depending on the dataset, you have to adjust the number of epochs (`--num-epochs`) and the scheduler step size (`--lr-step`), as listed in the commands above. You can see all available training options by running \n```bash\npython run_train.py -h\n```\n\n## 🧪 Testing\n\nFor test set evaluation, run\n\n```bash\npython run_eval.py --dataset \u003c...\u003e --data-dir ./datafolder/ --checkpoint ./save_dir/experiment_\u003c...\u003e/best_model.pth --scaling \u003c...\u003e\n```\nAgain, you can query all available options by running \n```bash\npython run_eval.py -h\n```\n\n## 🎓 Citation\n\n```\n@article{metzger2022guided,\n  title={Guided Depth Super-Resolution by Deep Anisotropic Diffusion},\n  author={Metzger, Nando and Daudt, Rodrigo Caye and Schindler, Konrad},\n  journal={arXiv preprint arXiv:2211.11592},\n  year={2022}\n}\n```\n\n## 🙏 Acknowledgement\n - A big thank you to the authors of the [Learned Graph Regularizer](https://github.com/prs-eth/graph-super-resolution) for providing the code framework.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprs-eth%2Fdiffusion-super-resolution","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprs-eth%2Fdiffusion-super-resolution","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprs-eth%2Fdiffusion-super-resolution/lists"}