{"id":13706288,"url":"https://github.com/ShuweiShao/IEBins","last_synced_at":"2025-05-05T20:30:39.729Z","repository":{"id":196519219,"uuid":"694969983","full_name":"ShuweiShao/IEBins","owner":"ShuweiShao","description":"[NeurIPS2023] IEBins: Iterative Elastic Bins for Monocular Depth Estimation","archived":false,"fork":false,"pushed_at":"2024-12-30T11:10:10.000Z","size":1445,"stargazers_count":84,"open_issues_count":3,"forks_count":5,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-12-30T12:20:07.785Z","etag":null,"topics":["computer-vision","deep-learning","depth-estimation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ShuweiShao.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-22T04:33:42.000Z","updated_at":"2024-12-30T11:10:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"df10f0bd-64f7-4cf6-a2d2-37fe880134d5","html_url":"https://github.com/ShuweiShao/IEBins","commit_stats":null,"previous_names":["shuweishao/iebins"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShuweiShao%2FIEBins","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShuweiShao%2FIEBins/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShuweiShao%2FIEBins/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShuweiShao%2FIEBins/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ShuweiShao","download_url":"https://codeload.github.com/ShuweiShao/IEBins/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252570866,"owners_count":21769735,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","deep-learning","depth-estimation"],"created_at":"2024-08-02T22:00:53.847Z","updated_at":"2025-05-05T20:30:39.709Z","avatar_url":"https://github.com/ShuweiShao.png","language":"Python","funding_links":[],"categories":["Papers"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003ch1\u003eIEBins: Iterative Elastic Bins for Monocular Depth Estimation\u003c/h1\u003e\n\n\u003cdiv\u003e\n    \u003ca href='https://scholar.google.com.hk/citations?hl=zh-CN\u0026user=ecZHSVQAAAAJ' target='_blank'\u003eShuwei Shao\u003c/a\u003e\u003csup\u003e1\u003c/sup\u003e\u0026emsp;\n    \u003ca target='_blank'\u003eZhongcai Pei\u003c/a\u003e\u003csup\u003e1\u003c/sup\u003e\u0026emsp;\n    \u003ca target='_blank'\u003eXingming Wu\u003c/a\u003e\u003csup\u003e1\u003c/sup\u003e\u0026emsp;\n    \u003ca target='_blank'\u003eZhong Liu\u003c/a\u003e\u003csup\u003e1\u003c/sup\u003e\u0026emsp;\n    \u003ca href='https://scholar.google.com.hk/citations?hl=zh-CN\u0026user=5PoZrcYAAAAJ' target='_blank'\u003eWeihai Chen\u003c/a\u003e\u003csup\u003e2\u003c/sup\u003e\u0026emsp;\n    \u003ca href='https://scholar.google.com.hk/citations?hl=zh-CN\u0026user=LiUX7WQAAAAJ' target='_blank'\u003eZhengguo Li\u003c/a\u003e\u003csup\u003e3\u003c/sup\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n    \u003csup\u003e1\u003c/sup\u003eBeihang University, \u003csup\u003e2\u003c/sup\u003eAnhui University, \u003csup\u003e3\u003c/sup\u003eA*STAR\n\u003c/div\u003e\n\n\n\u003cdiv\u003e\n    \u003ch4 align=\"center\"\u003e\n        • \u003ca href=\"https://arxiv.org/abs/2309.14137\" target='_blank'\u003eNeurIPS 2023\u003c/a\u003e •\n    \u003c/h4\u003e\n\u003c/div\u003e\n\n[![KITTI Benchmark](https://img.shields.io/badge/KITTI%20Benchmark-2nd%20among%20all%20at%20submission%20time-blue)](https://www.cvlibs.net/datasets/kitti/eval_depth.php?benchmark=depth_prediction)\n[![Hugging Space Badge](https://img.shields.io/badge/🤗-Open%20In%20Spaces-blue.svg)](https://huggingface.co/spaces/umuthopeyildirim/IEBins-Depth-Estimation)\n\n## Abstract\n\n\u003cdiv style=\"text-align:center\"\u003e\n\u003cimg src=\"assets/teaser.jpg\"  width=\"80%\" height=\"80%\"\u003e\n\u003c/div\u003e\n\n\u003c/div\u003e\n\u003cstrong\u003eWe propose a novel concept of iterative elastic bins for the classification-regression-based MDE. The proposed IEBins aims to search for high-quality depth by progressively optimizing the search range, which involves multiple stages and each stage performs a finer-grained depth search in the target bin on top of its previous stage. To alleviate the possible error accumulation during the iterative process, we utilize a novel elastic target bin to replace the original target bin, the width of which is adjusted elastically based on the depth uncertainty. \u003c/strong\u003e\n\n---\n\n\u003c/div\u003e\n\n## Installation\n```\nconda create -n iebins python=3.8\nconda activate iebins\nconda install pytorch=1.10.0 torchvision cudatoolkit=11.1\npip install matplotlib, tqdm, tensorboardX, timm, mmcv, open3d\n```\n\n## Datasets\nYou can prepare the datasets KITTI and NYUv2 according to [here](https://github.com/cleinc/bts/tree/master/pytorch) and download the SUN RGB-D dataset from [here](https://rgbd.cs.princeton.edu/), and then modify the data path in the config files to your dataset locations.\n\n\n## Training\nFirst download the pretrained encoder backbone from [here](https://github.com/microsoft/Swin-Transformer), and then modify the pretrain path in the config files. If you want to train the KITTI_Official model, first download the pretrained encoder backbone from [here](https://drive.google.com/file/d/1qjDnMwmEz0k0XWh7GP2aNPGiAjvOPF_5/view?usp=drive_link), which is provided by [MIM](https://github.com/SwinTransformer/MIM-Depth-Estimation).\n\nTraining the NYUv2 model:\n```\npython iebins/train.py configs/arguments_train_nyu.txt\n```\n\nTraining the KITTI_Eigen model:\n```\npython iebins/train.py configs/arguments_train_kittieigen.txt\n```\n\nTraining the KITTI_Official model:\n```\npython iebins_kittiofficial/train.py configs/arguments_train_kittiofficial.txt\n```\n\n## Evaluation\nEvaluate the NYUv2 model:\n```\npython iebins/eval.py configs/arguments_eval_nyu.txt\n```\n\nEvaluate the NYUv2 model on the SUN RGB-D dataset:\n```\npython iebins/eval_sun.py configs/arguments_eval_sun.txt\n```\n\nEvaluate the KITTI_Eigen model:\n```\npython iebins/eval.py configs/arguments_eval_kittieigen.txt\n```\n\nTo generate KITTI Online evaluation data for the KITTI_Official model:\n```\npython iebins_kittiofficial/test.py --data_path path to dataset --filenames_file ./data_splits/kitti_official_test.txt --max_depth 80 --checkpoint_path path to pretrained checkpoint  --dataset kitti --do_kb_crop\n```\n\n## Qualitative Depth and Point Cloud Results\nYou can download the qualitative depth results of [IEBins](https://arxiv.org/abs/2309.14137), [NDDepth](https://arxiv.org/abs/2309.10592), [NeWCRFs](https://openaccess.thecvf.com/content/CVPR2022/html/Yuan_Neural_Window_Fully-Connected_CRFs_for_Monocular_Depth_Estimation_CVPR_2022_paper.html), [PixelFormer](https://openaccess.thecvf.com/content/WACV2023/html/Agarwal_Attention_Attention_Everywhere_Monocular_Depth_Prediction_With_Skip_Attention_WACV_2023_paper.html), [AdaBins](https://openaccess.thecvf.com/content/CVPR2021/html/Bhat_AdaBins_Depth_Estimation_Using_Adaptive_Bins_CVPR_2021_paper.html) and [BTS](https://arxiv.org/abs/1907.10326) on the test sets of NYUv2 and KITTI_Eigen from [here](https://pan.baidu.com/s/1zaFe40mwpQ5cvdDlLZRrCQ?pwd=vfxd) and download the qualitative point cloud results of IEBins, NDDepth, NeWCRFS, PixelFormer, AdaBins and BTS on the NYUv2 test set from [here](https://pan.baidu.com/s/1WwpFuPBGBUaSGPEdThJ6Rw?pwd=n9rw). \n\nIf you want to derive these results by yourself, please refer to the test.py.\n\nIf you want to perform inference on a single image, run:\n```\npython iebins/inference_single_image.py --dataset kitti or nyu --image_path path to image --checkpoint_path path to pretrained checkpoint --max_depth 80 or 10\n```\nThen you can acquire the qualitative depth result.\n\n## Gradio Demo\n\n1. Install the gradio and other required libraries\n\n```python\npip install gradio gradio_imageslider timm -q\n```\n\n2. Run the demo\n\n```python\npython notebooks/app.py\n```\n\n\n## Models\n| Model | Abs Rel | Sq Rel | RMSE | a1 | a2 | a3| Link|\n| ------------ | :---: | :---: | :---: |  :---: |  :---: |  :---: |  :---: |\n|NYUv2 (Swin-L)| 0.087 | 0.040 | 0.314 | 0.936 | 0.992 | 0.998 |[[Google]](https://drive.google.com/file/d/14Rn-vxvpXO2EXRaWqCPmh2JufvOurwtl/view?usp=drive_link)|\n|NYUv2 (Swin-T)| 0.108 | 0.061 | 0.375 | 0.893 | 0.984 | 0.996 |[[Google]](https://drive.google.com/file/d/1eYkTb3grbDitQ9tJdg1DhAOaGmqgHWHK/view?usp=drive_link)|\n|KITTI_Eigen (Swin-L)| 0.050 | 0.142 | 2.011 | 0.978 | 0.998 | 0.999 |[[Google]](https://drive.google.com/file/d/1xaVLDq7zJ-C2GtFvABolSUtK7gzvNQNd/view?usp=drive_link)|\n|KITTI_Eigen (Swin-T)| 0.056 | 0.169 | 2.205 | 0.970 | 0.996 | 0.999 |[[Google]](https://drive.google.com/file/d/1s0LXZmS6_Q4_H_0hmbOldPcVhlRw8Dut/view?usp=drive_link)|\n\n| Model | SILog| Abs Rel | Sq Rel | RMSE | a1 | a2 | a3| Link|\n| ------------ | :---: | :---: | :---: | :---: |  :---: |  :---: |  :---: |  :---: |\n|KITTI_Official (Swinv2-L)| 7.48 | 5.20 | 0.79 | 2.34 | 0.974 | 0.996 | 0.999 |[[Google]](https://drive.google.com/file/d/19ARBiDTIvtZSWJVvhbEWBcZMonXsiOX1/view?usp=drive_link)|\n\n\n## Citation\n\nIf you find our work useful in your research please consider citing our paper:\n\n```\n@inproceedings{shao2023IEBins,\ntitle={IEBins: Iterative Elastic Bins for Monocular Depth Estimation},\nauthor={Shao, Shuwei and Pei, Zhongcai and Wu, Xingming and Liu, Zhong and Chen, Weihai and Li, Zhengguo},\nbooktitle={Advances in Neural Information Processing Systems (NeurIPS)},\nyear={2023}\n}\n```\n\n## Contact\n\nIf you have any questions, please feel free to contact swshao@buaa.edu.cn.\n\n\n## Acknowledgement\n\nOur code is based on the implementation of [NeWCRFs](https://github.com/aliyun/NeWCRFs) and [BTS](https://github.com/cleinc/bts). We thank their excellent works.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FShuweiShao%2FIEBins","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FShuweiShao%2FIEBins","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FShuweiShao%2FIEBins/lists"}