{"id":13535328,"url":"https://github.com/Ravi-Garg/Unsupervised_Depth_Estimation","last_synced_at":"2025-04-02T01:30:38.311Z","repository":{"id":215855544,"uuid":"64445270","full_name":"Ravi-Garg/Unsupervised_Depth_Estimation","owner":"Ravi-Garg","description":"Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue","archived":false,"fork":false,"pushed_at":"2019-03-11T16:37:15.000Z","size":25799,"stargazers_count":238,"open_issues_count":3,"forks_count":70,"subscribers_count":22,"default_branch":"master","last_synced_at":"2024-11-02T23:32:49.525Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ravi-Garg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-07-29T02:52:11.000Z","updated_at":"2024-10-25T14:49:40.000Z","dependencies_parsed_at":"2024-01-18T11:09:18.433Z","dependency_job_id":"ec4f7291-87c8-4fc8-adad-e9d417072558","html_url":"https://github.com/Ravi-Garg/Unsupervised_Depth_Estimation","commit_stats":null,"previous_names":["ravi-garg/unsupervised_depth_estimation"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ravi-Garg%2FUnsupervised_Depth_Estimation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ravi-Garg%2FUnsupervised_Depth_Estimation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ravi-Garg%2FUnsupervised_Depth_Estimation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ravi-Garg%2FUnsupervised_Depth_Estimation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ravi-Garg","download_url":"https://codeload.github.com/Ravi-Garg/Unsupervised_Depth_Estimation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246738392,"owners_count":20825775,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T08:00:53.804Z","updated_at":"2025-04-02T01:30:36.716Z","avatar_url":"https://github.com/Ravi-Garg.png","language":"C++","funding_links":[],"categories":["Computer Vision","2. Monocular Depth (Semi- / Un-Supervised)","2. 单目深度估计(半监督、无监督)"],"sub_categories":["Geometry","2.1 Stereo Consistency"],"readme":"# Realtime Unsupervised Depth Estimation from an Image\n\nThis is the caffe implementation of our paper \"Unsupervised CNN for single view depth estimation: Geometry to the rescue\" published in ECCV 2016 with minor modifications. In this variant, we train the network end-to-end instead of in coarse to fine manner with deeper network (Resnet 50) and TVL1 loss instead of HS loss. \n\nWith the implementation we share the sample Resnet50by2 model trained on KITTI training set:\n\nhttps://github.com/Ravi-Garg/Unsupervised_Depth_Estimation/blob/master/model/train_iter_40000.caffemodel\n\n\nShared model is a small variant of the 50 layer residual network from scratch on KITTI.\nOur model is **\u003c25 MB** and predicts depths on 160x608 resolution images **at over 30Hz** on Nvidia Geforce GTX980 (50Hz on TITAN X). It can be used with caffe without any modification and we provide a simple matlab wrapper for testing.\n\nClick on the image to watch preview of the results on youtube:\n\n[![Screenshot](https://img.youtube.com/vi/Fut885qvBvQ/0.jpg)](https://www.youtube.com/watch?v=Fut885qvBvQ)\n\nIf you use our model or the code for your research please cite:\n```\n@inproceedings{garg2016unsupervised,\n  title={Unsupervised CNN for single view depth estimation: Geometry to the rescue},\n  author={Garg, Ravi and Kumar, BG Vijay and Carneiro, Gustavo and Reid, Ian},\n  booktitle={European Conference on Computer Vision},\n  pages={740--756},\n  year={2016},\n  organization={Springer}\n}\n```\n\n# Training Procedure\nThis model was trained on 23200 raw stereo pairs of KITTI taken from city, residential and road sequences. Images from other sequences of KITTI were left untouched. A subset of 697 images from 28 sequences froms the testset, leaving the remaining 33 sequences from these categories which can be used for training.\n\nTo use the same training data use the splits spacified in the file 'train_test_split.mat'.\n\n\nOur model is trained end-to-end from scratch with adam solver (momentum1 = 0.9 , momentom2 = 0.999, learning rate =10e-3 ) for 40,000 iterations on 4 gpus with batchsize 14 per GPU. This model is a pre-release further tuning of hyperparameters should improve results. Only left-right flips as described in the paper were used to train the provided network. Other agumentations described in the paper and runtime shuffle were not used but should also lead to performance imrovement.\n\nHere is the training loss recorded per 20 iterations: \n\n\u003cimg src=\"https://github.com/Ravi-Garg/Unsupervised_Depth_Estimation/blob/master/model/train.png\" alt=\"loss per 20 iterations \" width=\"400\"\u003e\n\nNote: We have resized the KITTI images to 160x608 for training - which changes the aspect ratio of the images. Thus for proper evaluation on KITTI the images needs to be resized to this resolution and predicted disparities should be scaled by a factor of 608/width_of_input_image before computing depth. For ease in citing the results for further publications, we share the performance measures.\n\nOur model gives following results on KITTI test-set without any post processing: \n---------------------------------------------------------------------\n\nRMSE(linear):   4.400866\n\nRMSE(log)   :   0.233548\n\nRMSE(log10)   :   0.101441\n\nAbs rel diff:   0.137796\n\nSq rel diff :   0.824861\n\naccuracy THr 1.25 :   0.809765\n\naccuracy THr 1.25 sq:   0.935108\n\naccuracy THr 1.25 cube:   0.974739\n\n---------------------------------------------------------------------\n\nThe test-set consists of 697 images which was used in  https://www.cs.nyu.edu/~deigen/depth/kitti_depth_predictions.mat\nDepth Predictions were first clipped to depth values between 0 and 50 meters and evaluated only in the region spacified in the given mask.\n\n#Network Architecture\n\nArchitecture of our networks closely follow Residual networks scheme. We start from [resnet 50 by 2](https://github.com/jay-mahadeokar/pynetbuilder/tree/master/models/imagenet)\n architecture and have replaced strided convolutions with 2x2 MAX pooling layers like VGG. The first 7x7 convolution with stride 2 is replaced with the 7x7 convolution with no stride and the max-pooled output at ½ resolution is passed through an extra 3x3 convolutional (128 features)-\u003erelu-\u003e2x2 pooling block. Rest of the network followes resnet50 with half the parameters every layer.\n\nFor dense prediction we have followed the skip-connections as specified in FCN and our ECCV paper. \nWe have introduced a learnable scale layer with weight decay 0.01 before every 1x1 convolution of FCN skip-connections which allows us to merge mid-level features more efficiently by:\n\n* Adaptively selecting the mid-level features which are more correlated to depth of the scene.\n* Making 1x1 convolutions for projections more stable for end to end training.\n\nFurther analysis and visualizations of learned features will be released shortly on the arxiv:\nhttps://arxiv.org/pdf/1603.04992v2.pdf\n\n# Using the code\n\nTo train and finetune networks on your own data, you need to compile caffe with additional:\n* “AbsLoss” layer for L1 loss minimization, \n\n* “Warping” layer for image warpping given flow\n\n* and modified \"filler.hpp\" to compute image gradient with convolutions which we share here.\n\n\n# License\nFor academic usage, the code is released under the permissive BSD license. For any commercial purpose, please contact the authors.\n\n# Contact\nPlease report any known issues on this thread of to ravi.garg@adelaide.edu.au\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRavi-Garg%2FUnsupervised_Depth_Estimation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRavi-Garg%2FUnsupervised_Depth_Estimation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRavi-Garg%2FUnsupervised_Depth_Estimation/lists"}