{"id":13528730,"url":"https://github.com/changhao-chen/deep-learning-localization-mapping","last_synced_at":"2025-04-01T14:32:56.686Z","repository":{"id":42434129,"uuid":"245213913","full_name":"changhao-chen/deep-learning-localization-mapping","owner":"changhao-chen","description":"A collection of deep learning based localization models","archived":false,"fork":false,"pushed_at":"2024-03-13T06:44:06.000Z","size":241,"stargazers_count":637,"open_issues_count":4,"forks_count":127,"subscribers_count":31,"default_branch":"master","last_synced_at":"2024-11-02T15:36:14.577Z","etag":null,"topics":["deep-learning","localization","mapping","slam"],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/changhao-chen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-05T16:30:43.000Z","updated_at":"2024-10-16T06:15:01.000Z","dependencies_parsed_at":"2022-07-11T00:16:21.220Z","dependency_job_id":"af71a1b9-f130-43ff-b102-377bd6982d47","html_url":"https://github.com/changhao-chen/deep-learning-localization-mapping","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/changhao-chen%2Fdeep-learning-localization-mapping","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/changhao-chen%2Fdeep-learning-localization-mapping/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/changhao-chen%2Fdeep-learning-localization-mapping/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/changhao-chen%2Fdeep-learning-localization-mapping/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/changhao-chen","download_url":"https://codeload.github.com/changhao-chen/deep-learning-localization-mapping/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246655232,"owners_count":20812603,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","localization","mapping","slam"],"created_at":"2024-08-01T07:00:23.482Z","updated_at":"2025-04-01T14:32:55.562Z","avatar_url":"https://github.com/changhao-chen.png","language":null,"funding_links":[],"categories":["5. Learning based SLAM"],"sub_categories":["5.1 Survey"],"readme":"# Deep Learning for Localization and Mapping\n\n![image](image/concept_figure.png)\nThis repository is a collection of deep learning based localization and mapping approaches. A survey on Deep Learning for Visual Localization and Mapping is offered in the following paper:\n\n\u003eDeep Learning for Visual Localization and Mapping: A Survey\n \u003e\n \u003e[Changhao Chen](https://changhao-chen.github.io/), [Bing Wang](https://www.cs.ox.ac.uk/people/bing.wang/), [Chris Xiaoxuan Lu](https://christopherlu.github.io/), [Niki Trigoni](https://www.cs.ox.ac.uk/people/niki.trigoni/) and [Andrew Markham](https://www.cs.ox.ac.uk/people/andrew.markham/)\n \u003e\n \u003e**IEEE Transactions on Neural Networks and Learning Systems** [[PDF](https://arxiv.org/abs/2308.14039)]\n\nA survey on Deep Learning for Inertial Positioning is offered in the following paper:\n\n\u003eDeep Learning for Inertial Positioning: A Survey\n \u003e\n \u003e[Changhao Chen](https://changhao-chen.github.io/), Xianfei Pan\n \u003e\n \u003e**IEEE Transactions on Intelligent Transportation Systems** [[PDF](https://arxiv.org/abs/2303.03757)]\n\nPrevious Version.\n\n\u003eA Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence\n \u003e\n \u003e[Changhao Chen](https://changhao-chen.github.io/), [Bing Wang](https://www.cs.ox.ac.uk/people/bing.wang/), [Chris Xiaoxuan Lu](https://christopherlu.github.io/), [Niki Trigoni](https://www.cs.ox.ac.uk/people/niki.trigoni/) and [Andrew Markham](https://www.cs.ox.ac.uk/people/andrew.markham/)\n \u003e\n \u003e**arXiv:2006.12567** [[PDF](https://arxiv.org/abs/2006.12567)]\n\n## News\n### Update: Jun-22-2020\n- We released our survey paper.\n### Update: Aug-30-2023\n- Our Survey \"Deep Learning for Visual Localization and Mapping: A Survey\" was accepted to IEEE TNNLS.\n### Update: Mar-13-2024\n- Our Survey \"Deep Learning for Inertial Positioning: A Survey\" was accepted to IEEE TITS.\n\n## TO DO\n\n## Category\n- [Odometry Estimation](#Odometry-Estimation)\n  - [Visual Odometry](#Visual-Odometry)\n  - [Visual-Inertial Odometry](#Visual-Inertial-Odometry)\n  - [Inertial Odometry](#Inertial-Odometry)\n  - [LIDAR Odometry](#LIDAR-Odometry)\n- [Mapping](#Mapping)\n  - [Geometric Mapping](#Geometric-Mapping)\n  - [Semantic Mapping](#Semantic-Mapping)\n  - [General Mapping](#General-Mapping)\n- [Global localization](#Global-Localization)\n  - [2D-to-2D Localization](#2D-to-2D-Localization)\n  - [2D-to-3D Localization](#2D-to-3D-Localization)\n  - [3D-to-3D Localization](#3D-to-3D-Localization)\n- [Simultaneous Localization and Mapping (SLAM)](#SLAM)\n  - [Local Optimization](#Local-Optimization)\n  - [Global Optimization](#Global-Optimization)\n  - [Keyframe and Loop-closure Detection](#Keyframe-and-Loop-closure-Detection)\n  - [Uncertainty Estimation](#Uncertainty-Estimation)\n\n## If you find this repository useful, please cite our paper:\n \n    @misc{chen2020survey,\n    title={A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence},\n    author={Changhao Chen and Bing Wang and Chris Xiaoxuan Lu and Niki Trigoni and Andrew Markham},\n    year={2020},\n    eprint={2006.12567},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV}\n    }\n\n  \n## Categorized by Topic\n*The Date in the table denotes the publication date (e.g. date of conference).\n### Odometry Estimation\n#### Visual Odometry\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| Konda et al. | 2015 | VISAPP | [Learning visual odometry with a convolutional network](https://www.iro.umontreal.ca/~memisevr/pubs/VISAPP2015.pdf) | |\n| Costante et al. | 2016 | RA-L | [Exploring Representation Learning With CNNs for Frame-to-Frame Ego-Motion Estimation](https://ieeexplore.ieee.org/document/7347378) | |\n| Backprop KF | 2016 | NeurIPS | [Backprop KF: Learning Discriminative Deterministic State Estimators](https://arxiv.org/abs/1605.07148) | |\n| DeepVO | 2017 | ICRA | [DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks](https://arxiv.org/abs/1709.08429) | |\n| SfmLearner | 2017 | CVPR | [unsupervised learning of depth and ego-motion from video](http://openaccess.thecvf.com/content_cvpr_2018/papers/Mahjourian_Unsupervised_Learning_of_CVPR_2018_paper.pdf) | [TF](https://github.com/tinghuiz/SfMLearner) [PT](https://github.com/ClementPinard/SfmLearner-Pytorch)|\n| Yin et al. | 2017 | ICCV | [Scale Recovery for Monocular Visual Odometry Using Depth Estimated With Deep Convolutional Neural Fields](http://openaccess.thecvf.com/content_ICCV_2017/papers/Yin_Scale_Recovery_for_ICCV_2017_paper.pdf) | |\n| UnDeepVO | 2018 | ICRA | [UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning](https://arxiv.org/abs/1709.06841) | |\n| Barnes et al. | 2018 | ICRA | [Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments](https://arxiv.org/abs/1711.06623) | |\n| GeoNet | 2018 | CVPR | [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://arxiv.org/abs/1803.02276) | [TF](https://github.com/yzcjtr/GeoNet) |\n| Zhan et al. | 2018 | CVPR | [Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction](https://arxiv.org/abs/1803.03893) | [Caffe](https://github.com/Huangying-Zhan/Depth-VO-Feat) |\n| DPF | 2018 | RSS | [Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors](https://arxiv.org/abs/1805.11122) | [TF](https://github.com/tu-rbo/differentiable-particle-filters) |\n| Yang et al. | 2018 | ECCV | [Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry](https://arxiv.org/abs/1807.02570) | |\n| Zhao et al. | 2018 | IROS | [Learning monocular visual odometry with dense 3d mapping from dense 3d flow](https://arxiv.org/abs/1803.02286) | |\n| Turan et al. | 2018 | IROS | [Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots](https://arxiv.org/pdf/1803.01047.pdf) | |\n| Struct2Depth | 2019 | AAAI | [Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videos](https://arxiv.org/abs/1811.06152) | [TF](https://github.com/tensorflow/models/tree/master/research/struct2depth) |\n| Saputra et al.| 2019 | ICRA | [Learning monocular visual odometry through geometry-aware curriculum learning](https://arxiv.org/abs/1903.10543) | |\n| GANVO | 2019 | ICRA | [GANVO: Unsupervised deep monocular visual odometry and depth estimation with generative adversarial networks](https://arxiv.org/abs/1809.05786) | |\n| CNN-SVO | 2019 | ICRA | [CNN-SVO: Improving the Mapping in Semi-Direct Visual Odometry Using Single-Image Depth Prediction](https://ieeexplore.ieee.org/document/8794425) | [ROS](https://github.com/yan99033/CNN-SVO) |\n| Li et al. | 2019 | ICRA | [Pose graph optimization for unsupervised monocular visual odometry](https://arxiv.org/abs/1903.06315) | |\n| Xue et al.| 2019 | CVPR | [Beyond tracking: Selecting memory and refining poses for deep visual odometry](https://arxiv.org/abs/1904.01892) | |\n| Wang et al.| 2019 | CVPR | [Recurrent neural network for (un-) supervised learning of monocular video visual odometry and depth](https://arxiv.org/abs/1904.07087) | |\n| Li et al. | 2019 | ICCV | [Sequential adversarial learning for self-supervised deep visual odometry](https://arxiv.org/abs/1908.08704) | |\n| Saputra et al. | 2019 | ICCV | [Distilling knowledge from a deep pose regressor network](https://arxiv.org/abs/1908.00858) | |\n| Gordon et al. | 2019 | ICCV | [Depth from videos in the wild: Unsupervised monocular depth learning from unknown cameras](https://arxiv.org/abs/1904.04998) | [TF](https://github.com/google-research/google-research/tree/master/depth_from_video_in_the_wild) |\n|  Koumis et al. | 2019 | IROS | [Estimating Metric Scale Visual Odometry from Videos using 3D Convolutional Networks](https://jpreiss.github.io/pubs/Koumis_Preiss_3DCVO_IROS2019.pdf) | |\n| Bian et al. | 2019 | NeurIPS | [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://papers.nips.cc/paper/8299-unsupervised-scale-consistent-depth-and-ego-motion-learning-from-monocular-video.pdf) | [PT](https://github.com/JiawangBian/SC-SfMLearner-Release) |\n| D3VO | 2020 | CVPR | [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://arxiv.org/pdf/2003.01060.pdf#page=9\u0026zoom=100,412,902) | |\n| Jiang et al. | 2020 | CVPR | [Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization](https://arxiv.org/pdf/2002.11826.pdf) | |\n\n\n#### Visual-Inertial Odometry\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| VINet | 2017 | AAAI | [VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem](https://arxiv.org/abs/1701.08376) | |\n| VIOLearner | 2019 | TPAMI | [Unsupervised deep visual-inertial odometry with online error correction for rgb-d imagery](https://ieeexplore.ieee.org/document/8691513) | |\n| SelectFusion | 2019 | CVPR | [Selective Sensor Fusion for Neural Visual-Inertial Odometry](https://arxiv.org/abs/1903.01534) | |\n| DeepVIO | 2019 | IROS | [DeepVIO: Self-supervised deep learning of monocular visual inertial odometry using 3d geometric constraints](https://arxiv.org/abs/1906.11435) | |\n\n\n#### Inertial Odometry\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| IONet | 2018 | AAAI | [IONet: Learning to Cure the Curse of Drift in Inertial Odometry](https://arxiv.org/abs/1802.02209) | |\n| RIDI | 2018 | ECCV | [RIDI: Robust IMU Double Integration](https://arxiv.org/abs/1712.09004) | [Py](https://github.com/higerra/ridi_imu) |\n| Wagstaff et al. | 2018 | IPIN | [LSTM-Based Zero-Velocity Detection for Robust Inertial Navigation](https://ieeexplore.ieee.org/abstract/document/8533770) | [PT](https://github.com/utiasSTARS/pyshoe) |\n| Cortes et al. | 2019 | MLSP | [Deep Learning Based Speed Estimation for Constraining Strapdown Inertial Navigation on Smartphones](https://ieeexplore.ieee.org/abstract/document/8516710) | |\n| MotionTransformer| 2019 | AAAI | [MotionTransformer: Transferring Neural Inertial Tracking between Domains](https://www.aaai.org/ojs/index.php/AAAI/article/view/4802) | |\n| AbolDeepIO | 2019 | TITS | [AbolDeepIO: A Novel Deep Inertial Odometry Network for Autonomous Vehicles](https://ieeexplore.ieee.org/abstract/document/8693766) | |\n| Brossard et al. | 2019 | ICRA | [Learning wheel odometry and imu errors for localization](https://hal.archives-ouvertes.fr/hal-01874593/document) | |\n| OriNet | 2019 | RA-L | [OriNet: Robust 3-D Orientation Estimation With a Single Particular IMU](https://ieeexplore.ieee.org/abstract/document/8931590) | [PT](https://github.com/mbrossar/denoise-imu-gyro) |\n| L-IONet | 2020| IoT-J | [Deep Learning based Pedestrian Inertial Navigation: Methods, Dataset and On-Device Inference](https://arxiv.org/abs/2001.04061) | |\n\n#### LIDAR Odometry\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| Velas et al. | 2018 | ICARSC | [CNN for IMU Assisted Odometry Estimation using Velodyne LiDAR](https://arxiv.org/abs/1712.06352) | | \n| LO-Net | 2019 | CVPR | [LO-Net: Deep Real-time Lidar Odometry](https://arxiv.org/abs/1904.08242) | |\n| DeepPCO | 2019 | IROS | [DeepPCO: End-to-End Point Cloud Odometry through Deep Parallel Neural Network](https://arxiv.org/abs/1910.11088) | |\n| Valente et al. | 2019 | IROS | [Deep sensor fusion for real-time odometry estimation](https://ieeexplore.ieee.org/document/8967803) | |\n\n### Mapping\n#### Geometric Mapping\n##### Depth Representation\n* Joint learning of depth and ego-motion has been discussed in Visual Odometry. We do not include these works here, although they can produce depth representation.\n\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| Eigen et al. | 2014 | NeurIPS | [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://papers.nips.cc/paper/5539-depth-map-prediction-from-a-single-image-using-a-multi-scale-deep-network.pdf) | |\n| Liu et al. | 2015 | TPAMI | [Learning depth from single monocular images using deep convolutional neural fields](https://arxiv.org/abs/1502.07411) | |\n| Garg et al. | 2016 | ECCV | [Unsupervised cnn for single view depth estimation: Geometry to the rescue](https://arxiv.org/abs/1603.04992) | |\n| Demon | 2017 | CVPR | [Demon: Depth and motion network for learning monocular stereo](https://arxiv.org/abs/1612.02401) | |\n| Godard et al. | 2017 | CVPR | [Unsupervised monocular depth estimation with left-right consistency](https://arxiv.org/abs/1609.03677) | |\n| Wang et al. | 2018 | CVPR | [Learning depth from monocular videos using direct methods](https://arxiv.org/abs/1712.00175) | |\n\n##### Voxel Representation\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| SurfaceNet | 2017 | CVPR | [SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis](https://arxiv.org/abs/1708.01749) | |\n| Dai et al. | 2017 | CVPR | [Shape completion using 3d-encoder-predictor cnns and shape synthesis](https://arxiv.org/abs/1612.00101) | |\n| Hane et al. | 2017 | 3DV | [Hierarchical surface prediction for 3d object reconstruction](https://arxiv.org/abs/1704.00710) | |\n| OctNetFusion | 2017 | 3DV | [Octnetfusion: Learning depth fusion from data](https://arxiv.org/abs/1704.01047) | |\n| OGN | 2017 | ICCV | [Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs](https://arxiv.org/abs/1703.09438) | |\n| Kar et al. | 2017 | NeurIPS | [Learning a multi-view stereo machine](https://arxiv.org/abs/1708.05375) | |\n| RayNet | 2018 | CVPR | [RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials](https://arxiv.org/abs/1901.01535) | |\n\n\n##### Point Representation\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| Fan et al. | 2017 | CVPR | [A point set generation network for 3d object reconstruction from a single image](https://arxiv.org/abs/1612.00603) | |\n\n##### Mesh Representation\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| Ladicky et al. | 2017 | ICCV | [From point clouds to mesh using regression](https://ieeexplore.ieee.org/document/8237682) | |\n| Mukasa et al. | 2017 | ICCVW | [3d scene mesh from cnn depth predictions and sparse monocular slam](http://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w17/Mukasa_3D_Scene_Mesh_ICCV_2017_paper.pdf) | |\n| Wang et al. | 2018 | ECCV | [Pixel2mesh: Generating 3d mesh models from single rgb images](https://arxiv.org/abs/1804.01654) | |\n| Groueix et al. | 2018 | CVPR | [AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation](https://arxiv.org/abs/1802.05384) | |\n| Scan2Mesh | 2019 | CVPR | [Scan2mesh: From unstructured range scans to 3d meshes](https://arxiv.org/abs/1811.10464) | |\n| Bloesch et al. | 2019 | ICCV | [Learning meshes for dense visual SLAM](https://www.imperial.ac.uk/media/imperial-college/research-centres-and-groups/dyson-robotics-lab/mbloesch_etal_iccv2019.pdf) | |\n\n#### Semantic Mapping\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| SemanticFusion | 2017 | ICRA | [Semanticfusion: Dense 3d semantic mapping with convolutional neural networks](https://arxiv.org/abs/1609.05130) | |\n| DA-RNN | 2017 | RSS | [DA-RNN: Semantic mapping with data associated recurrent neural networks](https://arxiv.org/abs/1703.03098) | |\n| Ma et al. | 2017 | IROS | [Multi-view deep learning for consistent semantic mapping with rgb-d cameras](https://arxiv.org/abs/1703.08866) | |\n| Sunderhauf et al. | 2017 | IROS | [Meaningful maps with object-oriented semantic mapping](https://arxiv.org/abs/1609.07849) | |\n| Fusion++ | 2018 | 3DV | [Fusion++: Volumetric object-level SLAM](https://arxiv.org/abs/1808.08378) | |\n| Grinvald et al. | 2019 | RA-L | [Volumetric instance-aware semantic mapping and 3d object discovery](https://arxiv.org/abs/1903.00268) | |\n| PanopticFusion | 2019 | IROS | [Panopticfusion: Online volumetric semantic mapping at the level of stuff and things](https://arxiv.org/abs/1903.01177) | |\n\n#### General Mapping\n* neural scene representation, task-driven representation \n\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| Mirowski et al. | 2017 | ICLR | [Learning to navigate in complex environments](https://arxiv.org/abs/1611.03673) | |\n| Zhu et al. | 2017 | ICRA | [Target-driven visual navigation in indoor scenes using deep reinforcement learning](https://arxiv.org/abs/1609.05143) | |\n| Eslami et al. | 2018 | Science | [Neural scene representation and rendering](https://science.sciencemag.org/content/360/6394/1204) | |\n| CodeSLAM | 2018 | CVPR | [CodeSLAM — Learning a Compact, Optimisable Representation for Dense Visual SLAM](https://arxiv.org/abs/1804.00874) | |\n| Mirowski et al. | 2018 | NeurIPS | [Learning to navigate in cities without a map](https://arxiv.org/abs/1804.00168) | | \n| SRN | 2019 | NeurIPS | [Scene representation networks: Continuous 3d-structure-aware neural scene representations](https://arxiv.org/abs/1906.01618) | |\n| Tobin et al. | 2019 | NeurIPS | [Geometry-aware neural rendering](https://arxiv.org/abs/1911.04554) | |\n| Lim et al. | 2019 | NeurIPS | [Neural multisensory scene inference](https://arxiv.org/abs/1910.02344) | |\n\n\n### Global Localization\n#### 2D-to-2D Localization\n##### Implicit Map Based Localization\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| PoseNet  | 2015 | ICCV | [PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization](https://arxiv.org/abs/1505.07427)  | |\n| Bayesian PoseNet  | 2016 | ICRA | [Modelling uncertainty in deep learning for camera relocalization](https://arxiv.org/abs/1509.05909)  | |\n| BranchNet | 2017 | ICRA | [Delving deeper into convolutional neural networks for camera relocalization](http://ieeexplore.ieee.org/document/7989663/) | |\n| VidLoc   | 2017 | CVPR | [VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization](https://arxiv.org/abs/1702.06521)  | |\n| Geometric PoseNet | 2017 | CVPR | [Geometric loss functions for camera pose regression with deep learning](http://openaccess.thecvf.com/content_cvpr_2017/html/Kendall_Geometric_Loss_Functions_CVPR_2017_paper.html) | |\n| Naseer et al. | 2017 | IROS | [Deep Regression for Monocular Camera-based 6-DoF Global Localization in Outdoor Environments](http://ais.informatik.uni-freiburg.de/publications/papers/naseer17iros.pdf)| |\n| LSTM-PoseNet | 2017| ICCV | [Image-based localization using lstms for structured feature correlation](https://arxiv.org/abs/1611.07890) | |\n| Hourglass PoseNet | 2017| ICCV Workshops | [Image-based localization using hourglass networks](http://openaccess.thecvf.com/content_ICCV_2017_workshops/w17/html/Melekhov_Image-Based_Localization_Using_ICCV_2017_paper.html) | |\n| VLocNet | 2018 | ICRA | [Deep auxiliary learning for visual localization and odometry](https://arxiv.org/abs/1803.03642)| |\n| MapNet | 2018 | CVPR | [Geometry-Aware Learning of Maps for Camera Localization](https://arxiv.org/abs/1712.03342)| |\n| SPP-Net | 2018 | BMVC | [Synthetic view generation for absolute pose regression and image synthesis](http://bmvc2018.org/contents/papers/0221.pdf) | |\n| GPoseNet | 2018 | BMVC | [A hybrid probabilistic model for camera relocalization](http://bmvc2018.org/contents/papers/0799.pdf) | |\n| VLocNet++ | 2018 | RA-L | [Vlocnet++: Deep multitask learning for semantic visual localization and odometry](https://arxiv.org/abs/1804.08366)| |\n| Xue et al. | 2019 | ICCV |  [Local supports global: Deep camera relocalization with sequence enhancement](https://arxiv.org/abs/1908.04391)| |\n| Huang et al. | 2019 | ICCV | [Prior guided dropout for robust visual localization in dynamic environments](http://openaccess.thecvf.com/content_ICCV_2019/papers/Huang_Prior_Guided_Dropout_for_Robust_Visual_Localization_in_Dynamic_Environments_ICCV_2019_paper.pdf) | |\n| Bui et al. | 2019 | ICCVW | [Adversarial networks for camera pose regression and refinement](https://arxiv.org/abs/1903.06646) | |\n| GN-Net | 2020 | RA-L | [GN-Net: The Gauss-Newton Loss for Multi-Weather Relocalization](https://ieeexplore.ieee.org/abstract/document/8954808) | |\n| AtLoc | 2020 | AAAI | [AtLoc: Attention Guided Camera Localization](https://arxiv.org/abs/1909.03557) | |\n\n##### Explicit Map Based Localization\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| Laskar et al. | 2017| ICCV Workshops | [Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network](http://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w17/Laskar_Camera_Relocalization_by_ICCV_2017_paper.pdf) | |\n| DELS-3D | 2018 | CVPR | [Dels-3d: Deep localization and segmentation with a 3d semantic map](https://arxiv.org/abs/1805.04949) | |\n| AnchorNet | 2018 | BMVC | [Improved visual relocalization by discovering anchor points](https://arxiv.org/abs/1811.04370) | |\n| RelocNet | 2018 | ECCV | [RelocNet: Continuous Metric Learning Relocalisation using Neural Nets](http://openaccess.thecvf.com/content_ECCV_2018/papers/Vassileios_Balntas_RelocNet_Continous_Metric_ECCV_2018_paper.pdf) | |\n| CamNet | 2019 |  ICCV | [Camnet: Coarse-to-fine retrieval for camera re-localization](http://openaccess.thecvf.com/content_ICCV_2019/html/Ding_CamNet_Coarse-to-Fine_Retrieval_for_Camera_Re-Localization_ICCV_2019_paper.html)| |\n\n\n#### 2D-to-3D Localization\n##### Descriptor Matching\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| NetVLAD | 2016 | CVPR | [Netvlad: Cnn architecture for weakly supervised place recognition](https://arxiv.org/abs/1511.07247) | |\n| DELF | 2017 | CVPR | [Large-scale image retrieval with attentive deep local features](https://arxiv.org/abs/1612.06321) | | InLoc | 2018/06 | CVPR | [InLoc: Indoor Visual Localization with Dense Matching and View Synthesis](http://openaccess.thecvf.com/content_cvpr_2018/papers/Taira_InLoc_Indoor_Visual_CVPR_2018_paper.pdf) | |\n| Schonberger et al.| 2018/06 | CVPR | [Semantic Visual Localization](http://openaccess.thecvf.com/content_cvpr_2018/papers/Schonberger_Semantic_Visual_Localization_CVPR_2018_paper.pdf) | |\n| SuperPoint | 2018 | CVPRW | [Superpoint: Selfsupervised interest point detection and description](https://arxiv.org/abs/1712.07629) | |\n| NC-Net | 2018 | NeurIPS | [Neighbourhood consensus networks](https://arxiv.org/abs/1810.10510) | |\n| Sarlin et al. | 2019/06 | CVPR | [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](http://openaccess.thecvf.com/content_CVPR_2019/papers/Sarlin_From_Coarse_to_Fine_Robust_Hierarchical_Localization_at_Large_Scale_CVPR_2019_paper.pdf)| |\n| 2D3D-MatchNet | 2019 | ICRA | [2d3d-matchnet: learning to match keypoints across 2d image and 3d point cloud](https://ieeexplore.ieee.org/document/8794415) | |\n| D2-Net | 2019 | CVPR | [D2-net: A trainable cnn for joint description and detection of local features](https://arxiv.org/abs/1905.03561) | |\n| Speciale et al. | 2019 | CVPR | [Privacy preserving image-based localization](https://arxiv.org/abs/1903.05572) | |\n| OOI-Net | 2019 | CVPR | [Visual localization by learning objects-of-interest dense match regression](https://europe.naverlabs.com/wp-content/uploads/2019/05/Visual-Localization-by-Learning-Objects-of-Interest-Dense-Match-Regression.pdf) | |\n| Camposeco et al. | 2019 | CVPR | [scene compression for visual localization](https://arxiv.org/abs/1807.07512) | |\n| Cheng et al. | 2019 | CVPR | [Cascaded parallel filtering for memory-efficient image-based localization](https://arxiv.org/abs/1908.06141) | |\n|Taira et al. | 2019 | CVPR | [Is this the right place? geometric-semantic pose verification for indoor visual localization](https://arxiv.org/abs/1908.04598) | |\n| R2D2 | 2019| NeurIPS | [R2d2: Repeatable and reliable detector and descriptor](https://papers.nips.cc/paper/9407-r2d2-reliable-and-repeatable-detector-and-descriptor.pdf) | |\n| ASLFeat | 2020 | CVPR | [Aslfeat: Learning local features of accurate shape and localization](https://arxiv.org/abs/2003.10071) | |\n\n##### Scene Coordinate Regression\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| DSAC | 2017/07 | CVPR | [DSAC - Differentiable RANSAC for Camera Localization](http://openaccess.thecvf.com/content_cvpr_2017/html/Brachmann_DSAC_-_Differentiable_CVPR_2017_paper.html) | |\n| DSAC++ | 2018/06 | CVPR | [Learning less is more-6d camera localization via 3d surface regression](https://arxiv.org/abs/1711.10228) | |\n| Dense SCR | 2018/07 | RSS | [Full-Frame Scene Coordinate Regression for Image-Based Localization](https://arxiv.org/abs/1802.03237) | |\n| DSAC++ angle | 2018/09 | ECCV | [Scene coordinate regression with angle-based reprojection loss for camera relocalization](http://openaccess.thecvf.com/content_eccv_2018_workshops/w16/html/Li_Scene_Coordinate_Regression_with_Angle-Based_Reprojection_Loss_for_Camera_Relocalization_ECCVW_2018_paper.html) | |\n| Confidence SCR | 2018/09 | BMVC | [Scene Coordinate and Correspondence Learning for Image-Based Localization](https://arxiv.org/abs/1805.08443) | |\n| ESAC | 2019/10 |  ICCV | [Expert Sample Consensus Applied to Camera Re-Localization](http://openaccess.thecvf.com/content_ICCV_2019/html/Ding_CamNet_Coarse-to-Fine_Retrieval_for_Camera_Re-Localization_ICCV_2019_paper.html)| |\n| NG-RANSAC | 2019/06 | CVPR | [Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses](http://openaccess.thecvf.com/content_ICCV_2019/papers/Brachmann_Neural-Guided_RANSAC_Learning_Where_to_Sample_Model_Hypotheses_ICCV_2019_paper.pdf)| |\n| SANet | 2019/10 | ICCV | [SANet: scene agnostic network for camera localization](http://openaccess.thecvf.com/content_ICCV_2019/papers/Yang_SANet_Scene_Agnostic_Network_for_Camera_Localization_ICCV_2019_paper.pdf)| |\n| HSC-Net | 2020 | CVPR | [Hierarchical scene coordinate classification and regression for visual localization](https://arxiv.org/abs/1909.06216) | |\n| KF-Net | 2020 | CVPR | [Kfnet: Learning temporal camera relocalization using kalman filtering](https://arxiv.org/abs/2003.10629) | |\n\n#### 3D-to-3D Localization\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| LocNet | 2018 | IV | [Locnet: Global localization in 3d point clouds for mobile vehicles](https://arxiv.org/abs/1712.02165) | |\n| PointNetVLAD | 2018 | CVPR | [Pointnetvlad: Deep point cloud based retrieval for large-scale place recognition](https://arxiv.org/abs/1804.03492) | |\n| Barsan et al. | 2018 | CoRL | [Learning to localize using a lidar intensity map](http://proceedings.mlr.press/v87/barsan18a/barsan18a.pdf) | |\n| L3-Net | 2019 | CVPR | [L3-net: Towards learning based lidar localization for autonomous driving](https://songshiyu01.github.io/pdf/L3Net_W.Lu_Y.Zhou_S.Song_CVPR2019.pdf) | |\n| PCAN | 2019 | CVPR | [PCAN: 3D Attention Map Learning Using Contextual Information for Point Cloud Based Retrieval](https://arxiv.org/abs/1904.09793) | |\n| DeepICP | 2019 | CVPR | [Deepicp: An end-to-end deep neural network for 3d point cloud registration](https://arxiv.org/abs/1905.04153) | |\n| DCP | 2019 | CVPR | [Deep closest point: Learning representations for point cloud registration](https://arxiv.org/abs/1905.03304) | |\n| D3Feat | 2020 | CVPR | [D3feat: Joint learning of dense detection and description of 3d local features](https://arxiv.org/abs/2003.03164) | |\n\n### SLAM\n\n#### Local Optimization\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| LS-Net | 2018 | ECCV | [Learning to solve nonlinear least squares for monocular stereo](https://arxiv.org/abs/1809.02966) | |\n| BA-Net | 2019 | ICLR | [BA-Net: Dense bundle adjustment network](https://arxiv.org/abs/1806.04807) | |\n\n#### Global Optimization\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| CNN-SLAM | 2017 | CVPR | [CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction](https://arxiv.org/abs/1704.03489) | |\n| Li et al. | 2019 | ICRA | [Pose graph optimization for unsupervised monocular visual odometry](https://arxiv.org/abs/1903.06315) | |\n| DeepTAM | 2020 | IJCV | [DeepTAM: Deep Tracking and Mapping with Convolutional Neural Networks](https://lmb.informatik.uni-freiburg.de/Publications/2019/ZUB19a/) | |\n| DeepFactors | 2020 | RA-L | [DeepFactors: Real-Time Probabilistic Dense Monocular SLAM](https://arxiv.org/abs/2001.05049) | |\n\n\n#### Keyframe and Loop-closure Detection\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| Sunderhauf et al. | 2015 | RSS | [Place recognition with convnet landmarks: Viewpoint-robust, condition-robust, training-free](https://nikosuenderhauf.github.io/assets/papers/rss15_placeRec.pdf) | |\n| Gao et al. | 2017 | AR | [Unsupervised learning to detect loops using deep neural networks for visual slam system](https://dl.acm.org/citation.cfm?id=3040686) | |\n| Huang et al. | 2018 | RSS | [Lightweight unsupervised deep loop closure](https://arxiv.org/abs/1805.07703) | |\n| Sheng et al. | 2019 | ICCV | [Unsupervised Collaborative Learning of Keyframe Detection and Visual Odometry Towards Monocular Deep SLAM](http://openaccess.thecvf.com/content_ICCV_2019/html/Sheng_Unsupervised_Collaborative_Learning_of_Keyframe_Detection_and_Visual_Odometry_Towards_ICCV_2019_paper.html) | |\n| Memon et al. | 2020 | RAS | [Loop closure detection using supervised and unsupervised deep neural networks for monocular slam systems](https://www.sciencedirect.com/science/article/abs/pii/S0921889019308425) | |\n\n#### Uncertainty Estimation\n| Models   |Date| Publication| Paper | Code |\n|----------|----|------------|------|---|\n| Kendall et al. | 2016 | ICRA | [Modelling uncertainty in deep learning for camera relocalization](https://arxiv.org/abs/1509.05909) | |\n| Kendall et al. | 2017 | NeurIPS | [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://arxiv.org/abs/1703.04977) | |\n| VidLoc | 2017 | CVPR | [VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization](https://arxiv.org/abs/1702.06521) | |\n| Wang et al. | 2018 | IJRR | [End-to-end, sequenceto-sequence probabilistic visual odometry through deep neural networks](https://researchportal.hw.ac.uk/en/publications/end-to-end-sequence-to-sequence-probabilistic-visual-odometry-thr) | |\n| Chen et al. | 2019 | TMC | [Deep neural network based inertial odometry using low-cost inertial measurement units](http://www.cs.ox.ac.uk/files/11501/DNN_IONet.pdf) | |\n\nThis list is maintained by [Changhao Chen](http://www.cs.ox.ac.uk/people/changhao.chen/website/) and [Bing Wang](http://www.cs.ox.ac.uk/people/bing.wang/), Department of Computer Science, University of Oxford.\n\nPlease contact them (email: changhao.chen@cs.ox.ac.uk; bing.wang@cs.ox.ac.uk), if you have any question or would like to add your work on this list.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchanghao-chen%2Fdeep-learning-localization-mapping","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchanghao-chen%2Fdeep-learning-localization-mapping","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchanghao-chen%2Fdeep-learning-localization-mapping/lists"}