https://github.com/nianticlabs/rectified-features
[ECCV 2020] Single image depth prediction allows us to rectify planar surfaces in images and extract view-invariant local features for better feature matching
https://github.com/nianticlabs/rectified-features
depth-estimation feature-matching image-matching local-features monocular-depth-estimation rectification robotcar single-image-depth-prediction
Last synced: 3 months ago
JSON representation
[ECCV 2020] Single image depth prediction allows us to rectify planar surfaces in images and extract view-invariant local features for better feature matching
- Host: GitHub
- URL: https://github.com/nianticlabs/rectified-features
- Owner: nianticlabs
- License: other
- Created: 2020-08-04T12:33:10.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2021-05-11T12:36:39.000Z (almost 5 years ago)
- Last Synced: 2025-06-03T15:29:14.862Z (11 months ago)
- Topics: depth-estimation, feature-matching, image-matching, local-features, monocular-depth-estimation, rectification, robotcar, single-image-depth-prediction
- Homepage:
- Size: 2.69 MB
- Stars: 64
- Watchers: 8
- Forks: 7
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# [Single-Image Depth Prediction Makes Feature Matching Easier](https://arxiv.org/abs/2008.09497)
**[Carl Toft](https://scholar.google.com/citations?hl=en&user=vvgmWA0AAAAJ&view_op=list_works&sortby=pubdate), [Daniyar Turmukhambetov](http://dantkz.github.io/about), [Torsten Sattler](https://scholar.google.com/citations?user=jzx6_ZIAAAAJ&hl=en), [Fredrik Kahl](http://www.maths.lth.se/matematiklth/personal/fredrik/) and [Gabriel J. Brostow](http://www0.cs.ucl.ac.uk/staff/g.brostow/) – ECCV 2020**
[Link to paper](https://arxiv.org/abs/2008.09497)
[Link to supplementary pdf](https://storage.googleapis.com/niantic-lon-static/research/rectified-features/rectified-features-supplementary.pdf)
**Code is coming soon...**
Good local features improve the robustness of many 3D relocalization and multi-view reconstruction pipelines. The problem is that viewing angle and distance severely impact the recognizability of a local feature. Attempts to improve appearance invariance by choosing better local feature points or by leveraging outside information, have come with pre-requisites that made some of them impractical. In this paper, we propose a surprisingly effective enhancement to local feature extraction, which improves matching.
We show that CNN-based depths inferred from single RGB images are quite helpful, despite their flaws. They allow us to pre-warp images and rectify perspective distortions, to significantly enhance SIFT and BRISK features, enabling more good matches, even when cameras are looking at the same scene but in opposite directions.
Our pipeline finds planar patches according to estimated depth, and extracts features from rectified views of these patches. Non-rectified features are also extracted from regions that do not belong to planar patches.
## 💾 📸 Dataset
[Dataset README](https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/README.txt)
The "Strong Viewpoint Changes Dataset" is published as part of ECCV 2020 "Single-Image Depth Prediction Makes Feature Matching Easier" paper by
Carl Toft, Daniyar Turmukhambetov, Torsten Sattler, Fredrik Kahl and Gabriel J. Brostow.
Please cite the paper if you are using this dataset.
The images, file pairs for evaluation and ground truth poses for the 8 scenes are
available at:
```
https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/scene1.zip
https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/scene2.zip
https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/scene3.zip
https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/scene4.zip
https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/scene5.zip
https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/scene6.zip
https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/scene7.zip
https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/scene8.zip
```
The dataset is published with Attribution 4.0 International (CC BY 4.0) License, see:
https://storage.googleapis.com/niantic-lon-static/research/rectified-features/StrongViewpointChangesDataset/LICENSE.txt
## ✏️ 📄 Citation
If you find our work useful or interesting, please consider citing [our paper](https://arxiv.org/abs/2008.09497):
```
@inproceedings{toft-2020-rectified-features,
title = {Single-Image Depth Prediction Makes Feature Matching Easier},
author = {Carl Toft and
Daniyar Turmukhambetov and
Torsten Sattler and
Fredrik Kahl and
Gabriel J. Brostow
},
booktitle = {European Conference on Computer Vision ({ECCV})},
year = {2020}
}
```
# 👩⚖️ License
Copyright © Niantic, Inc. 2020. Patent Pending. All rights reserved. Please see the license file for terms.