{"id":15030195,"url":"https://github.com/cvg/hierarchical-localization","last_synced_at":"2025-05-13T21:08:10.338Z","repository":{"id":37457453,"uuid":"280084324","full_name":"cvg/Hierarchical-Localization","owner":"cvg","description":"Visual localization made easy with hloc","archived":false,"fork":false,"pushed_at":"2025-03-24T20:40:27.000Z","size":20832,"stargazers_count":3493,"open_issues_count":144,"forks_count":638,"subscribers_count":85,"default_branch":"master","last_synced_at":"2025-05-07T16:48:06.538Z","etag":null,"topics":["deep-learning","feature-matching","image-retrieval","pose-estimation","structure-from-motion","superglue","visual-localization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cvg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-16T07:25:35.000Z","updated_at":"2025-05-07T14:14:30.000Z","dependencies_parsed_at":"2023-12-06T11:06:20.991Z","dependency_job_id":"c1428ee8-7902-44cf-ac2e-b4edccf6587d","html_url":"https://github.com/cvg/Hierarchical-Localization","commit_stats":{"total_commits":173,"total_committers":31,"mean_commits":5.580645161290323,"dds":0.5491329479768786,"last_synced_commit":"b21ff203ead5a3257df3b6806214ff605e0c01c5"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2FHierarchical-Localization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2FHierarchical-Localization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2FHierarchical-Localization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2FHierarchical-Localization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cvg","download_url":"https://codeload.github.com/cvg/Hierarchical-Localization/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254028940,"owners_count":22002282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","feature-matching","image-retrieval","pose-estimation","structure-from-motion","superglue","visual-localization"],"created_at":"2024-09-24T20:12:43.221Z","updated_at":"2025-05-13T21:08:05.326Z","avatar_url":"https://github.com/cvg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# hloc - the hierarchical localization toolbox\n\nThis is `hloc`, a modular toolbox for state-of-the-art 6-DoF visual localization. It implements [Hierarchical Localization](https://arxiv.org/abs/1812.03506), leveraging image retrieval and feature matching, and is fast, accurate, and scalable. This codebase combines and makes easily accessible years of research on image matching and Structure-from-Motion.\n\nWith `hloc`, you can:\n\n- Reproduce state-of-the-art results on multiple indoor and outdoor visual localization benchmarks\n- Run Structure-from-Motion with SuperPoint+SuperGlue to localize with your own datasets\n- Evaluate your own local features or image retrieval for visual localization\n- Implement new localization pipelines and debug them easily 🔥\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://arxiv.org/abs/1812.03506\"\u003e\u003cimg src=\"doc/hloc.png\" width=\"60%\"/\u003e\u003c/a\u003e\n  \u003cbr /\u003e\u003cem\u003eHierachical Localization uses both image retrieval and feature matching\u003c/em\u003e\n\u003c/p\u003e\n\n##\n\n## Quick start ➡️ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1MrVs9b8aQYODtOGkoaGNF9Nji3sbCNMQ)\n\nBuild 3D maps with Structure-from-Motion and localize any Internet image right from your browser! **You can now run `hloc` and COLMAP in Google Colab with GPU for free.** The notebook [`demo.ipynb`](https://colab.research.google.com/drive/1MrVs9b8aQYODtOGkoaGNF9Nji3sbCNMQ) shows how to run SfM and localization in just a few steps. Try it with your own data and let us know!\n\n## Installation\n\n`hloc` requires Python \u003e=3.7 and PyTorch \u003e=1.1. Installing the package locally pulls the other dependencies:\n\n```bash\ngit clone --recursive https://github.com/cvg/Hierarchical-Localization/\ncd Hierarchical-Localization/\npython -m pip install -e .\n```\n\nAll dependencies are listed in `requirements.txt`. **Starting with `hloc-v1.3`, installing COLMAP is not required anymore.** This repository includes external local features as git submodules – don't forget to pull submodules with `git submodule update --init --recursive`.\n\nWe also provide a Docker image:\n```bash\ndocker build -t hloc:latest .\ndocker run -it --rm -p 8888:8888 hloc:latest  # for GPU support, add `--runtime=nvidia`\njupyter notebook --ip 0.0.0.0 --port 8888 --no-browser --allow-root\n```\n\n## General pipeline\n\nThe toolbox is composed of scripts, which roughly perform the following steps:\n\n1. Extract local features, like [SuperPoint](https://arxiv.org/abs/1712.07629) or [DISK](https://arxiv.org/abs/2006.13566), for all database and query images\n2. Build a reference 3D SfM model\n   1. Find covisible database images, with retrieval or a prior SfM model\n   2. Match these database pairs with [SuperGlue](https://psarlin.com/superglue/) or the faster [LightGlue](https://github.com/cvg/LightGlue)\n   3. Triangulate a new SfM model with COLMAP\n3. Find database images relevant to each query, using retrieval\n4. Match the query images\n5. Run the localization\n6. Visualize and debug\n\nThe localization can then be evaluated on [visuallocalization.net](https://www.visuallocalization.net/) for the supported datasets. When 3D Lidar scans are available, such as for the indoor dataset InLoc, step 2. can be skipped.\n\nStrcture of the toolbox:\n\n- `hloc/*.py` : top-level scripts\n- `hloc/extractors/` : interfaces for feature extractors\n- `hloc/matchers/` : interfaces for feature matchers\n- `hloc/pipelines/` : entire pipelines for multiple datasets\n\n`hloc` can be imported as an external package with `import hloc` or called from the command line with:\n```bash\npython -m hloc.name_of_script --arg1 --arg2\n```\n\n## Tasks\n\nWe provide step-by-step guides to localize with Aachen, InLoc, and to generate reference poses for your own data using SfM. Just download the datasets and you're reading to go!\n\n### Aachen – outdoor localization\n\nHave a look at [`pipeline_Aachen.ipynb`](https://nbviewer.jupyter.org/github/cvg/Hierarchical-Localization/blob/master/pipeline_Aachen.ipynb) for a step-by-step guide on localizing with Aachen. Play with the visualization, try new local features or matcher, and have fun! Don't like notebooks? You can also run all scripts from the command line.\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://nbviewer.jupyter.org/github/cvg/Hierarchical-Localization/blob/master/pipeline_Aachen.ipynb\"\u003e\u003cimg src=\"doc/loc_aachen.svg\" width=\"70%\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n### InLoc – indoor localization\n\nThe notebook [`pipeline_InLoc.ipynb`](https://nbviewer.jupyter.org/github/cvg/Hierarchical-Localization/blob/master/pipeline_InLoc.ipynb) shows the steps for localizing with InLoc. It's much simpler since a 3D SfM model is not needed.\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://nbviewer.jupyter.org/github/cvg/Hierarchical-Localization/blob/master/pipeline_InLoc.ipynb\"\u003e\u003cimg src=\"doc/loc_inloc.svg\" width=\"70%\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n### SfM reconstruction from scratch\n\nWe show in [`pipeline_SfM.ipynb`](https://nbviewer.jupyter.org/github/cvg/Hierarchical-Localization/blob/master/pipeline_SfM.ipynb) how to run 3D reconstruction for an unordered set of images. This generates reference poses, and a nice sparse 3D model suitable for localization with the same pipeline as Aachen.\n\n## Results\n\n- Supported local feature extractors: [SuperPoint](https://arxiv.org/abs/1712.07629), [DISK](https://arxiv.org/abs/2006.13566), [D2-Net](https://arxiv.org/abs/1905.03561), [SIFT](https://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf), and [R2D2](https://arxiv.org/abs/1906.06195).\n- Supported feature matchers: [SuperGlue](https://arxiv.org/abs/1911.11763), its faster follow-up [LightGlue](https://github.com/cvg/LightGlue), and nearest neighbor search with ratio test, distance test, and/or mutual check. hloc also supports dense matching with [LoFTR](https://github.com/zju3dv/LoFTR).\n- Supported image retrieval: [NetVLAD](https://arxiv.org/abs/1511.07247), [AP-GeM/DIR](https://github.com/naver/deep-image-retrieval), [OpenIBL](https://github.com/yxgeee/OpenIBL), and [MegaLoc](https://github.com/gmberton/MegaLoc).\n\nUsing NetVLAD for retrieval, we obtain the following best results:\n\n| Methods                                                      | Aachen day         | Aachen night       | Retrieval      |\n| ------------------------------------------------------------ | ------------------ | ------------------ | -------------- |\n| [SuperPoint + SuperGlue](https://www.visuallocalization.net/details/10931/) | 89.6 / 95.4 / 98.8 | 86.7 / 93.9 / 100  | NetVLAD top 50 |\n| [SuperPoint + NN](https://www.visuallocalization.net/details/10866/) | 85.4 / 93.3 / 97.2 | 75.5 / 86.7 / 92.9 | NetVLAD top 30 |\n| D2Net (SS) + NN                                              | 84.6 / 91.4 / 97.1 | 83.7 / 90.8 / 100  | NetVLAD top 30 |\n\n| Methods                                                      | InLoc DUC1         | InLoc DUC2         | Retrieval      |\n| ------------------------------------------------------------ | ------------------ | ------------------ | -------------- |\n| [SuperPoint + SuperGlue](https://www.visuallocalization.net/details/10936/) | 46.5 / 65.7 / 78.3 | 52.7 / 72.5 / 79.4 | NetVLAD top 40 |\n| [SuperPoint + SuperGlue (temporal)](https://www.visuallocalization.net/details/10937/) | 49.0 / 68.7 / 80.8 | 53.4 / 77.1 / 82.4 | NetVLAD top 40 |\n| [SuperPoint + NN](https://www.visuallocalization.net/details/10896/) | 39.9 / 55.6 / 67.2 | 37.4 / 57.3 / 70.2 | NetVLAD top 20 |\n| D2Net (SS) + NN                                              | 39.9 / 57.6 / 67.2 | 36.6 / 53.4 / 61.8 | NetVLAD top 20 |\n\nCheck out [visuallocalization.net/benchmark](https://www.visuallocalization.net/benchmark) for more details and additional baselines.\n\n## Supported datasets\n\nWe provide in [`hloc/pipelines/`](./hloc/pipelines) scripts to run the reconstruction and the localization on the following datasets: Aachen Day-Night (v1.0 and v1.1), InLoc, Extended CMU Seasons, RobotCar Seasons, 4Seasons, Cambridge Landmarks, and 7-Scenes. For example, after downloading the dataset [with the instructions given here](./hloc/pipelines/Aachen#installation), we can run the Aachen Day-Night pipeline with SuperPoint+SuperGlue using the command:\n```bash\npython -m hloc.pipelines.Aachen.pipeline [--outputs ./outputs/aachen]\n```\n\n## BibTex Citation\n\nIf you report any of the above results in a publication, or use any of the tools provided here, please consider citing both [Hierarchical Localization](https://arxiv.org/abs/1812.03506) and [SuperGlue](https://arxiv.org/abs/1911.11763) papers:\n\n```\n@inproceedings{sarlin2019coarse,\n  title     = {From Coarse to Fine: Robust Hierarchical Localization at Large Scale},\n  author    = {Paul-Edouard Sarlin and\n               Cesar Cadena and\n               Roland Siegwart and\n               Marcin Dymczyk},\n  booktitle = {CVPR},\n  year      = {2019}\n}\n\n@inproceedings{sarlin2020superglue,\n  title     = {{SuperGlue}: Learning Feature Matching with Graph Neural Networks},\n  author    = {Paul-Edouard Sarlin and\n               Daniel DeTone and\n               Tomasz Malisiewicz and\n               Andrew Rabinovich},\n  booktitle = {CVPR},\n  year      = {2020},\n}\n```\n\n## Going further\n\n### Debugging and Visualization\n\n\u003cdetails\u003e\n\u003csummary\u003e[Click to expand]\u003c/summary\u003e\n\nEach localization run generates a pickle log file. For each query, it contains the selected database images, their matches, and information from the pose solver, such as RANSAC inliers. It can thus be parsed to gather statistics and analyze failure modes or difficult scenarios. \n\nWe also provide some visualization tools in [`hloc/visualization.py`](./hloc/visualization.py) to visualize some attributes of the 3D SfM model, such as visibility of the keypoints, their track length, or estimated sparse depth (like below).\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"./pipeline_Aachen.ipynb\"\u003e\u003cimg src=\"doc/depth_aachen.svg\" width=\"60%\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003c/details\u003e\n\n### Using your own local features or matcher\n\n\u003cdetails\u003e\n\u003csummary\u003e[Click to expand]\u003c/summary\u003e\n\nIf your code is based on PyTorch: simply add a new interface in [`hloc/extractors/`](hloc/extractors/) or [`hloc/matchers/`](hloc/matchers/). It needs to inherit from `hloc.utils.base_model.BaseModel`, take as input a data dictionary, and output a prediction dictionary. Have a look at `hloc/extractors/superpoint.py` for an example. You can additionally define a standard configuration in [`hloc/extract_features.py`](hloc/extract_features.py) or [`hloc/match_features.py`](hloc/match_features.py) - it can then be called directly from the command line.\n\nIf your code is based on TensorFlow: you will need to either modify `hloc/extract_features.py` and `hloc/match_features.py`, or export yourself the features and matches to HDF5 files, described below.\n\nIn a feature file, each key corresponds to the relative path of an image w.r.t. the dataset root (e.g. `db/1.jpg` for Aachen), and has one dataset per prediction (e.g. `keypoints` and `descriptors`, with shape Nx2 and DxN).\n\nIn a match file, each key corresponds to the string `path0.replace('/', '-')+'_'+path1.replace('/', '-')` and has a dataset `matches0` with shape N. It indicates, for each keypoint in the first image, the index of the matching keypoint in the second image, or `-1` if the keypoint is unmatched.\n\u003c/details\u003e\n\n### Using your own image retrieval\n\n\u003cdetails\u003e\n\u003csummary\u003e[Click to expand]\u003c/summary\u003e\n\n`hloc` also provides an interface for image retrieval via `hloc/extract_features.py`. As previously, simply add a new interface to [`hloc/extractors/`](hloc/extractors/). Alternatively, you will need to export the global descriptors into an HDF5 file, in which each key corresponds to the relative path of an image w.r.t. the dataset root, and contains a dataset `global_descriptor` with size D. You can then export the images pairs with [`hloc/pairs_from_retrieval.py`](hloc/pairs_from_retrieval.py).\n\u003c/details\u003e\n\n### Reconstruction with known camera parameters\n\n\u003cdetails\u003e\n\u003csummary\u003e[Click to expand]\u003c/summary\u003e\n\nIf the calibration of the camera is known, for example from an external calibration system, you can tell hloc to use these parameters instead of estimating them from EXIF. The name of the camera models and their parameters are [defined by COLMAP](https://colmap.github.io/cameras.html). Python API:\n```python\nopts = dict(camera_model='SIMPLE_RADIAL', camera_params=','.join(map(str, (f, cx, cy, k))))\nmodel = reconstruction.main(..., image_options=opts)\n```\nCommand-line interface:\n```bash\npython -m hloc.reconstruction [...] --image_options camera_model='\"SIMPLE_RADIAL\"' camera_params='\"256,256,256,0\"'\n```\n\nBy default, hloc refines the camera parameters during the reconstruction process. To prevent this, add:\n```python\nreconstruction.main(..., mapper_options=dict(ba_refine_focal_length=False, ba_refine_extra_params=False))\n```\n```bash\npython -m hloc.reconstruction [...] --mapper_options ba_refine_focal_length=False ba_refine_extra_params=False\n```\n\n\u003c/details\u003e\n\n## Versions\n\n\u003cdetails\u003e\n\u003csummary\u003ev1.4 (July 2023)\u003c/summary\u003e\n\n- New front ends\n  - global features: OpenIBL (https://github.com/cvg/Hierarchical-Localization/pull/164), CosPlace (https://github.com/cvg/Hierarchical-Localization/pull/257)\n  - patch descriptors: SOSNet (https://github.com/cvg/Hierarchical-Localization/pull/161), HardNet (https://github.com/cvg/Hierarchical-Localization/pull/235)\n  - detector \u0026 descriptor: DISK (https://github.com/cvg/Hierarchical-Localization/pull/233, https://github.com/cvg/Hierarchical-Localization/pull/291)\n  - sparse matching: AdaLAM (https://github.com/cvg/Hierarchical-Localization/pull/229), LightGlue (https://github.com/cvg/Hierarchical-Localization/pull/285)\n  - dense matching: LoFTR (https://github.com/cvg/Hierarchical-Localization/pull/173, https://github.com/cvg/Hierarchical-Localization/pull/243, https://github.com/cvg/Hierarchical-Localization/pull/254)\n- Triangulation: use known camera poses for two-view geometric verification (https://github.com/cvg/Hierarchical-Localization/pull/178)\n- Control over COLMAP import and reconstruction options (https://github.com/cvg/Hierarchical-Localization/pull/210)\n- Performance\n  - More reliably skip existing pairs in a match file (https://github.com/cvg/Hierarchical-Localization/pull/159)\n  - Faster HDF5 write (https://github.com/cvg/Hierarchical-Localization/pull/194)\n  - Parallel reading and writing in match_features (https://github.com/cvg/Hierarchical-Localization/pull/242)\n- Add scalar detection uncertainty for LaMAR (https://github.com/cvg/Hierarchical-Localization/pull/158)\n- Documentation (https://github.com/cvg/Hierarchical-Localization/pull/294)\n- Updated requirements: tqdm\u003e=4.36.0, pycolmap\u003e=0.3.0, kornia\u003e=0.6.11\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ev1.3 (January 2022)\u003c/summary\u003e\n\n- Demo notebook in Google Colab\n- Use the new pycolmap Reconstruction objects and pipeline API\n  - Do not require an installation of COLMAP anymore - pycolmap is enough\n  - Faster model reading and writing\n  - Fine-grained control over camera sharing via the `camera_mode` parameter\n  - Localization with unknown or inaccurate focal length\n- Modular localization API with control over all estimator parameters\n- 3D visualizations or camera frustums and points with plotly\n- Package-specific logging in the hloc namespace\n- Store the extracted features by default as fp16 instead of fp32\n- Optionally fix a long-standing bug in SuperPoint descriptor sampling\n- Add script to compute exhaustive pairs for reconstruction or localization\n- Require pycolmap\u003e=0.1.0 and Python\u003e=3.7\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ev1.2 (December 2021)\u003c/summary\u003e\n\n- Bug fixes and usability improvements.\n- Support PIL backend for image resizing.\n- Add `__version__` attribute to check against future releases.\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ev1.1 (July 2021)\u003c/summary\u003e\n\n- **Breaking**: improved structure of the SfM folders (triangulation and reconstruction), see [#76](https://github.com/cvg/Hierarchical-Localization/pull/76)\n- Support for image retrieval (NetVLAD, DIR) and more local features (SIFT, R2D2)\n- Support for more datasets: Aachen v1.1, Extended CMU Seasons, RobotCar Seasons, Cambridge Landmarks, 7-Scenes\n- Simplified pipeline and API\n- Spatial matcher\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ev1.0 (July 2020)\u003c/summary\u003e\n\nInitial public version.\n\u003c/details\u003e\n\n## Contributions welcome!\n\nExternal contributions are very much welcome. Please follow the [PEP8 style guidelines](https://www.python.org/dev/peps/pep-0008/) using a linter like flake8. This is a non-exhaustive list of features that might be valuable additions:\n\n- [ ] support for GPS (extraction from EXIF + guided retrieval)\n- [ ] covisibility clustering for InLoc\n- [ ] visualization of the raw predictions (features and matches)\n- [ ] other local features or image retrieval\n\nCreated and maintained by [Paul-Edouard Sarlin](https://psarlin.com/) with the help of many contributors.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvg%2Fhierarchical-localization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcvg%2Fhierarchical-localization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvg%2Fhierarchical-localization/lists"}