{"id":15030914,"url":"https://github.com/cvg/lightglue","last_synced_at":"2025-04-11T06:18:08.577Z","repository":{"id":176465573,"uuid":"658277588","full_name":"cvg/LightGlue","owner":"cvg","description":"LightGlue: Local Feature Matching at Light Speed (ICCV 2023)","archived":false,"fork":false,"pushed_at":"2024-06-20T21:37:54.000Z","size":10304,"stargazers_count":3724,"open_issues_count":83,"forks_count":392,"subscribers_count":49,"default_branch":"main","last_synced_at":"2025-04-11T06:18:03.626Z","etag":null,"topics":["deep-learning","image-matching","pose-estimation","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cvg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-25T09:39:51.000Z","updated_at":"2025-04-10T14:21:51.000Z","dependencies_parsed_at":"2023-09-27T13:28:48.180Z","dependency_job_id":"b94b024e-318f-437c-a9aa-be396adce674","html_url":"https://github.com/cvg/LightGlue","commit_stats":null,"previous_names":["cvg/lightglue"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2FLightGlue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2FLightGlue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2FLightGlue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2FLightGlue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cvg","download_url":"https://codeload.github.com/cvg/LightGlue/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248351396,"owners_count":21089272,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","image-matching","pose-estimation","transformers"],"created_at":"2024-09-24T20:14:31.636Z","updated_at":"2025-04-11T06:18:08.553Z","avatar_url":"https://github.com/cvg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003e\u003cins\u003eLightGlue\u003c/ins\u003e ⚡️\u003cbr\u003eLocal Feature Matching at Light Speed\u003c/h1\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://www.linkedin.com/in/philipplindenberger/\"\u003ePhilipp Lindenberger\u003c/a\u003e\n    ·\n    \u003ca href=\"https://psarlin.com/\"\u003ePaul-Edouard\u0026nbsp;Sarlin\u003c/a\u003e\n    ·\n    \u003ca href=\"https://www.microsoft.com/en-us/research/people/mapoll/\"\u003eMarc\u0026nbsp;Pollefeys\u003c/a\u003e\n  \u003c/p\u003e\n  \u003ch2 align=\"center\"\u003e\n    \u003cp\u003eICCV 2023\u003c/p\u003e\n    \u003ca href=\"https://arxiv.org/pdf/2306.13643.pdf\" align=\"center\"\u003ePaper\u003c/a\u003e | \n    \u003ca href=\"https://colab.research.google.com/github/cvg/LightGlue/blob/main/demo.ipynb\" align=\"center\"\u003eColab\u003c/a\u003e | \n    \u003ca href=\"https://psarlin.com/assets/LightGlue_ICCV2023_poster_compressed.pdf\" align=\"center\"\u003ePoster\u003c/a\u003e | \n    \u003ca href=\"https://github.com/cvg/glue-factory\" align=\"center\"\u003eTrain your own!\u003c/a\u003e\n  \u003c/h2\u003e\n  \n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://arxiv.org/abs/2306.13643\"\u003e\u003cimg src=\"assets/easy_hard.jpg\" alt=\"example\" width=80%\u003e\u003c/a\u003e\n    \u003cbr\u003e\n    \u003cem\u003eLightGlue is a deep neural network that matches sparse local features across image pairs.\u003cbr\u003eAn adaptive mechanism makes it fast for easy pairs (top) and reduces the computational complexity for difficult ones (bottom).\u003c/em\u003e\n\u003c/p\u003e\n\n##\n\nThis repository hosts the inference code of LightGlue, a lightweight feature matcher with high accuracy and blazing fast inference. It takes as input a set of keypoints and descriptors for each image and returns the indices of corresponding points. The architecture is based on adaptive pruning techniques, in both network width and depth - [check out the paper for more details](https://arxiv.org/pdf/2306.13643.pdf).\n\nWe release pretrained weights of LightGlue with [SuperPoint](https://arxiv.org/abs/1712.07629), [DISK](https://arxiv.org/abs/2006.13566), [ALIKED](https://arxiv.org/abs/2304.03608) and [SIFT](https://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf) local features.\nThe training and evaluation code can be found in our library [glue-factory](https://github.com/cvg/glue-factory/).\n\n## Installation and demo [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/cvg/LightGlue/blob/main/demo.ipynb)\n\nInstall this repo using pip:\n\n```bash\ngit clone https://github.com/cvg/LightGlue.git \u0026\u0026 cd LightGlue\npython -m pip install -e .\n```\n\nWe provide a [demo notebook](demo.ipynb) which shows how to perform feature extraction and matching on an image pair.\n\nHere is a minimal script to match two images:\n\n```python\nfrom lightglue import LightGlue, SuperPoint, DISK, SIFT, ALIKED, DoGHardNet\nfrom lightglue.utils import load_image, rbd\n\n# SuperPoint+LightGlue\nextractor = SuperPoint(max_num_keypoints=2048).eval().cuda()  # load the extractor\nmatcher = LightGlue(features='superpoint').eval().cuda()  # load the matcher\n\n# or DISK+LightGlue, ALIKED+LightGlue or SIFT+LightGlue\nextractor = DISK(max_num_keypoints=2048).eval().cuda()  # load the extractor\nmatcher = LightGlue(features='disk').eval().cuda()  # load the matcher\n\n# load each image as a torch.Tensor on GPU with shape (3,H,W), normalized in [0,1]\nimage0 = load_image('path/to/image_0.jpg').cuda()\nimage1 = load_image('path/to/image_1.jpg').cuda()\n\n# extract local features\nfeats0 = extractor.extract(image0)  # auto-resize the image, disable with resize=None\nfeats1 = extractor.extract(image1)\n\n# match the features\nmatches01 = matcher({'image0': feats0, 'image1': feats1})\nfeats0, feats1, matches01 = [rbd(x) for x in [feats0, feats1, matches01]]  # remove batch dimension\nmatches = matches01['matches']  # indices with shape (K,2)\npoints0 = feats0['keypoints'][matches[..., 0]]  # coordinates in image #0, shape (K,2)\npoints1 = feats1['keypoints'][matches[..., 1]]  # coordinates in image #1, shape (K,2)\n```\n\nWe also provide a convenience method to match a pair of images:\n\n```python\nfrom lightglue import match_pair\nfeats0, feats1, matches01 = match_pair(extractor, matcher, image0, image1)\n```\n\n##\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://arxiv.org/abs/2306.13643\"\u003e\u003cimg src=\"assets/teaser.svg\" alt=\"Logo\" width=50%\u003e\u003c/a\u003e\n  \u003cbr\u003e\n  \u003cem\u003eLightGlue can adjust its depth (number of layers) and width (number of keypoints) per image pair, with a marginal impact on accuracy.\u003c/em\u003e\n\u003c/p\u003e\n\n## Advanced configuration\n\n\u003cdetails\u003e\n\u003csummary\u003e[Detail of all parameters - click to expand]\u003c/summary\u003e\n\n- ```n_layers```: Number of stacked self+cross attention layers. Reduce this value for faster inference at the cost of accuracy (continuous red line in the plot above). Default: 9 (all layers).\n- ```flash```: Enable FlashAttention. Significantly increases the speed and reduces the memory consumption without any impact on accuracy. Default: True (LightGlue automatically detects if FlashAttention is available).\n- ```mp```: Enable mixed precision inference. Default: False (off)\n- ```depth_confidence```: Controls the early stopping. A lower values stops more often at earlier layers. Default: 0.95, disable with -1.\n- ```width_confidence```: Controls the iterative point pruning. A lower value prunes more points earlier. Default: 0.99, disable with -1.\n- ```filter_threshold```: Match confidence. Increase this value to obtain less, but stronger matches. Default: 0.1\n\n\u003c/details\u003e\n\nThe default values give a good trade-off between speed and accuracy. To maximize the accuracy, use all keypoints and disable the adaptive mechanisms:\n```python\nextractor = SuperPoint(max_num_keypoints=None)\nmatcher = LightGlue(features='superpoint', depth_confidence=-1, width_confidence=-1)\n```\n\nTo increase the speed with a small drop of accuracy, decrease the number of keypoints and lower the adaptive thresholds:\n```python\nextractor = SuperPoint(max_num_keypoints=1024)\nmatcher = LightGlue(features='superpoint', depth_confidence=0.9, width_confidence=0.95)\n```\n\nThe maximum speed is obtained with a combination of:\n- [FlashAttention](https://arxiv.org/abs/2205.14135): automatically used when ```torch \u003e= 2.0``` or if [installed from source](https://github.com/HazyResearch/flash-attention#installation-and-features).\n- PyTorch compilation, available when ```torch \u003e= 2.0```:\n```python\nmatcher = matcher.eval().cuda()\nmatcher.compile(mode='reduce-overhead')\n```\nFor inputs with fewer than 1536 keypoints (determined experimentally), this compiles LightGlue but disables point pruning (large overhead). For larger input sizes, it automatically falls backs to eager mode with point pruning. Adaptive depths is supported for any input size.\n\n## Benchmark\n\n\n\u003cp align=\"center\"\u003e\n  \u003ca\u003e\u003cimg src=\"assets/benchmark.png\" alt=\"Logo\" width=80%\u003e\u003c/a\u003e\n  \u003cbr\u003e\n  \u003cem\u003eBenchmark results on GPU (RTX 3080). With compilation and adaptivity, LightGlue runs at 150 FPS @ 1024 keypoints and 50 FPS @ 4096 keypoints per image. This is a 4-10x speedup over SuperGlue. \u003c/em\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca\u003e\u003cimg src=\"assets/benchmark_cpu.png\" alt=\"Logo\" width=80%\u003e\u003c/a\u003e\n  \u003cbr\u003e\n  \u003cem\u003eBenchmark results on CPU (Intel i7 10700K). LightGlue runs at 20 FPS @ 512 keypoints. \u003c/em\u003e\n\u003c/p\u003e\n\nObtain the same plots for your setup using our [benchmark script](benchmark.py):\n```\npython benchmark.py [--device cuda] [--add_superglue] [--num_keypoints 512 1024 2048 4096] [--compile]\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e[Performance tip - click to expand]\u003c/summary\u003e\n\nNote: **Point pruning** introduces an overhead that sometimes outweighs its benefits.\nPoint pruning is thus enabled only when the there are more than N keypoints in an image, where N is hardware-dependent.\nWe provide defaults optimized for current hardware (RTX 30xx GPUs).\nWe suggest running the benchmark script and adjusting the thresholds for your hardware by updating `LightGlue.pruning_keypoint_thresholds['cuda']`.\n\n\u003c/details\u003e\n\n## Training and evaluation\n\nWith [Glue Factory](https://github.com/cvg/glue-factory), you can train LightGlue with your own local features, on your own dataset!\nYou can also evaluate it and other baselines on standard benchmarks like HPatches and MegaDepth.\n\n## Other links\n- [hloc - the visual localization toolbox](https://github.com/cvg/Hierarchical-Localization/): run LightGlue for Structure-from-Motion and visual localization.\n- [LightGlue-ONNX](https://github.com/fabio-sim/LightGlue-ONNX): export LightGlue to the Open Neural Network Exchange (ONNX) format with support for TensorRT and OpenVINO.\n- [Image Matching WebUI](https://github.com/Vincentqyw/image-matching-webui): a web GUI to easily compare different matchers, including LightGlue.\n- [kornia](https://kornia.readthedocs.io) now exposes LightGlue via the interfaces [`LightGlue`](https://kornia.readthedocs.io/en/latest/feature.html#kornia.feature.LightGlue) and [`LightGlueMatcher`](https://kornia.readthedocs.io/en/latest/feature.html#kornia.feature.LightGlueMatcher).\n\n## BibTeX citation\nIf you use any ideas from the paper or code from this repo, please consider citing:\n\n```txt\n@inproceedings{lindenberger2023lightglue,\n  author    = {Philipp Lindenberger and\n               Paul-Edouard Sarlin and\n               Marc Pollefeys},\n  title     = {{LightGlue: Local Feature Matching at Light Speed}},\n  booktitle = {ICCV},\n  year      = {2023}\n}\n```\n\n\n## License\nThe pre-trained weights of LightGlue and the code provided in this repository are released under the [Apache-2.0 license](./LICENSE). [DISK](https://github.com/cvlab-epfl/disk) follows this license as well but SuperPoint follows [a different, restrictive license](https://github.com/magicleap/SuperPointPretrainedNetwork/blob/master/LICENSE) (this includes its pre-trained weights and its [inference file](./lightglue/superpoint.py)). [ALIKED](https://github.com/Shiaoming/ALIKED) was published under a BSD-3-Clause license. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvg%2Flightglue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcvg%2Flightglue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvg%2Flightglue/lists"}