{"id":19517484,"url":"https://github.com/cvg/glue-factory","last_synced_at":"2025-05-16T13:06:02.062Z","repository":{"id":198513420,"uuid":"700919494","full_name":"cvg/glue-factory","owner":"cvg","description":"Training library for local feature detection and matching","archived":false,"fork":false,"pushed_at":"2024-08-06T02:20:30.000Z","size":1878,"stargazers_count":879,"open_issues_count":62,"forks_count":112,"subscribers_count":21,"default_branch":"main","last_synced_at":"2025-05-09T22:03:27.793Z","etag":null,"topics":["computer-vision","deep-learning","iccv2023","image-matching"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cvg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-05T14:50:04.000Z","updated_at":"2025-05-09T10:39:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"c357ef4e-9275-4882-83b2-63c966d9ecd1","html_url":"https://github.com/cvg/glue-factory","commit_stats":{"total_commits":19,"total_committers":7,"mean_commits":"2.7142857142857144","dds":0.5263157894736843,"last_synced_commit":"1f56839db2242929960d70f85bfac6c19ef2821c"},"previous_names":["cvg/glue-factory"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2Fglue-factory","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2Fglue-factory/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2Fglue-factory/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2Fglue-factory/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cvg","download_url":"https://codeload.github.com/cvg/glue-factory/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254535827,"owners_count":22087399,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","deep-learning","iccv2023","image-matching"],"created_at":"2024-11-11T00:03:55.382Z","updated_at":"2025-05-16T13:06:02.042Z","avatar_url":"https://github.com/cvg.png","language":"Python","funding_links":[],"categories":["Deep Learning Feature (Local Feature)"],"sub_categories":[],"readme":"# Glue Factory\nGlue Factory is CVG's library for training and evaluating deep neural network that extract and match local visual feature. It enables you to:\n- Reproduce the training of state-of-the-art models for point and line matching, like [LightGlue](https://github.com/cvg/LightGlue) and [GlueStick](https://github.com/cvg/GlueStick) (ICCV 2023)\n- Train these models on multiple datasets using your own local features or lines\n- Evaluate feature extractors or matchers on standard benchmarks like HPatches or MegaDepth-1500\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/cvg/LightGlue\"\u003e\u003cimg src=\"docs/lightglue_matches.svg\" width=\"60%\"/\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/cvg/GlueStick\"\u003e\u003cimg src=\"docs/gluestick_img.svg\" width=\"60%\"/\u003e\u003c/a\u003e\n  \u003cbr /\u003e\u003cem\u003ePoint and line matching with LightGlue and GlueStick.\u003c/em\u003e\n\u003c/p\u003e\n\n## Installation\nGlue Factory runs with Python 3 and [PyTorch](https://pytorch.org/). The following installs the library and its basic dependencies:\n```bash\ngit clone https://github.com/cvg/glue-factory\ncd glue-factory\npython3 -m pip install -e .  # editable mode\n```\nSome advanced features might require installing the full set of dependencies:\n```bash\npython3 -m pip install -e .[extra]\n```\n\nAll models and datasets in gluefactory have auto-downloaders, so you can get started right away!\n\n## License\nThe code and trained models in Glue Factory are released with an Apache-2.0 license. This includes LightGlue and an [open version of SuperPoint](https://github.com/rpautrat/SuperPoint). Third-party models that are not compatible with this license, such as SuperPoint (original) and SuperGlue, are provided in `gluefactory_nonfree`, where each model might follow its own, restrictive license.\n\n## Evaluation\n\n#### HPatches\nRunning the evaluation commands automatically downloads the dataset, by default to the directory `data/`. You will need about 1.8 GB of free disk space.\n\n\u003cdetails\u003e\n\u003csummary\u003e[Evaluating LightGlue]\u003c/summary\u003e\n\nTo evaluate the pre-trained SuperPoint+LightGlue model on HPatches, run:\n```bash\npython -m gluefactory.eval.hpatches --conf superpoint+lightglue-official --overwrite\n```\nYou should expect the following results\n```\n{'H_error_dlt@1px': 0.3515,\n 'H_error_dlt@3px': 0.6723,\n 'H_error_dlt@5px': 0.7756,\n 'H_error_ransac@1px': 0.3428,\n 'H_error_ransac@3px': 0.5763,\n 'H_error_ransac@5px': 0.6943,\n 'mnum_keypoints': 1024.0,\n 'mnum_matches': 560.756,\n 'mprec@1px': 0.337,\n 'mprec@3px': 0.89,\n 'mransac_inl': 130.081,\n 'mransac_inl%': 0.217,\n 'ransac_mAA': 0.5378}\n```\n\nThe default robust estimator is `opencv`, but we strongly recommend to use `poselib` instead:\n```bash\npython -m gluefactory.eval.hpatches --conf superpoint+lightglue-official --overwrite \\\n    eval.estimator=poselib eval.ransac_th=-1\n```\nSetting `eval.ransac_th=-1` auto-tunes the RANSAC inlier threshold by running the evaluation with a range of thresholds and reports results for the optimal value.\nHere are the results as Area Under the Curve (AUC) of the homography error at  1/3/5 pixels:\n\n| Methods                                                      | DLT         | [OpenCV](../gluefactory/robust_estimators/homography/opencv.py)       | [PoseLib](../gluefactory/robust_estimators/homography/poselib.py)      |\n| ------------------------------------------------------------ | ------------------ | ------------------ | ------------------ |\n| [SuperPoint + SuperGlue](gluefactory/configs/superpoint+superglue-official.yaml) | 32.1 / 65.0 / 75.7 | 32.9 / 55.7 / 68.0 | 37.0 / 68.2 / 78.7 |\n| [SuperPoint + LightGlue](gluefactory/configs/superpoint+lightglue-official.yaml) | 35.1 / 67.2 / 77.6 | 34.2 / 57.9 / 69.9 | 37.1 / 67.4 / 77.8 |\n\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e[Evaluating GlueStick]\u003c/summary\u003e\n\nTo evaluate GlueStick on HPatches, run:\n```bash\npython -m gluefactory.eval.hpatches --conf gluefactory/configs/superpoint+lsd+gluestick.yaml --overwrite\n```\nYou should expect the following results\n```\n{\"mprec@1px\": 0.245,\n \"mprec@3px\": 0.838,\n \"mnum_matches\": 1290.5,\n \"mnum_keypoints\": 2287.5,\n \"mH_error_dlt\": null,\n \"H_error_dlt@1px\": 0.3355,\n \"H_error_dlt@3px\": 0.6637,\n \"H_error_dlt@5px\": 0.7713,\n \"H_error_ransac@1px\": 0.3915,\n \"H_error_ransac@3px\": 0.6972,\n \"H_error_ransac@5px\": 0.7955,\n \"H_error_ransac_mAA\": 0.62806,\n \"mH_error_ransac\": null}\n```\n\nSince we use points and lines to solve for the homography, we use a different robust estimator here: [Hest](https://github.com/rpautrat/homography_est/). Here are the results as Area Under the Curve (AUC) of the homography error at  1/3/5 pixels:\n\n| Methods                                                      | DLT         | [Hest](gluefactory/robust_estimators/homography/homography_est.py)       |\n| ------------------------------------------------------------ | ------------------ | ------------------ |\n| [SP + LSD + GlueStick](gluefactory/configs/superpoint+lsd+gluestick.yaml) | 33.6 / 66.4 / 77.1 | 39.2 / 69.7 / 79.6 |\n\n\u003c/details\u003e\n\n\n#### MegaDepth-1500\n\nRunning the evaluation commands automatically downloads the dataset, which takes about 1.5 GB of disk space.\n\n\u003cdetails\u003e\n\u003csummary\u003e[Evaluating LightGlue]\u003c/summary\u003e\n\nTo evaluate the pre-trained SuperPoint+LightGlue model on MegaDepth-1500, run:\n```bash\npython -m gluefactory.eval.megadepth1500 --conf superpoint+lightglue-official\n# or the adaptive variant\npython -m gluefactory.eval.megadepth1500 --conf superpoint+lightglue-official \\\n    model.matcher.{depth_confidence=0.95,width_confidence=0.95}\n```\nThe first command should print the following results\n```\n{'mepi_prec@1e-3': 0.795,\n 'mepi_prec@1e-4': 0.15,\n 'mepi_prec@5e-4': 0.567,\n 'mnum_keypoints': 2048.0,\n 'mnum_matches': 613.287,\n 'mransac_inl': 280.518,\n 'mransac_inl%': 0.442,\n 'rel_pose_error@10°': 0.681,\n 'rel_pose_error@20°': 0.8065,\n 'rel_pose_error@5°': 0.5102,\n 'ransac_mAA': 0.6659}\n```\n\nTo use the PoseLib estimator:\n\n```bash\npython -m gluefactory.eval.megadepth1500 --conf superpoint+lightglue-official \\\n    eval.estimator=poselib eval.ransac_th=2.0\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e[Evaluating GlueStick]\u003c/summary\u003e\n\nTo evaluate the pre-trained SuperPoint+GlueStick model on MegaDepth-1500, run:\n```bash\npython -m gluefactory.eval.megadepth1500 --conf gluefactory/configs/superpoint+lsd+gluestick.yaml\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\nHere are the results as Area Under the Curve (AUC) of the pose error at  5/10/20 degrees:\n\n| Methods                                                      | [pycolmap](../gluefactory/robust_estimators/relative_pose/pycolmap.py)         | [OpenCV](../gluefactory/robust_estimators/relative_pose/opencv.py)       | [PoseLib](../gluefactory/robust_estimators/relative_pose/poselib.py)      |\n| ------------------------------------------------------------ | ------------------ | ------------------ | ------------------ |\n| [SuperPoint + SuperGlue](gluefactory/configs/superpoint+superglue-official.yaml) | 54.4 / 70.4 / 82.4 | 48.7 / 65.6 / 79.0 | 64.8 / 77.9 / 87.0 |\n| [SuperPoint + LightGlue](gluefactory/configs/superpoint+lightglue-official.yaml) | 56.7 / 72.4 / 83.7 | 51.0 / 68.1 / 80.7 | 66.8 / 79.3 / 87.9 |\n| [SIFT (2K) + LightGlue](gluefactory/configs/sift+lightglue-official.yaml) | ? / ? / ? | 43.5 / 61.5 / 75.9 | 60.4 / 74.3 / 84.5 |\n| [SIFT (4K) + LightGlue](gluefactory/configs/sift+lightglue-official.yaml) | ? / ? / ? | 49.9 / 67.3 / 80.3 | 65.9 / 78.6 / 87.4 |\n| [ALIKED + LightGlue](gluefactory/configs/aliked+lightglue-official.yaml) | ? / ? / ? | 51.5 / 68.1 / 80.4 | 66.3 / 78.7 / 87.5 |\n| [SuperPoint + GlueStick](gluefactory/configs/superpoint+lsd+gluestick.yaml) | 53.2 / 69.8 / 81.9 | 46.3 / 64.2 / 78.1 | 64.4 / 77.5 / 86.5 |\n\n\u003c/details\u003e\n\n\n#### ETH3D\n\nThe dataset will be auto-downloaded if it is not found on disk, and will need about 6 GB of free disk space.\n\n\u003cdetails\u003e\n\u003csummary\u003e[Evaluating GlueStick]\u003c/summary\u003e\n\nTo evaluate GlueStick on ETH3D, run:\n```bash\npython -m gluefactory.eval.eth3d --conf gluefactory/configs/superpoint+lsd+gluestick.yaml\n```\nYou should expect the following results\n```\nAP: 77.92\nAP_lines: 69.22\n```\n\n\u003c/details\u003e\n\n#### Image Matching Challenge 2021\nComing soon!\n\n#### Image Matching Challenge 2023\nComing soon!\n\n#### Visual inspection\n\u003cdetails\u003e\nTo inspect the evaluation visually, you can run:\n\n```bash\npython -m gluefactory.eval.inspect hpatches superpoint+lightglue-official\n```\n\nClick on a point to visualize matches on this pair.\n\nTo compare multiple methods on a dataset:\n\n```bash\npython -m gluefactory.eval.inspect hpatches superpoint+lightglue-official superpoint+superglue-official\n```\n\nAll current benchmarks are supported by the viewer.\n\u003c/details\u003e\n\nDetailed evaluation instructions can be found [here](./docs/evaluation.md).\n\n## Training\n\nWe generally follow a two-stage training:\n1. Pre-train on a large dataset of synthetic homographies applied to internet images. We use the 1M-image distractor set of the Oxford-Paris retrieval dataset. It requires about 450 GB of disk space.\n2. Fine-tune on the MegaDepth dataset, which is based on PhotoTourism pictures of popular landmarks around the world. It exhibits more complex and realistic appearance and viewpoint changes.  It requires about 420 GB of disk space.\n\nAll training commands automatically download the datasets.\n\n\u003cdetails\u003e\n\u003csummary\u003e[Training LightGlue]\u003c/summary\u003e\n\nWe show how to train LightGlue with [SuperPoint](https://github.com/magicleap/SuperPointPretrainedNetwork).\nWe first pre-train LightGlue on the homography dataset:\n```bash\npython -m gluefactory.train sp+lg_homography \\  # experiment name\n    --conf gluefactory/configs/superpoint+lightglue_homography.yaml\n```\nFeel free to use any other experiment name. By default the checkpoints are written to `outputs/training/`. The default batch size of 128 corresponds to the results reported in the paper and requires 2x 3090 GPUs with 24GB of VRAM each as well as PyTorch \u003e= 2.0 (FlashAttention).\nConfigurations are managed by [OmegaConf](https://omegaconf.readthedocs.io/) so any entry can be overridden from the command line.\nIf you have PyTorch \u003c 2.0 or weaker GPUs, you may thus need to reduce the batch size via:\n```bash\npython -m gluefactory.train sp+lg_homography \\\n    --conf gluefactory/configs/superpoint+lightglue_homography.yaml  \\\n    data.batch_size=32  # for 1x 1080 GPU\n```\nBe aware that this can impact the overall performance. You might need to adjust the learning rate accordingly.\n\nWe then fine-tune the model on the MegaDepth dataset:\n```bash\npython -m gluefactory.train sp+lg_megadepth \\\n    --conf gluefactory/configs/superpoint+lightglue_megadepth.yaml \\\n    train.load_experiment=sp+lg_homography\n```\n\nHere the default batch size is 32. To speed up training on MegaDepth, we suggest to cache the local features before training (requires around 150 GB of disk space):\n```bash\n# extract features\npython -m gluefactory.scripts.export_megadepth --method sp --num_workers 8\n# run training with cached features\npython -m gluefactory.train sp+lg_megadepth \\\n    --conf gluefactory/configs/superpoint+lightglue_megadepth.yaml \\\n    train.load_experiment=sp+lg_homography \\\n    data.load_features.do=True\n```\n\nThe model can then be evaluated using its experiment name:\n```bash\npython -m gluefactory.eval.megadepth1500 --checkpoint sp+lg_megadepth\n```\n\nYou can also run all benchmarks after each training epoch with the option `--run_benchmarks`.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e[Training GlueStick]\u003c/summary\u003e\n\nWe first pre-train GlueStick on the homography dataset:\n```bash\npython -m gluefactory.train gluestick_H --conf gluefactory/configs/superpoint+lsd+gluestick-homography.yaml --distributed\n```\nFeel free to use any other experiment name. Configurations are managed by [OmegaConf](https://omegaconf.readthedocs.io/) so any entry can be overridden from the command line.\n\nWe then fine-tune the model on the MegaDepth dataset:\n```bash\npython -m gluefactory.train gluestick_MD --conf gluefactory/configs/superpoint+lsd+gluestick-megadepth.yaml --distributed\n```\nNote that we used the training splits `train_scenes.txt` and `valid_scenes.txt` to train the original model, which contains some overlap with the IMC challenge. The new default splits are now `train_scenes_clean.txt` and `valid_scenes_clean.txt`, without this overlap.\n\n\u003c/details\u003e\n\n### Available models\nGlue Factory supports training and evaluating the following deep matchers:\n| Model     | Training? | Evaluation? |\n| --------- | --------- | ----------- |\n| [LightGlue](https://github.com/cvg/LightGlue) | ✅         | ✅           |\n| [GlueStick](https://github.com/cvg/GlueStick) | ✅         | ✅           |\n| [SuperGlue](https://github.com/magicleap/SuperGluePretrainedNetwork) | ✅         | ✅           |\n| [LoFTR](https://github.com/zju3dv/LoFTR)     | ❌         | ✅           |\n\nUsing the following local feature extractors:\n\n| Model     | LightGlue config |\n| --------- | --------- |\n| [SuperPoint (open)](https://github.com/rpautrat/SuperPoint) | `superpoint-open+lightglue_{homography,megadepth}.yaml` |\n| [SuperPoint (official)](https://github.com/magicleap/SuperPointPretrainedNetwork) | `superpoint+lightglue_{homography,megadepth}.yaml` |\n| SIFT (via [pycolmap](https://github.com/colmap/pycolmap)) | `sift+lightglue_{homography,megadepth}.yaml` |\n| [ALIKED](https://github.com/Shiaoming/ALIKED) | `aliked+lightglue_{homography,megadepth}.yaml` |\n| [DISK](https://github.com/cvlab-epfl/disk) | `disk+lightglue_{homography,megadepth}.yaml` |\n| Key.Net + HardNet | ❌ TODO |\n\n## Coming soon\n- [ ] More baselines (LoFTR, ASpanFormer, MatchFormer, SGMNet, DKM, RoMa)\n- [ ] Training deep detectors and descriptors like SuperPoint\n- [ ] IMC evaluations\n- [ ] Better documentation\n\n## BibTeX Citation\nPlease consider citing the following papers if you found this library useful:\n```bibtex\n@InProceedings{lindenberger_2023_lightglue,\n  title     = {{LightGlue: Local Feature Matching at Light Speed}},\n  author    = {Philipp Lindenberger and\n               Paul-Edouard Sarlin and\n               Marc Pollefeys},\n  booktitle = {International Conference on Computer Vision (ICCV)},\n  year      = {2023}\n}\n```\n```bibtex\n@InProceedings{pautrat_suarez_2023_gluestick,\n  title     = {{GlueStick: Robust Image Matching by Sticking Points and Lines Together}},\n  author    = {R{\\'e}mi Pautrat* and\n               Iago Su{\\'a}rez* and\n               Yifan Yu and\n               Marc Pollefeys and\n               Viktor Larsson},\n  booktitle = {International Conference on Computer Vision (ICCV)},\n  year      = {2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvg%2Fglue-factory","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcvg%2Fglue-factory","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvg%2Fglue-factory/lists"}