{"id":28666211,"url":"https://github.com/cvg/scrstudio","last_synced_at":"2025-10-11T03:59:11.082Z","repository":{"id":285468048,"uuid":"911108978","full_name":"cvg/scrstudio","owner":"cvg","description":"[CVPR 2025] A unified framework for Scene Coordinate Regression-based visual localization","archived":false,"fork":false,"pushed_at":"2025-04-23T13:32:58.000Z","size":358,"stargazers_count":88,"open_issues_count":2,"forks_count":0,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-04-23T14:33:35.706Z","etag":null,"topics":["3d-reconstruction","3d-vision","computer-vision","deep-learning","pose-estimation","scene-coordinate-regression","visual-localization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cvg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-02T09:08:02.000Z","updated_at":"2025-04-23T13:33:01.000Z","dependencies_parsed_at":"2025-03-31T23:22:24.111Z","dependency_job_id":"d175dd7f-6356-456e-b3ea-fd531ef9ccef","html_url":"https://github.com/cvg/scrstudio","commit_stats":null,"previous_names":["cvg/scrstudio"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cvg/scrstudio","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2Fscrstudio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2Fscrstudio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2Fscrstudio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2Fscrstudio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cvg","download_url":"https://codeload.github.com/cvg/scrstudio/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cvg%2Fscrstudio/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259659757,"owners_count":22891682,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction","3d-vision","computer-vision","deep-learning","pose-estimation","scene-coordinate-regression","visual-localization"],"created_at":"2025-06-13T14:13:51.515Z","updated_at":"2025-10-11T03:59:11.061Z","avatar_url":"https://github.com/cvg.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\n    \u003c!-- \u003cimg alt=\"scrstudio\" src=\"media/sdf_studio_4.png\" width=\"300\"\u003e --\u003e\n    \u003ch1 align=\"center\"\u003eSCRStudio \u003cbr\u003e A Unified Framework for Scene Coordinate Regression\u003c/h1\u003e\n        \u003ch3 align=\"center\"\u003e\u003ca href=\"https://drive.google.com/file/d/1yZw_ZeZkq6MjIZhTXfmQSPbVg8oohdKs/view?usp=sharing\"\u003ePaper\u003c/a\u003e \n    \u003cimg src=\"docs/module.png\" center width=\"95%\"/\u003e\n\u003c/p\u003e\n\n# About\nSCRStudio is a unified and modular framework for Scene Coordinate Regression (SCR)-based visual localization, built on top of the [nerfstudio](https://github.com/nerfstudio-project/nerfstudio) project.\n\nThis library provides an interpretable and modular implementation of SCRs, breaking down components such as input encoding, network architecture, and supervision strategies. It offers a unified implementation of three major SCR methods: **ACE, GLACE, and R-SCoRe**. SCRStudio supports various pretrained local encodings (both sparse and dense) while incorporating state-of-the-art techniques for integrating global encodings.\n\n\n# Quickstart\n\nThis guide will help you get started with the default R-SCoRe SCR model trained on the classic **Aachen** dataset.\n\n## 1. Installation: Setup the Environment\n\n### Create Environment\n\nWe recommend using **conda** to manage dependencies. Make sure to install [Conda](https://docs.conda.io/miniconda.html) before proceeding.\n\n\n### Install Dependencies\n\nInstall PyTorch with CUDA (tested with CUDA 12.1 and 12.4). PyTorch Geometric and cuML are also required for encoding preprocessing.\n\nFor **CUDA 12.4**:\n```bash\nconda create -n scrstudio python=3.10 pytorch=2.5.1 torchvision=0.20.1 pytorch-cuda=12.4 cuml=25.02 -c pytorch  -c rapidsai -c conda-forge -c nvidia\nconda activate scrstudio\npip install --upgrade pip\npip install torch_geometric\npip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.5.1+cu124.html\n```\n\n\n\n### Install SCRStudio\n\n```bash\ngit clone --recursive https://github.com/cvg/scrstudio.git\ncd scrstudio\npip install --upgrade pip setuptools\npip install -e .\n```\n\n## 2. Train Your First Model\n\nThe following steps will train a **scrfacto** model, our recommended model for large scenes.\n\n### Download the Data\n\n```bash\n# Download Aachen dataset:\nscr-download-data aachen\n# Download specific capture for NAVER Lab dataset:\nscr-download-data naver --capture-name dept_1F\n```\n\n### Preprocessing for Training\n\n**scrfacto** follows the methodology from **R-SCoRe** ([paper](https://arxiv.org/abs/2501.01421)), utilizing **PCA** for local encoding dimensionality reduction and **[Node2Vec](https://arxiv.org/abs/1607.00653)** for learning global encodings.\n\n#### Local Encoding: PCA Compression\nTo reduce GPU memory usage for local encoding buffer, apply PCA on local encodings:\n```bash\n# Compute PCA for Dedode local encodings on Aachen dataset\nscr-encoding-pca dedode --encoder.detector L --encoder.descriptor B --n_components 128 --data data/aachen\n```\n\nThis saves the PCA components as PyTorch state dictionary at: `data/aachen/proc/pcad3LB_128.pth`\n\n#### Global Encoding: Covisibility Graph \u0026 Node2Vec Training\nCompute the pose overlap score for training images:\n\n```bash\nscr-overlap-score --data data/aachen/train --max_depth 50\n```\nThis saves a sparse COO format overlap matrix at: `data/aachen/train/pose_overlap.npz`\n\nTrain a **Node2Vec** model on this graph:\n```bash\nscr-train node2vec --data data/aachen --pipeline.model.graph pose_overlap.npz --pipeline.model.edge_threshold 0.2\n```\nUse the trained global encoding:\n\n```bash\ncp outputs/aachen/node2vec/\u003ctimestamp\u003e/scrstudio_models/head.pt data/aachen/train/pose_n2c.pt\n```\n\n### Model Training\nNow, train the **scrfacto** model:\n```bash\nscr-train scrfacto --data data/aachen --pipeline.datamanager.train_dataset.feat_name pose_n2c.pt\n```\nResults are saved in `outputs/aachen/scrfacto/\u003ctimestamp\u003e`.\n\n### 3. Evaluation\n\n#### Preprocessing for Evaluation\n\nCompute NetVLAD retrieve features and compress them with Product Quantization (PQ):\n\n```bash\nscr-retrieval-feat --data data/aachen/train --pq\n```\nResults are saved in `data/aachen/train/netvlad_feats_pq.pkl`\n\n### Running Evaluation\nCompute retrieval features for test images and run evaluation:\n\n```bash\nscr-retrieval-feat --data data/aachen/test\nscr-eval --load-config outputs/aachen/scrfacto/\u003ctimestamp\u003e/config.yml --split test\n```\n\n\n# Publications\n\nThis code builds on previous camera relocalization pipelines, namely DSAC, DSAC++, DSAC*, ACE, GLACE, and R-SCoRe.\nPlease consider citing:\n\n```\n@inproceedings{brachmann2017dsac,\n  title={{DSAC}-{Differentiable RANSAC} for Camera Localization},\n  author={Brachmann, Eric and Krull, Alexander and Nowozin, Sebastian and Shotton, Jamie and Michel, Frank and Gumhold, Stefan and Rother, Carsten},\n  booktitle={CVPR},\n  year={2017}\n}\n\n@inproceedings{brachmann2018lessmore,\n  title={Learning less is more - {6D} camera localization via {3D} surface regression},\n  author={Brachmann, Eric and Rother, Carsten},\n  booktitle={CVPR},\n  year={2018}\n}\n\n@article{brachmann2021dsacstar,\n  title={Visual Camera Re-Localization from {RGB} and {RGB-D} Images Using {DSAC}},\n  author={Brachmann, Eric and Rother, Carsten},\n  journal={TPAMI},\n  year={2021}\n}\n\n@inproceedings{brachmann2023ace,\n    title={Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses},\n    author={Brachmann, Eric and Cavallari, Tommaso and Prisacariu, Victor Adrian},\n    booktitle={CVPR},\n    year={2023},\n}\n\n@inproceedings{wang2024glace,\n    title={Glace: Global local accelerated coordinate encoding},\n    author={Wang, Fangjinhua and Jiang, Xudong and Galliani, Silvano and Vogel, Christoph and Pollefeys, Marc},\n    booktitle={CVPR},\n    year={2024}\n}\n\n@inproceedings{jiang2025rscore,\n      title={R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localization},\n      author={Jiang, Xudong and Wang, Fangjinhua and Galliani, Silvano and Vogel, Christoph and Pollefeys, Marc},\n      booktitle = {CVPR},\n      year={2025}\n}\n```\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvg%2Fscrstudio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcvg%2Fscrstudio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcvg%2Fscrstudio/lists"}