{"id":24719675,"url":"https://github.com/markyu98/madpose","last_synced_at":"2025-04-06T09:06:17.319Z","repository":{"id":271760096,"uuid":"882337306","full_name":"MarkYu98/madpose","owner":"MarkYu98","description":"[CVPR 2025] Official implementation of the solvers and estimators proposed in the paper \"Relative Pose Estimation through Affine Corrections of Monocular Depth Priors\"","archived":false,"fork":false,"pushed_at":"2025-02-27T08:59:34.000Z","size":15574,"stargazers_count":116,"open_issues_count":3,"forks_count":6,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-30T08:07:11.313Z","etag":null,"topics":["monocular-depth-estimation","multi-view-geometry","relative-pose-estimation"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2501.05446","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MarkYu98.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-02T15:00:39.000Z","updated_at":"2025-03-28T18:15:25.000Z","dependencies_parsed_at":"2025-02-28T06:09:57.257Z","dependency_job_id":"a41d3824-b4f4-48a3-beb8-aa667e076eae","html_url":"https://github.com/MarkYu98/madpose","commit_stats":null,"previous_names":["markyu98/madpose"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkYu98%2Fmadpose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkYu98%2Fmadpose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkYu98%2Fmadpose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarkYu98%2Fmadpose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MarkYu98","download_url":"https://codeload.github.com/MarkYu98/madpose/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247457799,"owners_count":20941906,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["monocular-depth-estimation","multi-view-geometry","relative-pose-estimation"],"created_at":"2025-01-27T11:19:02.384Z","updated_at":"2025-04-06T09:06:17.300Z","avatar_url":"https://github.com/MarkYu98.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MADPose\n\n\u003ca href=\"https://arxiv.org/abs/2501.05446\"\u003e\u003cimg src='https://img.shields.io/badge/arXiv-Paper-red?logo=arxiv\u0026logoColor=white' alt='arXiv'\u003e\u003c/a\u003e\n\nThis repo contains the official implementation of the solvers and estimators proposed in the paper \"Relative Pose Estimation through Affine Corrections of Monocular Depth Priors\". The solvers and estimators are implemented using C++, and we provide easy-to-use Python bindings. \n\nNote: \"**MAD**\" is an acronym for \"**M**onocular **A**ffine **D**epth\".\n\n![](resources/pipeline.png)\n**Overview**: Our method takes a pair of images as input, runs off-the-shelf feature matching and monocular depth\nestimation, then jointly estimates the relative pose, scale and shift parameters of the two depth maps, and optionally the focal lengths.\n\n## 📢 News\n- 2025-02-27: Our paper is accepted to CVPR 2025! See you in Nashville, TN!\n  \n## Evaluation\nWe develop three solvers for relative pose estimation that explicitly account for independent affine (scale and shift) ambiguities, tailored for three setups of cameras: calibrated, shared-focal, and unknown focal lengths (two-focal). The solvers are further combined with classic point-based solvers and epipolar constraints in our hybrid RANSAC estimators. Our estimators show consistent improvements over classic point-based methods (e.g. [PoseLib](https://github.com/PoseLib/PoseLib)) and recent two-view reconstruction methods [DUSt3R](https://github.com/naver/dust3r) and [MASt3R](https://github.com/naver/mast3r), across different datasets with different feature matchers and monocular depth estimation models. \n\nSome highlight results (probably best viewed in light theme on GitHub!):\n\n#### Pose Error AUCs on ScanNet-1500 with shared-focal setting\n\u003ctable style=\"border-collapse: collapse; margin: 20px 0; font-size: 14px; font-family: -apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,Arial,sans-serif; min-width: 600px;\"\u003e\n    \u003cthead\u003e\n        \u003ctr style=\"border-bottom: 3px solid #ddd; text-align: center;\"\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eMatches\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eMethod\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eMD Model\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eAUC@5°\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eAUC@10°\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eAUC@20°\u003c/th\u003e\n        \u003c/tr\u003e\n    \u003c/thead\u003e\n    \u003ctbody\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd rowspan=\"2\" style=\"padding: 8px; text-align: center;\"\u003eSP+SG\u003c/td\u003e\n            \u003ctd style=\"padding: 8px;\"\u003ePoseLib-6pt\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e-\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e12.84\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e28.13\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e45.64\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 2px solid #eee;\"\u003e\n            \u003ctd style=\"padding: 8px;\"\u003eOurs-sf\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003eDA-met.\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e18.35\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e37.54\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e57.58\u003c/b\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd rowspan=\"2\" style=\"padding: 8px; text-align: center;\"\u003eRoMa\u003c/td\u003e\n            \u003ctd style=\"padding: 8px;\"\u003ePoseLib-6pt\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e-\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e27.17\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e49.24\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e67.42\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 2px solid #eee;\"\u003e\n            \u003ctd style=\"padding: 8px;\"\u003eOurs-sf\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003eDA-met.\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e29.81\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e53.11\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e71.15\u003c/b\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd rowspan=\"3\" style=\"padding: 8px; text-align: center;\"\u003eMASt3R\u003c/td\u003e\n            \u003ctd style=\"padding: 8px;\"\u003ePoseLib-6pt\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e-\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e30.28\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e54.16\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e72.87\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd rowspan=\"2\" style=\"padding: 8px;\"\u003eOurs-sf\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003eDA-met.\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003ci\u003e31.87\u003c/i\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003ci\u003e56.20\u003c/i\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003ci\u003e74.51\u003c/i\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003eMASt3R\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e32.58\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e56.99\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e74.91\u003c/b\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd colspan=\"3\" style=\"padding: 8px; text-align: center;\"\u003eReference entry - DUSt3R\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e25.90\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e48.45\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e68.03\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd colspan=\"3\" style=\"padding: 8px; text-align: center;\"\u003eReference entry - MASt3R\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e23.94\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e46.44\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e66.18\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/tbody\u003e\n\u003c/table\u003e\n\n#### Pose Error AUCs on 2D-3D-S sampled image pairs with unknown focal lengths\n\u003ctable style=\"border-collapse: collapse; margin: 20px 0; font-size: 14px; font-family: -apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,Arial,sans-serif; min-width: 700px;\"\u003e\n    \u003cthead\u003e\n        \u003ctr style=\"border-bottom: 3px solid #ddd; text-align: center;\"\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eMatches\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eMethod\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eMD Model\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eAUC@2°\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eAUC@5°\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eAUC@10°\u003c/th\u003e\n            \u003cth style=\"padding: 12px 8px; text-align: center;\"\u003eAUC@20°\u003c/th\u003e\n        \u003c/tr\u003e\n    \u003c/thead\u003e\n    \u003ctbody\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd rowspan=\"2\" style=\"padding: 8px; text-align: center;\"\u003eSP+LG\u003c/td\u003e\n            \u003ctd style=\"padding: 8px;\"\u003ePoseLib-7pt\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e-\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e5.85\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e13.95\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e21.94\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e30.71\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 2px solid #eee;\"\u003e\n            \u003ctd style=\"padding: 8px;\"\u003eOurs-tf\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003eDAv2-met.\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e9.15\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e22.22\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e32.80\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e43.26\u003c/b\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd rowspan=\"2\" style=\"padding: 8px; text-align: center;\"\u003eRoMa\u003c/td\u003e\n            \u003ctd style=\"padding: 8px;\"\u003ePoseLib-7pt\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e-\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e8.73\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e20.31\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e30.45\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e41.48\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 2px solid #eee;\"\u003e\n            \u003ctd style=\"padding: 8px;\"\u003eOurs-tf\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003eDAv2-met.\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e13.50\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e29.19\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e42.18\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e54.42\u003c/b\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd rowspan=\"3\" style=\"padding: 8px; text-align: center;\"\u003eMASt3R\u003c/td\u003e\n            \u003ctd style=\"padding: 8px;\"\u003ePoseLib-7pt\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e-\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e12.58\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e30.27\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e45.57\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e59.85\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd rowspan=\"2\" style=\"padding: 8px;\"\u003eOurs-tf\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003eDAv2-met.\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003ci\u003e18.05\u003c/i\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003ci\u003e39.92\u003c/i\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003ci\u003e56.64\u003c/i\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003ci\u003e70.86\u003c/i\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003eMASt3R\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e22.44\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e48.02\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e64.79\u003c/b\u003e\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e\u003cb\u003e76.55\u003c/b\u003e\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd colspan=\"3\" style=\"padding: 8px; text-align: center;\"\u003eReference entry - DUSt3R\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e6.43\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e24.47\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e42.39\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e58.36\u003c/td\u003e\n        \u003c/tr\u003e\n        \u003ctr style=\"border-bottom: 1px solid #eee;\"\u003e\n            \u003ctd colspan=\"3\" style=\"padding: 8px; text-align: center;\"\u003eReference entry - MASt3R\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e13.39\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e38.41\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e57.92\u003c/td\u003e\n            \u003ctd style=\"padding: 8px; text-align: center;\"\u003e71.91\u003c/td\u003e\n        \u003c/tr\u003e\n    \u003c/tbody\u003e\n\u003c/table\u003e\nPlease refer to the paper for more results and discussions.\n\n## Installation\n### Install from PyPI\nWe are working on setting up wheel for easy installation using PyPI. Currently please use the following method to install from source.\n\n### Install from source\n#### Install dependencies\n```bash\nsudo apt-get install libeigen3-dev libceres-dev libopencv-dev\n```\n_Note: The two-focal estimator currently relies on `cv::recoverPose` from OpenCV, we plan to remove dependency on OpenCV in future updates._\n\n#### Clone the repo\n```bash\ngit clone --recursive https://github.com/MarkYu98/madpose\n```\n\n#### Build and install the Python bindings\n```bash\npip install .\n```\nIf you would like to see the building process (e.g. CMake logs) you can add `-v` option to the above command.\n\nNote: MADPose also depends on [PoseLib](https://github.com/PoseLib/PoseLib). By default, CMake will automatically build PoseLib using `FetchContent`. Set `FETCH_POSELIB` CMake option to `OFF` if you prefer to use a self-installed version.\n\n#### Check the installation\n```bash\npython -c \"import madpose\"\n```\nYou should not see any errors if MADPose is successfully installed.\n\n## Usage\n### Estimators\nWe provide Python bindings of our 3 hybrid estimators for image pairs with calibrated cameras, shared-focal cameras, and cameras with unknown focal lengths (two-focal). \n\nThe estimators take `HybridLORansacOptions` and `EstimatorConfig` for related settings, some useful settings are:\n```python\nimport madpose\n\noptions = madpose.HybridLORansacOptions()\noptions.min_num_iterations = 100\noptions.max_num_iterations = 1000\noptions.success_probability = 0.9999\noptions.random_seed = 0 # for reproducibility\noptions.final_least_squares = True\noptions.threshold_multiplier = 5.0\noptions.num_lo_steps = 4\n# squared px thresholds for reprojection error and epipolar error\noptions.squared_inlier_thresholds = [reproj_pix_thres ** 2, epipolar_pix_thres ** 2]\n# weight when scoring for the two types of errors\noptions.data_type_weights = [1.0, epipolar_weight]\n\nest_config = madpose.EstimatorConfig()\n# if enabled, the input min_depth values are guaranteed to be positive with the estimated depth offsets (shifts), default: True\nest_config.min_depth_constraint = True\n# if disabled, will model the depth with only scale (only applicable to the calibrated camera case)\nest_config.use_shift = True\n# best set to the number of PHYSICAL CPU cores\nest_config.ceres_num_threads = 8\n```\n\n**Note on performance**: if speed is a primary concern, best set `est_config.ceres_num_threads` to the number of _physical_ CPU cores on your machine, also tune `options.min_num_iterations`, `options.max_num_iterations`, and `options.num_lo_steps` for trade-off between speed and accuracy. In addition, `export OPENBLAS_NUM_THREADS=1` to use single-thread dense solver in Ceres to avoid overheads due to the small-sized nature of the Jacobians.\n\nWe provide a example image pairs and code snippets in [examples/](examples/) to test the hybrid estimators. More demos and evaluations will be added in the future. \n\nWe use `numpy` and `cv2` to handle pre-computed data in Python in the examples, you can easily install them using pip:\n```bash\npip install numpy opencv-python\n```\n\n#### Calibrated estimator\n```python\npose, stats = madpose.HybridEstimatePoseScaleOffset(\n                  mkpts0, mkpts1, \n                  depth0, depth1,\n                  [depth_map0.min(), depth_map1.min()], \n                  K0, K1, options, est_config\n              )\n# rotation and translation of the estimated pose\nR_est, t_est = pose.R(), pose.t()\n# scale and offsets of the affine corrected depth maps\ns_est, o0_est, o1_est = pose.scale, pose.offset0, pose.offset1\n```\nThe parameters are: keypoint matches(`mkpts0`, `mkpts1`), their corresponding depth prior values(`depth0`, `depth1`), min depth values for both views (used when `est_config.min_depth_constraint` is `True`), camera intrinsics(`K0`, `K1`),`options`, and `est_config`.\n\nSee [examples/calibrated.py](examples/calibrated.py) for a complete code example, evaluation, and comparison with point-based estimation using PoseLib.\n\n#### Shared-focal estimator\n```python\npose, stats = madpose.HybridEstimatePoseScaleOffsetSharedFocal(\n                  mkpts0, mkpts1, \n                  depth0, depth1,\n                  [depth_map0.min(), depth_map1.min()], \n                  pp0, pp1, options, est_config\n              )\n# rotation and translation of the estimated pose\nR_est, t_est = pose.R(), pose.t()\n# scale and offsets of the affine corrected depth maps\ns_est, o0_est, o1_est = pose.scale, pose.offset0, pose.offset1\n# estimated shared focal length\nf_est = pose.focal\n```\nDifferent from the calibrated estimator, now instead of intrinsics(`K0`, `K1`), the shared-focal estimator now takes as input the principal points(`pp0`, `pp1`).\n\nSee [examples/shared_focal.py](examples/shared_focal.py) for complete example.\n\n#### Two-focal estimator\n```python\npose, stats = madpose.HybridEstimatePoseScaleOffsetTwoFocal(\n                  mkpts0, mkpts1, \n                  depth0, depth1,\n                  [depth_map0.min(), depth_map1.min()], \n                  pp0, pp1, options, est_config\n              )\n# rotation and translation of the estimated pose\nR_est, t_est = pose.R(), pose.t()\n# scale and offsets of the affine corrected depth maps\ns_est, o0_est, o1_est = pose.scale, pose.offset0, pose.offset1\n# the estimated two focal lengths\nf0_est, f1_est = pose.focal0, pose.focal1\n```\nThe parameters are same with the shared-focal estimator, but now the estimator will estimate two independent focal lengths.\n\nSee [examples/two_focal.py](examples/two_focal.py) for complete example.\n\n#### Point-based baseline\nYou can compare with point-based estimators from [PoseLib](https://github.com/PoseLib/PoseLib). You need to install the [Poselib's Python bindings](https://github.com/PoseLib/PoseLib?tab=readme-ov-file#python-bindings). \n\nThe corresponding point-based estimation are included in each of the three example scripts above.\n\n### Solvers\nOur hybrid estimators combine our newly proposed depth-aware solvers with the point-based solvers. The Python implementation of the solvers can be found under [solver_py/](solver_py/), which also include calls to the C++ solvers through Python bindings for comparison.\n\nAs described in the paper, the proposed solvers all work in a 2-step manner, first solve for the depth scale and shifts (affine parameters) of the depth maps and then recover the pose by finding the rigid transformation between the back-projected points. We provide Python bindings for the first step (solving scale and shifts) as well as combined wrapper for both the pose and the affine parameters.\n\nThe bindings for solving the affine parameters:\n```python\n# Calibrated case\naffine_params = madpose.solve_scale_and_shift(x1.T, x2.T, d1, d2) \n# Shared-focal\naffine_params = madpose.solve_scale_and_shift_shared_focal(x1.T, x2.T, d1, d2)\n# Two-focal\naffine_params = madpose.solve_scale_and_shift_two_focal(x1.T, x2.T, d1, d2)\n```\nThe input `x1` and `x2` are `np.array` (or list) of homogeneous normalized image coordinates (`[x, y, 1]`) of the minimal set of $M$ matched keypoints; `d1` and `d2` are `np.array` (or list) of the corresponding depth values. The returned `affine_params` is a list of candidate solutions `(a1, a2, b1, b2)` with `a1` always equal to 1 and therefore `a2, b1, b2` correspond to the scale and shift values $\\alpha, \\beta_1, \\beta_2$.\n\nOr we can use the combined wrapper that solves affine parameters and also recover the candidate poses:\n```python\n# Calibrated case\nposes = madpose.solve_scale_shift_pose(x1.T, x2.T, d1, d2)\n# Shared-focal\nposes = madpose.solve_scale_shift_pose_shared_focal(x1.T, x2.T, d1, d2)\n# Two-focal\nposes = madpose.solve_scale_shift_pose_two_focal(x1.T, x2.T, d1, d2)\n```\nNote that for simplicity the return solutions are all named `poses`, the list contains objects of different classes for the three cases (`madpose.PoseScaleOffset`, `madpose.PoseScaleOffsetSharedFocal`, and `madpose.PoseScaleOffsetTwoFocal`). Retrieve the pose, focal lengths, and affine parameters by:\n```python\nR_est, t_est = pose.R(), pose.t()\ns_est, o0_est, o1_est = pose.scale, pose.offset0, pose.offset1\n\n# for PoseScaleOffsetSharedFocal\nf_est = pose.focal\n\n# for PoseScaleOffsetTwoFocal\nf0_est, f1_est = pose.focal0, pose.focal1\n```\n\n## TODO List\n\n- [ ] Remove dependency on OpenCV.\n- [ ] Setup wheel for PyPI\n- [ ] Add experiment scirpts on datasets\n\n## Acknowledgement\nThis codebase is inspired by and built upon many research work and opensource projects, we thank the authors and contributors for their work.\n- [PoseLib](https://github.com/PoseLib/PoseLib)\n- [COLMAP](https://github.com/colmap/colmap)\n- [LIMAP](https://github.com/cvg/limap)\n- [SuperGlue](https://github.com/magicleap/SuperGluePretrainedNetwork)\n- [RansacLib](https://github.com/tsattler/RansacLib)\n- [pybind11](https://github.com/pybind/pybind11)\n\n## Citation\nIf you find our work useful in your research, please consider citing our paper:\n```\n@inproceedings{yu2025madpose,\n    title={Relative Pose Estimation through Affine Corrections of Monocular Depth Priors}, \n    author={Yifan Yu and Shaohui Liu and Rémi Pautrat and Marc Pollefeys and Viktor Larsson},\n    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n    year={2025}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkyu98%2Fmadpose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarkyu98%2Fmadpose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkyu98%2Fmadpose/lists"}