{"id":16250295,"url":"https://github.com/apple/ml-depth-pro","last_synced_at":"2025-05-13T22:12:25.815Z","repository":{"id":257820586,"uuid":"847933360","full_name":"apple/ml-depth-pro","owner":"apple","description":"Depth Pro: Sharp Monocular Metric Depth in Less Than a Second.","archived":false,"fork":false,"pushed_at":"2025-04-21T12:13:43.000Z","size":2549,"stargazers_count":4392,"open_issues_count":70,"forks_count":318,"subscribers_count":46,"default_branch":"main","last_synced_at":"2025-04-30T12:16:59.668Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apple.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-26T20:26:05.000Z","updated_at":"2025-04-30T10:16:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"7074cac4-0c54-4803-8919-6e9258033e8e","html_url":"https://github.com/apple/ml-depth-pro","commit_stats":{"total_commits":4,"total_committers":2,"mean_commits":2.0,"dds":0.25,"last_synced_commit":"b2cd0d51daa95e49277a9f642f7fd736b7f9e91d"},"previous_names":["apple/ml-depth-pro"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apple%2Fml-depth-pro","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apple%2Fml-depth-pro/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apple%2Fml-depth-pro/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apple%2Fml-depth-pro/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apple","download_url":"https://codeload.github.com/apple/ml-depth-pro/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254036843,"owners_count":22003654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-10T15:01:48.716Z","updated_at":"2025-05-13T22:12:20.800Z","avatar_url":"https://github.com/apple.png","language":"Python","funding_links":[],"categories":["Python","排行榜 [2025-03-18]","Depth Estimation"],"sub_categories":["Foundation Models"],"readme":"## Depth Pro: Sharp Monocular Metric Depth in Less Than a Second\n\nThis software project accompanies the research paper:\n**[Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://arxiv.org/abs/2410.02073)**, \n*Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R. Richter, and Vladlen Koltun*.\n\n![](data/depth-pro-teaser.jpg)\n\nWe present a foundation model for zero-shot metric monocular depth estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with unparalleled sharpness and high-frequency details. The predictions are metric, with absolute scale, without relying on the availability of metadata such as camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map in 0.3 seconds on a standard GPU. These characteristics are enabled by a number of technical contributions, including an efficient multi-scale vision transformer for dense prediction, a training protocol that combines real and synthetic datasets to achieve high metric accuracy alongside fine boundary tracing, dedicated evaluation metrics for boundary accuracy in estimated depth maps, and state-of-the-art focal length estimation from a single image.\n\n\nThe model in this repository is a reference implementation, which has been re-trained. Its performance is close to the model reported in the paper but does not match it exactly.\n\n## Getting Started\n\nWe recommend setting up a virtual environment. Using e.g. miniconda, the `depth_pro` package can be installed via:\n\n```bash\nconda create -n depth-pro -y python=3.9\nconda activate depth-pro\n\npip install -e .\n```\n\nTo download pretrained checkpoints follow the code snippet below:\n```bash\nsource get_pretrained_models.sh   # Files will be downloaded to `checkpoints` directory.\n```\n\n### Running from commandline\n\nWe provide a helper script to directly run the model on a single image:\n```bash\n# Run prediction on a single image:\ndepth-pro-run -i ./data/example.jpg\n# Run `depth-pro-run -h` for available options.\n```\n\n### Running from python\n\n```python\nfrom PIL import Image\nimport depth_pro\n\n# Load model and preprocessing transform\nmodel, transform = depth_pro.create_model_and_transforms()\nmodel.eval()\n\n# Load and preprocess an image.\nimage, _, f_px = depth_pro.load_rgb(image_path)\nimage = transform(image)\n\n# Run inference.\nprediction = model.infer(image, f_px=f_px)\ndepth = prediction[\"depth\"]  # Depth in [m].\nfocallength_px = prediction[\"focallength_px\"]  # Focal length in pixels.\n```\n\n\n### Evaluation (boundary metrics) \n\nOur boundary metrics can be found under `eval/boundary_metrics.py` and used as follows:\n\n```python\n# for a depth-based dataset\nboundary_f1 = SI_boundary_F1(predicted_depth, target_depth)\n\n# for a mask-based dataset (image matting / segmentation) \nboundary_recall = SI_boundary_Recall(predicted_depth, target_mask)\n```\n\n\n## Citation\n\nIf you find our work useful, please cite the following paper:\n\n```bibtex\n@inproceedings{Bochkovskii2024:arxiv,\n  author     = {Aleksei Bochkovskii and Ama\\\"{e}l Delaunoy and Hugo Germain and Marcel Santos and\n               Yichao Zhou and Stephan R. Richter and Vladlen Koltun},\n  title      = {Depth Pro: Sharp Monocular Metric Depth in Less Than a Second},\n  booktitle  = {International Conference on Learning Representations},\n  year       = {2025},\n  url        = {https://arxiv.org/abs/2410.02073},\n}\n```\n\n## License\nThis sample code is released under the [LICENSE](LICENSE) terms.\n\nThe model weights are released under the [LICENSE](LICENSE) terms.\n\n## Acknowledgements\n\nOur codebase is built using multiple opensource contributions, please see [Acknowledgements](ACKNOWLEDGEMENTS.md) for more details.\n\nPlease check the paper for a complete list of references and datasets used in this work.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapple%2Fml-depth-pro","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapple%2Fml-depth-pro","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapple%2Fml-depth-pro/lists"}