{"id":40707206,"url":"https://github.com/calledit/metric_depth_video_toolbox","last_synced_at":"2026-01-30T13:00:57.065Z","repository":{"id":274430649,"uuid":"922875233","full_name":"calledit/metric_depth_video_toolbox","owner":"calledit","description":"Python tools for rendering, viewing and generating metric 3D depth videos. Tools for recovering and exporting camera pose and 3D geometry to popular formats as well as tools for projecting depthvideo in to side by side stereo(sbs) 3D stereo video. ","archived":false,"fork":false,"pushed_at":"2026-01-24T15:39:10.000Z","size":664,"stargazers_count":126,"open_issues_count":1,"forks_count":11,"subscribers_count":9,"default_branch":"main","last_synced_at":"2026-01-24T23:51:11.747Z","etag":null,"topics":["3d-reconstruction","camera-pose-estimation","depth-estimation","monocular-depth-estimation","video-depth"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/calledit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-27T08:43:53.000Z","updated_at":"2026-01-24T15:39:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"83f5d061-1d77-4abe-ae3a-be64dd346301","html_url":"https://github.com/calledit/metric_depth_video_toolbox","commit_stats":null,"previous_names":["calledit/metric_video_depth_anything","calledit/depth_video_toolbox","calledit/metric_depth_video_toolbox"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/calledit/metric_depth_video_toolbox","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/calledit%2Fmetric_depth_video_toolbox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/calledit%2Fmetric_depth_video_toolbox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/calledit%2Fmetric_depth_video_toolbox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/calledit%2Fmetric_depth_video_toolbox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/calledit","download_url":"https://codeload.github.com/calledit/metric_depth_video_toolbox/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/calledit%2Fmetric_depth_video_toolbox/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28913288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-30T12:13:43.263Z","status":"ssl_error","status_checked_at":"2026-01-30T12:13:22.389Z","response_time":66,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction","camera-pose-estimation","depth-estimation","monocular-depth-estimation","video-depth"],"created_at":"2026-01-21T12:34:46.089Z","updated_at":"2026-01-30T13:00:57.060Z","avatar_url":"https://github.com/calledit.png","language":"Python","readme":"# Metric depth video toolbox (MDVToolbox)\n\nTools for Generating and working with metric 3D depth videos.\n\n![gif_banner](https://github.com/user-attachments/assets/4d737bb3-6fb6-4135-b01e-b35528371d22)\n\n_Banner created with 3d_view_depthfile.py_\n\n## 📽️ Demos\n\n**Showcase video:** https://youtu.be/nEiUloZ591Q\n\n**Movie → 3D conversion demo**: https://www.youtube.com/watch?v=PLFjoNgkZDY\n\n**Sample stereo clips:**\nhttps://github.com/calledit/metric_depth_video_toolbox/releases/tag/Showcase\n\n---\n\n## ✨ Features\n\n### Metric Depth Video Generation\n\nConvert rgb videos into metric depth videos using multiple SOTA models:\n\n- Depth-Anything series (`video_metric_convert.py`) and (`videoanythingmetric_video.py`)\n- MoGe  (`moge_video.py`)\n- UniDepth  (`unidepth_video.py`)\n- UniK3D  (`unik3d_video.py`)\n- DepthPro  (`depthpro_video.py`)\n- DepthCrafter  (`depthcrafter_video.py`)\n- MVSAnywhere  (`video_mvsa.py`)\n\n### Stereo / 3D Conversion\n\n- 2D movies → 3D (`movie_2_3D.py`)\n- Stereo rendering from depth and rgb video (`stereo_rerender.py`)\n- Parallax infill \u0026 diffusion-based stereo inpainting (`stereo_crafter_infill.py`)\n\n### Visualization\n\n- Real-time 3D viewer for metric depth videos (`3d_view_depthfile.py`)\n- Novel-view rendering from depth video (`3d_view_depthfile.py`)\n\n### Camera Tracking \u0026 3D Reconstruction\n\n- Depth-assisted camera tracking (`sam_track_video.py`) and (`align_3d_points.py`)\n- Long-term point tracking (CoTracker3) (`track_points_in_video.py`)\n- Pose extraction \u0026 alignment tools\n\n### Export Tools\n\nExport metric depth video to standard formats for use in other tools: (`convert_metric_depth_video_to_other_format.py`)\n\n- `.ply` point clouds\n- `.obj` meshes\n- Blender `.blend` \u0026 Alembic `.abc` camera tracks\n- 8-bit / 16-bit depth maps\n- Depth rescaling based on triangulation\n\n### Masking \u0026 Cleanup\n\n- Automatic main subject masking (`generate_video_mask.py`)\n- Subtitle/logo inpainting (`apply_inpainting.sh`)\n\n---\n\n## 📚 Documentation\n\n| Topic | Link |\n|---|---|\nBeginner guide | [`HOWTO.md`](docs/HOWTO.md)  \nMovie → 3D guide | [`HOWTO_movie2_3d.md`](docs/HOWTO_movie2_3d.md)  \nFull tool reference | [`USAGE.md`](docs/USAGE.md)  \nGUI tutorial video | https://youtu.be/BE_aJCI7DHI  \n\n---\n\n## 🧠 Depth Video Format\n\nMDVT uses **RGB-encoded 16-bit metric depth**:\n\n- **Red + Green** = upper 8 bits (duplicated for visibility)  \n- **Blue** = lower 8 bits  \n- Default range: **100 meters**\n- Resolution: **~1.5mm depth precision**\n\nFuture upgrade for more precision will be done with: **24-bit depth** or **log-encoded depth** for long-range accuracy.\n\n---\n\n## ⚙️ Installation\n\n### Windows\n1. install git https://git-scm.com/downloads/win\n2. Install miniconda https://docs.conda.io/en/latest/\n3. Open the Anacoda prompt(miniconda) from the start menu\n4. run\n```batch\ngit clone https://github.com/calledit/metric_depth_video_toolbox\ncd metric_depth_video_toolbox\nwindows_installer.bat\n```\n5. use `conda activate mdvt` to activate conda and use the tools. Some ML models are not supported on windows but the most esential ones are supported like Sterecrafter and depth anything.\n\n### Ubuntu/Debian and OSX\n\n```bash\n\n\ngit clone https://github.com/calledit/metric_depth_video_toolbox\ncd metric_depth_video_toolbox\n\n# on linux\nsudo apt-get install -y libgl1\n./install_mdvtoolbox.sh\n\n#Optional (only required for some tools)\n./install_mdvtoolbox.sh -megasam\n./install_mdvtoolbox.sh -geometrycrafter\n./install_mdvtoolbox.sh -unik3d\n./install_mdvtoolbox.sh -depthpro\n./install_mdvtoolbox.sh -stereocrafter\n./install_mdvtoolbox.sh -madpose\n./install_mdvtoolbox.sh -unidepth\n./install_mdvtoolbox.sh -moge\n./install_mdvtoolbox.sh -promptda\n\n# if using headless linux you need to start a virtual x11 server\napt-get install xvfb\nXvfb :2 \u0026\nexport DISPLAY=:2\n\n# OSX (OSX only supports post processing of depth videos not generation of them. As the ML models need CUDA)\n# (open3d requires python3.11 on OSX (as of 2025)))\npip3.11 install open3d numpy opencv-python\n\n```\n\n### Requirements\nThe tools that reuire ML models have been tested on machines with nvida 3090 cards that support Cuda 12.4 and Torch 2.5.1 on [vast.ai](https://cloud.vast.ai/?ref_id=148636) using \"template PyTorch (cuDNN Devel)\"\n\n### Next steps\n- Currently waiting for new stable and accurate depth models.\n\n### Contributing\nIs appreciated. Even for simple things like spelling.\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalledit%2Fmetric_depth_video_toolbox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcalledit%2Fmetric_depth_video_toolbox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalledit%2Fmetric_depth_video_toolbox/lists"}