{"id":19137494,"url":"https://github.com/dsaurus/diffustereo","last_synced_at":"2025-08-20T22:32:21.827Z","repository":{"id":46191026,"uuid":"515183014","full_name":"DSaurus/DiffuStereo","owner":"DSaurus","description":"This repository is the official implementation of DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Cameras.","archived":false,"fork":false,"pushed_at":"2022-08-05T16:00:02.000Z","size":9183,"stargazers_count":200,"open_issues_count":13,"forks_count":21,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-04-07T22:24:13.584Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DSaurus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-18T12:51:56.000Z","updated_at":"2025-03-11T08:48:45.000Z","dependencies_parsed_at":"2022-07-19T03:47:07.378Z","dependency_job_id":null,"html_url":"https://github.com/DSaurus/DiffuStereo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/DSaurus/DiffuStereo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DSaurus%2FDiffuStereo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DSaurus%2FDiffuStereo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DSaurus%2FDiffuStereo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DSaurus%2FDiffuStereo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DSaurus","download_url":"https://codeload.github.com/DSaurus/DiffuStereo/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DSaurus%2FDiffuStereo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271397965,"owners_count":24752641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-20T02:00:09.606Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T06:38:34.292Z","updated_at":"2025-08-20T22:32:21.260Z","avatar_url":"https://github.com/DSaurus.png","language":"Python","readme":"**News**\n\n* `31/07/2022` We plan to release the THUman5.0 dataset for acadamic use. The dataset contains 10 dynamic human sequences which are captured by 32 RGB cameras with resolution of 4000x3096. Please see [here](DATASET.md) for more detais. \n\n# DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Cameras\n### [Project Page](http://liuyebin.com/diffustereo/diffustereo.html) | [Paper](https://arxiv.org/pdf/2207.08000.pdf) | [Data](DATASET.md)\n\n![image](assets/teaser.jpg)\n\n\u003e [DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Cameras](https://arxiv.org/pdf/2207.08000.pdf)  \n\u003e Ruizhi Shao, Zerong Zheng, Hongwen Zhang, Jingxiang Sun, Yebin Liu\n\u003e ECCV 2022\n\nWe plan to release the training and testing code of DiffuStereo in this repository as soon as possible.  Any discussions or questions would be welcome!\n\n## Installation\n\nPlease see [INSTALL.md](INSTALL.md) for manual installation.\n\n## Pretrained model\n\nWe will provide the pretrained diffusion models for stereo matching including 20 degree and 45 degree as soon as possible. You can download and put them into the `checkpoints/` directory.\n\n[Download: pretrained model for 20 degree stereo matching](https://mailstsinghuaeducn-my.sharepoint.com/:u:/g/personal/shaorz20_mails_tsinghua_edu_cn/EUgJEhePS11On81j2r7NGj8Bj8XZmRc0LqhD7kxUrNJBJA?e=TXXoPg)\n\n## Run the code on the THUman2.0 dataset\n\nPlease see [THUman2.0](THUMAN2_0.md) to download DEMO samples and the dataset.\n\n### Structure of DEMO samples\n\n```\nthuman_demo/\n├── depth_db\n│   └── 0001\n│       ├── xxx.npz  --- the depth of view xxx (reconstructed by DoubleField)\n├── img\n│   └── 0001\n│       ├── xxx.png  --- the image of view xxx\n├── mask\n│   └── 0001\n│       ├── xxx.png  --- the mask of view xxx\n├── normal_db\n│   └── 0001\n│       ├── xxx.png  --- the normal of view xxx (reconstructed by DoubleField)\n└── parameter        --- We use perspective camera model to render images\n    └── 0001   \n        ├── xxx_extrinsic.npy  --- the extrinsic of view xxx (3x4 world-to-camera matrix)\n        ├── xxx_intrinsic.npy  --- the intrinsic of view xxx (3x3 matrix)\n```\n\n### Inference on a single stereo matching pair\nTo inference on a single stereo matching pair of DEMO samples, please run the following script.\n\n```\npython -m app.inference --config configs/thuman_demo.yaml --dataroot [the directory of DEMO samples] --view_list 0 20\n```\n\n### Visualization on a single stereo matching pair\nThe results will be saved in `results/thuman_demo/`. To visualize the results, use [MeshLab](https://www.meshlab.net/) to open `results/thuman_demo/fusion000.ply` and apply Possion Reconstruction with `depth=11` or `depth=10`.\n\n![video](assets/video_thuman_demo_single.gif)\n\n### Inference on multiple stereo matching pairs\nTo inference on multiple stereo matching pairs of DEMO samples, please run the following script.\n\n```\npython -m app.inference --config configs/thuman_demo_multi.yaml --dataroot [the directory of DEMO samples] --view_list 0 20 90 110 180 200 270 290 --use_db_normal\n```\n\nFor cases with 45 degree, please run the following script.\n\n```\npython -m app.inference --config configs/thuman_demo_multi_45.yaml --dataroot [the directory of DEMO samples] --view_list 0 45 90 135 180 225 270 315 --use_db_normal\n```\n\n### Naive multi-view fusion and visualization on the results\nThe results will be saved in `results/thuman_demo_multi/`. To simply fuse multi-view depth point clouds, use [MeshLab](https://www.meshlab.net/) to open `results/thuman_demo_multi/fusion000.ply` and apply Possion Reconstruction with `depth=11` or `depth=10`.\n\n![video](assets/video_thuman_demo.gif)\n\n## Run the code on the THUman5.0 dataset\n\nPlease see [DATASET.md](DATASET.md) to download DEMO samples and the dataset.\n\n### Structure of DEMO samples\n```\nreal_demo/\n├── depth_db\n│   └── 0001\n│       ├── xxx.npz  --- the depth of view xxx (reconstructed by DoubleField)\n├── img\n│   └── 0001\n│       ├── xxx.jpg  --- the image of view xxx (after undistortion)\n├── mask\n│   └── 0001\n│       ├── xxx.jpg  --- the mask of view xxx (after undistortion)\n├── normal_db\n│   └── 0001\n│       ├── xxx.png  --- the normal of view xxx (reconstructed by DoubleField)\n└── parameter        --- parameters of real world perspective camera model (after undistortion)\n    └── 0001   \n        ├── xxx_extrinsic.npy  --- the extrinsic of view xxx (3x4 world-to-camera matrix)\n        ├── xxx_intrinsic.npy  --- the intrinsic of view xxx (3x3 matrix)\n```\n\n### Inference on one stereo matching pair\nTo inference on a single stereo matching pair of DEMO samples, please run the following script.\n\n```\npython -m app.inference --config configs/real_demo.yaml --dataroot [the directory of DEMO samples] --view_list 0 1\n```\n\n### Visualization on one stereo matching pair\nThe results will be saved in `results/real_demo/`. To visualize the results, use [MeshLab]() to open `results/real_demo/fusion000.ply` and apply Possion Reconstruction with `depth=11` or `depth=10`.\n\n![video](assets/video_real_demo.gif)\n\n### Multi-view fusion\nOn account of calibration error and the complicated lighting environment in the real-world dataset, naive multi-view fusion based on `Poisson Reconstruction` will generate noise and artifacts. We recommend using traditional multi-view fusion algorithm or programming our proposed `light-weight multi-view fusion` to reproduce the final results.\n\n## Training on the 3D human scan dataset\n\nPlease see [THUman2.0](THUman2_0.md) to download DEMO samples and the dataset.\n\n### TODO\n\n## Run the code on the custom dataset\n\nTo run the code on the custom dataset, please see [CUSTOM](CUSTOM.md). We also clarify our camera model and provide some examples to render depth and normal maps.\n\n## Citation\n\nIf you find this code useful for your research, please use the following BibTeX entry.\n\n```\n@inproceedings{shao2022diffustereo,\n    author = {Shao, Ruizhi and Zheng, Zerong and Zhang, Hongwen and Sun, Jingxiang and Liu, Yebin},\n    title = {DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Cameras},\n    booktitle = {ECCV},\n    year = {2022}\n}\n```\n\n## Acknowledgments\n\nOur project is benefit from these great resources:\n\n- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://github.com/luost26/diffusion-point-cloud)\n- [Diffusion Models Beat GANS on Image Synthesis](https://github.com/openai/guided-diffusion)\n- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://github.com/princeton-vl/RAFT-Stereo)\n- [StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision](https://github.com/CrisHY1995/StereoPIFu_Code)\n- [POP: The Power of Points for Modeling Humans in Clothing](https://github.com/qianlim/POP)\n\nThanks for their sharing code.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdsaurus%2Fdiffustereo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdsaurus%2Fdiffustereo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdsaurus%2Fdiffustereo/lists"}