{"id":13934703,"url":"https://github.com/zju3dv/mvpose","last_synced_at":"2025-04-05T08:06:57.661Z","repository":{"id":42020186,"uuid":"165477360","full_name":"zju3dv/mvpose","owner":"zju3dv","description":"Code for \"Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views\" (CVPR 2019, T-PAMI 2021)","archived":false,"fork":false,"pushed_at":"2021-07-30T12:06:35.000Z","size":52201,"stargazers_count":520,"open_issues_count":20,"forks_count":79,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-03-29T07:06:40.448Z","etag":null,"topics":["cvpr"],"latest_commit_sha":null,"homepage":"https://zju3dv.github.io/mvpose/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zju3dv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-13T07:19:10.000Z","updated_at":"2025-02-24T11:14:42.000Z","dependencies_parsed_at":"2022-08-12T02:31:26.068Z","dependency_job_id":null,"html_url":"https://github.com/zju3dv/mvpose","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2Fmvpose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2Fmvpose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2Fmvpose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2Fmvpose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zju3dv","download_url":"https://codeload.github.com/zju3dv/mvpose/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247305934,"owners_count":20917208,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cvpr"],"created_at":"2024-08-07T23:01:11.011Z","updated_at":"2025-04-05T08:06:57.644Z","avatar_url":"https://github.com/zju3dv.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"# Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views\n\n\n\n\u003e [Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views](https://arxiv.org/pdf/1901.04111.pdf)  \n\u003e Junting Dong, Wen Jiang, Qixing Huang, Hujun Bao, Xiaowei Zhou   \n\u003e CVPR 2019, T-PAMI 2021 \n\u003e [Project Page](https://zju3dv.github.io/mvpose/)\n\nAny questions or discussions are welcomed!\n\n\n## Installation\n\n - Set up python environment\n\n```   \npip install -r requirements.txt\n```    \n\n - Compile the `backend/tf_cpn/lib` and `backend/light_head_rcnn/lib`\n```\ncd mvpose/backend/tf_cpn/lib/\nmake\ncd ./lib_kernel/lib_nms\nbash compile.sh\n```\n```\ncd mvpose/backend/light_head_rcnn/lib/\nbash make.sh\n```\n\n Since they use py-faster-rcnn as backbone. Many people using faster-rcnn meet with some problems when compiling those components. Suggestions on google can be helpful.\n\n - Compile the pictorial function for acceleration\n```\ncd mvpose/src/m_lib/\npython setup.py build_ext --inplace\n```\n\n## Prepare models and datasets\n\n -  **Prepare models**: \nPlease put [light-head-rcnn models](https://drive.google.com/file/d/1klpM_DEIn2Ln4ZN-xWHdvwp40dYpQ05b/view?usp=sharing) to `backend/light_head_rcnn/output/model_dump`, \n    [backend/tf_cpn/log/model_dump](https://drive.google.com/file/d/1DJF4p-SC_PokGtt7TbCVPgo-EWRQYhGi/view?usp=sharing) to `backend/tf_cpn/log/model_dump`,\n    and [CamStyle model](https://drive.google.com/file/d/1FRAu6sr0Bd39ZliCscum69mwuZ1j502b/view?usp=sharing) trained by myself to `backend/CamStyle/logs`\n\n - **Prepare the datasets**: \nPut datasets such as Shelf and CampusSeq1 to `./datasets/`\nDownload [Campus](http://campar.cs.tum.edu/files/belagian/multihuman/CampusSeq1.tar.bz2) and [Shelf](http://campar.cs.tum.edu/files/belagian/multihuman/Shelf.tar.bz2) datasets. Then, put datasets such as Shelf and CampusSeq1 to datasets/\n\n - **Generate the camera parameters**: \n   Since each dataset uses different way to obtain the camera parameters, we show an example to deal with the Campus dataset: \n   - Add following code to `.datasets/CampusSeq1/Calibration/producePmat.m`\n   ```\n   K = cell(1,3);\n   K{1} = K1; K{2} = K2; K{3} = K3;\n   m_RT = cell(1,3);\n   m_RT{1} = RT1; m_RT{2} = RT2; m_RT{3} = RT3;\n   save('intrinsic.mat','K');\n   save('m_RT.mat', 'm_RT');\n   save('P.mat', 'P');\n   save('prjectionMat','P');\n   ```\n   - generate the `camera_parameter.pickle`\n   ```\n   python ./src/tools/mat2pickle.py /parameter/dir ./datasets/CampusSeq1\n   ```\n   Here, we also provide the camera_parameter.pickle of [Campus](https://drive.google.com/file/d/1BvIyB53Jb_asZ2gEoIRh8gYUvHxPDcPA/view?usp=sharing) and [Shelf](https://drive.google.com/file/d/1mWe9CpyYrsU7t2FFf5PMJw5CqFE-3Otj/view?usp=sharing). You can generate the .pickle file for your datasets using the same way.\n## Demo and Evaluate\n\n### Run the demo\n\n```\npython ./src/m_utils/demo.py -d Campus\npython ./src/m_utils/demo.py -d Shelf\n```\n\nIf all the configuration is OK, you may see the visualization of following items.\n\n![matching](./assets/result.png)\n\n\n\n### Evaluate on the Campus/Shelf datasets\n\n```\npython ./src/m_utils/evaluate.py -d Campus\npython ./src/m_utils/evaluate.py -d Shelf\n```\nAs long as the progress bar finished, you may see a beautified table of evaluation result and a csv file for the evaluation result will be save in `./result` directory.\n\n### Accelerate the evaluation\n\nSince the 2D pose estimator (CPN) is a little slow, we can save the predicted 2D poses and heatmaps and then start with these saved files.\n\n 1. produce the files\n```\npython src/tools/preprocess.py -d Campus -dump_dir ./datasets/Campus_processed\npython src/tools/preprocess.py -d Shelf -dump_dir ./datasets/Shelf_processed\n```\n 2. evaluate with saved 2D poses and heatmaps\n```\npython ./src/m_utils/evaluate.py -d Campus -dumped ./datasets/Campus_processed\npython ./src/m_utils/evaluate.py -d Shelf -dumped ./datasets/Shelf_processed\n```\nNote: for the sake of convenience, we do not optimize on the size of dumped file.\nTherefore, the size of `Campus_processed` is around `4.0G` and the size of `Shelf_processed` is around `234G`. Please make sure your disk have `200+G` free space. Any pull request to solve this issues will be welcomed.\n\n## Citation\n\nIf you find this code useful for your research, please use the following BibTeX entry.\n\n```\n@inproceedings{dong2019fast,\n  title={Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views},\n  author={Dong, Junting and Jiang, Wen and Huang, Qixing and Bao, Hujun and Zhou, Xiaowei},\n  journal={CVPR},\n  year={2019}\n}\n\n@article{dong2021fastpami,\n  title={Fast and Robust Multi-Person 3D Pose Estimation and Tracking from Multiple Views},\n  author={Dong, Junting and Fang, Qi and Jiang, Wen and Yang, Yurou and Bao, Hujun and Zhou, Xiaowei},\n  booktitle={T-PAMI},\n  year={2021}\n}\n```\n## Acknowledgements\nThis code uses these code ([Light head rcnn](https://github.com/zengarden/light_head_rcnn), [Cascaded Pyramid Network](https://github.com/chenyilun95/tf-cpn), [CamStyle](https://github.com/zhunzhong07/CamStyle)) as backbone. We gratefully appreciate the impact it had on our work. If you use our code, please consider citing the original paper as well.\n\n## Copyright\n\nThis work is affliated with ZJU-SenseTime Joint Lab of 3D Vision, and its intellectual property belongs to SenseTime Group Ltd.\n\n```\nCopyright SenseTime. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzju3dv%2Fmvpose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzju3dv%2Fmvpose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzju3dv%2Fmvpose/lists"}