{"id":13958386,"url":"https://github.com/SangbumChoi/MobileHumanPose","last_synced_at":"2025-07-20T23:31:19.640Z","repository":{"id":44423461,"uuid":"303889047","full_name":"SangbumChoi/MobileHumanPose","owner":"SangbumChoi","description":"This repo is official PyTorch implementation of MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices(CVPRW 2021).","archived":false,"fork":false,"pushed_at":"2022-05-19T13:46:30.000Z","size":29388,"stargazers_count":231,"open_issues_count":13,"forks_count":28,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-08-09T13:18:44.379Z","etag":null,"topics":["3d-human-pose","mobile-app","pytorch-implementation","tflite-conversion"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SangbumChoi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-14T03:09:36.000Z","updated_at":"2024-07-17T11:32:24.000Z","dependencies_parsed_at":"2022-08-12T11:10:49.440Z","dependency_job_id":null,"html_url":"https://github.com/SangbumChoi/MobileHumanPose","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SangbumChoi%2FMobileHumanPose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SangbumChoi%2FMobileHumanPose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SangbumChoi%2FMobileHumanPose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SangbumChoi%2FMobileHumanPose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SangbumChoi","download_url":"https://codeload.github.com/SangbumChoi/MobileHumanPose/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226845006,"owners_count":17691140,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-human-pose","mobile-app","pytorch-implementation","tflite-conversion"],"created_at":"2024-08-08T13:01:31.296Z","updated_at":"2024-11-28T01:31:55.426Z","avatar_url":"https://github.com/SangbumChoi.png","language":"Python","funding_links":[],"categories":["人像\\姿势\\3D人脸"],"sub_categories":["网络服务_其他"],"readme":"# Github Code of \"MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices\"\n\n#### [2021.11.23] There will be massive refactoring and optimization expected. It will be released as soon as possible including new model.pth, Please wait for the model!(expecting end of December)\n#### [2022.05.19] Dummy dataloader is added. This will make reduce about to 100x faster that user to generate dummy pth.tar file of MobileHumanPose model for their PoC.\n\n## Introduction\n\nThis repo is official **[PyTorch](https://pytorch.org)** implementation of **[MobileHumanPose: Toward real-time 3D human pose estimation in mobile devices(CVPRW 2021)](https://openaccess.thecvf.com/content/CVPR2021W/MAI/html/Choi_MobileHumanPose_Toward_Real-Time_3D_Human_Pose_Estimation_in_Mobile_Devices_CVPRW_2021_paper.html)**.\n\n## Dependencies\n* [PyTorch](https://pytorch.org)\n* [CUDA](https://developer.nvidia.com/cuda-downloads)\n* [cuDNN](https://developer.nvidia.com/cudnn)\n* [Anaconda](https://www.anaconda.com/download/)\n* [COCO API](https://github.com/cocodataset/cocoapi)\n\nThis code is tested under Ubuntu 16.04, CUDA 11.2 environment with two NVIDIA RTX or V100 GPUs.\n\nPython 3.6.5 version with virtualenv is used for development.\n\n## Directory\n\n### Root\nThe `${ROOT}` is described as below.\n```\n${ROOT}\n|-- data\n|-- demo\n|-- common\n|-- main\n|-- tool\n|-- vis\n`-- output\n```\n* `data` contains data loading codes and soft links to images and annotations directories.\n* `demo` contains demo codes.\n* `common` contains kernel codes for 3d multi-person pose estimation system. Also custom backbone is implemented in this repo\n* `main` contains high-level codes for training or testing the network.\n* `tool` contains data pre-processing codes. You don't have to run this code. I provide pre-processed data below.\n* `vis` contains scripts for 3d visualization.\n* `output` contains log, trained models, visualized outputs, and test result.\n\n### Data\nYou need to follow directory structure of the `data` as below.\n```\n${POSE_ROOT}\n|-- data\n|   |-- Human36M\n|   |   |-- bbox_root\n|   |   |   |-- bbox_root_human36m_output.json\n|   |   |-- images\n|   |   |-- annotations\n|   |-- MPII\n|   |   |-- images\n|   |   |-- annotations\n|   |-- MSCOCO\n|   |   |-- bbox_root\n|   |   |   |-- bbox_root_coco_output.json\n|   |   |-- images\n|   |   |   |-- train2017\n|   |   |   |-- val2017\n|   |   |-- annotations\n|   |-- MuCo\n|   |   |-- data\n|   |   |   |-- augmented_set\n|   |   |   |-- unaugmented_set\n|   |   |   |-- MuCo-3DHP.json\n|   |-- MuPoTS\n|   |   |-- bbox_root\n|   |   |   |-- bbox_mupots_output.json\n|   |   |-- data\n|   |   |   |-- MultiPersonTestSet\n|   |   |   |-- MuPoTS-3D.json\n```\n* Download Human3.6M parsed data [[data](https://drive.google.com/drive/folders/1kgVH-GugrLoc9XyvP6nRoaFpw3TmM5xK?usp=sharing)]\n* Download MPII parsed data [[images](http://human-pose.mpi-inf.mpg.de/)][[annotations](https://drive.google.com/drive/folders/1MmQ2FRP0coxHGk0Ntj0JOGv9OxSNuCfK?usp=sharing)]\n* Download MuCo parsed and composited data [[data](https://drive.google.com/drive/folders/1yL2ey3aWHJnh8f_nhWP--IyC9krAPsQN?usp=sharing)]\n* Download MuPoTS parsed data [[images](http://gvv.mpi-inf.mpg.de/projects/SingleShotMultiPerson/)][[annotations](https://drive.google.com/drive/folders/1WmfQ8UEj6nuamMfAdkxmrNcsQTrTfKK_?usp=sharing)]\n* All annotation files follow [MS COCO format](http://cocodataset.org/#format-data).\n* If you want to add your own dataset, you have to convert it to [MS COCO format](http://cocodataset.org/#format-data).\n\n### Output\nYou need to follow the directory structure of the `output` folder as below.\n```\n${POSE_ROOT}\n|-- output\n|-- |-- log\n|-- |-- model_dump\n|-- |-- result\n`-- |-- vis\n```\n* Creating `output` folder as soft link form is recommended instead of folder form because it would take large storage capacity.\n* `log` folder contains training log file.\n* `model_dump` folder contains saved checkpoints for each epoch.\n* `result` folder contains final estimation files generated in the testing stage.\n* `vis` folder contains visualized results.\n\n### 3D visualization\n* Run `$DB_NAME_img_name.py` to get image file names in `.txt` format.\n* Place your test result files (`preds_2d_kpt_$DB_NAME.mat`, `preds_3d_kpt_$DB_NAME.mat`) in `single` or `multi` folder.\n* Run `draw_3Dpose_$DB_NAME.m`\n\n\u003cp align=\"middle\"\u003e\n\u003cimg src=\"assets/test.JPG\"\u003e\n\u003c/p\u003e\n\n## Running 3DMPPE_POSENET\n\n### Requirements\n\n```shell\ncd main\npip install -r requirements.txt\n```\n\n### Setup Training\n* In the `main/config.py`, you can change settings of the model including dataset to use, network backbone, and input size and so on.\n\n### Train\nIn the `main` folder, run\n```bash\npython train.py --gpu 0-1 --backbone LPSKI\n```\nto train the network on the GPU 0,1. \n\nIf you want to continue experiment, run \n```bash\npython train.py --gpu 0-1 --backbone LPSKI --continue\n```\n`--gpu 0,1` can be used instead of `--gpu 0-1`.\n\n### Test\nPlace trained model at the `output/model_dump/`.\n\nIn the `main` folder, run \n```bash\npython test.py --gpu 0-1 --test_epoch 20-21 --backbone LPSKI\n```\nto test the network on the GPU 0,1 with 20th and 21th epoch trained model. `--gpu 0,1` can be used instead of `--gpu 0-1`. For the backbone you can either choose \nBACKBONE_DICT = {\n    'LPRES':LpNetResConcat,\n    'LPSKI':LpNetSkiConcat,\n    'LPWO':LpNetWoConcat\n    }\n\n#### Human3.6M dataset using protocol 1\nFor the evaluation, you can run `test.py` or there are evaluation codes in `Human36M`.\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/protocol1.JPG\"\u003e\n\u003c/p\u003e\n\n#### Human3.6M dataset using protocol 2\nFor the evaluation, you can run `test.py` or there are evaluation codes in `Human36M`.\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/protocol2.JPG\"\u003e\n\u003c/p\u003e\n\n#### MuPoTS-3D dataset\nFor the evaluation, run `test.py`.  After that, move `data/MuPoTS/mpii_mupots_multiperson_eval.m` in `data/MuPoTS/data`. Also, move the test result files (`preds_2d_kpt_mupots.mat` and `preds_3d_kpt_mupots.mat`) in `data/MuPoTS/data`. Then run `mpii_mupots_multiperson_eval.m` with your evaluation mode arguments.\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/mupots.JPG\"\u003e\n\u003c/p\u003e\n\n#### TFLite inference\nFor the inference in mobile devices we also tested in mobile devices which converting PyTorch implementation through onnx and finally serving into TFlite.\nOfficial demo app is available in [here](https://github.com/tucan9389/PoseEstimation-TFLiteSwift)\n\n## Reference\n\n**What this repo cames from:**\nTraining section and is based on following paper and github\n* [PyTorch](https://pytorch.org) implementation of [Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image (ICCV 2019)](https://arxiv.org/abs/1907.11346).\n* Flexible and simple code.\n* Compatibility for most of the publicly available 2D and 3D, single and multi-person pose estimation datasets including **[Human3.6M](http://vision.imar.ro/human3.6m/description.php), [MPII](http://human-pose.mpi-inf.mpg.de/), [MS COCO 2017](http://cocodataset.org/#home), [MuCo-3DHP](http://gvv.mpi-inf.mpg.de/projects/SingleShotMultiPerson/) and [MuPoTS-3D](http://gvv.mpi-inf.mpg.de/projects/SingleShotMultiPerson/)**.\n* Human pose estimation visualization code.\n\n```\n@InProceedings{Choi_2021_CVPR,\n    author    = {Choi, Sangbum and Choi, Seokeon and Kim, Changick},\n    title     = {MobileHumanPose: Toward Real-Time 3D Human Pose Estimation in Mobile Devices},\n    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},\n    month     = {June},\n    year      = {2021},\n    pages     = {2328-2338}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSangbumChoi%2FMobileHumanPose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSangbumChoi%2FMobileHumanPose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSangbumChoi%2FMobileHumanPose/lists"}