{"id":15034675,"url":"https://github.com/zhec/realtime_multi-person_pose_estimation","last_synced_at":"2025-05-14T16:01:49.820Z","repository":{"id":37432571,"uuid":"76280357","full_name":"ZheC/Realtime_Multi-Person_Pose_Estimation","owner":"ZheC","description":"Code repo for realtime multi-person pose estimation in CVPR'17 (Oral)","archived":false,"fork":false,"pushed_at":"2020-03-21T13:01:08.000Z","size":46830,"stargazers_count":5112,"open_issues_count":107,"forks_count":1361,"subscribers_count":258,"default_branch":"master","last_synced_at":"2025-04-12T01:52:33.639Z","etag":null,"topics":["caffe","computer-vision","cpp11","cvpr-2017","deep-learning","human-behavior-understanding","human-pose-estimation","matlab","python","realtime"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ZheC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-12-12T17:40:12.000Z","updated_at":"2025-04-11T01:41:00.000Z","dependencies_parsed_at":"2022-07-12T13:33:53.691Z","dependency_job_id":null,"html_url":"https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZheC%2FRealtime_Multi-Person_Pose_Estimation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZheC%2FRealtime_Multi-Person_Pose_Estimation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZheC%2FRealtime_Multi-Person_Pose_Estimation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZheC%2FRealtime_Multi-Person_Pose_Estimation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ZheC","download_url":"https://codeload.github.com/ZheC/Realtime_Multi-Person_Pose_Estimation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248505873,"owners_count":21115354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["caffe","computer-vision","cpp11","cvpr-2017","deep-learning","human-behavior-understanding","human-pose-estimation","matlab","python","realtime"],"created_at":"2024-09-24T20:25:58.477Z","updated_at":"2025-04-12T01:52:43.165Z","avatar_url":"https://github.com/ZheC.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Realtime Multi-Person Pose Estimation\nBy [Zhe Cao](https://people.eecs.berkeley.edu/~zhecao/), [Tomas Simon](http://www.cs.cmu.edu/~tsimon/), [Shih-En Wei](https://scholar.google.com/citations?user=sFQD3k4AAAAJ\u0026hl=en), [Yaser Sheikh](http://www.cs.cmu.edu/~yaser/).\n\n## Introduction\nCode repo for winning 2016 MSCOCO Keypoints Challenge, 2016 ECCV Best Demo Award, and 2017 CVPR Oral paper.  \n\nWatch our video result in [YouTube](https://www.youtube.com/watch?v=pW6nZXeWlGM\u0026t=77s) or [our website](http://posefs1.perception.cs.cmu.edu/Users/ZheCao/humanpose.mp4). \n\nWe present a bottom-up approach for realtime multi-person pose estimation, without using any person detector. For more details, refer to our [CVPR'17 paper](https://arxiv.org/abs/1611.08050), our [oral presentation video recording](https://www.youtube.com/watch?v=OgQLDEAjAZ8\u0026list=PLvsYSxrlO0Cl4J_fgMhj2ElVmGR5UWKpB) at CVPR 2017 or our [presentation slides](http://image-net.org/challenges/talks/2016/Multi-person%20pose%20estimation-CMU.pdf) at ILSVRC and COCO workshop 2016.\n\n\u003cp align=\"left\"\u003e\n\u003cimg src=\"https://github.com/ZheC/Multi-Person-Pose-Estimation/blob/master/readme/dance.gif\", width=\"720\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"left\"\u003e\n\u003cimg src=\"https://github.com/ZheC/Multi-Person-Pose-Estimation/blob/master/readme/shake.gif\", width=\"720\"\u003e\n\u003c/p\u003e\n\nThis project is licensed under the terms of the [license](LICENSE).\n\n## Other Implementations\nThank you all for the efforts for the reimplementation! If you have new implementation and want to share with others, feel free to make a pull request or email me! \n- Our new C++ library [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) (testing only)\n- Tensorflow [[version 1]](https://github.com/ildoonet/tf-openpose) | [[version 2]](https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation) | [[version 3]](https://github.com/anatolix/keras_Realtime_Multi-Person_Pose_Estimation) | [[version 4]](https://github.com/raymon-tian/keras_Realtime_Multi-Person_Pose_Estimation) | [[version 5]](https://github.com/tensorlayer/openpose) | [[version 6]](https://github.com/YangZeyu95/unofficial-implement-of-openpose)  | [[version 7 - TF2.1]](https://github.com/MikeOfZen/Yet-Another-Openpose-Implementation) \n- Pytorch [[version 1]](https://github.com/tensorboy/pytorch_Realtime_Multi-Person_Pose_Estimation) | [[version 2]](https://github.com/last-one/Pytorch_Realtime_Multi-Person_Pose_Estimation) | [[version 3]](https://github.com/CVBox/PyTorchCV) \n- Caffe2 [[version 1]](https://github.com/eddieyi/caffe2-pose-estimation)\n- Chainer [[version 1]](https://github.com/DeNA/Chainer_Realtime_Multi-Person_Pose_Estimation)\n- MXnet [[version 1]](https://github.com/dragonfly90/mxnet_Realtime_Multi-Person_Pose_Estimation)\n- MatConvnet [[version 1]](https://github.com/coocoky/matconvnet_Realtime_Multi-Person_Pose_Estimation)\n- CNTK [[version 1]](https://github.com/Hzzone/CNTK_Realtime_Multi-Person_Pose_Estimation)\n\n\n## Contents\n1. [Testing](#testing)\n2. [Training](#training)\n3. [Citation](#citation)\n\n## Testing\n\n### C++ (realtime version, for demo purpose)\n- Please use [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose), now it can run in CPU/ GPU and windows /Ubuntu.\n- Three input options: images, video, webcam\n\n### Matlab (slower, for COCO evaluation)\n- Compatible with general [Caffe](http://caffe.berkeleyvision.org/). Compile matcaffe. \n- Run `cd testing; get_model.sh` to retrieve our latest MSCOCO model from our web server.\n- Change the caffepath in the `config.m` and run `demo.m` for an example usage.\n\n### Python\n- `cd testing/python`\n- `ipython notebook`\n- Open `demo.ipynb` and execute the code\n\n## Training\n\n### Network Architecture\n![Teaser?](https://github.com/ZheC/Multi-Person-Pose-Estimation/blob/master/readme/arch.png)\n\n### Training Steps \n- Run `cd training; bash getData.sh` to obtain the COCO images in `dataset/COCO/images/`, keypoints annotations in `dataset/COCO/annotations/` and [COCO official toolbox](https://github.com/pdollar/coco) in `dataset/COCO/coco/`. \n- Run `getANNO.m` in matlab to convert the annotation format from json to mat in `dataset/COCO/mat/`.\n- Run `genCOCOMask.m` in matlab to obatin the mask images for unlabeled person. You can use 'parfor' in matlab to speed up the code.\n- Run `genJSON('COCO')` to generate a json file in `dataset/COCO/json/` folder. The json files contain raw informations needed for training.\n- Run `python genLMDB.py` to generate your LMDB. (You can also download our LMDB for the COCO dataset (189GB file) by: `bash get_lmdb.sh`)\n- Download our modified caffe: [caffe_train](https://github.com/CMU-Perceptual-Computing-Lab/caffe_train). Compile pycaffe. It will be merged with caffe_rtpose (for testing) soon.\n- Run `python setLayers.py --exp 1` to generate the prototxt and shell file for training.\n- Download [VGG-19 model](https://gist.github.com/ksimonyan/3785162f95cd2d5fee77), we use it to initialize the first 10 layers for training.\n- Run `bash train_pose.sh 0,1` (generated by setLayers.py) to start the training with two gpus. \n\n## Citation\nPlease cite the paper in your publications if it helps your research:\n\n    \n    \n    @inproceedings{cao2017realtime,\n      author = {Zhe Cao and Tomas Simon and Shih-En Wei and Yaser Sheikh},\n      booktitle = {CVPR},\n      title = {Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields},\n      year = {2017}\n      }\n\t  \n    @inproceedings{wei2016cpm,\n      author = {Shih-En Wei and Varun Ramakrishna and Takeo Kanade and Yaser Sheikh},\n      booktitle = {CVPR},\n      title = {Convolutional pose machines},\n      year = {2016}\n      }\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhec%2Frealtime_multi-person_pose_estimation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhec%2Frealtime_multi-person_pose_estimation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhec%2Frealtime_multi-person_pose_estimation/lists"}