{"id":15703868,"url":"https://github.com/daniil-osokin/lightweight-human-pose-estimation.pytorch","last_synced_at":"2025-05-15T03:07:44.526Z","repository":{"id":37664834,"uuid":"173498120","full_name":"Daniil-Osokin/lightweight-human-pose-estimation.pytorch","owner":"Daniil-Osokin","description":"Fast and accurate human pose estimation in PyTorch. Contains implementation of \"Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose\" paper.","archived":false,"fork":false,"pushed_at":"2024-04-30T09:54:40.000Z","size":213,"stargazers_count":2161,"open_issues_count":2,"forks_count":481,"subscribers_count":34,"default_branch":"master","last_synced_at":"2025-05-15T03:07:37.018Z","etag":null,"topics":["coco-keypoints-detection","deep-learning","human-pose-estimation","lightweight","lightweight-openpose","mscoco-keypoint","openpose","openvino","pose-estimation","pytorch","real-time"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Daniil-Osokin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-02T20:57:11.000Z","updated_at":"2025-05-12T06:23:34.000Z","dependencies_parsed_at":"2022-07-12T16:42:40.501Z","dependency_job_id":"195b7ad5-bfbe-4fc5-a796-18da7aecbc2a","html_url":"https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch","commit_stats":{"total_commits":22,"total_committers":1,"mean_commits":22.0,"dds":0.0,"last_synced_commit":"d23c284b09acf27a163e1febd511e7482cac25ed"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Daniil-Osokin%2Flightweight-human-pose-estimation.pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Daniil-Osokin%2Flightweight-human-pose-estimation.pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Daniil-Osokin%2Flightweight-human-pose-estimation.pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Daniil-Osokin%2Flightweight-human-pose-estimation.pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Daniil-Osokin","download_url":"https://codeload.github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254264769,"owners_count":22041794,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coco-keypoints-detection","deep-learning","human-pose-estimation","lightweight","lightweight-openpose","mscoco-keypoint","openpose","openvino","pose-estimation","pytorch","real-time"],"created_at":"2024-10-03T20:07:28.231Z","updated_at":"2025-05-15T03:07:39.500Z","avatar_url":"https://github.com/Daniil-Osokin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose\n\nThis repository contains training code for the paper [Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose](https://arxiv.org/pdf/1811.12004.pdf). This work heavily optimizes the [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) approach to reach real-time inference on CPU with negliable accuracy drop. It detects a skeleton (which consists of keypoints and connections between them) to identify human poses for every person inside the image. The pose may contain up to 18 keypoints: ears, eyes, nose, neck, shoulders, elbows, wrists, hips, knees, and ankles. On COCO 2017 Keypoint Detection validation set this code achives 40% AP for the single scale inference (no flip or any post-processing done). The result can be reproduced using this repository. *This repo significantly overlaps with https://github.com/opencv/openvino_training_extensions, however contains just the necessary code for human pose estimation.*\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"data/preview.jpg\" /\u003e\n\u003c/p\u003e\n\n:fire: Check out our [new work](https://github.com/Daniil-Osokin/gccpm-look-into-person-cvpr19.pytorch) on accurate (and still fast) single-person pose estimation, which ranked 10\u003csup\u003eth\u003c/sup\u003e on CVPR'19 [Look-Into-Person](http://47.100.21.47:9999/index.php) challenge.\n\n:fire::fire: Check out our lightweight [3D pose estimation](https://github.com/Daniil-Osokin/lightweight-human-pose-estimation-3d-demo.pytorch), which is based on [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://arxiv.org/pdf/1712.03453.pdf) paper and this work.\n\n## Table of Contents\n\n* [Requirements](#requirements)\n* [Prerequisites](#prerequisites)\n* [Training](#training)\n* [Validation](#validation)\n* [Pre-trained model](#pre-trained-model)\n* [C++ demo](#cpp-demo)\n* [Python demo](#python-demo)\n* [Citation](#citation)\n\n### Other Implementations\n\n* TensorFlow by [murdockhou](https://github.com/murdockhou/lightweight_openpose).\n* OpenVINO by [Pavel Druzhkov](https://github.com/openvinotoolkit/open_model_zoo/pull/1718/).\n\n## Requirements\n\n* Ubuntu 16.04\n* Python 3.6\n* PyTorch 0.4.1 (should also work with 1.0, but not tested)\n\n## Prerequisites\n\n1. Download COCO 2017 dataset: [http://cocodataset.org/#download](http://cocodataset.org/#download) (train, val, annotations) and unpack it to `\u003cCOCO_HOME\u003e` folder.\n2. Install requirements `pip install -r requirements.txt`\n\n## Training\n\nTraining consists of 3 steps (given AP values for full validation dataset):\n* Training from MobileNet weights. Expected AP after this step is ~38%.\n* Training from weights, obtained from previous step. Expected AP after this step is ~39%.\n* Training from weights, obtained from previous step and increased number of refinement stages to 3 in network. Expected AP after this step is ~40% (for the network with 1 refinement stage, two next are discarded).\n\n1. Download pre-trained MobileNet v1 weights `mobilenet_sgd_68.848.pth.tar` from: [https://github.com/marvis/pytorch-mobilenet](https://github.com/marvis/pytorch-mobilenet) (sgd option). If this doesn't work, download from [GoogleDrive](https://drive.google.com/file/d/18Ya27IAhILvBHqV_tDp0QjDFvsNNy-hv/view?usp=sharing).\n\n2. Convert train annotations in internal format. Run `python scripts/prepare_train_labels.py --labels \u003cCOCO_HOME\u003e/annotations/person_keypoints_train2017.json`. It will produce `prepared_train_annotation.pkl` with converted in internal format annotations.\n\n   [OPTIONAL] For fast validation it is recommended to make *subset* of validation dataset. Run `python scripts/make_val_subset.py --labels \u003cCOCO_HOME\u003e/annotations/person_keypoints_val2017.json`. It will produce `val_subset.json` with annotations just for 250 random images (out of 5000).\n\n3. To train from MobileNet weights, run `python train.py --train-images-folder \u003cCOCO_HOME\u003e/train2017/ --prepared-train-labels prepared_train_annotation.pkl --val-labels val_subset.json --val-images-folder \u003cCOCO_HOME\u003e/val2017/ --checkpoint-path \u003cpath_to\u003e/mobilenet_sgd_68.848.pth.tar --from-mobilenet`\n\n4. Next, to train from checkpoint from previous step, run `python train.py --train-images-folder \u003cCOCO_HOME\u003e/train2017/ --prepared-train-labels prepared_train_annotation.pkl --val-labels val_subset.json --val-images-folder \u003cCOCO_HOME\u003e/val2017/ --checkpoint-path \u003cpath_to\u003e/checkpoint_iter_420000.pth --weights-only`\n\n5. Finally, to train from checkpoint from previous step and 3 refinement stages in network, run `python train.py --train-images-folder \u003cCOCO_HOME\u003e/train2017/ --prepared-train-labels prepared_train_annotation.pkl --val-labels val_subset.json --val-images-folder \u003cCOCO_HOME\u003e/val2017/ --checkpoint-path \u003cpath_to\u003e/checkpoint_iter_280000.pth --weights-only --num-refinement-stages 3`. We took checkpoint after 370000 iterations as the final one.\n\nWe did not perform the best checkpoint selection at any step, so similar result may be achieved after less number of iterations.\n\n#### Known issue\n\nWe observe this error with maximum number of open files (`ulimit -n`) equals to 1024:\n\n```\n  File \"train.py\", line 164, in \u003cmodule\u003e\n    args.log_after, args.val_labels, args.val_images_folder, args.val_output_name, args.checkpoint_after, args.val_after)\n  File \"train.py\", line 77, in train\n    for _, batch_data in enumerate(train_loader):\n  File \"/\u003cpath\u003e/python3.6/site-packages/torch/utils/data/dataloader.py\", line 330, in __next__\n    idx, batch = self._get_batch()\n  File \"/\u003cpath\u003e/python3.6/site-packages/torch/utils/data/dataloader.py\", line 309, in _get_batch\n    return self.data_queue.get()\n  File \"/\u003cpath\u003e/python3.6/multiprocessing/queues.py\", line 337, in get\n    return _ForkingPickler.loads(res)\n  File \"/\u003cpath\u003e/python3.6/site-packages/torch/multiprocessing/reductions.py\", line 151, in rebuild_storage_fd\n    fd = df.detach()\n  File \"/\u003cpath\u003e/python3.6/multiprocessing/resource_sharer.py\", line 58, in detach\n    return reduction.recv_handle(conn)\n  File \"/\u003cpath\u003e/python3.6/multiprocessing/reduction.py\", line 182, in recv_handle\n    return recvfds(s, 1)[0]\n  File \"/\u003cpath\u003e/python3.6/multiprocessing/reduction.py\", line 161, in recvfds\n    len(ancdata))\nRuntimeError: received 0 items of ancdata\n```\n\nTo get rid of it, increase the limit to bigger number, e.g. 65536, run in the terminal: `ulimit -n 65536`\n\n## Validation\n\n1. Run `python val.py --labels \u003cCOCO_HOME\u003e/annotations/person_keypoints_val2017.json --images-folder \u003cCOCO_HOME\u003e/val2017 --checkpoint-path \u003cCHECKPOINT\u003e`\n\n## Pre-trained model \u003ca name=\"pre-trained-model\"/\u003e\n\nThe model expects normalized image (mean=[128, 128, 128], scale=[1/256, 1/256, 1/256]) in planar BGR format.\nPre-trained on COCO model is available at: https://download.01.org/opencv/openvino_training_extensions/models/human_pose_estimation/checkpoint_iter_370000.pth, it has 40% of AP on COCO validation set (38.6% of AP on the val *subset*).\n\n#### Conversion to OpenVINO format\n\n1. Convert PyTorch model to ONNX format: run script in terminal `python scripts/convert_to_onnx.py --checkpoint-path \u003cCHECKPOINT\u003e`. It produces `human-pose-estimation.onnx`.\n2. Convert ONNX model to OpenVINO format with Model Optimizer: run in terminal `python \u003cOpenVINO_INSTALL_DIR\u003e/deployment_tools/model_optimizer/mo.py --input_model human-pose-estimation.onnx --input data --mean_values data[128.0,128.0,128.0] --scale_values data[256] --output stage_1_output_0_pafs,stage_1_output_1_heatmaps`. This produces model `human-pose-estimation.xml` and weights `human-pose-estimation.bin` in single-precision floating-point format (FP32).\n\n## C++ Demo \u003ca name=\"cpp-demo\"/\u003e\n\nC++ demo can be found in the Intel\u0026reg; OpenVINO\u0026trade; toolkit, the corresponding model is `human-pose-estimation-0001`. Please follow the [official instruction](https://docs.openvino.ai/2024/omz_demos_human_pose_estimation_demo_cpp.html) to run it.\n\n## Python Demo \u003ca name=\"python-demo\"/\u003e\n\nWe provide python demo just for the quick results preview. Please, consider c++ demo for the best performance. To run the python demo from a webcam:\n* `python demo.py --checkpoint-path \u003cpath_to\u003e/checkpoint_iter_370000.pth --video 0`\n\n## Citation:\n\nIf this helps your research, please cite the paper:\n\n```\n@inproceedings{osokin2018lightweight_openpose,\n    author={Osokin, Daniil},\n    title={Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose},\n    booktitle = {arXiv preprint arXiv:1811.12004},\n    year = {2018}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaniil-osokin%2Flightweight-human-pose-estimation.pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdaniil-osokin%2Flightweight-human-pose-estimation.pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaniil-osokin%2Flightweight-human-pose-estimation.pytorch/lists"}