Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/yas-sim/human-pose-estimation-2d-demo
Deep learning based human pose estimation demo in Python using Intel OpenVINO toolkit
https://github.com/yas-sim/human-pose-estimation-2d-demo
deep-learning demo dl-models edge-computing hms human-pose-estimation inference inferencing-engine intel openvino openvino-toolkit pafs pair-affinity-fields person-pose-estimation pose-estimation python
Last synced: about 1 month ago
JSON representation
Deep learning based human pose estimation demo in Python using Intel OpenVINO toolkit
- Host: GitHub
- URL: https://github.com/yas-sim/human-pose-estimation-2d-demo
- Owner: yas-sim
- License: apache-2.0
- Created: 2020-05-05T06:23:56.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2022-04-09T14:07:19.000Z (over 2 years ago)
- Last Synced: 2024-04-23T00:09:51.935Z (8 months ago)
- Topics: deep-learning, demo, dl-models, edge-computing, hms, human-pose-estimation, inference, inferencing-engine, intel, openvino, openvino-toolkit, pafs, pair-affinity-fields, person-pose-estimation, pose-estimation, python
- Language: Python
- Size: 16.6 MB
- Stars: 18
- Watchers: 1
- Forks: 4
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Human Pose Estimation 2D Demo (Python version)
This program demonstrates how to use the [`human-pose-estimation-0001`](https://docs.openvinotoolkit.org/latest/_models_intel_human_pose_estimation_0001_description_human_pose_estimation_0001.html) model from OpenVINO [Open Model Zoo](https://docs.openvinotoolkit.org/latest/_models_intel_index.html). The program can extract the pose (a set of coordinations which consists of 18 points from a person) from the people in an image.
The DL model generates [heatmaps (HMs) and pair affinity fields (PAFs)](https://arvrjourney.com/human-pose-estimation-using-openpose-with-tensorflow-part-2-e78ab9104fc8) form the imput image. The HMs and PAFs are kind of images in 57x32 resolution, and the HMs consists of 18 images and the PAFs consists of 18x2 images.
The user program has to perform image analytics as a postprocess to reconstruct the human pose from the PAFs and HMs.
You can write this process in Python but it's too heavy for Python and could be the bottleneck of the program.
Fortunately, Intel provides this postprocess code in a Python module written in C++. The module is designed for `3d-human-pose-estimation` demo program but the basics are the same and it can be reused for reconstructing human pose form the PAFs and HMs generated by a 2D human pose model.
This project contains a small script file to copy and build the `pose_extractor` module and a Python demo program which demonstrates how to use the `pose_extractor` in a Python code.
このプログラムはOpenVINO [Open Model Zoo](https://docs.openvinotoolkit.org/latest/_models_intel_index.html)の [`human-pose-estimation-0001`](https://docs.openvinotoolkit.org/latest/_models_intel_human_pose_estimation_0001_description_human_pose_estimation_0001.html)モデルの使い方のデモプログラムです。プログラムは入力画像から人の姿勢(一人あたり18の座標点)を抽出します。
DLモデル自身は入力画像を推論し、[heatmaps (HMs)とpair affinity fields (PAFs)](https://arvrjourney.com/human-pose-estimation-using-openpose-with-tensorflow-part-2-e78ab9104fc8)を出力します。HMsとPAFsは一種の画像で57x32の解像度を持ち、HMsは18枚、PAFsは18x2枚の画像からなります。ユーザープログラムは後処理としてPAFsとHMsの画像解析を行い、人の姿勢(各点の座標)を再構築するアルゴリズムを記述する必要があります。
この処理をPythonで記述することも可能ですが、重い処理であり、プログラム全体のボトルネックとなります。
幸いなことにインテルがこの後処理を行うC++で記述された`pose_extractor` Pythonモジュールを提供していますのでこれを利用することが可能です。このモジュールは3D-human-pose-estimationデモ用の物ですが基本の処理は同じですのでこれを2Dモデルに再利用することが可能です。
このプロジェクトは`pose_extractor`後処理モジュールをビルドするための小さなスクリプトと、このモジュールを利用する方法を示すためのPythonコードからなります。
**[NEW 09-APR-2022] OpenVINO 2022.1 Support (API 2.0 support)**
### Human Pose Estimation Result
### Required DL Models to Run This Demo
The demo expects the following models in the Intermediate Representation (IR) format:
* `human-pose-estimation-0001`
You can download this model from OpenVINO [Open Model Zoo](https://github.com/opencv/open_model_zoo).
In the `models.lst` is the list of appropriate models for this demo that can be obtained via `Model downloader`.
Please see more information about `Model downloader` [here](../../../tools/downloader/README.md).
## How to Run
### 0. Prerequisites
- **OpenVINO 2020.2**
- If you haven't installed it, go to the OpenVINO web page and follow the [*Get Started*](https://software.intel.com/en-us/openvino-toolkit/documentation/get-started) guide to do it.
- **Microsoft Visual Studio 2019**
- Community version works. MS VS 2017/2015 may also work. You need to modify the `build-poseextractor.bat` accordingly.
- This is requied to build the `pose_extractor` module.
### 1. Install dependencies
The demo depends on:
- `opencv-python`
- `numpy`
To install all the required Python modules you can use:
``` sh
(Linux) pip3 install -r requirements.txt
(Win10) pip install -r requirements.txt
```
### 2. Download DL models from OMZ
Use `Model Downloader` to download the required models.
``` sh
(Linux) python3 $INTEL_OPENVINO_DIR/deployment_tools/tools/model_downloader/downloader.py --list models.lst
(Win10) python "%INTEL_OPENVINO_DIR%\deployment_tools\tools\model_downloader\downloader.py" --list models.lst
```
### 3. Build `pose_extracor` C++ module
The `build-poseextractor.[sh|bat]` will copy the `pose_extractor` C++ code from the OpenVINO demo program directory and build it. You'll have `pose_extractor.pyd` (Win10) or `pose_extractor.so` (Ubuntu) if you could successfully build the module.
``` sh
(Linux)
$ chmod +x build-poseextractor.sh
$ ./build-poseextractor.sh
(Win10)
* Open 'Developer Command Prompt for VS 2019' instead of the 'cmd.exe'. Or the build process will fail because the PATH is not properly set for `msbuild` command.
> build-poseextractor.bat
```
### 4. Run the demo app
Attach a USB webCam as input of the demo program, then run the program. If you want to use a movie file as an input, you can modify the source code to do it.
Also, you can change inference devide by modifying `ie.load_network(net_hp, 'CPU')` in the code. You can use `CPU`, `GPU`, `MYRIAD`, `HDDL`, `HETERO:FPGA,CPU`, `MULTI:CPU,GPU`, and so on. Please refer to the [OpenVINO document web site](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_supported_plugins_Supported_Devices.html) for details.
``` sh
(Linux) python3 human-pose-estimation-2d.py
(Win10) python human-pose-estimation-2d.py
```
## Demo Output
The application draws the results on the input image.
## Tested Environment
- Windows 10 x64 1909 and Ubuntu 18.04 LTS
- Intel(r) Distribution of OpenVINO(tm) toolkit 2021.3
- Python 3.6.5 x64
## See Also
* [Using Open Model Zoo demos](../../README.md)
* [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
* [Model Downloader](../../../tools/downloader/README.md)