Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/stefanopini/simple-hrnet
Multi-person Human Pose Estimation with HRNet in Pytorch
https://github.com/stefanopini/simple-hrnet
coco-dataset computer-vision deep-learning hrnet human-pose-estimation keypoint-detection machine-learning pytorch yolov3
Last synced: 3 days ago
JSON representation
Multi-person Human Pose Estimation with HRNet in Pytorch
- Host: GitHub
- URL: https://github.com/stefanopini/simple-hrnet
- Owner: stefanopini
- License: gpl-3.0
- Created: 2019-05-07T12:43:37.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2022-12-29T18:17:05.000Z (about 2 years ago)
- Last Synced: 2025-01-11T19:09:28.166Z (10 days ago)
- Topics: coco-dataset, computer-vision, deep-learning, hrnet, human-pose-estimation, keypoint-detection, machine-learning, pytorch, yolov3
- Language: Python
- Homepage:
- Size: 76.3 MB
- Stars: 584
- Watchers: 9
- Forks: 110
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Multi-person Human Pose Estimation with HRNet in PyTorch
This is an unofficial implementation of the paper
[*Deep High-Resolution Representation Learning for Human Pose Estimation*](https://arxiv.org/abs/1902.09212).
The code is a simplified version of the [official code](https://github.com/leoxiaobin/deep-high-resolution-net.pytorch)
with the ease-of-use in mind.
The code is fully compatible with the
[official pre-trained weights](https://github.com/leoxiaobin/deep-high-resolution-net.pytorch) and the results are the
same of the original implementation (only slight differences on gpu due to CUDA).
It supports both Windows and Linux.
This repository provides:
- A simple ``HRNet`` implementation in PyTorch (>=1.0) - compatible with official weights (``pose_hrnet_*``).
- A simple class (``SimpleHRNet``) that loads the HRNet network for the human pose estimation, loads the pre-trained weights,
and make human predictions on a single image or a batch of images.
- Support for "SimpleBaselines" model based on ResNet - compatible with official weights (``pose_resnet_*``).
- Support for multi-GPU inference.
- Add options for retrieving yolo bounding boxes and HRNet heatmaps.
- **NEW** Multi-person support with
[YOLOv3](https://github.com/eriklindernoren/PyTorch-YOLOv3/tree/47b7c912877ca69db35b8af3a38d6522681b3bb3)
(enabled by default), YOLOv3-tiny, or [YOLOv5](https://github.com/ultralytics/yolov5) by Ultralytics.
- A reference code that runs a live demo reading frames from a webcam or a video file.
- A relatively-simple code for training and testing the HRNet network.
- A specific script for training the network on the COCO dataset.
- **NEW** An updated [Jupyter Notebook](https://github.com/stefanopini/simple-HRNet/blob/master/SimpleHRNet_notebook.ipynb) compatible with Google Colab showcasing how to use this repository.
- 
[Click here](https://colab.research.google.com/github/stefanopini/simple-HRNet/blob/master/SimpleHRNet_notebook.ipynb) to open the notebook on Colab!
- Thanks to [@basicvisual](https://github.com/basicvisual) and [@wuyenlin](https://github.com/wuyenlin) for the initial notebook.
- **NEW** Support for TensorRT (thanks to [@gpastal24](https://github.com/gpastal24), see [#99](https://github.com/stefanopini/simple-HRNet/pull/99) and [#100](https://github.com/stefanopini/simple-HRNet/pull/100)).
If you are interested in **HigherHRNet**, please look at [*simple-HigherHRNet*](https://github.com/stefanopini/simple-HigherHRNet)
### Examples
### Class usage
```
import cv2
from SimpleHRNet import SimpleHRNet
model = SimpleHRNet(48, 17, "./weights/pose_hrnet_w48_384x288.pth")
image = cv2.imread("image.png", cv2.IMREAD_COLOR)
joints = model.predict(image)
```
The most useful parameters of the `__init__` function are:
cnumber of channels (HRNet: 32, 48; PoseResNet: resnet size)
nof_jointsnumber of joints (COCO: 17, MPII: 16)
checkpoint_pathpath of the (official) weights to be loaded
model_name'HRNet' or 'PoseResNet'
resolutionimage resolution, it depends on the loaded weights
multipersonenable multiperson prediction
return_heatmapsthe `predict` method returns also the heatmaps
return_bounding_boxesthe `predict` method returns also the bounding boxes (useful in conjunction with
`multiperson`)
max_batch_sizemaximum batch size used in hrnet inference
devicedevice (cpu or cuda)
### Running the live demo
From a connected camera:
```
python scripts/live-demo.py --camera_id 0
```
From a saved video:
```
python scripts/live-demo.py --filename video.mp4
```
For help:
```
python scripts/live-demo.py --help
```
### Extracting keypoints:
From a saved video:
```
python scripts/extract-keypoints.py --format csv --filename video.mp4
```
For help:
```
python scripts/extract-keypoints.py --help
```
### Converting the model to TensorRT:
Warning: require the installation of TensorRT (see Nvidia website) and onnx.
On some platforms, they can be installed with
```
pip install tensorrt onnx
```
Converting in FP16:
```
python scripts/export-tensorrt-model.py --device 0 --half
```
For help:
```
python scripts/export-tensorrt-model.py --help
```
### Running the training script
```
python scripts/train_coco.py
```
For help:
```
python scripts/train_coco.py --help
```
### Installation instructions
- Clone the repository
``git clone https://github.com/stefanopini/simple-HRNet.git``
- Install the required packages
``pip install -r requirements.txt``
- Download the official pre-trained weights from
[https://github.com/leoxiaobin/deep-high-resolution-net.pytorch](https://github.com/leoxiaobin/deep-high-resolution-net.pytorch)
Direct links ([official Drive folder](https://drive.google.com/drive/folders/1hOTihvbyIxsm5ygDpbUuJ7O_tzv4oXjC), [official OneDrive folder](https://1drv.ms/f/s!AhIXJn_J-blW231MH2krnmLq5kkQ)):
- COCO w48 384x288 (more accurate, but slower) - Used as default in `live_demo.py` and the other scripts
[pose_hrnet_w48_384x288.pth](https://drive.google.com/open?id=1UoJhTtjHNByZSm96W3yFTfU5upJnsKiS)
- COCO w32 256x192 (less accurate, but faster)
[pose_hrnet_w32_256x192.pth](https://drive.google.com/open?id=1zYC7go9EV0XaSlSBjMaiyE_4TcHc_S38)
- MPII w32 256x256 (MPII human joints)
[pose_hrnet_w32_256x256.pth](https://drive.google.com/open?id=1_wn2ifmoQprBrFvUCDedjPON4Y6jsN-v)
Remember to set the parameters of SimpleHRNet accordingly (in particular `c`, `nof_joints`, and `resolution`).
- For multi-person support:
- Get YOLOv3:
- Clone [YOLOv3](https://github.com/eriklindernoren/PyTorch-YOLOv3/tree/47b7c912877ca69db35b8af3a38d6522681b3bb3)
in the folder ``./models/detectors`` and change the folder name from ``PyTorch-YOLOv3`` to ``yolo``
OR
- Update git submodules
``git submodule update --init --recursive``
- Install YOLOv3 required packages
``pip install -r requirements.txt`` (from folder `./models/detectors/yolo`)
- Download the pre-trained weights running the script ``download_weights.sh`` from the ``weights`` folder
- (Optional) Download the [COCO dataset](http://cocodataset.org/#download) and save it in ``./datasets/COCO``
- Your folders should look like:
```
simple-HRNet
├── datasets (datasets - for training only)
│ └── COCO (COCO dataset)
├── losses (loss functions)
├── misc (misc)
│ └── nms (CUDA nms module - for training only)
├── models (pytorch models)
│ └── detectors (people detectors)
│ └── yolo (PyTorch-YOLOv3 repository)
│ ├── ...
│ └── weights (YOLOv3 weights)
├── scripts (scripts)
├── testing (testing code)
├── training (training code)
└── weights (HRnet weights)
```
- If you want to run the training script on COCO `scripts/train_coco.py`, you have to build the `nms` module first.
Please note that a linux machine with CUDA is currently required.
Build it with either:
- `cd misc; make` or
- `cd misc/nms; python setup_linux.py build_ext --inplace`
You may need to add the `./misc/nms` directory in the `PYTHONPATH` variable:
`export PYTHONPATH="/misc/nms:$PYTHONPATH"`
### Google Colab notebook
Thanks to the great work of [@basicvisual](https://github.com/basicvisual) and [@wuyenlin](https://github.com/wuyenlin), you can also try this repository online on Google Colab.
More details and the notebook URL are available in [this issue](https://github.com/stefanopini/simple-HRNet/issues/84#issuecomment-908199736).
Please make sure to make a copy on your own Google Drive and to change the Colab "Runtime type" from CPU to GPU or TPU.