https://github.com/nvidia-ai-iot/trt_pose
Real-time pose estimation accelerated with NVIDIA TensorRT
https://github.com/nvidia-ai-iot/trt_pose
human-pose human-pose-estimation jetson jetson-nano jetson-xavier live-demo pretrained-models pytorch real-time tensorrt torch2trt
Last synced: 3 months ago
JSON representation
Real-time pose estimation accelerated with NVIDIA TensorRT
- Host: GitHub
- URL: https://github.com/nvidia-ai-iot/trt_pose
- Owner: NVIDIA-AI-IOT
- License: mit
- Created: 2019-07-29T19:42:14.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2022-08-12T00:45:06.000Z (about 3 years ago)
- Last Synced: 2024-07-31T19:23:21.575Z (about 1 year ago)
- Topics: human-pose, human-pose-estimation, jetson, jetson-nano, jetson-xavier, live-demo, pretrained-models, pytorch, real-time, tensorrt, torch2trt
- Language: Python
- Homepage:
- Size: 1.97 MB
- Stars: 966
- Watchers: 41
- Forks: 288
- Open Issues: 136
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# trt_pose
> Want to detect hand poses? Check out the new [trt_pose_hand](http://github.com/NVIDIA-AI-IOT/trt_pose_hand) project for real-time hand pose and gesture recognition!
trt_pose is aimed at enabling real-time pose estimation on NVIDIA Jetson. You may find it useful for other NVIDIA platforms as well. Currently the project includes
- Pre-trained models for human pose estimation capable of running in real time on Jetson Nano. This makes it easy to detect features like ``left_eye``, ``left_elbow``, ``right_ankle``, etc.
- Training scripts to train on any keypoint task data in [MSCOCO](https://cocodataset.org/#home) format. This means you can experiment with training trt_pose for keypoint detection tasks other than human pose.
To get started, follow the instructions below. If you run into any issues please [let us know](../../issues).
## Getting Started
To get started with trt_pose, follow these steps.
### Step 1 - Install Dependencies
1. Install PyTorch and Torchvision. To do this on NVIDIA Jetson, we recommend following [this guide](https://forums.developer.nvidia.com/t/72048)
2. Install [torch2trt](https://github.com/NVIDIA-AI-IOT/torch2trt)
```python
git clone https://github.com/NVIDIA-AI-IOT/torch2trt
cd torch2trt
sudo python3 setup.py install --plugins
```
3. Install other miscellaneous packages
```python
sudo pip3 install tqdm cython pycocotools
sudo apt-get install python3-matplotlib
```
### Step 2 - Install trt_pose
```python
git clone https://github.com/NVIDIA-AI-IOT/trt_pose
cd trt_pose
sudo python3 setup.py install
```
### Step 3 - Run the example notebook
We provide a couple of human pose estimation models pre-trained on the MSCOCO dataset. The throughput in FPS is shown for each platform
| Model | Jetson Nano | Jetson Xavier | Weights |
|-------|-------------|---------------|---------|
| resnet18_baseline_att_224x224_A | 22 | 251 | [download (81MB)](https://drive.google.com/open?id=1XYDdCUdiF2xxx4rznmLb62SdOUZuoNbd) |
| densenet121_baseline_att_256x256_B | 12 | 101 | [download (84MB)](https://drive.google.com/open?id=13FkJkx7evQ1WwP54UmdiDXWyFMY1OxDU) |
To run the live Jupyter Notebook demo on real-time camera input, follow these steps
1. Download the model weights using the link in the above table.
2. Place the downloaded weights in the [tasks/human_pose](tasks/human_pose) directory
3. Open and follow the [live_demo.ipynb](tasks/human_pose/live_demo.ipynb) notebook
> You may need to modify the notebook, depending on which model you use
## See also
- [trt_pose_hand](http://github.com/NVIDIA-AI-IOT/trt_pose_hand) - Real-time hand pose estimation based on trt_pose
- [torch2trt](http://github.com/NVIDIA-AI-IOT/torch2trt) - An easy to use PyTorch to TensorRT converter
- [JetBot](http://github.com/NVIDIA-AI-IOT/jetbot) - An educational AI robot based on NVIDIA Jetson Nano
- [JetRacer](http://github.com/NVIDIA-AI-IOT/jetracer) - An educational AI racecar using NVIDIA Jetson Nano
- [JetCam](http://github.com/NVIDIA-AI-IOT/jetcam) - An easy to use Python camera interface for NVIDIA Jetson
## References
The trt_pose model architectures listed above are inspired by the following works, but are not a direct replica. Please review the open-source code and configuration files in this repository for architecture details. If you have any questions feel free to reach out.
* _Cao, Zhe, et al. "Realtime multi-person 2d pose estimation using part affinity fields." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017._
* _Xiao, Bin, Haiping Wu, and Yichen Wei. "Simple baselines for human pose estimation and tracking." Proceedings of the European Conference on Computer Vision (ECCV). 2018._