{"id":13435638,"url":"https://github.com/michalfaber/tensorflow_Realtime_Multi-Person_Pose_Estimation","last_synced_at":"2025-03-18T11:31:51.083Z","repository":{"id":39739846,"uuid":"207535653","full_name":"michalfaber/tensorflow_Realtime_Multi-Person_Pose_Estimation","owner":"michalfaber","description":"Multi-Person Pose Estimation project for Tensorflow 2.0 with a small and fast model based on MobilenetV3 ","archived":false,"fork":false,"pushed_at":"2022-11-21T21:19:07.000Z","size":21435,"stargazers_count":216,"open_issues_count":25,"forks_count":65,"subscribers_count":11,"default_branch":"master","last_synced_at":"2024-10-27T18:58:25.728Z","etag":null,"topics":["android","cmu-model","computer-vision","convolutional-neural-networks","deep-learning","human-pose-estimation","mobile","mobilenet","mobilenetv3","pose-estimation","singlenet","tensorflow","tensorflow2","tflite"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/michalfaber.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-10T11:00:01.000Z","updated_at":"2024-10-10T17:22:53.000Z","dependencies_parsed_at":"2023-01-21T04:00:21.877Z","dependency_job_id":null,"html_url":"https://github.com/michalfaber/tensorflow_Realtime_Multi-Person_Pose_Estimation","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michalfaber%2Ftensorflow_Realtime_Multi-Person_Pose_Estimation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michalfaber%2Ftensorflow_Realtime_Multi-Person_Pose_Estimation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michalfaber%2Ftensorflow_Realtime_Multi-Person_Pose_Estimation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michalfaber%2Ftensorflow_Realtime_Multi-Person_Pose_Estimation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/michalfaber","download_url":"https://codeload.github.com/michalfaber/tensorflow_Realtime_Multi-Person_Pose_Estimation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244211092,"owners_count":20416580,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android","cmu-model","computer-vision","convolutional-neural-networks","deep-learning","human-pose-estimation","mobile","mobilenet","mobilenetv3","pose-estimation","singlenet","tensorflow","tensorflow2","tflite"],"created_at":"2024-07-31T03:00:37.673Z","updated_at":"2025-03-18T11:31:46.066Z","avatar_url":"https://github.com/michalfaber.png","language":"Jupyter Notebook","funding_links":[],"categories":["Sample Codes / Projects \u003ca name=\"sample\" /\u003e ⛏️📐📁"],"sub_categories":["Reinforcement Learning \u003ca name=\"RL\" /\u003e🔮"],"readme":"# Tensorflow 2.0 Realtime Multi-Person Pose Estimation\n\n## What's New\n\n**Oct 5, 2020**\n* Converted models to the new [string notation representation](https://github.com/michalfaber/tf_netbuilder)\n* Added a new openpose singlenet model based on Mobilenet V3 [Single-Network Whole-Body Pose Estimation](https://arxiv.org/abs/1909.13423).\n* Added dependency to the library [tf_netbuilder](https://github.com/michalfaber/tf_netbuilder)\n* Old code is available under the tag: \"v1.0\"\n\n#\n\n\nThis repo contains a new upgraded version of the **keras_Realtime_Multi-Person_Pose_Estimation** project plus some extra scripts and new models.\n\n\nI added a visualization of final heatmaps and pafs in the Tensorboard.\nEvery 100 iterations, a single image is passed to the model. The predicted heatmaps and pafs are logged in the Tensorboard.\nYou can check this visual representation of prediction every few hours as it gives a good sense of how the training performs.\n\n# Scripts and notebooks\n\nThis project contains the following scripts and jupyter notebooks:\n\n**train_singlenet_mobilenetv3.py** - training code for the new model presented in this paper [Single-Network Whole-Body Pose Estimation](https://arxiv.org/abs/1909.13423). I replaced VGG with Mobilenet V3. Simplified model with just 3 pafs and 1 heatmap.\n\n**train_2br_vgg.py** - training code for the old CMU model (2017). This is a new version of the training code from the old repo *keras_Realtime_Multi-Person_Pose_Estimation*. It has been upgraded to Tensorflow 2.0.\n\n**convert_to_tflite.py** - conversion of trained models into *TFLite*.\n\n**demo_image.py** - pose estimation on the provided image.\n\n**demo_video.py** - pose estimation on the provided video.\n\n**inspect_dataset.ipynb** - helper notebook to get more insights into what is generated from the datasets.\n\n**test_openpose_singlenet_model.ipynb** - helper notebook to preview the predictions from the singlenet model.\n\n**test_openpose_2br_vgg_model.ipynb** - helper notebook to preview the predictions from the original vgg-based model.\n\n**test_tflite_models.ipynb** - helper notebook to verify exported *TFLite* model.\n  \n\n# Installation\n\n## Prerequisites\n\n* download [dataset and annotations](http://cocodataset.org/#download) into a separate folder datasets, outside of this project:\n```bash\n    ├── datasets\n    │   └── coco_2017_dataset\n    │       ├── annotations\n    │       │   ├── person_keypoints_train2017.json\n    │       │   └── person_keypoints_val2017.json\n    │       ├── train2017/*\n    │       └── val2017/*\n    └── tensorflow_Realtime_Multi-Person_Pose_Estimation/*\n```\n                \n* install [CUDNN](https://developer.nvidia.com/cudnn) and [CUDA](https://developer.nvidia.com/cuda-downloads)\n\n\n## Install\n\n**Virtualenv**\n\n```bash\npip install virtualenv\nvirtualenv .venv\nsource .venv/bin/activate\npip install -r requirements.txt\n```\n\n## Examples\n```bash\npython convert_to_tflite.py --weights=[path to saved weights] --tflite-path=openpose_singlenet.tflite --create-model-fn=create_openpose_singlenet\npython demo_image.py --image=resources/ski_224.jpg --output-image=out1.png --create-model-fn=create_openpose_singlenet\npython demo_image.py --image=resources/ski_368.jpg --output-image=out2.png --create-model-fn=create_openpose_2branches_vgg\npython demo_video.py --video=resources/sample1.mp4 --output-video=sample1_out1.mp4 --create-model-fn=create_openpose_2branches_vgg --input-size=368 --output-resize-factor=8 --paf-idx=10 --heatmap-idx=11\npython demo_video.py --video=resources/sample1.mp4 --output-video=sample1_out2.mp4 --create-model-fn=create_openpose_singlenet --input-size=224 --output-resize-factor=8 --paf-idx=2 --heatmap-idx=3\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichalfaber%2Ftensorflow_Realtime_Multi-Person_Pose_Estimation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmichalfaber%2Ftensorflow_Realtime_Multi-Person_Pose_Estimation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichalfaber%2Ftensorflow_Realtime_Multi-Person_Pose_Estimation/lists"}