{"id":13435694,"url":"https://github.com/tucan9389/tf2-mobile-2d-single-pose-estimation","last_synced_at":"2025-07-06T10:33:31.560Z","repository":{"id":37443484,"uuid":"177510440","full_name":"tucan9389/tf2-mobile-2d-single-pose-estimation","owner":"tucan9389","description":" :dancer: Pose estimation for iOS and android using TensorFlow 2.0","archived":false,"fork":false,"pushed_at":"2022-11-22T07:43:56.000Z","size":40697,"stargazers_count":168,"open_issues_count":39,"forks_count":42,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-05-07T19:02:57.420Z","etag":null,"topics":["ios","mobile","pose-estimation","tensorflow-lite","tensorflow2"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tucan9389.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-25T03:50:32.000Z","updated_at":"2025-03-21T03:42:55.000Z","dependencies_parsed_at":"2023-01-20T21:19:15.850Z","dependency_job_id":null,"html_url":"https://github.com/tucan9389/tf2-mobile-2d-single-pose-estimation","commit_stats":null,"previous_names":["tucan9389/tf2-mobile-pose-estimation"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tucan9389%2Ftf2-mobile-2d-single-pose-estimation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tucan9389%2Ftf2-mobile-2d-single-pose-estimation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tucan9389%2Ftf2-mobile-2d-single-pose-estimation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tucan9389%2Ftf2-mobile-2d-single-pose-estimation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tucan9389","download_url":"https://codeload.github.com/tucan9389/tf2-mobile-2d-single-pose-estimation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252940918,"owners_count":21828769,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ios","mobile","pose-estimation","tensorflow-lite","tensorflow2"],"created_at":"2024-07-31T03:00:38.142Z","updated_at":"2025-05-07T19:09:33.379Z","avatar_url":"https://github.com/tucan9389.png","language":"Python","funding_links":[],"categories":["Sample Codes / Projects \u003ca name=\"sample\" /\u003e ⛏️📐📁"],"sub_categories":["Reinforcement Learning \u003ca name=\"RL\" /\u003e🔮"],"readme":"# 💃 Mobile 2D Single Person (Or Your Own Object) Pose Estimation for TensorFlow 2.0\n\u003e ~~This repository is forked from [edvardHua/PoseEstimationForMobile](https://github.com/edvardHua/PoseEstimationForMobile) when the original repository was closed.~~ \u003cbr\u003e[edvardHua/PoseEstimationForMobile](https://github.com/edvardHua/PoseEstimationForMobile) repository is reopened! I'll maintain it separately. 👍\n\n\nThis repository currently implemented the Hourglass model using TensorFlow 2.0 with Keras API.\n\n## Table of contents\n\n- [Goals](#goals)\n- [Getting Started](#getting-started)\n- [Results](#results)\n- [Converting To Mobile Model](#converting-to-mobile-model)\n- [Tuning](#tuning)\n- [Details](#details)\n    - [Folder Structure](#folder-structure)\n    - [Main Components](#main-components)\n- [TODO](#todo)\n- [Related Projects](#related-projects)\n- [Acknowledgements](#acknowledgements)\n- [Reference](#reference)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Goals\n\n- 📚 Easy to train\n- 🏃‍ Easy to use the model on mobile device\n\n## Getting Started\n\n### Install Anaconda (~10 min)\n\n- [How To Install Anaconda on Ubuntu 18.04 [Quickstart]](https://www.digitalocean.com/community/tutorials/how-to-install-anaconda-on-ubuntu-18-04-quickstart)\n- [How to Install Anaconda on CentOS 7](https://linuxize.com/post/how-to-install-anaconda-on-centos-7/)\n\n### Create Virtual Environment (~2 min)\n\nCreate new environment.\n```shell\nconda create -n {env_name} python={python_version} anaconda\n# in my case\n# conda create -n mpe-env-tf2-alpha0 python=3.7 anaconda\n```\n\nStart the environment.\n```shell\nsource activate {env_name}\n# in my case\n# source activate mpe-env-tf2-alpha0\n```\n\n### Install the requirements (~1 min)\n```shell\ncd {tf2-mobile-pose-estimation_path}\npip install -r requirements.txt\npip install git+https://github.com/philferriere/cocoapi.git@2929bd2ef6b451054755dfd7ceb09278f935f7ad#subdirectory=PythonAPI\n```\n\n\n\u003cdetails\u003e\u003csummary\u003eDownload original COCO dataset.\u003c/summary\u003e\n\u003cp\u003e\n\n### Download original COCO dataset\n\nSpecial script that will help you to download and unpack\nneeded COCO datasets. Please fill COCO_DATASET_PATH with path\nthat is used in current version of repository.\nYou can check needed path in file train.py\n\n**Warning** Your system should have approximately 40gb of free space for datasets\n\n```shell\npython downloader.py --download-path=COCO_DATASET_PATH\n```\n\n\u003c/p\u003e\n\u003c/details\u003e\n\n## Run The Project\n\nIn order to use the project you have to:\n1. Prepare the dataset([ai_challenger dataset](https://drive.google.com/file/d/1rZng2KiEuyb-dev3HxJFYcZU4Il1VHqj/view?usp=sharing)) and unzip.\n2. Run the model using:\n```shell\npython train.py \\\n--dataset_config config/dataset/coco_single_person_only-gpu.cfg \\\n--experiment_config config/training/coco_single_experiment01-cpm-sg4-gpu.cfg\n```\n\n## Compatiable Datasets\n\nDataset Name | Doanload | Size | Number of images\u003cbr\u003etrain/valid | Number of Keypoints | Note\n--- | --- | --- | --- | --- | ---\nai challenge | [google drive](https://drive.google.com/file/d/1rZng2KiEuyb-dev3HxJFYcZU4Il1VHqj/view?usp=sharing) | 2GB | 22k/1.5k | 14 | default dataset of this repo\ncoco single person only | [google drive](https://drive.google.com/file/d/1lwt3smqdJ2-ZuVCzgImEp8gw-RHuG-YR/view?usp=sharing) | 4GB | 25k/1k | 17 | filtered by showing only one person in an image which is from coco 2017 keypoint dataset\n\n- ai challenge's keypoint names: `['top_head', 'neck', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'left_hip', 'right_hip', 'left_knee', 'right_knee', 'left_ankle', 'right_ankle']`\n- coco's keypoint names: `['nose', 'left_eye', 'right_eye', 'left_ear', 'right_ear', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'left_hip', 'right_hip', 'left_knee', 'right_knee', 'left_ankle', 'right_ankle']`\n\n## Results\n\n### AI Challenge Dataset\n\nModel Name | Backbone | Stage Or Depth | PCH@.5 | Size | Total Epoch | Total Training Time | Note\n--- | --- | --- | --- | --- | --- | --- | ---\nMobileNetV2 based CPM | cpm-b0 | Stage 1 | .. | .. | .. | .. | Default CPM\nMobileNetV2 based CPM | cpm-b0 | Stage 2 | .. | .. | .. | ..\nMobileNetV2 based CPM | cpm-b0 | Stage 3 | .. | .. | .. | ..\nMobileNetV2 based CPM | cpm-b0 | Stage 4 | .. | .. | .. | ..\nMobileNetV2 based CPM | cpm-b0 | Stage 5 | .. | .. | .. | ..\nMobileNetV2 based Hourglass | hg-b0 | Depth 4 | .. | .. | .. | .. | Default Hourglass\n\n### COCO Single persononly Dataset\n\nModel Name | Backbone | Stage Or Depth | OKS | Size | Total Epoch | Total Training Time | Note\n--- | --- | --- | --- | --- | --- | --- | ---\nMobileNetV2 based CPM | cpm-b0 | Stage 1 | .. | .. | .. | .. | Default CPM\nMobileNetV2 based CPM | cpm-b0 | Stage 2 | .. | .. | .. | ..\nMobileNetV2 based CPM | cpm-b0 | Stage 3 | .. | .. | .. | ..\nMobileNetV2 based CPM | cpm-b0 | Stage 4 | .. | .. | .. | ..\nMobileNetV2 based CPM | cpm-b0 | Stage 5 | .. | .. | .. | ..\nMobileNetV2 based Hourglass | hg-b0 | Depth 4 | .. | .. | .. | .. | Default Hourglass\n\n## Converting To Mobile Model\n\n### TensorFLow Lite\n\nIf you train the model, it will create tflite models per evaluation step.\n\n### Core ML\n\nCheck `convert_to_coreml.py` script. The converted `.mlmodel` support iOS14+.\n\n## Details\n\n\u003e This section will be separated to other `.md` file.\n\n### Folder Structure\n\n```\ntf2-mobile-pose-estimation\n├── config\n|   ├── model_config.py\n|   └── train_config.py\n├── data_loader\n|   ├── data_loader.py\n|   ├── dataset_augment.py\n|   ├── dataset_prepare.py\n|   └── pose_image_processor.py\n├── models\n|   ├── common.py\n|   ├── mobilenet.py\n|   ├── mobilenetv2.py\n|   ├── mobilenetv3.py\n|   ├── resnet.py\n|   ├── resneta.py\n|   ├── resnetd.py\n|   ├── senet.py\n|   ├── simplepose_coco.py\n|   └── simpleposemobile_coco.py\n├── train.py            - the main training script\n├── common.py \n├── requirements.txt\n└── outputs             - this folder will be generated automatically when start training\n    ├── 20200312-sp-ai_challenger\n    |   ├── saved_model\n    |   └── image_results\n    └── 20200312-sp-ai_challenger\n        └── ...\n\nMy SSD    \n└── datasets            - this folder contains the datasets of the project.\n    └── ai_challenger\n        ├── train.json\n        ├── valid.json\n        ├── train\n        └── valid\n\n```\n\n## TODO\n\n- ~~Save model to saved_model~~\n- ~~Convert the model(saved_model) to TFLite model(`.tflite`)~~\n- ~~Convert the model(saved_model) to Core ML model(`.mlmodel`)~~\n- ~~Run the model on iOS~~\n- Release 1.0 models\n- Support distributed GPUs training\n- Make DEMO gif running on mobile device\n- Run the model on Android\n\n## Reference\n\n[1] [Paper of Convolutional Pose Machines](https://arxiv.org/abs/1602.00134) \u003cbr/\u003e\n[2] [Paper of Stack Hourglass](https://arxiv.org/abs/1603.06937) \u003cbr/\u003e\n[3] [Paper of MobileNet V2](https://arxiv.org/pdf/1801.04381.pdf) \u003cbr/\u003e\n[4] [Repository PoseEstimation-CoreML](https://github.com/tucan9389/PoseEstimation-CoreML) \u003cbr/\u003e\n[5] [Repository of tf-pose-estimation](https://github.com/ildoonet/tf-pose-estimation) \u003cbr\u003e\n[6] [Devlope guide of TensorFlow Lite](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/docs_src/mobile/tflite) \u003cbr/\u003e\n[7] [Mace documentation](https://mace.readthedocs.io)\n\n### Related Projects\n- [tucan9389/PoseEstimation-CoreML](https://github.com/tucan9389/PoseEstimation-CoreML)\n- [tucan9389/PoseEstimation-TFLiteSwift](https://github.com/tucan9389/PoseEstimation-TFLiteSwift) (Preparing...)\n- [tucan9389/KeypointAnnotation](https://github.com/tucan9389/KeypointAnnotation)\n- [osmr/imgclsmob](https://github.com/osmr/imgclsmob)\n- [edvardHua/PoseEstimationForMobile](https://github.com/edvardHua/PoseEstimationForMobile)\n- [jwkanggist/tf-tiny-pose-estimation](https://github.com/jwkanggist/tf-tiny-pose-estimatio)\n- [dongseokYang/Body-Pose-Estimation-Android-gpu](https://github.com/dongseokYang/Body-Pose-Estimation-Android-gpu)\n\n### Other Pose Estimation Projects\n\n- [cbsudux/awesome-human-pose-estimation](https://github.com/cbsudux/awesome-human-pose-estimation)\n\n## Contributing\n\n\u003e This section will be separated to other `.md` file.\n\nAny contributions are welcome including improving the project.\n\n# License\n\n[Apache License 2.0](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftucan9389%2Ftf2-mobile-2d-single-pose-estimation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftucan9389%2Ftf2-mobile-2d-single-pose-estimation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftucan9389%2Ftf2-mobile-2d-single-pose-estimation/lists"}