{"id":16509596,"url":"https://github.com/joonb14/tfliteposeestimation","last_synced_at":"2026-04-14T15:33:34.877Z","repository":{"id":111832741,"uuid":"345618688","full_name":"joonb14/TFLitePoseEstimation","owner":"joonb14","description":"TensorFlow Lite Pose Estimation Python Implementation","archived":false,"fork":false,"pushed_at":"2021-03-08T11:20:54.000Z","size":12342,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-02T06:14:43.426Z","etag":null,"topics":["computer-vision","interpreter","mobilenet","pose-estimation","python","tensorflow","tflite","tflite-model","tflite-pose-estimation","tflite-python","vision"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joonb14.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-08T10:34:23.000Z","updated_at":"2022-09-30T10:02:43.000Z","dependencies_parsed_at":"2023-06-04T08:15:26.727Z","dependency_job_id":null,"html_url":"https://github.com/joonb14/TFLitePoseEstimation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/joonb14/TFLitePoseEstimation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joonb14%2FTFLitePoseEstimation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joonb14%2FTFLitePoseEstimation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joonb14%2FTFLitePoseEstimation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joonb14%2FTFLitePoseEstimation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joonb14","download_url":"https://codeload.github.com/joonb14/TFLitePoseEstimation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joonb14%2FTFLitePoseEstimation/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31803479,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-14T11:13:53.975Z","status":"ssl_error","status_checked_at":"2026-04-14T11:13:53.299Z","response_time":153,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","interpreter","mobilenet","pose-estimation","python","tensorflow","tflite","tflite-model","tflite-pose-estimation","tflite-python","vision"],"created_at":"2024-10-11T15:51:21.980Z","updated_at":"2026-04-14T15:33:34.861Z","avatar_url":"https://github.com/joonb14.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TFLitePoseEstimation\n\nThis code snipset is heavily based on \u003cb\u003e\u003ca href=\"https://www.tensorflow.org/lite/examples/pose_estimation/overview\"\u003eTensorFlow Lite Pose Estimation\u003c/a\u003e\u003c/b\u003e\u003cbr\u003e\nThe detection model can be downloaded from above link.\u003cbr\u003e\nFor the realtime implementation on Android look into the \u003ca href=\"https://github.com/tensorflow/examples/tree/master/lite/examples/posenet/android\"\u003eAndroid Pose Estimation Example\u003c/a\u003e\u003cbr\u003e\nFollow the \u003ca href=\"https://github.com/joonb14/TFLitePoseEstimation/blob/main/pose%20estimation.ipynb\"\u003epose estimation.ipynb\u003c/a\u003e to get information about how to use the TFLite model in your Python environment.\u003cbr\u003e\n\n### Details\nThe \u003cb\u003eposenet_mobilenet_v1_100_257x257_multi_kpt_stripped.tflite\u003c/b\u003e file's input takes normalized 257x257x3 shape image. And the output is composed of 4 different outputs. The 1st output contains the heatmaps, 2nd output contains the offsets, 3rd output contains the forward_displacements, 4th output contains the backward_displacements.\u003cbr\u003e\n\nFor model inference, we need to load, resize, typecast the image.\u003cbr\u003e\nIn my case for convenience used pillow library to load and just applied /255 for all values then cast the numpy array to float32.\u003cbr\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/30307587/110313718-2f093580-804a-11eb-8961-0d67383be16e.png\" width=400px/\u003e\u003cbr\u003e\nThen if you follow the correct instruction provided by Google in \u003ca href=\"https://www.tensorflow.org/lite/guide/inference#load_and_run_a_model_in_python\"\u003eload_and_run_a_model_in_python\u003c/a\u003e, you would get output in below shape\u003cbr\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/30307587/110313834-5cee7a00-804a-11eb-8182-943423d0c6c2.png\" width=600px/\u003e\u003cbr\u003e\nNow we need to process this output to use it for pose estimation\u003cbr\u003e\n\n##### Extract Key points\n```python\nimport math\n\ndef sigmoid(x):\n    return 1 / (1 + math.exp(-x))\n    \nheight = heatmaps[0].shape[0]\nwidth = heatmaps[0][0].shape[0]\nnumKeypoints = heatmaps[0][0][0].shape[0]\n\nkeypointPositions = []\n\nfor keypoint in range(numKeypoints):\n    maxVal = heatmaps[0][0][0][keypoint]\n    maxRow = 0\n    maxCol = 0\n    for row in range(height):\n        for col in range(width):\n            if (heatmaps[0][row][col][keypoint] \u003e maxVal):\n                maxVal = heatmaps[0][row][col][keypoint]\n                maxRow = row\n                maxCol = col\n    keypointPositions.append([maxRow,maxCol])\n\n\nconfidenceScores=[]\nyCoords = []\nxCoords = []\nfor idx, position in enumerate(keypointPositions):\n    positionY = keypointPositions[idx][0]\n    positionX = keypointPositions[idx][1]\n    yCoords.append(position[0] / (height - 1) * 257 + offsets[0][positionY][positionX][idx])\n    xCoords.append(position[1] / (width - 1) * 257 + offsets[0][positionY][positionX][idx + numKeypoints])\n    confidenceScores.append(sigmoid(heatmaps[0][positionY][positionX][idx]))\n#     yCoords.append()\nscore = np.average(confidenceScores)\nscore\n```\n\n##### Visualize Key points and Body joints\n```python\nfrom enum import Enum\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as patches\nfrom PIL import Image\n\n\nclass BodyPart(Enum):\n    NOSE = 0\n    LEFT_EYE = 1\n    RIGHT_EYE = 2\n    LEFT_EAR = 3\n    RIGHT_EAR= 4\n    LEFT_SHOULDER = 5\n    RIGHT_SHOULDER = 6\n    LEFT_ELBOW = 7\n    RIGHT_ELBOW = 8\n    LEFT_WRIST = 9\n    RIGHT_WRIST = 10\n    LEFT_HIP = 11\n    RIGHT_HIP = 12\n    LEFT_KNEE = 13\n    RIGHT_KNEE = 14\n    LEFT_ANKLE = 15\n    RIGHT_ANKLE = 16\n  \nbodyJoints = np.array(\n    [(BodyPart.LEFT_WRIST, BodyPart.LEFT_ELBOW),\n    (BodyPart.LEFT_ELBOW, BodyPart.LEFT_SHOULDER),\n    (BodyPart.LEFT_SHOULDER, BodyPart.RIGHT_SHOULDER),\n    (BodyPart.RIGHT_SHOULDER, BodyPart.RIGHT_ELBOW),\n    (BodyPart.RIGHT_ELBOW, BodyPart.RIGHT_WRIST),\n    (BodyPart.LEFT_SHOULDER, BodyPart.LEFT_HIP),\n    (BodyPart.LEFT_HIP, BodyPart.RIGHT_HIP),\n    (BodyPart.RIGHT_HIP, BodyPart.RIGHT_SHOULDER),\n    (BodyPart.LEFT_HIP, BodyPart.LEFT_KNEE),\n    (BodyPart.LEFT_KNEE, BodyPart.LEFT_ANKLE),\n    (BodyPart.RIGHT_HIP, BodyPart.RIGHT_KNEE),\n    (BodyPart.RIGHT_KNEE, BodyPart.RIGHT_ANKLE)]\n)\n\nminConfidence = 0.5\n\nfig, ax = plt.subplots(figsize=(10,10))\n\nif (score \u003e minConfidence):\n    ax.imshow(res_im)\n    for line in bodyJoints:\n        plt.plot([xCoords[line[0].value],xCoords[line[1].value]],[yCoords[line[0].value],yCoords[line[1].value]],'k-')\n    ax.scatter(xCoords, yCoords, s=30,color='r')\n    plt.show()\n```\n\n\u003cimg src=\"https://user-images.githubusercontent.com/30307587/110314123-cd959680-804a-11eb-84f2-198a45a50618.png\" width=600px/\u003e\u003cbr\u003e\nI believe you can modify the rest of the code as you want by yourself.\u003cbr\u003e\nThank you!\u003cbr\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoonb14%2Ftfliteposeestimation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoonb14%2Ftfliteposeestimation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoonb14%2Ftfliteposeestimation/lists"}