{"id":20543759,"url":"https://github.com/isarandi/metrabs","last_synced_at":"2025-04-05T04:15:02.168Z","repository":{"id":37675493,"uuid":"289547668","full_name":"isarandi/metrabs","owner":"isarandi","description":"Estimate absolute 3D human poses from RGB images.","archived":false,"fork":false,"pushed_at":"2024-04-22T21:51:48.000Z","size":6023,"stargazers_count":499,"open_issues_count":14,"forks_count":72,"subscribers_count":22,"default_branch":"master","last_synced_at":"2025-03-29T03:11:42.197Z","etag":null,"topics":["3d-human-pose","computer-vision","deep-learning","human-pose-estimation","machine-learning","motion-capture","tensorflow","tensorflow2"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2007.07227","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/isarandi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-22T18:48:44.000Z","updated_at":"2025-03-27T07:19:13.000Z","dependencies_parsed_at":"2025-02-16T15:11:16.872Z","dependency_job_id":"963f5a89-6a35-439c-9a6a-fbfab2c552ec","html_url":"https://github.com/isarandi/metrabs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isarandi%2Fmetrabs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isarandi%2Fmetrabs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isarandi%2Fmetrabs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isarandi%2Fmetrabs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/isarandi","download_url":"https://codeload.github.com/isarandi/metrabs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247284954,"owners_count":20913704,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-human-pose","computer-vision","deep-learning","human-pose-estimation","machine-learning","motion-capture","tensorflow","tensorflow2"],"created_at":"2024-11-16T01:40:10.457Z","updated_at":"2025-04-05T04:15:02.145Z","avatar_url":"https://github.com/isarandi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MeTRAbs Absolute 3D Human Pose Estimator\n\n\u003ca href=\"https://colab.research.google.com/github/isarandi/metrabs/blob/master/metrabs_demo.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\u003cbr\u003e\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/metrabs-metric-scale-truncation-robust/3d-human-pose-estimation-on-3d-poses-in-the)](https://paperswithcode.com/sota/3d-human-pose-estimation-on-3d-poses-in-the?p=metrabs-metric-scale-truncation-robust)\n\n\u003cp align=\"center\"\u003e\u003cimg src=img/demo.gif width=\"60%\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003e\u003ca href=\"https://youtu.be/4VFKiiW9RCQ\"\u003e\u003cimg src=img/thumbnail_video_qual.png width=\"30%\"\u003e\u003c/a\u003e\n\u003ca href=\"https://youtu.be/BemM8-Lx47g\"\u003e\u003cimg src=img/thumbnail_video_conf.png width=\"30%\"\u003e\u003c/a\u003e\u003c/p\u003e\n\nThis repository contains code for the following paper:\n\n**[MeTRAbs: Metric-Scale Truncation-Robust Heatmaps for Absolute 3D Human Pose Estimation](https://arxiv.org/abs/2007.07227)** \u003cbr\u003e\n*by István Sárándi, Timm Linder, Kai O. Arras, Bastian Leibe*\u003cbr\u003e\nIEEE Transactions on Biometrics, Behavior, and Identity Science (T-BIOM), Selected Best Works From\nAutomated Face and Gesture Recognition 2020.\n\nThe repo has been updated to an improved version employed in the following paper: \n\n**[Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats ](https://arxiv.org/abs/2212.14474)** \u003cbr\u003e\n*by István Sárándi, Alexander Hermans, Bastian Leibe*\u003cbr\u003e\nIEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023.\n\n\n## News\n\n* [2023-08-02] Major codebase refactoring, models as described in our [WACV'23 paper](https://istvansarandi.com/dozens), several components factored out into separate repos, PyTorch support for inference, and more.\n* [2021-12-03] Added new backbones, including the ResNet family from ResNet-18 to ResNet-152\n* [2021-10-19] Released new best-performing [models](docs/MODELS.md) based on EfficientNetV2 and super fast\n  ones using MobileNetV3, simplified [API](docs/API.md), multiple skeleton conventions, support for\n  radial/tangential distortion, improved antialiasing, plausibility filtering and other new\n  features.\n* [2021-10-19] Full codebase migrated to TensorFlow 2 and Keras\n* [2020-11-19] Oral presentation at the IEEE Conference on Automatic Face and Gesture Recognition\n  (FG'20) ([Talk Video](https://youtu.be/BemM8-Lx47g)\n  and [Slides](https://vision.rwth-aachen.de/media/papers/203/slides_metrabs.pdf))\n* [2020-11-16] Training and evaluation code now released along with dataset pre-processing scripts!\n  Code and models upgraded to Tensorflow 2.\n* [2020-10-06] [Journal paper](https://arxiv.org/abs/2007.07227) accepted for publication in the\n  IEEE Transactions on Biometrics, Behavior, and Identity Science (T-BIOM), Best of FG Special Issue\n* [2020-08-23] Short presentation at ECCV2020's 3DPW\n  workshop ([slides](https://vision.rwth-aachen.de/media/papers/203/metrabs_3dpw_slides.pdf))\n* [2020-08-06] Our method has won\n  the **[3DPW Challenge](https://virtualhumans.mpi-inf.mpg.de/3DPW_Challenge/)**\n\n## Inference Code\n\nWe release **standalone TensorFlow models** (SavedModel) to allow easy application in downstream\nresearch. After loading the model, you can run inference in a single line of Python **without having\nthis codebase as a dependency**. Try it in action in\n[Google Colab](\nhttps://colab.research.google.com/github/isarandi/metrabs/blob/master/metrabs_demo.ipynb).\n\n### Gist of Usage\n\n```python\nimport tensorflow as tf\nimport tensorflow_hub as tfhub\n\nmodel = tfhub.load('https://bit.ly/metrabs_l')\nimage = tf.image.decode_jpeg(tf.io.read_file('img/test_image_3dpw.jpg'))\npred = model.detect_poses(image)\npred['boxes'], pred['poses2d'], pred['poses3d']\n```\n\nSee also the [demos](demos/) folder for more examples.\n\nNOTE: The models can only be used for **non-commercial** purposes due to the licensing of the used\ntraining datasets.\n\nAlternatively, you can try the experimental PyTorch version:\n\n```bash\nwget -O - https://bit.ly/metrabs_l_pt | tar -xzvf -\npython -m metrabs_pytorch.scripts.demo_image --model-dir metrabs_eff2l_384px_800k_28ds_pytorch --image img/test_image_3dpw.jpg\n```\n\n### Demos\n\n* [```./demo.py```](demos/demo.py) to auto-download the model, predict on a sample image and display the\n  result with Matplotlib or [PoseViz](https://github.com/isarandi/poseviz) (if installed).\n* [```./demo_video.py```](demos/demo_video.py)``` filepath-or-url-to-video.mp4``` to run inference on a video.\n\n### Documentation\n\n- **[How-to Guide with Examples](docs/INFERENCE_GUIDE.md)**\n- **[Full API Reference](docs/API.md)**\n\n### Feature Summary\n\n- **Several skeleton conventions** supported through the keyword argument ```skeleton``` (e.g. COCO,\n  SMPL, H36M)\n- **Multi-image (batched) and single-image** predictions both supported\n- **Advanced, parallelized cropping** logic behind the scenes\n    - Anti-aliasing through image pyramid and\n      supersampling, [gamma-correct rescaling](http://www.ericbrasseur.org/gamma.html).\n    - GPU-accelerated undistortion of pinhole perspective (homography) and radial/tangential lens\n      distortions\n- Estimates returned  in **3D world space** (when calibration is provided) and **2D pixel space**\n- Built-in, configurable **test-time augmentation** (TTA) with rotation, flip and brightness (keyword\n  argument ```num_aug``` sets the number of TTA crops per detection)\n- Automatic **suppression of implausible poses** and non-max suppression on the 3D pose level (can be turned off)\n- **Multiple backbones** with different speed-accuracy trade-off (EfficientNetV2, MobileNetV3)\n\n## Training and Evaluation\n\nSee the docs directory.\n\n## BibTeX\n\nIf you find this work useful in your research, please cite it as:\n\n```bibtex\n@article{sarandi2021metrabs,\n  title={{MeTRAbs:} Metric-Scale Truncation-Robust Heatmaps for Absolute 3{D} Human Pose Estimation},\n  author={S\\'ar\\'andi, Istv\\'an and Linder, Timm and Arras, Kai O. and Leibe, Bastian},\n  journal={IEEE Transactions on Biometrics, Behavior, and Identity Science},\n  year={2021},\n  volume={3},\n  number={1},\n  pages={16-30},\n  doi={10.1109/TBIOM.2020.3037257}\n}\n```\n\nThe above paper is an extended journal version of the FG'2020 conference paper:\n\n```bibtex\n@inproceedings{Sarandi20FG,\n  title={Metric-Scale Truncation-Robust Heatmaps for 3{D} Human Pose Estimation},\n  author={S\\'ar\\'andi, Istv\\'an and Linder, Timm and Arras, Kai O. and Leibe, Bastian},\n  booktitle={IEEE International Conference on Automatic Face and Gesture Recognition},\n  pages={677-684},\n  year={2020}\n}\n```\n\nThe newer large-scale models correspond to the WACV'23 paper:\n\n```bibtex\n@inproceedings{Sarandi2023dozens,\n    author = {S\\'ar\\'andi, Istv\\'an and Hermans, Alexander and Leibe, Bastian},\n    title = {Learning {3D} Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats},\n    booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},\n    year = {2023}\n} \n```\n\n## Contact\n\nCode in this repository was written by [István Sárándi](https://isarandi.github.io) (RWTH Aachen\nUniversity) unless indicated otherwise.\n\nGot any questions or feedback? Drop a mail to sarandi@vision.rwth-aachen.de!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fisarandi%2Fmetrabs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fisarandi%2Fmetrabs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fisarandi%2Fmetrabs/lists"}