{"id":13790429,"url":"https://github.com/jeffffffli/HybrIK","last_synced_at":"2025-05-12T09:32:48.161Z","repository":{"id":37388355,"uuid":"317186983","full_name":"jeffffffli/HybrIK","owner":"jeffffffli","description":"Official code of \"HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation\", CVPR 2021","archived":false,"fork":false,"pushed_at":"2025-01-08T10:01:45.000Z","size":97469,"stargazers_count":1370,"open_issues_count":144,"forks_count":159,"subscribers_count":23,"default_branch":"main","last_synced_at":"2025-04-14T03:59:02.392Z","etag":null,"topics":["3d-pose-estimation","cvpr","cvpr21","inverse-kinematics","pose-estimation","pytorch","smpl"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jeffffffli.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-30T10:24:30.000Z","updated_at":"2025-04-13T14:21:41.000Z","dependencies_parsed_at":"2024-12-29T10:06:07.596Z","dependency_job_id":"1fb2a285-89c6-4656-b4c6-1d57de9c3d6e","html_url":"https://github.com/jeffffffli/HybrIK","commit_stats":null,"previous_names":["jeffffffli/hybrik","jeff-sjtu/hybrik"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffffffli%2FHybrIK","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffffffli%2FHybrIK/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffffffli%2FHybrIK/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffffffli%2FHybrIK/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jeffffffli","download_url":"https://codeload.github.com/jeffffffli/HybrIK/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253709354,"owners_count":21951123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-pose-estimation","cvpr","cvpr21","inverse-kinematics","pose-estimation","pytorch","smpl"],"created_at":"2024-08-03T22:00:43.636Z","updated_at":"2025-05-12T09:32:43.134Z","avatar_url":"https://github.com/jeffffffli.png","language":"Python","funding_links":[],"categories":["Single-Person 3D Mesh Recovery","🧭 Pose Estimation, Extraction, and Processing"],"sub_categories":["2021"],"readme":"# HybrIK: Hybrid Analytical-Neural Inverse Kinematics for Body Mesh Recovery\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/hybrik_dance1.gif\" height=\"160\"\u003e \u003cimg src=\"assets/hybrik_dance2.gif\" height=\"160\"\u003e \u003cimg src=\"assets/hybrik_dance3.gif\" height=\"160\"\u003e\n\u003c/br\u003e\n\u003cimg src=\"assets/hybrikx_dance1.gif\" height=\"160\"\u003e \u003cimg src=\"assets/hybrikx_basketball.gif\" height=\"160\"\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003ca href=\"https://pytorch.org/get-started/locally/\"\u003e\u003cimg alt=\"PyTorch\" src=\"https://img.shields.io/badge/PyTorch-ee4c2c?logo=pytorch\u0026logoColor=white\"\u003e\u003c/a\u003e\n\u003ca href='https://colab.research.google.com/drive/1n41l7I2NxWseuruVQEU8he2XqzSXhu2f?usp=sharing' style='padding-left: 0.5rem;'\u003e\u003cimg src='https://colab.research.google.com/assets/colab-badge.svg' alt='Google Colab'\u003e\u003c/a\u003e\n\u003ca href='https://paperswithcode.com/sota/3d-human-pose-estimation-on-agora?p=hybrik-x-hybrid-analytical-neural-inverse' style='padding-left: 0.5rem;'\u003e\u003cimg src='https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/hybrik-x-hybrid-analytical-neural-inverse/3d-human-pose-estimation-on-agora'\u003e\n\u003c/a\u003e\n\u003c/br\u003e\n\u003ca href=\"https://arxiv.org/abs/2011.14672\"\u003e\n    \u003cimg src='https://img.shields.io/badge/Paper-HybrIK-green?style=for-the-badge\u0026logo=adobeacrobatreader\u0026logoWidth=20\u0026logoColor=white\u0026labelColor=66cc00\u0026color=94DD15' alt='Paper PDF'\u003e\n\u003c/a\u003e\n\u003ca href=\"https://arxiv.org/abs/2304.05690\"\u003e\n    \u003cimg src='https://img.shields.io/badge/Paper-HybrIK--X-green?style=for-the-badge\u0026logo=adobeacrobatreader\u0026logoWidth=20\u0026logoColor=white\u0026labelColor=3CB371\u0026color=40E0D0' alt='Paper PDF'\u003e\n\u003c/a\u003e\n\u003ca href=\"https://youtu.be/xbdNFw2eZ4o\"\u003e\n    \u003cimg alt=\"youtube views\" src=\"https://img.shields.io/badge/Demo-HybrIK-red?logo=youtube\u0026logoColor=white\u0026labelColor=D35400\u0026style=for-the-badge\"/\u003e\n\u003c/a\u003e\n\u003ca href=\"https://youtu.be/VzCXKdQSZ7c\"\u003e\n    \u003cimg alt=\"youtube views\" src=\"https://img.shields.io/badge/Demo-HybrIK--X-red?logo=youtube\u0026logoColor=white\u0026labelColor=D35400\u0026style=for-the-badge\"/\u003e\n\u003c/a\u003e\n\n\u003c/br\u003e\n\u003c/div\u003e\n\n\nThis repo contains the code of our papers:\n\n**HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation**, In CVPR 2021\n\n**HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery**, ArXiv 2023\n\n\n\n\u003c!-- ![hybrik](assets/hybrik.png) --\u003e\n\n\u003c!-- \n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"assets/decompose.gif\", width=\"600\" alt\u003e\u003cbr\u003e\n    Twist-and-Swing Decomposition\n\u003c/div\u003e --\u003e\n\n## News :triangular_flag_on_post:\n[2023/06/02] Demo code for whole-body [HybrIK-X](https://arxiv.org/abs/2304.05690) is released.\n\n[2022/12/03] HybrIK for Blender [add-on](https://github.com/Jeff-sjtu/HybrIK/releases/download/add-on/hybrik_blender_addon.zip) is now available for download. The output of HybrIK can be imported to Blender and saved as `fbx`.\n\n[2022/08/16] [Pretrained model](https://drive.google.com/file/d/1C-jRnay38mJG-0O4_um82o1t7unC1zeT/view?usp=sharing) with HRNet-W48 backbone is available.\n\n[2022/07/31] Training code with predicted camera is released.\n\n[2022/07/25] [HybrIK](https://github.com/Jeff-sjtu/HybrIK) is now supported in [Alphapose](https://github.com/MVIG-SJTU/AlphaPose)! Multi-person demo with pose-tracking is available.\n\n[2022/04/27] \u003ca href='https://colab.research.google.com/drive/1n41l7I2NxWseuruVQEU8he2XqzSXhu2f?usp=sharing' style='padding-left: 0.5rem;'\u003e\u003cimg src='https://colab.research.google.com/assets/colab-badge.svg' alt='Google Colab'\u003e\u003c/a\u003e is ready to use.\n\n[2022/04/26] Achieve SOTA results by adding the 3DPW dataset for training.\n\n[2022/04/25] The demo code is released!\n\n\n## Key idea: Inverse Kinematics\n\nHybrIK and HybrIK-X are based on a hybrid inverse kinematics (IK) to convert accurate 3D keypoints to parametric body meshes.\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"assets/ik.gif\", width=\"600\" alt\u003e\u003cbr\u003e\n    Twist-and-Swing Decomposition\n\u003c/div\u003e\n\n\n## Installation instructions\n\n``` bash\n# 1. Create a conda virtual environment.\nconda create -n hybrik python=3.8 -y\nconda activate hybrik\n\n# 2. Install PyTorch\nconda install pytorch==1.9.1 torchvision==0.10.1 -c pytorch\n\n# 3. Install PyTorch3D (Optional, only for visualization)\nconda install -c fvcore -c iopath -c conda-forge fvcore iopath\nconda install -c bottler nvidiacub\npip install git+ssh://git@github.com/facebookresearch/pytorch3d.git@stable\n\n# 4. Pull our code\ngit clone https://github.com/Jeff-sjtu/HybrIK.git\ncd HybrIK\n\n# 5. Install\npip install pycocotools\npython setup.py develop  # or \"pip install -e .\"\n```\n\nDownload necessary model files from [[Google Drive](https://drive.google.com/file/d/1un9yAGlGjDooPwlnwFpJrbGHRiLaBNzV/view?usp=sharing) | [Baidu](https://pan.baidu.com/s/1hVrUOt2QX_UTs4QuAgN2Lg?pwd=2u3c) (code: `2u3c`) ] and un-zip them in the `${ROOT}` directory.\n\n\n## MODEL ZOO\n\n### HybrIK (SMPL)\n\n| Backbone | Training Data |     PA-MPJPE (3DPW)     | MPJPE (3DPW) | PA-MPJPE (Human3.6M) | MPJPE (Human3.6M) |  Download | Config |  \n|----------|----------|------------|------------|-------|-----------|--------|--------------|\n| ResNet-34          | w/ 3DPW | 44.5 | 72.4 | 33.8 | 55.5 | [model](https://drive.google.com/file/d/19ktHbERz0Un5EzJYZBdzdzTrFyd9gLCx/view?usp=share_link) | [cfg](./configs/256x192_adam_lr1e-3-res34_smpl_3d_cam_2x_mix_w_pw3d.yaml)    |\n| HRNet-W48           | w/o 3DPW | 48.6 | 88.0 | 29.5 | 50.4 | [model](https://drive.google.com/file/d/1o3z99bebm2XImElc3XEUzTNVhQboGJE9/view?usp=share_link) | [cfg](./configs/256x192_adam_lr1e-3-hrw48_cam_2x_wo_pw3d.yaml)    |\n| HRNet-W48          | w/ 3DPW | 41.8 | 71.3 | 29.8 | 47.1 | [model](https://drive.google.com/file/d/1gp3549vIEKfbc8SDQ-YF3Idi1aoR3DkW/view?usp=share_link) | [cfg](configs/256x192_adam_lr1e-3-hrw48_cam_2x_w_pw3d_3dhp.yaml)    |\n\n### HybrIK-X (SMPL-X)\n| Backbone |     MVE (AGORA Test)     | MPJPE (AGORA Test) |  Download | Config |  \n|----------|------------|------------|-------|--------------|\n| HRNet-W48           | 134.1 | 127.5 | [model](https://drive.google.com/file/d/1bKIPD60z_Im4S3W2-rew6YtOtUGff6-v/view?usp=sharing) | [cfg](configs/smplx/256x192_hrnet_smplx_kid.yaml)    |\n| HRNet-W48 + [RLE](https://github.com/Jeff-sjtu/res-loglikelihood-regression/tree/203dc3195ee5a11ed6f47c066ffdb83247511359)          | 112.1 | 107.6 | [model](https://drive.google.com/file/d/1R0WbySXs_vceygKg_oWeLMNAZCEoCadG/view?usp=sharing) | [cfg](configs/smplx/256x192_hrnet_rle_smplx_kid.yaml)    |\n\n\n## Demo\nFirst make sure you download the pretrained model (with predicted camera) and place it in the `${ROOT}/pretrained_models` directory, i.e., `./pretrained_models/hybrik_hrnet.pth` and `./pretrained_models/hybrikx_rle_hrnet.pth`.\n\n### SMPL\n\n* Visualize HybrIK on **videos** (run in single frame) and save results:\n\n``` bash\npython scripts/demo_video.py --video-name examples/dance.mp4 --out-dir res_dance --save-pk --save-img\n```\nThe saved results in `./res_dance/res.pk` can be imported to Blender with our [add-on](https://github.com/Jeff-sjtu/HybrIK/releases/download/add-on/hybrik_blender_addon.zip).\n\n* Visualize HybrIK on **images**:\n\n``` bash\npython scripts/demo_image.py --img-dir examples --out-dir res\n```\n\n### SMPL-X\n\n``` bash\npython scripts/demo_video_x.py --video-name examples/dance.mp4 --out-dir res_dance --save-pk --save-img\n```\n\n## Fetch data\nDownload *Human3.6M*, *MPI-INF-3DHP*, *3DPW* and *MSCOCO* datasets. You need to follow directory structure of the `data` as below. Thanks to the great job done by Moon *et al.*, we use the Human3.6M images provided in [PoseNet](https://github.com/mks0601/3DMPPE_POSENET_RELEASE).\n```\n|-- data\n`-- |-- h36m\n    `-- |-- annotations\n        `-- images\n`-- |-- pw3d\n    `-- |-- json\n        `-- imageFiles\n`-- |-- 3dhp\n    `-- |-- annotation_mpi_inf_3dhp_train.json\n        |-- annotation_mpi_inf_3dhp_test.json\n        |-- mpi_inf_3dhp_train_set\n        `-- mpi_inf_3dhp_test_set\n`-- |-- coco\n    `-- |-- annotations\n        |   |-- person_keypoints_train2017.json\n        |   `-- person_keypoints_val2017.json\n        |-- train2017\n        `-- val2017\n```\n* Download Human3.6M parsed annotations. [ [Google](https://drive.google.com/drive/folders/1tLA_XeZ_32Qk86lR06WJhJJXDYrlBJ9r?usp=sharing) | [Baidu](https://pan.baidu.com/s/1bqfVOlQWX0Rfc0Yl1a5VRA) ]\n* Download 3DPW parsed annotations. [ [Google](https://drive.google.com/drive/folders/1f7DyxyvlC9z6SFT37eS6TTQiUOXVR9rK?usp=sharing) | [Baidu](https://pan.baidu.com/s/1d42QyQmMONJgCJvHIU2nsA) ]\n* Download MPI-INF-3DHP parsed annotations. [ [Google](https://drive.google.com/drive/folders/1Ms3s7nZ5Nrux3spLxmMMAQWc5aAIecmv?usp=sharing) | [Baidu](https://pan.baidu.com/s/1aVBDudbDRT1w_ZxQc9zicA) ]\n\n\n## Train from scratch\n\n``` bash\n./scripts/train_smpl_cam.sh test_3dpw configs/256x192_adam_lr1e-3-res34_smpl_3d_cam_2x_mix_w_pw3d.yaml\n```\n\n## Evaluation\nDownload the pretrained model ([ResNet-34](https://drive.google.com/file/d/16Y_MGUynFeEzV8GVtKTE5AtkHSi3xsF9/view?usp=sharing) or [HRNet-W48](https://drive.google.com/file/d/1C-jRnay38mJG-0O4_um82o1t7unC1zeT/view?usp=sharing)).\n``` bash\n./scripts/validate_smpl_cam.sh ./configs/256x192_adam_lr1e-3-hrw48_cam_2x_w_pw3d_3dhp.yaml ./pretrained_hrnet.pth\n```\n\n\n## Citing\nIf our code helps your research, please consider citing the following paper:\n\n    @inproceedings{li2021hybrik,\n        title={Hybrik: A hybrid analytical-neural inverse kinematics solution for 3d human pose and shape estimation},\n        author={Li, Jiefeng and Xu, Chao and Chen, Zhicun and Bian, Siyuan and Yang, Lixin and Lu, Cewu},\n        booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n        pages={3383--3393},\n        year={2021}\n    }\n\n    @article{li2023hybrik,\n        title={HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery},\n        author={Li, Jiefeng and Bian, Siyuan and Xu, Chao and Chen, Zhicun and Yang, Lixin and Lu, Cewu},\n        journal={arXiv preprint arXiv:2304.05690},\n        year={2023}\n    }","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeffffffli%2FHybrIK","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjeffffffli%2FHybrIK","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeffffffli%2FHybrIK/lists"}