{"id":18612967,"url":"https://github.com/haofanwang/cliff","last_synced_at":"2025-09-04T07:36:57.950Z","repository":{"id":60472991,"uuid":"526855769","full_name":"haofanwang/CLIFF","owner":"haofanwang","description":"This repo equips the official CLIFF [ECCV 2022 Oral] with better detector, better tracker. Support multi-person, motion interpolation, motion smooth and SMPLify fitting.","archived":false,"fork":false,"pushed_at":"2023-01-31T03:06:59.000Z","size":4518,"stargazers_count":165,"open_issues_count":8,"forks_count":17,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-25T06:22:57.465Z","etag":null,"topics":["3d-human-pose","cliff","human-pose-estimation","multi-person-pose-estimation","pose-estimation","pytorch","smpl"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/haofanwang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-20T07:22:02.000Z","updated_at":"2025-03-21T09:10:34.000Z","dependencies_parsed_at":"2023-02-16T15:15:47.799Z","dependency_job_id":null,"html_url":"https://github.com/haofanwang/CLIFF","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haofanwang%2FCLIFF","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haofanwang%2FCLIFF/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haofanwang%2FCLIFF/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haofanwang%2FCLIFF/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/haofanwang","download_url":"https://codeload.github.com/haofanwang/CLIFF/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248316175,"owners_count":21083394,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-human-pose","cliff","human-pose-estimation","multi-person-pose-estimation","pose-estimation","pytorch","smpl"],"created_at":"2024-11-07T03:19:18.096Z","updated_at":"2025-04-10T23:31:38.289Z","avatar_url":"https://github.com/haofanwang.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CLIFF [ECCV 2022 Oral]\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/cliff-carrying-location-information-in-full/3d-human-pose-estimation-on-3dpw)](https://paperswithcode.com/sota/3d-human-pose-estimation-on-3dpw?p=cliff-carrying-location-information-in-full)\n\n\u003cp float=\"left\"\u003e\n   \u003cimg src=\"https://github.com/huawei-noah/noah-research/blob/master/CLIFF/assets/teaser.gif\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n\n## Introduction\nThis repo is highly built on the official [CLIFF](https://github.com/huawei-noah/noah-research/tree/master/CLIFF) and contains an inference demo, and further adds accurate detector and multi-person tracker. For post-processing, motion interpolation and smooth are supported for better visualization results.\n\n\n[**CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation**](\n    https://arxiv.org/abs/2208.00571\n).\n\n## Major features\n- [x] **[08/20/22]** Support YOLOX as single-person detector, better performance on single frame.\n- [x] **[08/20/22]** Support ByteTrack as multi-person tracker, better performance on person Re-ID.\n- [x] **[08/20/22]** Support linear interpolation as motion completion method, especially for occlusion.\n- [x] **[08/20/22]** Support Smooth-Net as post-processing motion smooth for decresing motion jittering.\n- [x] **[09/29/22]** Support SMPLify fitting given GT/Pred 2D keypoints for improving the quality of estimated SMPL params.\n- [x] **[01/31/23]** Further support motion smooth for SMPL pose and translation besides of 3D joints.\n\n## Preparation\n```bash\nconda create -n cliff python=3.10\npip install -r requirements.txt\n```\n\n1. Download [the SMPL models](https://smpl.is.tue.mpg.de) for rendering the reconstructed meshes\n2. Download the pretrained checkpoints to run the demo [[Google Drive](\n    https://drive.google.com/drive/folders/1EmSZwaDULhT9m1VvH7YOpCXwBWgYrgwP?usp=sharing)]\n3. Install MMDetection and download [the pretrained checkpoints](https://github.com/open-mmlab/mmdetection/tree/master/configs/yolox)\n4. Install MMTracking and download [the pretrained checkpoints](https://github.com/open-mmlab/mmtracking/tree/master/configs/mot/bytetrack)\n\nFinally put these data following the directory structure as below:\n```\n${ROOT}\n|-- data\n    smpl_mean_params.npz\n    |-- ckpt\n        |-- hr48-PA43.0_MJE69.0_MVE81.2_3dpw.pt\n        |-- res50-PA45.7_MJE72.0_MVE85.3_3dpw.pt\n        |-- hr48-PA53.7_MJE91.4_MVE110.0_agora_val.pt\n    |-- smpl\n        |-- SMPL_FEMALE.pkl\n        |-- SMPL_MALE.pkl\n        |-- SMPL_NEUTRAL.pkl\n|-- mmdetection\n    |-- checkpoints\n        |-- yolox_x_8x8_300e_coco_20211126_140254-1ef88d67.pth\n|-- mmtracking\n    |-- checkpoints\n        |-- bytetrack_yolox_x_crowdhuman_mot17-private-half_20211218_205500-1985c9f0.pth\n```\n\n## Demo\n\nWe provide demos for single-person and multi-person video.\n### Single-person\nRun the following command to test CLIFF on a single-person video:\n```\npython demo.py --ckpt data/ckpt/hr48-PA43.0_MJE69.0_MVE81.2_3dpw.pt \\\n               --backbone hr48 \\\n               --input_path test_samples/01e222117f63f874010370037f551497ac_258.mp4 \\\n               --input_type video \\\n               --save_results \\\n               --make_video \\\n               --frame_rate 30\n```\n### Multi-person\nUse the `--multi` flag to support multi-person tracking, `--infill` flag to support motion infill, `--smooth` flag to support motion smooth. Run the following command to test CLIFF on a multi-person video with post-processing:\n```\npython demo.py --ckpt data/ckpt/hr48-PA43.0_MJE69.0_MVE81.2_3dpw.pt \\\n               --backbone hr48 \\\n               --input_path test_samples/62883594000000000102c16c.mp4 \\\n               --input_type video \\\n               --multi \\\n               --infill \\\n               --smooth \\\n               --save_results \\\n               --make_video \\\n               --frame_rate 30\n```\n\n## SMPLify Fitting\n\nAs the same as [SPIN](https://github.com/nkolot/SPIN), we apply SMPLify fitting after CLIFF, OpenPose format 2D Keypoints are required for convinence.\n\n```\npython3 demo_fit.py --img=examples/im1010.jpg \\ \n                    --openpose=examples/im1010_openpose.json\n```\n\n## Citing\n```\n@Inproceedings{li2022cliff,\n  Title     = {CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation},\n  Author    = {Li, Zhihao and Liu, Jianzhuang and Zhang, Zhensong and Xu, Songcen and Yan, Youliang},\n  Booktitle = {ECCV},\n  Year      = {2022}\n}\n```\n\n## Contact\nIf you have problems about usage, feel free to open an issue or directly contact me via: haofanwang.ai@gmail.com. But please note that I'm NOT the author of CLIFF, so for any question about the paper, contact the author.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaofanwang%2Fcliff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhaofanwang%2Fcliff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaofanwang%2Fcliff/lists"}