{"id":16108362,"url":"https://github.com/zh-plus/video-to-pose3d","last_synced_at":"2025-04-04T19:09:54.258Z","repository":{"id":37789547,"uuid":"195204743","full_name":"zh-plus/video-to-pose3D","owner":"zh-plus","description":"Convert video to 3D pose in one-key.","archived":false,"fork":false,"pushed_at":"2023-06-07T02:39:09.000Z","size":11758,"stargazers_count":642,"open_issues_count":21,"forks_count":130,"subscribers_count":19,"default_branch":"master","last_synced_at":"2024-10-10T19:26:19.518Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zh-plus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-07-04T08:50:53.000Z","updated_at":"2024-09-26T12:29:01.000Z","dependencies_parsed_at":"2024-01-14T07:02:54.550Z","dependency_job_id":"1808dd81-0bbd-448f-b92a-5b654b21386c","html_url":"https://github.com/zh-plus/video-to-pose3D","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zh-plus%2Fvideo-to-pose3D","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zh-plus%2Fvideo-to-pose3D/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zh-plus%2Fvideo-to-pose3D/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zh-plus%2Fvideo-to-pose3D/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zh-plus","download_url":"https://codeload.github.com/zh-plus/video-to-pose3D/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247234921,"owners_count":20905854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-09T19:26:33.276Z","updated_at":"2025-04-04T19:09:54.234Z","avatar_url":"https://github.com/zh-plus.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Video to Pose3D\n\n\u003e Predict 3d human pose from video\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"asset/kunkun_alphapose.gif\" width=\"100%\" alt=\"\" /\u003e\u003c/p\u003e\n\n## Prerequisite\n\n1. Environment\n   - Linux system\n   - Python \u003e 3.6 distribution\n2. Dependencies\n   - **Packages**\n      - Pytorch \u003e 1.0.0\n      - [torchsample](https://github.com/MVIG-SJTU/AlphaPose/issues/71#issuecomment-398616495)\n      - [ffmpeg](https://ffmpeg.org/download.html)\n      - tqdm\n      - pillow\n      - scipy\n      - pandas\n      - h5py\n      - visdom\n      - nibabel\n      - opencv-python (install with pip)\n      - matplotlib\n   - **2D Joint detectors**\n     - Alphapose (Recommended)\n       - Download **duc_se.pth** from ([Google Drive](https://drive.google.com/open?id=1OPORTWB2cwd5YTVBX-NE8fsauZJWsrtW) | [Baidu pan](https://pan.baidu.com/s/15jbRNKuslzm5wRSgUVytrA)),\n         place to `./joints_detectors/Alphapose/models/sppe`\n       - Download **yolov3-spp.weights** from ([Google Drive](https://drive.google.com/open?id=1D47msNOOiJKvPOXlnpyzdKA3k6E97NTC) | [Baidu pan](https://pan.baidu.com/s/1Zb2REEIk8tcahDa8KacPNA)),\n         place to `./joints_detectors/Alphapose/models/yolo`\n     - HR-Net (Bad 3d joints performance in my testing environment)\n       - Download **pose_hrnet*** from [Google Drive](https://drive.google.com/drive/folders/1nzM_OBV9LbAEA7HClC0chEyf_7ECDXYA) | [Baidu pan](https://pan.baidu.com/s/1v6dov-TyPVOtejTNF1HXrA)), \n         place to `./joints_detectors/hrnet/models/pytorch/pose_coco/`\n       - Download **yolov3.weights** from [here](https://pjreddie.com/media/files/yolov3.weights),\n         place to `./joints_detectors/hrnet/lib/detector/yolo`\n     - OpenPose (Not tested, PR to README.md is highly appreciated )\n     - Mediapipe\n       - Install mediapipe from pypi: `pip install mediapipe`\n   - **3D Joint detectors**\n      - Download **pretrained_h36m_detectron_coco.bin** from [here](https://dl.fbaipublicfiles.com/video-pose-3d/pretrained_h36m_detectron_coco.bin),\n        place it into `./checkpoint` folder\n   - **2D Pose trackers (Optional)**\n      - PoseFlow (Recommended)\n        No extra dependences\n      - LightTrack (Bad 2d tracking performance in my testing environment)\n        - See [original README](https://github.com/Guanghan/lighttrack), and perform same *get started step* on `./pose_trackers/lighttrack`\n\n\n\n## Usage\n\n0. place your video into `./outputs` folder. (I've prepared a test video).\n\n##### Single person video\n\n1. change the `video_path` in the `./videopose.py`\n2. Run it! You will find the rendered output video in the `./outputs` folder.\n\n##### Multiple person video (Not implemented yet)\n\n1. For developing, check `./videopose_multi_person`\n\n   ```python\n   video = 'kobe.mp4'\n   \n   handle_video(f'outputs/{video}') \n   # Run AlphaPose, save the result into ./outputs/alpha_pose_kobe\n   \n   track(video)\t\t\t\t\t \n   # Taking the result from above as the input of PoseTrack, output poseflow-results.json # into the same directory of above. \n   # The visualization result is save in ./outputs/alpha_pose_kobe/poseflow-vis\n   \n   # TODO: Need more action:\n   #  1. \"Improve the accuracy of tracking algorithm\" or \"Doing specific post processing \n   #     after getting the track result\".\n   #  2. Choosing person(remove the other 2d points for each frame)\n   ```\n\n\n\n\n##### Tips\n0. The [PyCharm](https://www.jetbrains.com/pycharm/) is recommended since it is the IDE I'm using during development.\n1. If you're using PyCharm, mark `./joints_detectors/Alphapose`, `./joints_detectors/hrnet` and `./pose_trackers` as source root.\n2. If your're trying to run in command line, add these paths mentioned above to the sys.path at the head of `./videopose.py`\n\n## Advanced\n\nAs this script is based on the [VedioPose3D](https://github.com/facebookresearch/VideoPose3D) provided by Facebook, and automated in the following way:\n\n```python\nargs = parse_args()\n\nargs.detector_2d = 'alpha_pose'\ndir_name = os.path.dirname(video_path)\nbasename = os.path.basename(video_path)\nvideo_name = basename[:basename.rfind('.')]\nargs.viz_video = video_path\nargs.viz_output = f'{dir_name}/{args.detector_2d}_{video_name}.gif'\n\nargs.evaluate = 'pretrained_h36m_detectron_coco.bin'\n\nwith Timer(video_path):\n    main(args)\n```\n\nThe meaning of arguments can be found [here](https://github.com/facebookresearch/VideoPose3D/blob/master/DOCUMENTATION.md), you can customize it conveniently by changing the `args` in `./videopose.py`.\n\n\n\n## Acknowledgement\n\nThe 2D pose to 3D pose and visualization part is from [VideoPose3D](https://github.com/facebookresearch/VideoPose3D).\n\nSome of the \"In the wild\" script is adapted from the other [fork](https://github.com/tobiascz/VideoPose3D).\n\nThe project structure and `./videopose.py` running script is adapted from [this repo](https://github.com/lxy5513/videopose)\n\n\n\n## Coming soon\n\nThe other feature will be added to improve accuracy in the future:\n\n- [x] Human completeness check.\n- [x] Object Tracking to the first complete human covering largest area.\n- [x] Change 2D pose estimation method such as [AlphaPose](https://github.com/MVIG-SJTU/AlphaPose).\n- [x] Test HR-Net as 2d joints detector.\n- [x] Test LightTrack as pose tracker.\n- [ ] Multi-person video(complex) support.\n- [ ] Data augmentation to solve \"high-speed with low-rate\" problem: [SLOW-MO](https://github.com/avinashpaliwal/Super-SloMo).\n\n\n## Citation\n```\n@misc{videotopose2021,\n  author = {Zheng, Hao},\n  title = {video-to-pose3D},\n  year = {2021},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https://github.com/zh-plus/video-to-pose3D}},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzh-plus%2Fvideo-to-pose3d","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzh-plus%2Fvideo-to-pose3d","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzh-plus%2Fvideo-to-pose3d/lists"}