{"id":19263741,"url":"https://github.com/calciferzh/minimal-hand","last_synced_at":"2025-04-12T21:34:04.971Z","repository":{"id":46022957,"uuid":"246044510","full_name":"CalciferZh/minimal-hand","owner":"CalciferZh","description":"A minimal solution to hand motion capture from a single color camera at over 100fps. Easy to use, plug to run.","archived":false,"fork":false,"pushed_at":"2022-01-12T12:48:45.000Z","size":4936,"stargazers_count":1000,"open_issues_count":6,"forks_count":173,"subscribers_count":40,"default_branch":"master","last_synced_at":"2025-04-04T01:09:37.340Z","etag":null,"topics":["3d-hand-pose-estimation","computer-vision","deep-learning","hand-motion-capture","hand-tracking"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CalciferZh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-09T13:35:08.000Z","updated_at":"2025-04-01T17:30:04.000Z","dependencies_parsed_at":"2022-08-30T10:02:08.265Z","dependency_job_id":null,"html_url":"https://github.com/CalciferZh/minimal-hand","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalciferZh%2Fminimal-hand","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalciferZh%2Fminimal-hand/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalciferZh%2Fminimal-hand/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalciferZh%2Fminimal-hand/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CalciferZh","download_url":"https://codeload.github.com/CalciferZh/minimal-hand/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248636870,"owners_count":21137527,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-hand-pose-estimation","computer-vision","deep-learning","hand-motion-capture","hand-tracking"],"created_at":"2024-11-09T19:36:41.920Z","updated_at":"2025-04-12T21:34:04.952Z","avatar_url":"https://github.com/CalciferZh.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Minimal Hand\n\nA minimal solution to hand motion capture from a single color camera at over 100fps.\nEasy to use, plug to run.\n\n![teaser](teaser.gif)\n\nThis project provides the core components for hand motion capture:\n1. estimating joint **locations** from a monocular RGB image (DetNet)\n1. estimating joint **rotations** from locations (IKNet)\n\nWe focus on:\n1. ease of use (all you need is a webcam)\n1. time efficiency (on our 1080Ti, 8.9ms for DetNet, 0.9ms for IKNet)\n1. robustness to occlusion, hand-object interaction, fast motion, changing scale and view point\n\nSome links:\n[\\[video\\]](https://youtu.be/OIRulRoBdL4)\n[\\[paper\\]](https://calciferzh.github.io/files/zhou2020monocular.pdf)\n[\\[supp doc\\]](https://calciferzh.github.io/files/zhou2020monocular_supp.pdf)\n[\\[webpage\\]](https://calciferzh.github.io/publications/zhou2020monocular)\n\nThe author is too busy to collect the training code for release.\nOn the other hand, it should not be difficult to implement the training part.\nFeel free to open an issue for any encountered problems.\n\n### Pytorch Version\n[Here](https://github.com/MengHao666/Minimal-Hand-pytorch) is a pytorch version implemented by @MengHao666.\nI didn't personally check it but I believe it worth trying.\nMany thanks to @MengHao666 !\n\n### With Unity\n[Here](https://github.com/vinnik-dmitry07/minimal-hand) is a project that connects this repo to unity.\nIt looks very cool and many thanks to @vinnik-dmitry07 !\n\n\n## Usage\n\n### Install dependencies\nPlease check `requirements.txt`. All dependencies are available via pip and conda.\n\n### Prepare MANO hand model\n1. Download MANO model from [here](https://mano.is.tue.mpg.de/) and unzip it.\n1. In `config.py`, set `OFFICIAL_MANO_PATH` to the **left hand** model.\n1. Run `python prepare_mano.py`, you will get the converted MANO model that is compatible with this project at `config.HAND_MESH_MODEL_PATH`.\n\n### Prepare pre-trained network models\n1. Download models from [here](https://github.com/CalciferZh/minimal-hand/releases/download/v1/cvpr_2020_hand_model_v1.zip).\n1. Put `detnet.ckpt.*` in `model/detnet`, and `iknet.ckpt.*` in `model/iknet`.\n1. Check `config.py`, make sure all required files are there.\n\n### Run the demo for webcam input\n1. `python app.py`\n1. Put your **right hand** in front of the camera. The pre-trained model is for left hand, but the input would be flipped internally.\n1. Press `ESC` to quit.\n1. Although the model is robust to variant scales, most ideally the image should be 1.3x larger than the hand bounding box. A good bounding box may result in better accuracy. You can track the bounding box with the 2D predictions of the model.\n\nWe found that the model may fail on some \"simple\" poses. We think this is because such poses were no presented in the training data. We are working on a v2 version with further extended data to tackle this problem.\n\n### Use the models in your project\nPlease check `wrappers.py`.\n\n## IKNet Alternative\nWe also provide an optimization-based IK solver [here](https://github.com/CalciferZh/Minimal-IK).\n\n## Dataset\nThe detection model is trained with following datasets:\n* [CMU HandDB](http://domedb.perception.cs.cmu.edu/handdb.html)\n* [Rendered Handpose Dataset](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html)\n* [GANerated Hands Dataset](https://handtracker.mpi-inf.mpg.de/projects/GANeratedHands/GANeratedDataset.htm)\n\nThe IK model is trained with the poses shipped with [MANO](https://mano.is.tue.mpg.de/).\n\n## Citation\n\nThis is the official implementation of the paper \"Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data\" (CVPR 2020).\n\nThe quantitative numbers reported in the paper can be found in `plot.py`.\n\nIf you find the project helpful, please consider citing us:\n```\n@InProceedings{zhou2020monocular,\n  author = {Zhou, Yuxiao and Habermann, Marc and Xu, Weipeng and Habibie, Ikhsanul and Theobalt, Christian and Xu, Feng},\n  title = {Monocular Real-Time Hand Shape and Motion Capture Using Multi-Modal Data},\n  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n  month = {June},\n  year = {2020}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalciferzh%2Fminimal-hand","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcalciferzh%2Fminimal-hand","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalciferzh%2Fminimal-hand/lists"}