{"id":20295099,"url":"https://github.com/agnjason/xhand","last_synced_at":"2025-10-15T12:09:28.138Z","repository":{"id":248541424,"uuid":"828983645","full_name":"agnJason/XHand","owner":"agnJason","description":"Official pytorch  implementation of \"XHand: Real-time Expressive Hand Avatar\"","archived":false,"fork":false,"pushed_at":"2024-07-31T05:46:28.000Z","size":16290,"stargazers_count":75,"open_issues_count":2,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-29T15:11:17.742Z","etag":null,"topics":["hand","hand-reconstruction"],"latest_commit_sha":null,"homepage":"https://agnjason.github.io/XHand-page/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/agnJason.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-15T14:15:01.000Z","updated_at":"2025-02-23T15:53:50.000Z","dependencies_parsed_at":"2024-11-14T15:47:14.335Z","dependency_job_id":null,"html_url":"https://github.com/agnJason/XHand","commit_stats":null,"previous_names":["agnjason/xhand"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agnJason%2FXHand","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agnJason%2FXHand/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agnJason%2FXHand/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agnJason%2FXHand/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/agnJason","download_url":"https://codeload.github.com/agnJason/XHand/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250240168,"owners_count":21397826,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hand","hand-reconstruction"],"created_at":"2024-11-14T15:33:07.013Z","updated_at":"2025-10-15T12:09:23.102Z","avatar_url":"https://github.com/agnJason.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003ch3\u003eXHand: Real-time Expressive Hand Avatar\u003c/h3\u003e\n\n[Qijun Gan](https://github.com/agnJason), Zijie Zhou, [Jianke Zhu](https://scholar.google.cz/citations?user=SC-WmzwAAAAJ)\u003csup\u003e:email:\u003c/sup\u003e\n \nZhejiang University\n\n(\u003csup\u003e:email:\u003c/sup\u003e) corresponding author.\n\n[ArXiv Preprint](https://arxiv.org/abs/2407.21002) \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; [Project Page](https://agnjason.github.io/XHand-page/) \n\n\u003c/div\u003e\n\n### News\n* `Jul. 31st, 2024`: 🔥 We have released our code. Try it now! Please give us a star! ⭐️⭐️⭐️ 😄\n* Our source code is coming soon. Please stay tuned! ☕️\n\n## Abstract\n\nHand avatars play a pivotal role in a wide array of digital interfaces, enhancing user immersion and facilitating natural interaction within virtual environments. While previous studies have focused on photo-realistic hand rendering, little attention has been paid to reconstruct the hand geometry with fine details, which is essential to rendering quality. In the realms of extended reality and gaming, on-the-fly rendering becomes imperative. To this end, we introduce an expressive hand avatar, named XHand, that is designed to comprehensively generate hand shape, appearance, and deformations in real-time. To obtain fine-grained hand meshes, we make use of three feature embedding modules to predict hand deformation displacements, albedo, and linear blending skinning weights, respectively. To achieve photo-realistic hand rendering on fine-grained meshes, our method employs a mesh-based neural renderer by leveraging mesh topological consistency and latent codes from embedding modules. During training, a part-aware Laplace smoothing strategy is proposed by incorporating the distinct levels of regularization to effectively maintain the necessary details and eliminate the undesired artifacts. The experimental evaluations on InterHand2.6M and DeepHandMesh datasets demonstrate the efficacy of XHand, which is able to obtain high-fidelity geometry and texture for hand animations across diverse poses in real-time.\n\n## Introduction\n\n![framework](assets/teaser.png \"framework\")\n\u003cdiv align=\"center-align\"\u003eWe present XHand, a rigged hand avatar that captures the  geometry, appearance and poses of the hand. XHand is created from multi-view videos and utilizes MANO pose parameters (the first image in each group of (a)) to generate high-detail meshes (the second) and renderings (the third). XHand generates photo-realistic hand images in real-time for a given pose sequence (b). (c) is an example of animated personalized hand avatars according to poses in the wild images.\u003c/div\u003e\n\n**Notes**: \n\n- All the experiments are performed on 1 NVIDIA GeForce RTX 3090Ti GPU.\n\n\n## Getting Started\n\n### Install \n\n**a. Create a conda virtual environment and install required packages.**\n```shell\ngit clone git@github.com:agnJason/XHand.git\nconda create -n xhand python=3.10 -y\nconda activate xhand\n\npip install torch==1.13.1+cu116 torchvision==0.14.1+cu116 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu116\npip install -r requirements.txt\n```\n\n**b. Prepare MANO models.**\n\nBesides, you also need to download the MANO model. Please visit the [MANO website](https://mano.is.tue.mpg.de/) and register to get access to the downloads section. You need to put MANO_RIGHT.pkl and MANO_LEFT.pkl under the ./mano folder.\n\n### Training\nEdit your Interhand2.6M PATH in [conf/ih_sfsseq.conf](conf/ih_sfsseq.conf)-\u003edata_path, which should contain ./images and ./annotations.\n```bash\npython sfs_lbs_train.py --conf conf/ih_sfsseq.conf\n```\nThe output should be in `./interhand_out`.\n\n### Inference\n[Pretrained model](https://drive.google.com/file/d/1xDtMKb08aNDarDPSBOQzpTfXrF8aR5uA/view?usp=sharing)\n```bash\npython sfs_lbs_test.py --exp_path interhand_out/Capture0_ROM03_RT_No_Occlusion/xhand\n```\nTry `--cam_id (cam400004)`, `--test_data_name (0002_good_luck)` and `--test_capture_name (Capture1)` tu inference with different views or poses. Save visualized results with `--save_vis` and save detailed meshes with `--save_mesh`.\n\n## Citation\nIf you find our project is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.\n```bibtex\n@misc{gan2024xhandrealtimeexpressivehand,\n      title={XHand: Real-time Expressive Hand Avatar}, \n      author={Qijun Gan and Zijie Zhou and Jianke Zhu},\n      year={2024},\n      eprint={2407.21002},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV},\n      url={https://arxiv.org/abs/2407.21002}, \n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagnjason%2Fxhand","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagnjason%2Fxhand","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagnjason%2Fxhand/lists"}