{"id":20295103,"url":"https://github.com/agnjason/fmhr","last_synced_at":"2026-03-06T18:11:43.977Z","repository":{"id":211543997,"uuid":"729431624","full_name":"agnJason/FMHR","owner":"agnJason","description":"Fine-grained Multi-view Hand Reconstruction Using Inverse Rendering","archived":false,"fork":false,"pushed_at":"2024-07-08T09:47:33.000Z","size":54384,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-22T17:55:03.856Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/agnJason.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-09T07:52:11.000Z","updated_at":"2025-05-25T15:21:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"b13f0bbc-4c4f-4118-bf08-e0f4d7fe8f87","html_url":"https://github.com/agnJason/FMHR","commit_stats":null,"previous_names":["agnjason/fmhr"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/agnJason/FMHR","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agnJason%2FFMHR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agnJason%2FFMHR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agnJason%2FFMHR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agnJason%2FFMHR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/agnJason","download_url":"https://codeload.github.com/agnJason/FMHR/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agnJason%2FFMHR/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30189662,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T17:33:53.563Z","status":"ssl_error","status_checked_at":"2026-03-06T17:33:51.678Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T15:33:10.479Z","updated_at":"2026-03-06T18:11:43.943Z","avatar_url":"https://github.com/agnJason.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\u003ch1\u003eFine-Grained Multi-View Hand Reconstruction Using Inverse Rendering\u003c/h1\u003e\n\n[Qijun Gan](https://github.com/agnJason), [Wentong Li](https://cslwt.github.io/), [Jinwei Ren ](https://github.com/zijinxuxu), [Jianke Zhu](https://scholar.google.cz/citations?user=SC-WmzwAAAAJ) \u003csup\u003e:email:\u003c/sup\u003e\n\nZhejiang University\n\n(\u003csup\u003e:email:\u003c/sup\u003e) corresponding author.\n\n[Paper](https://ojs.aaai.org/index.php/AAAI/article/download/27946/27912)\n\n\u003c/div\u003e\n\n#\n### News\n* **`July. 8th, 2024`:** 🌟We released our source code!\n\n## Abstract\n\nReconstructing high-fidelity hand models with intricate tex- tures plays a crucial role in enhancing human-object interac- tion and advancing real-world applications. Despite the state- of-the-art methods excelling in texture generation and image rendering, they often face challenges in accurately capturing geometric details. Learning-based approaches usually offer better robustness and faster inference, which tend to produce smoother results and require substantial amounts of training data. To address these issues, we present a novel fine-grained multi-view hand mesh reconstruction method that leverages inverse rendering to restore hand poses and intricate details. Firstly, our approach predicts a parametric hand mesh model through Graph Convolutional Networks (GCN) based method from multi-view images. We further introduce a novel Hand Albedo and Mesh (HAM) optimization module to refine both the hand mesh and textures, which is capable of preserv- ing the mesh topology. In addition, we suggest an effec- tive mesh-based neural rendering scheme to simultaneously generate photo-realistic image and optimize mesh geometry by fusing the pre-trained rendering network with vertex fea- tures. We conduct the comprehensive experiments on Inter- Hand2.6M, DeepHandMesh and dataset collected by ourself, whose promising results show that our proposed approach outperforms the state-of-the-art methods on both reconstruc- tion accuracy and rendering quality.\n## Introduction\n\n![framework](assets/teaser.png \"framework\")\n\nOverview of our coarse-to-fine framework. Given a set of calibrated images, we initialize MANO parameters and refine the mesh using our proposed HAM module and inverse rendering to achieve geometric details. By jointly optimizing the mesh using a model-based neural rendering, a fine-grained mesh can be obtained along with its hyper-realistic rendered images.\n\n**Notes**: \n\n- All the experiments are performed on 1 NVIDIA GeForce RTX 3090Ti GPU.\n\n\n## Getting Started\n\n### Install \n\n**a. Create a conda virtual environment and install required packages.**\n```shell\ngit clone git@github.com:agnJason/FMHR.git\nconda create -n FMHR python=3.10 -y\nconda activate FMHR\n\npip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu118\npip install -r requirement.txt\n```\n\n**b. Prepare MANO models.**\n\nBesides, you also need to download the MANO model. Please visit the [MANO website](https://mano.is.tue.mpg.de/) and register to get access to the downloads section. You need to put MANO_RIGHT.pkl and MANO_LEFT.pkl under the ./mano folder.\n\n### Data from Interhand2.6M\nEdit your Interhand2.6M PATH in [conf/ih_sfs.conf](conf/ih_sfs.conf)-\u003edata_path, which should contain ./images and ./annotations.\n```bash\n# Mesh optim with MANO annotations, change capture/name in conf/ih_sfs.conf\npython mesh_sfs_optim.py --conf conf/ih_sfs.conf --scan_id 0\n\n# Train Neural renderer\npython neural_render.py --conf conf/ih_sfs.conf --scan_id 0  --net_type mlp\n```\nThe output should be in `./interhand_out`.\n\n### Data from Capture Room\n\nPrepare MANO paramters.\n```bash\n# optim 3d pose\npython pose_optim.py --data_path ./demo_data --scan_id 1 --out_path ./demo_out\n# optim mano para\npython mano_optim.py --data_path ./demo_data --scan_id 1 --out_path ./demo_out\n```\nMesh optim.\n```\n# Mesh optim\npython mesh_sfs_optim.py --conf ./conf/demo_sfs.conf --data_path ./demo_data --scan_id 1\n```\nThe output will be in `./demo_out`.\n\n(Optional) MANO parameters can also be predicted by GCN-base network.\n```bash\npython multihands_mano.py --conf ./conf/demo_sfs.conf --data_path ./demo_data --scan_id 1\n```\n\n## Citation\nIf you find our project is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.\n```bibtex\n@inproceedings{gan2024fine,\n  title={Fine-Grained Multi-View Hand Reconstruction Using Inverse Rendering},\n  author={Gan, Qijun and Li, Wentong and Ren, Jinwei and Zhu, Jianke},\n  booktitle={AAAI},\n  pages={1779--1787},\n  year={2024}\n}\n\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagnjason%2Ffmhr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagnjason%2Ffmhr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagnjason%2Ffmhr/lists"}