{"id":18614448,"url":"https://github.com/pfnet-research/surface-aligned-nerf","last_synced_at":"2025-04-13T10:27:08.254Z","repository":{"id":41065826,"uuid":"474046660","full_name":"pfnet-research/surface-aligned-nerf","owner":"pfnet-research","description":null,"archived":false,"fork":false,"pushed_at":"2022-07-11T03:38:02.000Z","size":27933,"stargazers_count":66,"open_issues_count":4,"forks_count":4,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-03-27T01:51:26.444Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pfnet-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-25T14:40:42.000Z","updated_at":"2025-02-12T08:20:36.000Z","dependencies_parsed_at":"2022-09-09T17:53:27.284Z","dependency_job_id":null,"html_url":"https://github.com/pfnet-research/surface-aligned-nerf","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfnet-research%2Fsurface-aligned-nerf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfnet-research%2Fsurface-aligned-nerf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfnet-research%2Fsurface-aligned-nerf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pfnet-research%2Fsurface-aligned-nerf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pfnet-research","download_url":"https://codeload.github.com/pfnet-research/surface-aligned-nerf/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248697394,"owners_count":21147320,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T03:25:56.873Z","updated_at":"2025-04-13T10:27:08.215Z","avatar_url":"https://github.com/pfnet-research.png","language":"Python","funding_links":[],"categories":["Papers"],"sub_categories":["NeRF Related Tasks"],"readme":"# Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis (CVPR 2022)\n\n### [Project page](https://pfnet-research.github.io/surface-aligned-nerf/) | [Video](https://youtu.be/cD3hMbkFk9Y) | [Paper](https://arxiv.org/pdf/2201.01683.pdf)\n\n![pipeline](https://pfnet-research.github.io/surface-aligned-nerf/files/pipeline.png)\n\n## Installation and Data Preparation\n\nPlease see [INSTALL.md](INSTALL.md) for installation and data preparation.\n\n## Run the code on ZJU-MoCap\nTake the subject \"313\" as an example.\n\n### Download the pre-trained model\nFor a quick start, you can download the pretrained model from [Google Drive](https://drive.google.com/drive/folders/1K-sTF26We3xC6Z1qrXn3PCd-DRTve1kc?usp=sharing) **(to be updated)**, and put it to `$ROOT/data/trained_model/sa-nerf/test_313/latest.pth`.\n\n### Evaluation\nTest on training human poses:\n```\npython run.py --type evaluate --cfg_file configs/zju_mocap_exp/multi_view_313.yaml exp_name test_313\n```\nTest on unseen human poses:\n```\npython run.py --type evaluate --cfg_file configs/zju_mocap_exp/multi_view_313.yaml exp_name test_313 test_novel_pose True\n```\nThen you can get a quantitative evaluation in terms of PSNR and SSIM. (The results may slightly differ from those reported in the paper, because we found that we originally used [the incorrect ray sampling code](https://github.com/zju3dv/neuralbody/commit/c47c36554c2991c4cbaf2a370c42fdee51bfb451) in the Neural Body pipeline. Here we use the updated code.)\n\n### Novel view synthesis (rotate camera)\n\n\u003cimg src=\"https://pfnet-research.github.io/surface-aligned-nerf/files/rotate.gif\" width=\"30%\"\u003e\n\n```\npython run.py --type visualize --cfg_file configs/zju_mocap_exp/multi_view_313.yaml exp_name test_313 vis_novel_view True\n```\n\n### Shape control (change SMPL shape parameters)\n\n\u003cimg src=\"https://pfnet-research.github.io/surface-aligned-nerf/files/shape_control.gif\" width=\"30%\"\u003e\n\nHere we use the list of `[pc, val]` to specify the principal components of SMPL. \nHere, `pc` represents the number of principal components and `val` represents the value of principal components. \nFor example, if you want to set `pc1` to `4` and `pc2` to `-2`, run:\n```\npython run.py --type visualize --cfg_file configs/zju_mocap_exp/multi_view_313.yaml exp_name test_313 vis_novel_view True shape_control \"[[1, 4], [2, -2]]\"\n```\n\n###  Changing clothes\n\n\u003cimg src=\"https://pfnet-research.github.io/surface-aligned-nerf/files/changing_clothes.gif\" width=\"30%\"\u003e\n\nFor example, if you want to use the upper body of subject \"377\" and lower body of subject \"390\" for rendering, run:\n```\npython run.py --type visualize --cfg_file configs/zju_mocap_exp/multi_view_313.yaml exp_name test_313 vis_novel_view True upper_body test_313 lower_body test_377\n```\nYou may need to adjust the threshold [here](lib/networks/renderer/if_clight_renderer.py#L77) for a better segmentation.\n\n### Training on ZJU-MoCap from scratch\n```\npython train_net.py --cfg_file configs/zju_mocap_exp/multi_view_313.yaml exp_name test_313 resume False\n```\n\n## Implementation of the dispersed projection\nIf you want to learn more about the implementation details of the proposed dispersed projection, check [here](lib/networks/projection/map.py).\n\n## Citation\n```bibtex\n@inproceedings{xu2022sanerf,\n    author    = {Xu, Tianhan and Fujita, Yasuhiro and Matsumoto, Eiichi},\n    title     = {Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis},\n    booktitle = {CVPR},\n    year      = {2022},\n}\n```\n\n## Acknowledgement\n\nOur implementation is based on the code for the [Neural Body (Peng et al., CVPR 2021)](https://zju3dv.github.io/neuralbody/).\nWe thank the authors for releasing the code, and please also consider citing their work.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpfnet-research%2Fsurface-aligned-nerf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpfnet-research%2Fsurface-aligned-nerf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpfnet-research%2Fsurface-aligned-nerf/lists"}