{"id":18610209,"url":"https://github.com/iamncj/nrhints","last_synced_at":"2025-04-04T22:03:13.875Z","repository":{"id":179193576,"uuid":"658630068","full_name":"iamNCJ/NRHints","owner":"iamNCJ","description":"Official Code Release for [SIGGRAPH 2023] Relighting Neural Radiance Fields with Shadow and Highlight Hints","archived":false,"fork":false,"pushed_at":"2025-02-28T09:12:43.000Z","size":1542,"stargazers_count":168,"open_issues_count":4,"forks_count":12,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-28T21:02:27.522Z","etag":null,"topics":["nerf","neural-rendering","neural-representation","relighting","siggraph"],"latest_commit_sha":null,"homepage":"https://nrhints.github.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iamNCJ.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-26T07:28:34.000Z","updated_at":"2025-03-16T08:51:39.000Z","dependencies_parsed_at":"2025-01-26T03:01:51.452Z","dependency_job_id":"f29affff-c32a-4872-8a30-dc21ee4b4565","html_url":"https://github.com/iamNCJ/NRHints","commit_stats":null,"previous_names":["iamncj/nrhints"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamNCJ%2FNRHints","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamNCJ%2FNRHints/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamNCJ%2FNRHints/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamNCJ%2FNRHints/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iamNCJ","download_url":"https://codeload.github.com/iamNCJ/NRHints/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247256104,"owners_count":20909240,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nerf","neural-rendering","neural-representation","relighting","siggraph"],"created_at":"2024-11-07T03:08:53.659Z","updated_at":"2025-04-04T22:03:13.845Z","avatar_url":"https://github.com/iamNCJ.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n\n  \u003ch1 align=\"center\"\u003eRelighting Neural Radiance Fields with Shadow and Highlight Hints\u003c/h1\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://www.chong-zeng.com/\"\u003e\u003cstrong\u003eChong Zeng\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://www.microsoft.com/en-us/research/people/guoch/\"\u003e\u003cstrong\u003eGuojun Chen\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://yuedong.shading.me/\"\u003e\u003cstrong\u003eYue Dong\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://www.cs.wm.edu/~ppeers/\"\u003e\u003cstrong\u003ePieter Peers\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://svbrdf.github.io/\"\u003e\u003cstrong\u003eHongzhi Wu\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://www.microsoft.com/en-us/research/people/xtong/\"\u003e\u003cstrong\u003eXin Tong\u003c/strong\u003e\u003c/a\u003e\n  \u003c/p\u003e\n  \u003ch2 align=\"center\"\u003eSIGGRAPH 2023 Conference Proceedings\u003c/h2\u003e\n  \u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"nrhints-teaser.png\"\u003e\n  \u003c/div\u003e\n\n  \u003cp align=\"center\"\u003e\n  \u003cbr\u003e\n    \u003ca href=\"https://nrhints.github.io/\"\u003e\u003cstrong\u003eProject Page\u003c/strong\u003e\u003c/a\u003e\n    |\n    \u003ca href=\"https://nrhints.github.io/pdfs/nrhints-sig23.pdf\"\u003e\u003cstrong\u003ePaper\u003c/strong\u003e\u003c/a\u003e\n    |\n    \u003ca href=\"https://arxiv.org/abs/2308.13404\"\u003e\u003cstrong\u003earXiv\u003c/strong\u003e\u003c/a\u003e\n    |\n    \u003ca href=\"#data-and-models\"\u003e\u003cstrong\u003eData\u003c/strong\u003e\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\n---\n\n# Setup\n\n## Environment\n\nThe code is developed and tested on Linux servers with NVIDIA GPU(s). We support Python 3.8+ and PyTorch 1.11+. After getting a required Python environment, you can setup the rest of requirements by running:\n\n```bash\ngit clone https://github.com/iamNCJ/NRHints.git\ncd NRHints\npip install -r requirements.txt\n```\n\n## Data\n\nOur data is compatible with `NeRF Blender Dataset`, except that we have extra fields in each frame for point light position.\n\nYou can download our data [here](#data-and-models).\n\n# Usage\n\n## Configuration System\n\nWe use [tyro](https://github.com/brentyi/tyro) for configuration management. Description to all configurations can be found by running `python main.py -h`.\n\n## Training\n\n```bash\npython3 main.py config:nr-hints --config.data.path /path/to/data/ --config.scene-name XXX\n```\n\nRefer to [train_synthetic.sh](scripts/train_synthetic.sh) and [train_real.sh](scripts/train_real.sh) for training on synthetic and real data, respectively.\n\n\u003e **Note**: \n\u003e 1. Our code automatically detects the number of GPUs and uses all of them for training. If you want to use a subset of GPUs, you can set the `CUDA_VISIBLE_DEVICES` environment variable.\n\u003e 2. For training on real captured scenes, we recommend turning on camera optimization by using `config:nr-hints-cam-opt`, which can significantly reduce the blurry effects. Since this is an improvement after the paper submission, details are described in [the author's version](https://arxiv.org/abs/2308.13404).\n\n## Testing\n\n```bash\npython3 main.py config:nr-hints --config.data.path /path/to/data/ --config.scene-name XXX --config.evaluation-only True\n```\n\nRefer to [eval_synthetic.sh](scripts/eval_synthetic.sh) and [eval_real.sh](scripts/eval_real.sh) for testing on synthetic and real data, respectively.\n\nOur pretrained models can be downloaded [here](#data-and-models).\n\n\n# Data and Models\n\n## Real Captured Scenes\n\n| Object      | Data        | Pre-trained model        |\n| ----------- | :----------: | :-----------: |\n| Cat | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Real/Cat.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Real/Cat_step_1000000.ckpt) |\n| Cluttered Scene  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Real/FurScene.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Real/FurScene_step_1000000.ckpt) |\n| Pixiu Statuette   | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Real/Pixiu.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Real/Pixiu_step_1000000.ckpt) |\n| Ornamental Fish | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Real/Fish.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Real/Fish_step_1000000.ckpt) |\n| Cat on Decor   | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Real/CatSmall.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Real/CatSmall_step_1000000.ckpt) |\n| Cup and Fabric   | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Real/CupFabric.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Real/CupFabric_step_1000000.ckpt) |\n| Pikachu Statuette   | [Part1](https://igwebhost.azurewebsites.net/NRHints/Data/Real/Pikachu.zip.001) [Part2](https://igwebhost.azurewebsites.net/NRHints/Data/Real/Pikachu.zip.002)       | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Real/Pikachu_step_1000000.ckpt) |\n\n## Synthetic Rendered Scenes\n\n\u003e **Note**:\n\u003e Our synthetic data rendering scripts are released at [here](https://github.com/iamNCJ/bpy-helper/tree/main/examples/nrhints-data).\n\n| Object      | Data        | Pre-trained model        |\n| ----------- | :----------: | :-----------: |\n|  Diffuse  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_Diffuse_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_Diffuse_PL_500_step_1000000.ckpt) |\n|  Metallic  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_Metal_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_Metal_PL_500_step_1000000.ckpt) |\n|  Glossy-Metal  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_Metal_Rough_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_Metal_Rough_PL_500_step_1000000.ckpt) |\n|  Rough-Metal  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_Metal_VeryRough_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_Metal_VeryRough_PL_500_step_1000000.ckpt) |\n|  Anisotropic-Metal  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_Metal_Aniso_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_Metal_Aniso_PL_500_step_1000000.ckpt) |\n|  Plastic  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_NonMetal_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_NonMetal_PL_500_step_1000000.ckpt) |\n|  Glossy-Plastic  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_NonMetal_Rough_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_NonMetal_Rough_PL_500_step_1000000.ckpt) |\n|  Rough-Plastic  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_NonMetal_VeryRough_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_NonMetal_VeryRough_PL_500_step_1000000.ckpt) |\n|  Short-Fur  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_ShortFur_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_ShortFur_PL_500_step_1000000.ckpt) |\n|  Long-Fur  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_LongFur_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_LongFur_PL_500_step_1000000.ckpt) |\n|  Translucent  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Cup_Plane_SSS_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Cup_Plane_SSS_PL_500_step_1000000.ckpt) |\n|  Fur-Ball  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/FurBall_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/FurBall_PL_500_step_1000000.ckpt) |\n|  Basket  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Basket_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Basket_PL_500_step_1000000.ckpt) |\n|  Layered Woven Ball  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Complex_Ball_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Complex_Ball_PL_500_step_1000000.ckpt) |\n|  Drums  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Drums_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Drums_PL_500_step_1000000.ckpt) |\n|  Hotdog | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Hotdog_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Hotdog_PL_500_step_1000000.ckpt) |\n|  Lego  | [Link](https://igwebhost.azurewebsites.net/NRHints/Data/Synthetic/Lego_PL_500.zip)        | [Link](https://igwebhost.azurewebsites.net/NRHints/Model/Synthetic/Lego_PL_500_step_1000000.ckpt) |\n\nYou can use the script [download_data.sh](scripts/download_data.sh) to download all data.\n\n# Citation\n\nCite as below if you find this repository is helpful to your project:\n\n```\n@inproceedings {zeng2023nrhints,\n    title      = {Relighting Neural Radiance Fields with Shadow and Highlight Hints},\n    author     = {Chong Zeng and Guojun Chen and Yue Dong and Pieter Peers and Hongzhi Wu and Xin Tong},\n    booktitle  = {ACM SIGGRAPH 2023 Conference Proceedings},\n    year       = {2023}\n}\n```\n\n# Acknowledgement\n\nSome code snippets are borrowed from [NeuS](https://github.com/Totoro97/NeuS) and [Nerfstudio](https://nerf.studio/). Thanks for these great projects.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamncj%2Fnrhints","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiamncj%2Fnrhints","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamncj%2Fnrhints/lists"}