{"id":18614324,"url":"https://github.com/mjmjeong/InfoNeRF","last_synced_at":"2025-04-11T00:30:37.472Z","repository":{"id":38372861,"uuid":"440456414","full_name":"mjmjeong/InfoNeRF","owner":"mjmjeong","description":null,"archived":false,"fork":false,"pushed_at":"2022-06-07T12:17:36.000Z","size":50214,"stargazers_count":143,"open_issues_count":6,"forks_count":7,"subscribers_count":10,"default_branch":"main","last_synced_at":"2024-11-07T03:31:01.742Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mjmjeong.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-21T09:15:00.000Z","updated_at":"2024-09-22T16:16:53.000Z","dependencies_parsed_at":"2022-08-26T17:01:11.037Z","dependency_job_id":null,"html_url":"https://github.com/mjmjeong/InfoNeRF","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mjmjeong%2FInfoNeRF","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mjmjeong%2FInfoNeRF/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mjmjeong%2FInfoNeRF/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mjmjeong%2FInfoNeRF/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mjmjeong","download_url":"https://codeload.github.com/mjmjeong/InfoNeRF/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248322208,"owners_count":21084333,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T03:25:54.624Z","updated_at":"2025-04-11T00:30:32.456Z","avatar_url":"https://github.com/mjmjeong.png","language":"Python","funding_links":[],"categories":["Papers"],"sub_categories":["NeRF"],"readme":"\n# InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering \n### CVPR 2022\n### [Project](https://cv.snu.ac.kr/research/InfoNeRF/) | [Paper](https://arxiv.org/abs/2112.15399) | [Video](https://www.youtube.com/embed/WNWuc281cQQ) \n\nPytorch implementation of our method for regularizing nerual radiance fields for few-shot neural volume rendering.\n\n\u003ctable\u003e\n  \u003cthead\u003e\n    \u003ctr\u003e\n      \u003cth rowspan=\"2\" colspan=\"1\"\u003e\u003c/th\u003e\n      \u003cth colspan=\"2\"\u003eNeRF (baseline) \u003c/td\u003e\n      \u003cth colspan=\"2\"\u003eInfoNeRF\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003e RGB \u003c/th\u003e\n      \u003cth\u003e Depth \u003c/th\u003e\n      \u003cth\u003e RGB \u003c/th\u003e\n      \u003cth\u003e Depth \u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003cth\u003eLego\u003c/th\u003e\n      \u003ctd\u003e \u003cimg src=\"teaser/lego/nerf_lego_rgb.gif\"  width=\"200\" /\u003e \u003c/td\u003e\n      \u003ctd\u003e \u003cimg src=\"teaser/lego/nerf_lego_depth.gif\"  width=\"200\" /\u003e \u003c/td\u003e\n      \u003ctd\u003e \u003cimg src=\"teaser/lego/ours_lego_rgb.gif\"  width=\"200\" /\u003e \u003c/td\u003e\n      \u003ctd\u003e \u003cimg src=\"teaser/lego/ours_lego_depth.gif\"  width=\"200\" /\u003e \u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003cth\u003eFicus\u003c/th\u003e\n      \u003ctd\u003e \u003cimg src=\"teaser/ficus/nerf_ficus_rgb.gif\"  width=\"200\" /\u003e  \u003c/td\u003e\n      \u003ctd\u003e \u003cimg src=\"teaser/ficus/nerf_ficus_depth.gif\"  width=\"200\" /\u003e \u003c/td\u003e\n      \u003ctd\u003e \u003cimg src=\"teaser/ficus/ours_ficus_rgb.gif\"  width=\"200\" /\u003e \u003c/td\u003e\n      \u003ctd\u003e \u003cimg src=\"teaser/ficus/ours_ficus_depth.gif\"  width=\"200\" /\u003e \u003c/td\u003e\n    \u003c/tr\u003e\n    \u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003cb\u003e InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering \u003c/b\u003e\n\n[Mijeong Kim](https://mjmjeong.github.io/), [Seonguk Seo](https://seoseong.uk/), [Bohyung Han](https://cv.snu.ac.kr/~bhhan/)\n\nSeoul National University\n\n\u003e \u003cp align=\"center\"\u003e  \u003cfigcaption align=\"center\"\u003e\u003cb\u003e\u003c/b\u003e\u003c/figcaption\u003e\n\u003e We present an information-theoretic regularization technique for few-shot novel view synthesis based on neural implicit representation. \n\u003e The proposed approach minimizes potential reconstruction inconsistency that happens due to insufficient viewpoints by imposing the entropy constraint of the density in each ray. \n\u003e In addition, to alleviate the potential degenerate issue when all training images are acquired from almost redundant viewpoints,\n\u003e we further incorporate the spatially smoothness constraint into the estimated images by restricting information gains from a pair of rays with slightly different viewpoints. \n\u003e The main idea of our algorithm is to make reconstructed scenes compact along individual rays and consistent across rays in the neighborhood. \n\u003e The proposed regularizers can be plugged into most of existing neural volume rendering techniques based on NeRF in a straightforward way. \n\u003e Despite its simplicity, we achieve consistently improved performance compared to existing neural view synthesis methods by large margins on multiple standard benchmarks. \n\n\n---\n\n## Installation\n```\ngit clone https://github.com/mjmjeong/InfoNeRF.git\ncd InfoNeRF\npip install -r requirements.txt\n```\nor\n```\ngit clone https://github.com/mjmjeong/InfoNeRF.git\ncd InfoNeRF\nconda env create -f environment.yaml\nconda activate infonerf\n```\n\n## How  to Run?\n\n### Quick Start \n\nTo train a InfoNeRF on the example `lego` scene:\n```\npython run_nerf.py --config configs/infonerf/synthetic/lego.txt\n```\n\nTo train a NeRF (baseline) on the example `lego` scene:\n```\npython run_nerf.py --config configs/nerf/synthetic/lego.txt\n```\n\nSince our setting is few-shot learning, it is enough to train the model with only 50000 iterations.\n\n### More Datasets\nTo play with other scenes presented in the paper, download the data [here](https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1). Place the downloaded dataset according to the following directory structure:\n```\n├── configs  \n│   ├── ...    \n│                                                                                      \n├── data \n|   ├── nerf_synthetic\n|   |   └── lego\n|   |   └── ship    # downloaded synthetic dataset\n|   |   └── ...\n│   ├── DTU\n│   │   └── scan1   # downloaded DTU dataset\n│   │   └── scan2   # downloaded DTU dataset\n|   |   └── ...\n```\n\n### Training Sequentially\n\nTo repeat the same experiments on other scenes in the same dataset, follow these steps:\n\nFirst, duplicate the config files using `duplicate_config.py` file:\n```\npython duplicate_config.py \n```\n\nSecond, change and run `train.sh` for sequential training:\n```\nbash train.sh\n```\n---\n                \n## Citation\n\nIf you find our work useful in your research, please cite:\n\n```\n@inproceedings{kim2022infonerf,\n            author = {Mijeong Kim and Seonguk Seo and Bohyung Han},\n            booktitle = {CVPR},\n            title = {InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering},\n            year = {2022}\n        }\n```\n\n## Acknowlegements\n\nThis code heavily borrows from [nerf-pytorch](https://github.com/yenchenlin/nerf-pytorch) and [DS-NeRF](https://github.com/dunbar12138/DSNeRF).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmjmjeong%2FInfoNeRF","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmjmjeong%2FInfoNeRF","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmjmjeong%2FInfoNeRF/lists"}