{"id":13690860,"url":"https://github.com/openxrlab/xrnerf","last_synced_at":"2025-10-14T12:59:48.284Z","repository":{"id":58417312,"uuid":"529149437","full_name":"openxrlab/xrnerf","owner":"openxrlab","description":"OpenXRLab Neural Radiance Field (NeRF) Toolbox and Benchmark","archived":false,"fork":false,"pushed_at":"2023-02-03T06:44:06.000Z","size":5769,"stargazers_count":584,"open_issues_count":7,"forks_count":50,"subscribers_count":16,"default_branch":"main","last_synced_at":"2025-05-02T11:43:44.485Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openxrlab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-08-26T06:55:10.000Z","updated_at":"2025-04-14T07:09:27.000Z","dependencies_parsed_at":"2023-02-16T06:01:10.722Z","dependency_job_id":null,"html_url":"https://github.com/openxrlab/xrnerf","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/openxrlab/xrnerf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openxrlab%2Fxrnerf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openxrlab%2Fxrnerf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openxrlab%2Fxrnerf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openxrlab%2Fxrnerf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openxrlab","download_url":"https://codeload.github.com/openxrlab/xrnerf/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openxrlab%2Fxrnerf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279018562,"owners_count":26086583,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T17:00:35.042Z","updated_at":"2025-10-14T12:59:48.254Z","avatar_url":"https://github.com/openxrlab.png","language":"C++","funding_links":[],"categories":["Related Resources"],"sub_categories":["Open Source Framework (for Nerf)"],"readme":"# XRNeRF\n\n\u003cdiv align=\"left\"\u003e\n\n[![actions](https://github.com/openxrlab/xrnerf/workflows/build/badge.svg)](https://github.com/openxrlab/xrnerf/actions) [![LICENSE](https://img.shields.io/github/license/openxrlab/xrnerf.svg)](https://github.com/openxrlab/xrnerf/blob/main/LICENSE)\n\n\u003c!-- [![codecov](https://codecov.io/gh/openxrlab/xrnerf/branch/main/graph/badge.svg)](https://codecov.io/gh/openxrlab/xrnerf) --\u003e\n\n\u003c/div\u003e\n\n## Introduction\n\nEnglish | [简体中文](README_CN.md)\n\nXRNeRF is an open-source PyTorch-based codebase for Neural Radiance Field (NeRF). It is a part of the [OpenXRLab](https://github.com/orgs/openxrlab/repositories) project.\n\nhttps://user-images.githubusercontent.com/24294293/187131048-5977c929-e136-4328-ad1f-7da8e7a566ff.mp4\n\nThis page provides basic tutorials about the usage of XRNeRF.\nFor installation instructions, please see [installation.md](docs/en/installation.md).\n\n\u003c!-- TOC --\u003e\n\n- [XRNeRF](#xrnerf)\n  - [Introduction](#introduction)\n  - [Benchmark](#benchmark)\n  - [Datasets](#datasets)\n  - [Installation](#installation)\n  - [Build a Model](#build-a-model)\n    - [Basic Concepts](#basic-concepts)\n    - [Write a new network](#write-a-new-network)\n  - [Train a Model](#train-a-model)\n    - [Iteration Controls](#iteration-controls)\n    - [Train](#train)\n    - [Test](#test)\n  - [Tutorials](#tutorials)\n  - [Other Documents](#other-documents)\n  - [Citation](#citation)\n  - [License](#license)\n  - [Contributing](#contributing)\n  - [Acknowledgement](#acknowledgement)\n  - [Projects in OpenXRLab](#projects-in-openxrlab)\n\n\u003c!-- TOC --\u003e\n\n## Benchmark\n\nMore details can be found in [benchmark.md](docs/en/benchmark.md).\n\nSupported scene-NeRF methods:\n\n\u003cdetails open\u003e\n\u003csummary\u003e(click to collapse)\u003c/summary\u003e\n\n- [X] [NeRF](https://www.matthewtancik.com/nerf) (ECCV'2020)\n- [X] [Mip-NeRF](https://jonbarron.info/mipnerf/) (ICCV'2021)\n- [X] [KiloNeRF](https://arxiv.org/abs/2103.13744) (ICCV'2021)\n- [X] [Instant NGP](https://nvlabs.github.io/instant-ngp/) (SIGGRAPH'2022)\n- [X] [BungeeNeRF](https://city-super.github.io/citynerf/) (ECCV'2022)\n\nSupported human-NeRF methods:\n\n\u003cdetails open\u003e\n\u003csummary\u003e(click to collapse)\u003c/summary\u003e\n\n- [X] [NeuralBody](https://zju3dv.github.io/neuralbody) (CVPR'2021)\n- [X] [AniNeRF](https://zju3dv.github.io/animatable_nerf/) (ICCV'2021)\n- [X] [GNR](https://generalizable-neural-performer.github.io/)\n\nWanna see more methods supported? Post method you want see in XRNeRF on our [wishlist](https://github.com/openxrlab/xrnerf/discussions/11).\n\n\u003c/details\u003e\n\n\u003c/details\u003e\n\n## Datasets\n\nIt is recommended to symlink the dataset root to `$PROJECT/data`.\nIf your folder structure is different, you may need to change the corresponding paths in config files.\n\n```\nxrnerf\n├── xrnerf\n├── docs\n├── configs\n├── test\n├── extensions\n├── data\n│   ├── nerf_llff_data\n│   ├── nerf_synthetic\n│   ├── multiscale\n│   ├── multiscale_google\n│   ├── ...\n```\n\nFor more information on data preparation, please see [dataset_preparation.md](docs/en/dataset_preparation.md)\n\n## Installation\n\nWe provide detailed [installation tutorial](docs/en/installation.md) for XRNeRF, users can install from scratch or use provided [dockerfile](docker/Dockerfile).\n\nIt is recommended to start by creating a docker image:\n\n```shell\ndocker build -f ./docker/Dockerfile --rm -t xrnerf .\n```\n\nFor more information, please follow our [installation tutorial](docs/en/installation.md).\n\n## Build a Model\n\n### Basic Concepts\n\nIn XRNeRF, model components are basically categorized as 4 types.\n\n- network: the whole nerf model pipeline, usually contains a embedder, mlp and render.\n- embedder: convert point-position and viewdirection data into embedded data, embedder can be function only or with trainable paramters.\n- mlp: use the output of embedder as input, and output raw data (the rgb and density value at sampled position) for render, usually contains FC layers.\n- render: receive mlp's raw data, output the rgb value at a pixel.\n\nFollowing some basic pipelines (e.g., `NerfNetwork`), the model structure\ncan be customized through config files with no pains.\n\n### Write a new network\n\nTo write a new nerf network, you need to inherit from `BaseNerfNetwork`,\nwhich defines the following abstract methods.\n\n- `train_step()`: forward method of the training mode.\n- `val_step()`: forward method of the testing mode.\n\n[NerfNetwork](xrnerf/models/networks/nerf.py) is a good example which show how to do that.\n\nTo be specific, if we want to implement some new components, there are several things to do.\n\n1. create a new file in `xrnerf/models/networks/my_networks.py`.\n\n   ```python\n   from ..builder import NETWORKS\n   from .nerf import NerfNetwork\n\n   @NETWORKS.register_module()\n   class MyNerfNetwork(NerfNetwork):\n\n       def __init__(self, cfg, mlp=None, mlp_fine=None, render=None):\n           super().__init__(cfg, mlp, mlp_fine, render)\n\n       def forward(self, data):\n           ....\n\n       def train_step(self, data, optimizer, **kwargs):\n           ....\n\n       def val_step(self, data, optimizer=None, **kwargs):\n           ....\n   ```\n2. Import the module in `xrnerf/models/networks/__init__.py`\n\n   ```python\n   from .my_networks import MyNerfNetwork\n   ```\n3. modify the [config file](configs/nerf/nerf_blender_base01.py) from\n\n   ```python\n   model = dict(\n       type='NerfNetwork',\n       ....\n   ```\n\n   to\n\n   ```python\n   model = dict(\n       type='MyNerfNetwork',\n       ....\n   ```\n\nTo implement some new components for embedder/mlp/render, procedure is similar to above.\n\n* To write a new nerf embedder, you need to inherit from `nn.Module` or `BaseEmbedder`, and define the `forward` method. [BaseEmbedder](xrnerf/models/embedders/base.py) is a good example.\n* To write a new nerf mlp, you need to inherit from `nn.Module` or `BaseMLP`, and define the `forward` method. [NerfMLP](xrnerf/models/mlps/nerf_mlp.py) is a good example.\n* To write a new nerf render, you need to inherit from `nn.Module` or `BaseRender`, and define the `forward` method. [NerfRender](xrnerf/models/renders/nerf_render.py) is a good example.\n\n## Train a Model\n\n### Iteration Controls\n\nXRNeRF use `mmcv.runner.IterBasedRunner` to control training, and `mmcv.runner.EpochBasedRunner` to for test mode.\n\nIn training mode, the `max_iters` in config file decide how many iters.\nIn test mode, `max_iters` is forced to change to 1, which represents only 1 epoch to test.\n\n### Train\n\n```shell\npython run_nerf.py --config configs/nerf/nerf_blender_base01.py --dataname lego\n```\n\nArguments are:\n\n- `--config`: config file path.\n- `--dataname`: select which data under dataset directory.\n\n### Test\n\nWe have provided model ``iter_200000.pth`` for test, download from [here](https://drive.google.com/file/d/147wRy3TFlRVrZdWqAgHNak7s6jiMZA1-/view?usp=sharing)\n\n```shell\npython run_nerf.py --config configs/nerf/nerf_blender_base01.py --dataname lego --test_only --load_from iter_200000.pth\n```\n\nArguments are:\n\n- `--config`: config file path.\n- `--dataname`: select which data under dataset directory.\n- `--test_only`: influence on whole testset once.\n- `--load_from`: load which checkpoint to test, this will overwrite the original `load_from` in config file to for convenience.\n\n## Tutorials\n\nCurrently, we provide some tutorials for users to\n\n* [learn about configs](docs/en/tutorials/config.md)\n* [customize data pipelines](docs/en/tutorials/data_pipeline.md)\n* [model definition](docs/en/tutorials/model.md)\n\n## Other Documents\n\nExcept for that，The document also includes the following\n\n* [api](docs/en/api.md)\n* [dataset](docs/en/dataset_preparation.md)\n* [installation](docs/en/installation.md)\n* [benchmark](docs/en/benchmark.md)\n* [FAQ](docs/en/faq.md)\n\n## Citation\n\nIf you find this project useful in your research, please consider cite:\n\n```bibtex\n@misc{xrnerf,\n    title={OpenXRLab Neural Radiance Field Toolbox and Benchmark},\n    author={XRNeRF Contributors},\n    howpublished = {\\url{https://github.com/openxrlab/xrnerf}},\n    year={2022}\n}\n```\n\n## License\n\nThe license of our codebase is [Apache-2.0](LICENSE). Note that this license only applies to code in our library, the dependencies of which are separate and individually licensed. We would like to pay tribute to open-source implementations to which we rely on. Please be aware that using the content of dependencies may affect the license of our codebase. Some supported methods may carry [additional licenses](docs/en/additional_licenses.md).\n\n\n## Contributing\n\nWe appreciate all contributions to improve XRNeRF. Please refer to [CONTRIBUTING.md](docs/en/CONTRIBUTING.md) for the contributing guideline.\n\n## Acknowledgement\n\nXRNeRF is an open source project that is contributed by researchers and engineers from both the academia and the industry.\nWe appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks.\nWe wish that the framework and benchmark could serve the growing research community by providing a flexible framework to reimplement existing methods and develop their own new models.\n\n## Projects in OpenXRLab\n\n- [XRPrimer](https://github.com/openxrlab/xrprimer): OpenXRLab foundational library for XR-related algorithms.\n- [XRSLAM](https://github.com/openxrlab/xrslam): OpenXRLab Visual-inertial SLAM Toolbox and Benchmark.\n- [XRSfM](https://github.com/openxrlab/xrsfm): OpenXRLab Structure-from-Motion Toolbox and Benchmark.\n- [XRLocalization](https://github.com/openxrlab/xrlocalization): OpenXRLab Visual Localization Toolbox and Server.\n- [XRMoCap](https://github.com/openxrlab/xrmocap): OpenXRLab Multi-view Motion Capture Toolbox and Benchmark.\n- [XRMoGen](https://github.com/openxrlab/xrmogen): OpenXRLab Human Motion Generation Toolbox and Benchmark.\n- [XRNeRF](https://github.com/openxrlab/xrnerf): OpenXRLab Neural Radiance Field (NeRF) Toolbox and Benchmark.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenxrlab%2Fxrnerf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenxrlab%2Fxrnerf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenxrlab%2Fxrnerf/lists"}