Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/apchenstu/TensoRF
[ECCV 2022] Tensorial Radiance Fields, a novel approach to model and reconstruct radiance fields
https://github.com/apchenstu/TensoRF
3d-modelling 3d-reconstruction 3d-rendering
Last synced: about 22 hours ago
JSON representation
[ECCV 2022] Tensorial Radiance Fields, a novel approach to model and reconstruct radiance fields
- Host: GitHub
- URL: https://github.com/apchenstu/TensoRF
- Owner: apchenstu
- License: mit
- Created: 2022-03-17T18:21:32.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-09-27T06:08:20.000Z (about 1 year ago)
- Last Synced: 2024-10-29T23:23:35.238Z (9 days ago)
- Topics: 3d-modelling, 3d-reconstruction, 3d-rendering
- Language: Python
- Homepage:
- Size: 32.2 KB
- Stars: 1,171
- Watchers: 20
- Forks: 154
- Open Issues: 59
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-NeRF - Torch
- awesome-NeRF - Torch
README
# TensoRF
## [Project page](https://apchenstu.github.io/TensoRF/) | [Paper](https://arxiv.org/abs/2203.09517)
This repository contains a pytorch implementation for the paper: [TensoRF: Tensorial Radiance Fields](https://arxiv.org/abs/2203.09517). Our work present a novel approach to model and reconstruct radiance fields, which achieves super
**fast** training process, **compact** memory footprint and **state-of-the-art** rendering quality.
https://user-images.githubusercontent.com/16453770/158920837-3fafaa17-6ed9-4414-a0b1-a80dc9e10301.mp4
## Installation
#### Tested on Ubuntu 20.04 + Pytorch 1.10.1
Install environment:
```
conda create -n TensoRF python=3.8
conda activate TensoRF
pip install torch torchvision
pip install tqdm scikit-image opencv-python configargparse lpips imageio-ffmpeg kornia lpips tensorboard
```
## Dataset
* [Synthetic-NeRF](https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1)
* [Synthetic-NSVF](https://dl.fbaipublicfiles.com/nsvf/dataset/Synthetic_NSVF.zip)
* [Tanks&Temples](https://dl.fbaipublicfiles.com/nsvf/dataset/TanksAndTemple.zip)
* [Forward-facing](https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1)
## Quick Start
The training script is in `train.py`, to train a TensoRF:
```
python train.py --config configs/lego.txt
```
we provide a few examples in the configuration folder, please note:
`dataset_name`, choices = ['blender', 'llff', 'nsvf', 'tankstemple'];
`shadingMode`, choices = ['MLP_Fea', 'SH'];
`model_name`, choices = ['TensorVMSplit', 'TensorCP'], corresponding to the VM and CP decomposition.
You need to uncomment the last a few rows of the configuration file if you want to training with the TensorCP model；
`n_lamb_sigma` and `n_lamb_sh` are string type refer to the basis number of density and appearance along XYZ
dimension;
`N_voxel_init` and `N_voxel_final` control the resolution of matrix and vector;
`N_vis` and `vis_every` control the visualization during training;
You need to set `--render_test 1`/`--render_path 1` if you want to render testing views or path after training.
More options refer to the `opt.py`.
### For pretrained checkpoints and results please see:
[https://1drv.ms/u/s!Ard0t_p4QWIMgQ2qSEAs7MUk8hVw?e=dc6hBm](https://1drv.ms/u/s!Ard0t_p4QWIMgQ2qSEAs7MUk8hVw?e=dc6hBm)
## Rendering
```
python train.py --config configs/lego.txt --ckpt path/to/your/checkpoint --render_only 1 --render_test 1
```
You can just simply pass `--render_only 1` and `--ckpt path/to/your/checkpoint` to render images from a pre-trained
checkpoint. You may also need to specify what you want to render, like `--render_test 1`, `--render_train 1` or `--render_path 1`.
The rendering results are located in your checkpoint folder.
## Extracting mesh
You can also export the mesh by passing `--export_mesh 1`:
```
python train.py --config configs/lego.txt --ckpt path/to/your/checkpoint --export_mesh 1
```
Note: Please re-train the model and don't use the pretrained checkpoints provided by us for mesh extraction,
because some render parameters has changed.
## Training with your own data
We provide two options for training on your own image set:
1. Following the instructions in the [NSVF repo](https://github.com/facebookresearch/NSVF#prepare-your-own-dataset), then set the dataset_name to 'tankstemple'.
2. Calibrating images with the script from [NGP](https://github.com/NVlabs/instant-ngp/blob/master/docs/nerf_dataset_tips.md):
`python dataLoader/colmap2nerf.py --colmap_matcher exhaustive --run_colmap`, then adjust the datadir in `configs/your_own_data.txt`. Please check the `scene_bbox` and `near_far` if you get abnormal results.
## Citation
If you find our code or paper helps, please consider citing:
```
@INPROCEEDINGS{Chen2022ECCV,
author = {Anpei Chen and Zexiang Xu and Andreas Geiger and Jingyi Yu and Hao Su},
title = {TensoRF: Tensorial Radiance Fields},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2022}
}
```