Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Asparagus15/GaussianShader
code for GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces
https://github.com/Asparagus15/GaussianShader
Last synced: 3 months ago
JSON representation
code for GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces
- Host: GitHub
- URL: https://github.com/Asparagus15/GaussianShader
- Owner: Asparagus15
- License: other
- Created: 2023-11-30T16:12:22.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2024-01-25T02:58:54.000Z (10 months ago)
- Last Synced: 2024-01-25T03:43:20.667Z (10 months ago)
- Language: C++
- Size: 109 MB
- Stars: 160
- Watchers: 6
- Forks: 7
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
- awesome-3dgs - Code
README
GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces
Yingwenqi Jiang,
Jiadong Tu,
Yuan Liu,
Xifeng Gao,
Xiaoxiao Long*,
Wenping Wang,
Yuexin Ma*
*Corresponding authors
Paper | Project Page
## Introduction
The advent of neural 3D Gaussians has recently brought about a revolution in the field of neural rendering, facilitating the generation of high-quality renderings at real-time speeds. However, the explicit and discrete representation encounters challenges when applied to scenes featuring reflective surfaces. In this paper, we present **GaussianShader**, a novel method that applies a simplified shading function on 3D Gaussians to enhance the neural rendering in scenes with reflective surfaces while preserving the training and rendering efficiency.
GaussianShader maintains real-time rendering speed and renders high-fidelity images for both general and reflective surfaces. GaussianShader enables free-viewpoint rendering objects under distinct lighting environments.
GaussianShader initiates with the neural 3D Gaussian spheres that integrate both conventional attributes and the newly introduced
shading attributes to accurately capture view-dependent appearances. We incorporate a differentiable environment lighting map to simulate
realistic lighting. The end-to-end training leads to a model that reconstructs both reflective and diffuse surfaces, achieving high material
and lighting fidelity.
## Installation
Provide installation instructions for your project. Include any dependencies and commands needed to set up the project.
```shell
# Clone the repository
git clone https://github.com/Asparagus15/GaussianShader.git
cd GaussianShader
# Install dependencies
conda env create --file environment.yml
conda activate gaussian_shader
```
## Running
Download the [example data](https://drive.google.com/file/d/1bSv0soQtjbRj9S9Aq9uQ27EW4wwY--6q/view?usp=sharing) and put it to the ``data`` folder. Execute the optimizer using the following command:
```shell
python train.py -s data/horse_blender --eval -m output/horse_blender -w --brdf_dim 0 --sh_degree -1 --lambda_predicted_normal 2e-1 --brdf_env 512
```
## Rendering
```shell
python render.py -m output/horse_blender --brdf_dim 0 --sh_degree -1 --brdf_mode envmap --brdf_env 512
```
## Dataset
We mainly evaluate our method on [NeRF Synthetic](https://github.com/bmild/nerf), [Tanks&Temples](https://www.tanksandtemples.org), [Shiny Blender](https://github.com/google-research/multinerf) and [Glossy Synthetic](https://github.com/liuyuan-pal/NeRO). You can use ``nero2blender.py`` to convert the Glossy Synthetic data into Blender format.
## More features
The repo is still being under construction, thanks for your patience.
- [ ] Arguments explanation.
- [ ] Residual color training code.
## Acknowledgement
We have intensively borrow codes from the following repositories. Many thanks to the authors for sharing their codes.
- [gaussian splatting](https://github.com/graphdeco-inria/gaussian-splatting)
- [Ref-NeRF](https://github.com/google-research/multinerf)
- [nvdiffrec](https://github.com/NVlabs/nvdiffrec)
- [Point-NeRF](https://github.com/Xharlie/pointnerf)
## Citation
If you find this repository useful in your project, please cite the following work. :)
```
@article{jiang2023gaussianshader,
title={GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces},
author={Jiang, Yingwenqi and Tu, Jiadong and Liu, Yuan and Gao, Xifeng and Long, Xiaoxiao and Wang, Wenping and Ma, Yuexin},
journal={arXiv preprint arXiv:2311.17977},
year={2023}
}
```