Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ajhamdi/ges-splatting
Original reference implementation of "GES : Generalized Exponential Splatting for Efficient Radiance Field Rendering" [CVPR 2024]
https://github.com/ajhamdi/ges-splatting
3d cvpr2024 deep-learning gaussian-splatting nerf
Last synced: 7 days ago
JSON representation
Original reference implementation of "GES : Generalized Exponential Splatting for Efficient Radiance Field Rendering" [CVPR 2024]
- Host: GitHub
- URL: https://github.com/ajhamdi/ges-splatting
- Owner: ajhamdi
- License: other
- Created: 2024-02-14T12:34:34.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2024-06-13T12:02:59.000Z (8 months ago)
- Last Synced: 2024-06-13T14:41:50.525Z (8 months ago)
- Topics: 3d, cvpr2024, deep-learning, gaussian-splatting, nerf
- Language: Python
- Homepage: https://abdullahamdi.com/ges/
- Size: 6.42 MB
- Stars: 175
- Watchers: 10
- Forks: 5
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering [CVPR 2024]
[webpage](https://abdullahamdi.com/ges/) | [arXiv](https://arxiv.org/abs/2402.10128) | [Video](https://youtu.be/edSvNy3roV8)
[Abdullah Hamdi](https://abdullahamdi.com/) 1, [Luke Melas-Kyriazi](https://lukemelas.github.io/) 1, [Guocheng Qian](https://guochengqian.github.io/) 2,4, [Jinjie Mai](https://cemse.kaust.edu.sa/people/person/jinjie-mai) 2, [Ruoshi Liu](https://ruoshiliu.github.io/) 3, [Carl Vondrick](https://www.cs.columbia.edu/~vondrick/) 3, [Bernard Ghanem](https://www.bernardghanem.com/) 2, [Andrea Vedaldi](https://www.robots.ox.ac.uk/~vedaldi/) 1
1 [Visual Geometry Group, University of Oxford](http://www.robots.ox.ac.uk/~vgg/)
2 [KAUST](https://www.kaust.edu.sa/),
3 [Columbia University](https://www.columbia.edu/),
4 [Snap Inc.](https://www.snap.com/),
## Overview
We provide a PyTorch implementation of our Generalized Exponential Splatting (GES) method, as well as the Gaussian Splatting method for comparison. We also provide the code to reproduce the results in our paper. The code is heavily based on [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/). The `.ply` files output of the GES are __exactly__ the same type as the Gaussian Splatting `.ply` files. You can use the __same rendering code and utilities__ to visualize the results or edit them. The extra parameter [`shape`](https://github.com/ajhamdi/ges-splatting/blob/f66ba3457f0bc15231e2009e219d34894130abac/scene/laplacian_model.py#L160) is fused with the other gaussian parameters before saving the `.ply` file.
### Colab Notebook
A great way to get started with GES is to use the online [notebook](https://github.com/camenduru/ges-splatting-jupyter) produced by [camenduru](https://github.com/camenduru) with no requirments.
### Hardware Requirements
- CUDA-ready GPU with Compute Capability 7.0+
- 24 GB VRAM (to train to paper evaluation quality)
### Software Requirements
- Conda (recommended for easy setup)
- C++ Compiler for PyTorch extensions (we used Visual Studio 2019 for Windows)
- CUDA SDK 11 for PyTorch extensions, install *after* Visual Studio (we used 11.8, **known issues with 11.6**)
- C++ Compiler and CUDA SDK must be compatible
### Setup
Our default, provided install method is based on Conda package and environment management:
```shell
SET DISTUTILS_USE_SDK=1 # Windows only
conda env create --file environment.yml
conda activate ges
```
Please note that this process assumes that you have CUDA SDK **11** installed, not **12**. For modifications, see below. You can also use the same conda environment of the [Gaussian Splatting repo](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/).
## Running
Download the datasets from the [original repository](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/), and place them in the `tandt_db` and `nerf_360` directories.
For example, let's assume you download `tandt_db` dataset, please run following to reproduce Gaussian splatting results, but under the setup of GES.
```
python train_gaussian.py -s ./tandt_db/tandt/train -m ./outputs/train --eval
```
And run following to reproduce our main GES results with default settings.
```
python train_ges.py -s ./tandt_db/tandt/train -m ./outputs/train --eval
```
To reproduce all the results in our paper, prepare your datasets according to the script, and then run it:
```
bash ges_full_eval.sh # for our GES implementation
```
The detailed numerical results of GES (per scene) can be found in [here](https://github.com/ajhamdi/ges-splatting/blob/main/assets/GES_results_data.xlsx)
## Numerical Simulation for Generlized Exponential Function (GEF)
Check the notebook `simulation.ipynb` [here](https://github.com/ajhamdi/ges-splatting/blob/main/simulation.ipynb) for the numerical simulation of the Generalized Exponential Function (GEF), that GES is based on, as well as additional visualizations from the paper.
## Cite
If you find our work useful in your research, please consider citing:
```bibtex
@InProceedings{Hamdi_2024_CVPR,
author = {Hamdi, Abdullah and Melas-Kyriazi, Luke and Mai, Jinjie and Qian, Guocheng and Liu, Ruoshi and Vondrick, Carl and Ghanem, Bernard and Vedaldi, Andrea},
title = {GES : Generalized Exponential Splatting for Efficient Radiance Field Rendering},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2024},
pages = {19812-19822}
}
}
```