https://github.com/yangjiheng/3DGS_and_Beyond_Docs
This is a collective repository for all 3DGS related progresses in research and industry world
https://github.com/yangjiheng/3DGS_and_Beyond_Docs
Last synced: 10 months ago
JSON representation
This is a collective repository for all 3DGS related progresses in research and industry world
- Host: GitHub
- URL: https://github.com/yangjiheng/3DGS_and_Beyond_Docs
- Owner: yangjiheng
- License: mit
- Created: 2024-06-06T17:40:58.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2025-01-19T14:32:47.000Z (over 1 year ago)
- Last Synced: 2025-03-08T11:15:20.251Z (over 1 year ago)
- Size: 823 KB
- Stars: 575
- Watchers: 22
- Forks: 27
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- World-Simulator - 3DGS and Beyond Docs
README
#
`3DGS and Beyond Docs`
This is a collection of documents and topics NeRF/3DGS & Beyond channel accumulated, as well as papers in literaure. Since there are lots of papers out there, so we split them into two seperate repositories: [NeRF and Beyond Docs](https://github.com/yangjiheng/nerf_and_beyond_docs) and [3DGS and Beyond Docs](https://github.com/yangjiheng/3DGS_and_Beyond_Docs). Please choose accordingly recarding to your preference.
Some papers we discussed in the group, will be added to the back of the paper with a **Notes** link. You can follow the link to check whether there is topic you are interested in. If not, welcome to join us and ask the question to the crowd. The mighty community might have your answers.
We are actively maintaining this page trying to stay up-to-date and gather important works in a daily basis. We would also like to put as many notes as possible to some works, trying to make it easier to catch up.
Please feel free to join us on WeChat group or start a discussion topic here.
## NeRF/3DGS Book
I have recently published a book with PHEI(Publishing House of Electronics Industry) on NeRF/3DGS. This would not have been possible without the help of the whole 3D vision community. It is now available on jd.com ([Checkout here](https://item.jd.com/10110615626932.html)) and it should be suitable as a reference handbook for NeRF/3DGS beginners or engineers in related areas. I sincerely hope the book can be helpful in any perspective.
For those of you who have already purchased the book, all references can be downloaded [HERE](./assets/book_references.pdf). If you experience any issue reading the book or have any suggestions to improve it, please contact me through my email address: `jiheng.yang@gmail.com`, or directly concact me on WeChat: `jiheng_yang`. I'm looking forward to talk to anyone reaching out to me, thanks in advance.
## How to join us
For now, you can join us in the following ways
* [Bilibili Channel](https://space.bilibili.com/455056488) where we post near daily updates (primarily) on NeRF.
* WeChat group, due to the limitation of WeChat group, you can add my personal account: `jiheng_yang`, and I will add you to the chat groups.
* If you want to view this from a timeline perspective, please refer to this [ProcessOn Diagram](https://www.processon.com/view/link/643f8907f1144c215788f3e2)
* If you think something is not correct or you think we could do better in some way, please write to us through all possible channels or drop an issue. All suggestions are appreciated!
* For other discussed techniques that's related to 3D reconstruction and NeRF, please refer to [link](./NeRF_Related_Tech.md), we are constantly trying to add more resource to this document.
## NeRF Progresses
For NeRF related progress, you can refer to [NeRF and Beyond Docs](https://github.com/yangjiheng/nerf_and_beyond_docs)
Table of Content
- [`3DGS and Beyond Docs`](#3dgs-and-beyond-docs)
- [NeRF/3DGS Book](#nerf3dgs-book)
- [How to join us](#how-to-join-us)
- [NeRF Progresses](#nerf-progresses)
- [3DGS Original Paper](#3dgs-original-paper)
- [3DGS Surveys](#3dgs-surveys)
- [3DGS Frameworks](#3dgs-frameworks)
- [3DGS Profiling](#3dgs-profiling)
- [3DGS Distributed Training](#3dgs-distributed-training)
- [3DGS Quality Enhancement](#3dgs-quality-enhancement)
- [3DGS Quality Assessment](#3dgs-quality-assessment)
- [3DGS with Lower Memory Footprint](#3dgs-with-lower-memory-footprint)
- [3DGS with Ray Tracing](#3dgs-with-ray-tracing)
- [3DGS Acceleration](#3dgs-acceleration)
- [3DGS Geometry Reconstruction](#3dgs-geometry-reconstruction)
- [3DGS+Mesh For Reconstruction](#3dgsmesh-for-reconstruction)
- [3DGS Based Dynamic Scene](#3dgs-based-dynamic-scene)
- [3DGS + Depth](#3dgs--depth)
- [3DGS Based Depth Estimation](#3dgs-based-depth-estimation)
- [3DGS Few-shot Reconstruction](#3dgs-few-shot-reconstruction)
- [3DGS Weak Camera Pose](#3dgs-weak-camera-pose)
- [3DGS Object Pose Estimation/Tracking/Detection](#3dgs-object-pose-estimationtrackingdetection)
- [3DGS-NeRF Transfer](#3dgs-nerf-transfer)
- [3DGS Generalization](#3dgs-generalization)
- [Generalizable 3DGS with Feed-forward Networks](#generalizable-3dgs-with-feed-forward-networks)
- [3DGS Indoor Scene Reconstruction](#3dgs-indoor-scene-reconstruction)
- [3DGS Based Wild Scene Reconstruction](#3dgs-based-wild-scene-reconstruction)
- [3DGS Based Large Scene Reconstruction](#3dgs-based-large-scene-reconstruction)
- [3DGS Autonomous Driving](#3dgs-autonomous-driving)
- [3DGS Based Occupancy Prediction](#3dgs-based-occupancy-prediction)
- [3DGS Based on Diffusion](#3dgs-based-on-diffusion)
- [3DGS Based AIGC](#3dgs-based-aigc)
- [3DGS Model Compression](#3dgs-model-compression)
- [3DGS Model Compression Surveys](#3dgs-model-compression-surveys)
- [3DGS Model Compression Progresses](#3dgs-model-compression-progresses)
- [3DGS Streaming](#3dgs-streaming)
- [3DGS Based Relighting](#3dgs-based-relighting)
- [3DGS Robotics](#3dgs-robotics)
- [3DGS Robotics Surveys](#3dgs-robotics-surveys)
- [3DGS Robotics Progresses](#3dgs-robotics-progresses)
- [3DGS Avatar Generation](#3dgs-avatar-generation)
- [3DGS Avatar Generation Survey](#3dgs-avatar-generation-survey)
- [3DGS Avatar Generation Progresses](#3dgs-avatar-generation-progresses)
- [3DGS Clothes/Garment](#3dgs-clothesgarment)
- [3DGS Scene Editing and Animation](#3dgs-scene-editing-and-animation)
- [3D Scene Editing Surveys](#3d-scene-editing-surveys)
- [3D Scene Editing Progresses](#3d-scene-editing-progresses)
- [3DGS Stylization](#3dgs-stylization)
- [3DGS Based Video Editing](#3dgs-based-video-editing)
- [3DGS for Computer Graphics](#3dgs-for-computer-graphics)
- [3DGS Based Scene Understanding](#3dgs-based-scene-understanding)
- [3DGS based Segmentation](#3dgs-based-segmentation)
- [3DGS + Specular](#3dgs--specular)
- [3DGS Based SLAM](#3dgs-based-slam)
- [3DGS SLAM Surveys and Benchmarks](#3dgs-slam-surveys-and-benchmarks)
- [3DGS SLAM Progresses](#3dgs-slam-progresses)
- [3DGS Based 3D Point Tracking](#3dgs-based-3d-point-tracking)
- [3DGS Based Inverse Rendering](#3dgs-based-inverse-rendering)
- [3DGS Imaging Tasks](#3dgs-imaging-tasks)
- [3DGS for Reflective and Transparent Objects](#3dgs-for-reflective-and-transparent-objects)
- [3DGS Superresolution](#3dgs-superresolution)
- [3DGS with/for Point Cloud](#3dgs-withfor-point-cloud)
- [3DGS for CV Tasks](#3dgs-for-cv-tasks)
- [3DGS with Hardware](#3dgs-with-hardware)
- [3DGS Applications](#3dgs-applications)
- [3DGS Application in XR/AR](#3dgs-application-in-xrar)
- [3DGS Application in Animal Reconstruction](#3dgs-application-in-animal-reconstruction)
- [3DGS Application in Medical Imaging](#3dgs-application-in-medical-imaging)
- [3DGS Application in Underwater Scenario](#3dgs-application-in-underwater-scenario)
- [3DGS Application in Agriculture/Forestry Scenario](#3dgs-application-in-agricultureforestry-scenario)
- [3DGS Application in other Areas](#3dgs-application-in-other-areas)
- [3DGS Artifact Detection](#3dgs-artifact-detection)
- [3DGS Copyright/Safety](#3dgs-copyrightsafety)
- [3DGS Applications in UAV/MAV](#3dgs-applications-in-uavmav)
- [3DGS Applications in Satellite Images](#3dgs-applications-in-satellite-images)
- [3DGS Network Applications](#3dgs-network-applications)
- [3DGS for Acoustic](#3dgs-for-acoustic)
- [3DGS with Panorama](#3dgs-with-panorama)
- [3DGS with Thermal](#3dgs-with-thermal)
- [3DGS with Fisheye Camera](#3dgs-with-fisheye-camera)
- [3DGS with Compressive Sensing](#3dgs-with-compressive-sensing)
- [Other NVS Methods](#other-nvs-methods)
- [Other Upstream work(Occasionally Came Across)](#other-upstream-workoccasionally-came-across)
- [Other Surveys](#other-surveys)
- [Contributors](#contributors)
- [License](#license)
## 3DGS Original Paper
:fire:**3D Gaussian Splatting for Real-Time Radiance Field Rendering**
*Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis*
ACM ToG 2023, 8 August, 2023
Abstract
The emergence of 3D Gaussian Splatting (3DGS) has greatly accelerated the rendering speed of novel view synthesis. Unlike neural implicit representations like Neural Radiance Fields (NeRF) that represent a 3D scene with position and viewpoint-conditioned neural networks, 3D Gaussian Splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images. Apart from the fast rendering speed, the explicit representation of 3D Gaussian Splatting facilitates editing tasks like dynamic reconstruction, geometry editing, and physical simulation. Considering the rapid change and growing number of works in this field, we present a literature review of recent 3D Gaussian Splatting methods, which can be roughly classified into 3D reconstruction, 3D editing, and other downstream applications by functionality. Traditional point-based rendering methods and the rendering formulation of 3D Gaussian Splatting are also illustrated for a better understanding of this technique. This survey aims to help beginners get into this field quickly and provide experienced researchers with a comprehensive overview, which can stimulate the future development of the 3D Gaussian Splatting representation.
[[arXiv](https://arxiv.org/abs/2308.04079)] [[Project](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/)] [[Github](https://github.com/graphdeco-inria/gaussian-splatting)]
## 3DGS Surveys
**A Survey on 3D Gaussian Splatting**
*Guikun Chen, Wenguan Wang*
arXiv preprint, 8 Jan 2024
[[arXiv](https://arxiv.org/abs/2401.03890)]
**3D Gaussian as a New Vision Era: A Survey**
*Ben Fei, Jingyi Xu, Rui Zhang, Qingyuan Zhou, Weidong Yang, Ying He*
arXiv preprint, 11 Feb 2024
[[arXiv](https://arxiv.org/abs/2402.07181)]
:fire:**Recent Advances in 3D Gaussian Splatting**
*Tong Wu, Yu-Jie Yuan, Ling-Xiao Zhang, Jie Yang, Yan-Pei Cao, Ling-Qi Yan, Lin Gao*
arXiv preprint, 17 Mar 2024
Abstract
The emergence of 3D Gaussian Splatting (3DGS) has greatly accelerated the rendering speed of novel view synthesis. Unlike neural implicit representations like Neural Radiance Fields (NeRF) that represent a 3D scene with position and viewpoint-conditioned neural networks, 3D Gaussian Splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images. Apart from the fast rendering speed, the explicit representation of 3D Gaussian Splatting facilitates editing tasks like dynamic reconstruction, geometry editing, and physical simulation. Considering the rapid change and growing number of works in this field, we present a literature review of recent 3D Gaussian Splatting methods, which can be roughly classified into 3D reconstruction, 3D editing, and other downstream applications by functionality. Traditional point-based rendering methods and the rendering formulation of 3D Gaussian Splatting are also illustrated for a better understanding of this technique. This survey aims to help beginners get into this field quickly and provide experienced researchers with a comprehensive overview, which can stimulate the future development of the 3D Gaussian Splatting representation.
[[arXiv](https://arxiv.org/abs/2403.11134)]
**Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Review**
*Anurag Dalal, Daniel Hagen, Kjell G. Robbersmyr, Kristian Muri Knausgård*
arXiv preprint, 6 May 2024
[[arXiv](https://arxiv.org/abs/2405.03417)]
**Survey on Fundamental Deep Learning 3D Reconstruction Techniques**
*Yonge Bai, LikHang Wong, TszYin Twan*
arXiv preprint, 11 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.08137)]
**3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities**
*Yanqi Bao, Tianyu Ding, Jing Huo, Yaoli Liu, Yuxin Li, Wenbin Li, Yang Gao, Jiebo Luo*
arXiv preprint, 24 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.17418)]
**3D Representation Methods: A Survey**
*Zhengren Wang*
arXiv preprint, 9 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.06475)]
## 3DGS Frameworks
:fire:**GauStudio: A Modular Framework for 3D Gaussian Splatting and Beyond**
*Chongjie Ye, Yinyu Nie, Jiahao Chang, Yuantao Chen, Yihao Zhi, Xiaoguang Han*
arXiv preprint, 28 Mar 2024
Abstract
We present GauStudio, a novel modular framework for modeling 3D Gaussian Splatting (3DGS) to provide standardized, plug-and-play components for users to easily customize and implement a 3DGS pipeline. Supported by our framework, we propose a hybrid Gaussian representation with foreground and skyball background models. Experiments demonstrate this representation reduces artifacts in unbounded outdoor scenes and improves novel view synthesis. Finally, we propose Gaussian Splatting Surface Reconstruction (GauS), a novel render-then-fuse approach for high-fidelity mesh reconstruction from 3DGS inputs without fine-tuning. Overall, our GauStudio framework, hybrid representation, and GauS approach enhance 3DGS modeling and rendering capabilities, enabling higher-quality novel view synthesis and surface reconstruction.
[[arXiv](https://arxiv.org/abs/2403.19632)] [[Code](https://github.com/GAP-LAB-CUHK-SZ/gaustudio)]
**gsplat: An Open-Source Library for Gaussian Splatting**
*Vickie Ye, Ruilong Li, Justin Kerr, Matias Turkulainen, Brent Yi, Zhuoyang Pan, Otto Seiskari, Jianbo Ye, Jeffrey Hu, Matthew Tancik, Angjoo Kanazawa*
arXiv preprint, 10 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.06765)]
**SuperSplat - 3D Gaussian Splat Editor**
PlayCanvas
[[Code](https://github.com/playcanvas/supersplat)]
## 3DGS Profiling
**NerfBaselines: Consistent and Reproducible Evaluation of Novel View Synthesis Methods**
*Jonas Kulhanek, Torsten Sattler*
arXiv preprint, 25 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.17345)] [[Project](https://jkulhanek.com/nerfbaselines/)]
## 3DGS Distributed Training
**RetinaGS: Scalable Training for Dense Scene Rendering with Billion-Scale 3D Gaussians**
*Bingling Li, Shengyi Chen, Luchao Wang, Kaimin He, Sijie Yan, Yuanjun Xiong*
arXiv preprint, 17 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.11836)]
**On Scaling Up 3D Gaussian Splatting Training**
*Hexu Zhao, Haoyang Weng, Daohan Lu, Ang Li, Jinyang Li, Aurojit Panda, Saining Xie*
arXiv preprint, 26 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.18533)] [[Project](https://daohanlu.github.io/scaling-up-3dgs)] [[Code](https://github.com/nyu-systems/Grendel-GS)]
## 3DGS Quality Enhancement
:fire:**Mip-Splatting: Alias-free 3D Gaussian Splatting**
*Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, Andreas Geiger*
arXiv preprint, 27 Nov 2023
Abstract
Recently, 3D Gaussian Splatting has demonstrated impressive novel view synthesis results, reaching high fidelity and efficiency. However, strong artifacts can be observed when changing the sampling rate, \eg, by changing focal length or camera distance. We find that the source for this phenomenon can be attributed to the lack of 3D frequency constraints and the usage of a 2D dilation filter. To address this problem, we introduce a 3D smoothing filter which constrains the size of the 3D Gaussian primitives based on the maximal sampling frequency induced by the input views, eliminating high-frequency artifacts when zooming in. Moreover, replacing 2D dilation with a 2D Mip filter, which simulates a 2D box filter, effectively mitigates aliasing and dilation issues. Our evaluation, including scenarios such a training on single-scale images and testing on multiple scales, validates the effectiveness of our approach.
[[arXiv](https://arxiv.org/abs/2311.16493)] [[Project](https://niujinshuchong.github.io/mip-splatting/)]
**Multi-Scale 3D Gaussian Splatting for Anti-Aliased Rendering**
*Zhiwen Yan, Weng Fei Low, Yu Chen, Gim Hee Lee*
arXiv preprint, 28 Nov 2023
[[arXiv](https://arxiv.org/abs/2311.17089)] [[Project](https://jokeryan.github.io/projects/ms-gs/)] [[Code](https://github.com/JokerYan/MS-GS/tree/main)] [[Video](https://youtu.be/q77p5nKnpJw)]
:fire:**Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering**
*Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, Bo Dai*
arXiv preprint, 30 Nov 2023
Abstract
Neural rendering methods have significantly advanced photo-realistic 3D scene rendering in various academic and industrial applications. The recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed combining the benefits of both primitive-based representations and volumetric representations. However, it often leads to heavily redundant Gaussians that try to fit every training view, neglecting the underlying scene geometry. Consequently, the resulting model becomes less robust to significant view changes, texture-less area and lighting effects. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians, and predicts their attributes on-the-fly based on viewing direction and distance within the view frustum. Anchor growing and pruning strategies are developed based on the importance of neural Gaussians to reliably improve the scene coverage. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering. We also demonstrates an enhanced capability to accommodate scenes with varying levels-of-detail and view-dependent observations, without sacrificing the rendering speed.
[[arXiv](https://arxiv.org/abs/2312.00109)] [[Project](https://city-super.github.io/scaffold-gs/)]
**Gaussian Splitting Algorithm with Color and Opacity Depended on Viewing Direction**
*Dawid Malarz, Weronika Smolak, Jacek Tabor, Sławomir Tadeja, Przemysław Spurek*
arXiv preprint, 21 Dec 2023
[[arXiv](https://arxiv.org/abs/2312.13729)]
:fire:**TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering**
*Linus Franke, Darius Rückert, Laura Fink, Marc Stamminger*
Eurographics 2024, 11 Jan 2024
Abstract
Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al. 2023] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [Rückert et al. 2022] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud.
In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions.
Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage.
The project page is located at: this https URL
[[arXiv](https://arxiv.org/abs/2401.06003)] [[Project](https://lfranke.github.io/trips/)] [[Code](https://github.com/lfranke/trips)] [[Video](https://www.youtube.com/watch?v=Nw4A1tIcErQ&feature=youtu.be)]
**On the Error Analysis of 3D Gaussian Splatting and an Optimal Projection Strategy**
*Letian Huang, Jiayang Bai, Jie Guo, Yanwen Guo*
ECCV 2024, 1 Feb 2024
[[arXiv](https://arxiv.org/abs/2402.00752)] [[Project](https://letianhuang.github.io/op43dgs/)] [[Code](https://github.com/LetianHuang/op43dgs)]
**FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization**
*Jiahui Zhang, Fangneng Zhan, Muyu Xu, Shijian Lu, Eric Xing*
CVPR 2024, 11 Mar 2024
[[arXiv](https://arxiv.org/abs/2403.06908)] [[Project](https://rogeraigc.github.io/FreGS-Page/)]
:fire:**Analytic-Splatting: Anti-Aliased 3D Gaussian Splatting via Analytic Integration**
*Zhihao Liang, Qi Zhang, Wenbo Hu, Ying Feng, Lei Zhu, Kui Jia*
ECCV 2024, 16 Mar 2024
Abstract
The 3D Gaussian Splatting (3DGS) gained its popularity recently by combining the advantages of both primitive-based and volumetric 3D representations, resulting in improved quality and efficiency for 3D scene rendering. However, 3DGS is not alias-free, and its rendering at varying resolutions could produce severe blurring or jaggies. This is because 3DGS treats each pixel as an isolated, single point rather than as an area, causing insensitivity to changes in the footprints of pixels. Consequently, this discrete sampling scheme inevitably results in aliasing, owing to the restricted sampling bandwidth. In this paper, we derive an analytical solution to address this issue. More specifically, we use a conditioned logistic function as the analytic approximation of the cumulative distribution function (CDF) in a one-dimensional Gaussian signal and calculate the Gaussian integral by subtracting the CDFs. We then introduce this approximation in the two-dimensional pixel shading, and present Analytic-Splatting, which analytically approximates the Gaussian integral within the 2D-pixel window area to better capture the intensity response of each pixel. Moreover, we use the approximated response of the pixel window integral area to participate in the transmittance calculation of volume rendering, making Analytic-Splatting sensitive to the changes in pixel footprint at different resolutions. Experiments on various datasets validate that our approach has better anti-aliasing capability that gives more details and better fidelity.
[[arXiv](https://arxiv.org/abs/2403.11056)] [[Project](https://lzhnb.github.io/project-pages/analytic-splatting/)] [[Code](https://github.com/lzhnb/Analytic-Splatting)]
:fire:**Mini-Splatting: Representing Scenes with a Constrained Number of Gaussians**
*Guangchi Fang, Bing Wang*
ECCV 2024, 21 Mar 2024
Abstract
In this study, we explore the challenge of efficiently representing scenes with a constrained number of Gaussians. Our analysis shifts from traditional graphics and 2D computer vision to the perspective of point clouds, highlighting the inefficient spatial distribution of Gaussian representation as a key limitation in model performance. To address this, we introduce strategies for densification including blur split and depth reinitialization, and simplification through intersection preserving and sampling. These techniques reorganize the spatial positions of the Gaussians, resulting in significant improvements across various datasets and benchmarks in terms of rendering quality, resource consumption, and storage compression. Our Mini-Splatting integrates seamlessly with the original rasterization pipeline, providing a strong baseline for future research in Gaussian-Splatting-based works. \href{this https URL}{Code is available}.
[[arXiv](https://arxiv.org/abs/2403.14166)] [[Code](https://github.com/fatPeter/mini-splatting)]
:fire:**Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splatting**
*Zheng Zhang, Wenbo Hu, Yixing Lao, Tong He, Hengshuang Zhao*
ECCV 2024, 22 Mar 2024
Abstract
3D Gaussian Splatting (3DGS) has demonstrated impressive novel view synthesis results while advancing real-time rendering performance. However, it relies heavily on the quality of the initial point cloud, resulting in blurring and needle-like artifacts in areas with insufficient initializing points. This is mainly attributed to the point cloud growth condition in 3DGS that only considers the average gradient magnitude of points from observable views, thereby failing to grow for large Gaussians that are observable for many viewpoints while many of them are only covered in the boundaries. To this end, we propose a novel method, named Pixel-GS, to take into account the number of pixels covered by the Gaussian in each view during the computation of the growth condition. We regard the covered pixel numbers as the weights to dynamically average the gradients from different views, such that the growth of large Gaussians can be prompted. As a result, points within the areas with insufficient initializing points can be grown more effectively, leading to a more accurate and detailed reconstruction. In addition, we propose a simple yet effective strategy to scale the gradient field according to the distance to the camera, to suppress the growth of floaters near the camera. Extensive experiments both qualitatively and quantitatively demonstrate that our method achieves state-of-the-art rendering quality while maintaining real-time rendering speed, on the challenging Mip-NeRF 360 and Tanks & Temples datasets.
[[arXiv](https://arxiv.org/abs/2403.15530)] [[Project](https://pixelgs.github.io/)] [[Code](https://github.com/zhengzhang01/Pixel-GS)]
:fire:**SA-GS: Scale-Adaptive Gaussian Splatting for Training-Free Anti-Aliasing**
*Xiaowei Song, Jv Zheng, Shiran Yuan, Huan-ang Gao, Jingwei Zhao, Xiang He, Weihao Gu, Hao Zhao*
arXiv preprint, 28 Mar 2024
Abstract
In this paper, we present a Scale-adaptive method for Anti-aliasing Gaussian Splatting (SA-GS). While the state-of-the-art method Mip-Splatting needs modifying the training procedure of Gaussian splatting, our method functions at test-time and is training-free. Specifically, SA-GS can be applied to any pretrained Gaussian splatting field as a plugin to significantly improve the field's anti-alising performance. The core technique is to apply 2D scale-adaptive filters to each Gaussian during test time. As pointed out by Mip-Splatting, observing Gaussians at different frequencies leads to mismatches between the Gaussian scales during training and testing. Mip-Splatting resolves this issue using 3D smoothing and 2D Mip filters, which are unfortunately not aware of testing frequency. In this work, we show that a 2D scale-adaptive filter that is informed of testing frequency can effectively match the Gaussian scale, thus making the Gaussian primitive distribution remain consistent across different testing frequencies. When scale inconsistency is eliminated, sampling rates smaller than the scene frequency result in conventional jaggedness, and we propose to integrate the projected 2D Gaussian within each pixel during testing. This integration is actually a limiting case of super-sampling, which significantly improves anti-aliasing performance over vanilla Gaussian Splatting. Through extensive experiments using various settings and both bounded and unbounded scenes, we show SA-GS performs comparably with or better than Mip-Splatting. Note that super-sampling and integration are only effective when our scale-adaptive filtering is activated. Our codes, data and models are available at this https URL.
[[arXiv](https://arxiv.org/abs/2403.19615)] [[Project](https://kevinsong729.github.io/project-pages/SA-GS/)] [[Code](https://github.com/zsy1987/SA-GS/)]
**Robust Gaussian Splatting**
*François Darmon, Lorenzo Porzi, Samuel Rota-Bulò, Peter Kontschieder*
arXiv preprint, 5 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.04211)]
**Revising Densification in Gaussian Splatting**
*Samuel Rota Bulò, Lorenzo Porzi, Peter Kontschieder*
arXiv preprint, 9 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.06109)]
**EGGS: Edge Guided Gaussian Splatting for Radiance Fields**
*Yuanhao Gong*
arXiv preprint, 14 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.09105)]
:fire:**3D Gaussian Splatting as Markov Chain Monte Carlo**
*Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Jeff Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi*
arXiv preprint, 15 Apr 2024
Abstract
While 3D Gaussian Splatting has recently become popular for neural rendering, current methods rely on carefully engineered cloning and splitting strategies for placing Gaussians, which can lead to poor-quality renderings, and reliance on a good initialization. In this work, we rethink the set of 3D Gaussians as a random sample drawn from an underlying probability distribution describing the physical representation of the scene-in other words, Markov Chain Monte Carlo (MCMC) samples. Under this view, we show that the 3D Gaussian updates can be converted as Stochastic Gradient Langevin Dynamics (SGLD) updates by simply introducing noise. We then rewrite the densification and pruning strategies in 3D Gaussian Splatting as simply a deterministic state transition of MCMC samples, removing these heuristics from the framework. To do so, we revise the 'cloning' of Gaussians into a relocalization scheme that approximately preserves sample probability. To encourage efficient use of Gaussians, we introduce a regularizer that promotes the removal of unused Gaussians. On various standard evaluation scenes, we show that our method provides improved rendering quality, easy control over the number of Gaussians, and robustness to initialization.
[[arXiv](https://arxiv.org/abs/2404.09591)] [[Project](https://ubc-vision.github.io/3dgs-mcmc/)] [[Code](https://github.com/ubc-vision/3dgs-mcmc)]
**AbsGS: Recovering Fine Details for 3D Gaussian Splatting**
*Zongxin Ye, Wenyu Li, Sidun Liu, Peng Qiao, Yong Dou*
arXiv preprint, 16 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.10484)] [[Project](https://ty424.github.io/AbsGS.github.io/)] [[Code](https://github.com/TY424/AbsGS)]
**Gaussian Splatting Decoder for 3D-aware Generative Adversarial Networks**
*Florian Barthel, Arian Beckmann, Wieland Morgenstern, Anna Hilsmann, Peter Eisert*
CVPRW 2024, 16 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.10625)]
**Does Gaussian Splatting need SFM Initialization?**
*Yalda Foroutan, Daniel Rebain, Kwang Moo Yi, Andrea Tagliasacchi*
arXiv preprint, 18 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.12547)] [[Project](https://theialab.github.io/nerf-3dgs/)]
**Bootstrap 3D Reconstructed Scenes from 3D Gaussian Splatting**
*Yifei Gao, Jie Ou, Lei Wang, Jun Cheng*
arXiv preprint, 29 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.18669)]
**Feature Splatting for Better Novel View Synthesis with Low Overlap**
*T. Berriel Martins, Javier Civera*
arXiv preprint, 24 May 2024
[[arXiv](https://arxiv.org/abs/2405.15518)] [[Code](https://github.com/tberriel/FeatSplat)]
**NegGS: Negative Gaussian Splatting**
*Artur Kasymov, Bartosz Czekaj, Marcin Mazur, Przemysław Spurek*
arXiv preprint, 28 May 2024
[[arXiv](https://arxiv.org/abs/2405.18163)]
**3D-HGS: 3D Half-Gaussian Splatting**
*Haolin Li, Jinyang Liu, Mario Sznaier, Octavia Camps*
arXiv preprint, 4 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.02720)]
**Gaussian Splatting with Localized Points Management**
*Haosen Yang, Chenhao Zhang, Wenqing Wang, Marco Volino, Adrian Hilton, Li Zhang, Xiatian Zhu*
arXiv preprint, 6 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.04251)] [[Code](https://github.com/Surrey-UP-Lab/GS-LPM)]
**Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting**
*Junha Hyung, Susung Hong, Sungwon Hwang, Jaeseong Lee, Jaegul Choo, Jin-Hwa Kim*
arXiv preprint, 17 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.11672)] [[Project](https://junhahyung.github.io/erankgs.github.io/)]
**Taming 3DGS: High-Quality Radiance Fields with Limited Resources**
*Saswat Subhajyoti Mallick, Rahul Goel, Bernhard Kerbl, Francisco Vicente Carrasco, Markus Steinberger, Fernando De La Torre*
arXiv preprint, 21 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.15643)]
**SpotlessSplats: Ignoring Distractors in 3D Gaussian Splatting**
*Sara Sabour, Lily Goli, George Kopanas, Mark Matthews, Dmitry Lagun, Leonidas Guibas, Alec Jacobson, David J. Fleet, Andrea Tagliasacchi*
arXiv preprint, 28 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.20055)]
**Textured-GS: Gaussian Splatting with Spatially Defined Color and Opacity**
*Zhentao Huang, Minglun Gong*
arXiv preprint, 13 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.09733)]
**Splatfacto-W: A Nerfstudio Implementation of Gaussian Splatting for Unconstrained Photo Collections**
*Congrong Xu, Justin Kerr, Angjoo Kanazawa*
arXiv preprint, 17 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.12306)] [[Code](https://github.com/KevinXu02/splatfacto-w)]
**MVG-Splatting: Multi-View Guided Gaussian Splatting with Adaptive Quantile-Based Geometric Consistency Densification**
*Zhuoxiao Li, Shanliang Yao, Yijie Chu, Angel F. Garcia-Fernandez, Yong Yue, Eng Gee Lim, Xiaohui Zhu*
arXiv preprint, 16 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.11840)] [[Project](https://mvgsplatting.github.io/)]
**3iGS: Factorised Tensorial Illumination for 3D Gaussian Splatting**
*Zhe Jun Tang, Tat-Jen Cham*
ECCV 2024, 7 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.03753)]
**Mipmap-GS: Let Gaussians Deform with Scale-specific Mipmap for Anti-aliasing Rendering**
*Jiameng Li, Yue Shi, Jiezhang Cao, Bingbing Ni, Wenjun Zhang, Kai Zhang, Luc Van Gool*
arXiv preprint, 12 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.06286)]
**FLoD: Integrating Flexible Level of Detail into 3D Gaussian Splatting for Customizable Rendering**
*Yunji Seo, Young Sun Choi, Hyun Seung Son, Youngjung Uh*
arXiv preprint, 23 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.12894)] [[Project](https://3dgs-flod.github.io/flod.github.io/)] [[Code](https://github.com/3DGS-FLoD/flod)]
**Robust 3D Gaussian Splatting for Novel View Synthesis in Presence of Distractors**
*Paul Ungermann, Armin Ettenhofer, Matthias Nießner, Barbara Roessle*
GCPR 2024, 21 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.11697)] [[Project](https://paulungermann.github.io/Robust3DGaussians/)] [[Video](https://www.youtube.com/watch?v=P9unyR7yK3E)] [[Code](https://github.com/paulungermann/Robust3DGaussians)]
**Implicit Gaussian Splatting with Efficient Multi-Level Tri-Plane Representation**
*Minye Wu, Tinne Tuytelaars*
arXiv preprint, 19 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.10041)]
**Correspondence-Guided SfM-Free 3D Gaussian Splatting for NVS**
*Wei Sun, Xiaosong Zhang, Fang Wan, Yanzhao Zhou, Yuan Li, Qixiang Ye, Jianbin Jiao*
arXiv preprint, 16 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.08723)]
**Sources of Uncertainty in 3D Scene Reconstruction**
*Marcus Klasson, Riccardo Mereu, Juho Kannala, Arno Solin*
ECCV 2024, 10 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.06407)] [[Project](https://aaltoml.github.io/uncertainty-nerf-gs/)] [[Code](https://github.com/AaltoML/uncertainty-nerf-gs)]
**Spectral-GS: Taming 3D Gaussian Splatting with Spectral Entropy**
*Letian Huang, Jie Guo, Jialin Dan, Ruoyu Fu, Shujie Wang, Yuanqi Li, Yanwen Guo*
arXiv preprint, 19 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.12771)]
**GStex: Per-Primitive Texturing of 2D Gaussian Splatting for Decoupled Appearance and Geometry Modeling**
*Victor Rong, Jingxiang Chen, Sherwin Bahmani, Kiriakos N. Kutulakos, David B. Lindell*
arXiv preprint, 19 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.12954)] [[Project](https://lessvrong.com/cs/gstex)]
**Frequency-based View Selection in Gaussian Splatting Reconstruction**
*Monica M.Q. Li, Pierre-Yves Lajoie, Giovanni Beltrame*
arXiv preprint, 24 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.16470)]
**MVGS: Multi-view-regulated Gaussian Splatting for Novel View Synthesis**
*Xiaobiao Du, Yida Wang, Xin Yu*
arXiv preprint, 2 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.02103)] [[Project](https://xiaobiaodu.github.io/mvgs-project/)] [[Code](https://github.com/xiaobiaodu/MVGS)]
**6DGS: Enhanced Direction-Aware Gaussian Splatting for Volumetric Rendering**
*Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri, Terrence Chen, Ziyan Wu*
arXiv preprint, 7 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.04974)] [[Project](https://gaozhongpai.github.io/6dgs/)]
**PH-Dropout: Prctical Epistemic Uncertainty Quantification for View Synthesis**
*Chuanhao Sun, Thanos Triantafyllou, Anthos Makris, Maja Drmač, Kai Xu, Luo Mai, Mahesh K. Marina*
arXiv preprint, 7 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.05468)]
**Variational Bayes Gaussian Splatting**
*Toon Van de Maele, Ozan Catal, Alexander Tschantz, Christopher L. Buckley, Tim Verbelen*
arXiv preprint, 4 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.03592)]
**VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Points**
*Linus Franke, Laura Fink, Marc Stamminger*
arXiv preprint, 23 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.17932)] [[Project](https://lfranke.github.io/vr_splatting/)]
**ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings**
*Suyoung Lee, Jaeyoung Chung, Jaeyoo Huh, Kyoung Mu Lee*
arXiv preprint, 28 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.20686)] [[Code](https://github.com/esw0116/ODGS)]
**Projecting Gaussian Ellipsoids While Avoiding Affine Projection Approximation**
*Han Qi, Tao Cai, Xiyue Han*
arXiv preprint, 12 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.07579)]
**SplatFormer: Point Transformer for Robust 3D Gaussian Splatting**
*Yutong Chen, Marko Mihajlovic, Xiyi Chen, Yiming Wang, Sergey Prokudin, Siyu Tang*
arXiv preprint, 10 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.06390)] [[Project](https://sergeyprokudin.github.io/splatformer/)] [[Code](https://github.com/ChenYutongTHU/SplatFormer)]
**BillBoard Splatting (BBSplat): Learnable Textured Primitives for Novel View Synthesis**
*David Svitov, Pietro Morerio, Lourdes Agapito, Alessio Del Bue*
arXiv preprint, 13 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.08508)] [[Project](https://david-svitov.github.io/BBSplat_project_page/)] [[Video](https://youtu.be/uRM7WFo5vVg)] [[Code](https://github.com/david-svitov/BBSplat)]
**Mini-Splatting2: Building 360 Scenes within Minutes via Aggressive Gaussian Densification**
*Guangchi Fang, Bing Wang*
19 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.12788)]
**Beyond Gaussians: Fast and High-Fidelity 3D Splatting with Linear Kernels**
*Haodong Chen, Runnan Chen, Qiang Qu, Zhaoqing Wang, Tongliang Liu, Xiaoming Chen, Yuk Ying Chung*
19 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.12440)] [[Project](https://hche8927.github.io/3DLS/)]
**Textured Gaussians for Enhanced 3D Scene Appearance Modeling**
*Brian Chao, Hung-Yu Tseng, Lorenzo Porzi, Chen Gao, Tuotuo Li, Qinbo Li, Ayush Saraf, Jia-Bin Huang, Johannes Kopf, Gordon Wetzstein, Changil Kim*
27 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.18625)] [[Project](https://textured-gaussians.github.io/)]
**3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes**
*Jan Held, Renaud Vandeghen, Abdullah Hamdi, Adrien Deliege, Anthony Cioppa, Silvio Giancola, Andrea Vedaldi, Bernard Ghanem, Marc Van Droogenbroeck*
22 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.14974)] [[Project](https://convexsplatting.github.io/)] [[Video](https://www.youtube.com/watch?v=5N3OFHH7lbU)] [[Code](https://github.com/convexsplatting/convex-splatting)]
**Deformable Radial Kernel Splatting**
*Yi-Hua Huang, Ming-Xian Lin, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, Xiaojuan Qi*
16 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.11752)]
**Pushing Rendering Boundaries: Hard Gaussian Splatting**
*Qingshan Xu, Jiequan Cui, Xuanyu Yi, Yuxuan Wang, Yuan Zhou, Yew-Soon Ong, Hanwang Zhang*
6 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.04826)]
**ResGS: Residual Densification of 3D Gaussian for Efficient Detail Recovery**
*Yanzhe Lyu, Kai Cheng, Xin Kang, Xuejin Chen*
10 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.07494)]
**GS-ProCams: Gaussian Splatting-based Projector-Camera Systems**
*Qingyue Deng, Jijiang Li, Haibin Ling, Bingyao Huang*
16 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.11762)]
**GeoTexDensifier: Geometry-Texture-Aware Densification for High-Quality Photorealistic 3D Gaussian Splatting**
*Hanqing Jiang, Xiaojun Xiang, Han Sun, Hongjie Li, Liyang Zhou, Xiaoyu Zhang, Guofeng Zhang*
22 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.16809)]
**Topology-Aware 3D Gaussian Splatting: Leveraging Persistent Homology for Optimized Structural Integrity**
*Tianqi Shen, Shaohua Liu, Jiaqi Feng, Ziye Ma, Ning An*
21 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.16619)]
**EasySplat: View - Adaptive Learning makes 3D Gaussian Splatting Easy**
*Ao Gao, Luosong Guo, Tao Chen, Zhao Wang, Ying Tai, Jian Yang, Zhenyu Zhang*
2 Jan 2025
[[arXiv](https://arxiv.org/abs/2501.01003)]
## 3DGS Quality Assessment
**Evaluating Human Perception of Novel View Synthesis: Subjective Quality Assessment of GaussianSplatting and NeRF in Dynamic Scenes**
*Yuhang Zhang, Joshua Maraval, Zhengyu Zhang, Nicolas Ramin, Shishun Tian, Lu Zhang*
13 Jan 2025
[[arXiv](https://arxiv.org/abs/2501.08072)]
**NVS-SQA: Exploring Self-Supervised Quality Representation Learning for Neurally Synthesized Scenes without References**
*Qiang Qu, Yiran Shen, Xiaoming Chen, Yuk Ying Chung, Weidong Cai, Tongliang Liu*
11 Jan 2025
[[arXiv](https://arxiv.org/abs/2501.06488)]
## 3DGS with Lower Memory Footprint
:fire:**Spectrally Pruned Gaussian Fields with Neural Compensation**
*Runyi Yang, Zhenxin Zhu, Zhou Jiang, Baijun Ye, Xiaoxue Chen, Yifei Zhang, Yuantao Chen, Jian Zhao, Hao Zhao*
arXiv preprint, 1 May 2024
Abstract
Recently, 3D Gaussian Splatting, as a novel 3D representation, has garnered attention for its fast rendering speed and high rendering quality. However, this comes with high memory consumption, e.g., a well-trained Gaussian field may utilize three million Gaussian primitives and over 700 MB of memory. We credit this high memory footprint to the lack of consideration for the relationship between primitives. In this paper, we propose a memory-efficient Gaussian field named SUNDAE with spectral pruning and neural compensation. On one hand, we construct a graph on the set of Gaussian primitives to model their relationship and design a spectral down-sampling module to prune out primitives while preserving desired signals. On the other hand, to compensate for the quality loss of pruning Gaussians, we exploit a lightweight neural network head to mix splatted features, which effectively compensates for quality losses while capturing the relationship between primitives in its weights. We demonstrate the performance of SUNDAE with extensive results. For example, SUNDAE can achieve 26.80 PSNR at 145 FPS using 104 MB memory while the vanilla Gaussian splatting algorithm achieves 25.60 PSNR at 160 FPS using 523 MB memory, on the Mip-NeRF360 dataset. Codes are publicly available at this https URL.
[[arXiv](https://arxiv.org/abs/2405.00676)] [[Project](https://runyiyang.github.io/projects/SUNDAE/)] [[Code](https://github.com/RunyiYang/SUNDAE)]
**PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting**
*Alex Hanson, Allen Tu, Vasu Singla, Mayuka Jayawardhana, Matthias Zwicker, Tom Goldstein*
arXiv preprint, 14 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.10219)]
**Object-Centric 2D GaussianSplatting: Background Removal and Occlusion-Aware Pruning for Compact Object Models**
*Marcel Rogge, Didier Stricker*
ICPRAM 2025, 14 Jan 2025
[[arXiv](https://arxiv.org/abs/2501.08174)]
**MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting**
*Sangwoon Kwak, Joonsoo Kim, Jun Young Jeong, Won-Sik Cheong, Jihyong Oh, Munchurl Kim*
7 Jan 2025
[[arXiv](https://arxiv.org/abs/2501.03714)] [[Project](https://kaist-viclab.github.io/MoDecGS-site/)] [[Video](https://youtu.be/5L6gzc5-cw8)]
## 3DGS with Ray Tracing
**Don't Splat your Gaussians: Volumetric Ray-Traced Primitives for Modeling and Rendering Scattering and Emissive Media**
*Jorge Condor, Sebastien Speierer, Lukas Bode, Aljaz Bozic, Simon Green, Piotr Didyk, Adrian Jarabo*
arXiv preprint, 24 May 2024
[[arXiv](https://arxiv.org/abs/2405.15425)]
**Unified Gaussian Primitives for Scene Representation and Rendering**
*Yang Zhou, Songyin Wu, Ling-Qi Yan*
arXiv preprint, 14 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.09733)]
**3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes**
*Nicolas Moenne-Loccoz, Ashkan Mirzaei, Or Perel, Riccardo de Lutio, Janick Martinez Esturo, Gavriel State, Sanja Fidler, Nicholas Sharp, Zan Gojcic*
arXiv preprint, 9 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.07090)]
**RayGauss: Volumetric Gaussian-Based Ray Casting for Photorealistic Novel View Synthesis**
*Hugo Blanc, Jean-Emmanuel Deschaud, Alexis Paljic*
arXiv preprint, 6 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.03356)] [[Project](https://raygauss.github.io/)]
:fire:**EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis**
*Alexander Mai, Peter Hedman, George Kopanas, Dor Verbin, David Futschik, Qiangeng Xu, Falko Kuester, Jon Barron, Yinda Zhang*
arXiv preprint, 2 Oct 2024
Abstract
We present Exact Volumetric Ellipsoid Rendering (EVER), a method for real-time differentiable emission-only volume rendering. Unlike recent rasterization based approach by 3D Gaussian Splatting (3DGS), our primitive based representation allows for exact volume rendering, rather than alpha compositing 3D Gaussian billboards. As such, unlike 3DGS our formulation does not suffer from popping artifacts and view dependent density, but still achieves frame rates of ∼30 FPS at 720p on an NVIDIA RTX4090. Since our approach is built upon ray tracing it enables effects such as defocus blur and camera distortion (e.g. such as from fisheye cameras), which are difficult to achieve by rasterization. We show that our method is more accurate with fewer blending issues than 3DGS and follow-up work on view-consistent rendering, especially on the challenging large-scale scenes from the Zip-NeRF dataset where it achieves sharpest results among real-time techniques.
[[arXiv](https://arxiv.org/abs/2410.01804)] [[Project](https://half-potato.gitlab.io/posts/ever/)] [[Video](https://www.bilibili.com/video/BV1atxXeREWg/)]
## 3DGS Acceleration
**EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS**
*Sharath Girish, Kamal Gupta, Abhinav Shrivastava*
arXiv preprint, 7 Dec, 2023
[[arXiv](https://arxiv.org/abs/2312.04564)] [[Project](https://efficientgaussian.github.io/)] [[Code](https://github.com/Sharath-girish/efficientgaussian)]
:fire:**StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering**
*Lukas Radl, Michael Steiner, Mathias Parger, Alexander Weinrauch, Bernhard Kerbl, Markus Steinberger*
SIGGRAPH 2024, 1 Feb 2024
Abstract
Gaussian Splatting has emerged as a prominent model for constructing 3D representations from images across diverse domains. However, the efficiency of the 3D Gaussian Splatting rendering pipeline relies on several simplifications. Notably, reducing Gaussian to 2D splats with a single view-space depth introduces popping and blending artifacts during view rotation. Addressing this issue requires accurate per-pixel depth computation, yet a full per-pixel sort proves excessively costly compared to a global sort operation. In this paper, we present a novel hierarchical rasterization approach that systematically resorts and culls splats with minimal processing overhead. Our software rasterizer effectively eliminates popping artifacts and view inconsistencies, as demonstrated through both quantitative and qualitative measurements. Simultaneously, our method mitigates the potential for cheating view-dependent effects with popping, ensuring a more authentic representation. Despite the elimination of cheating, our approach achieves comparable quantitative results for test images, while increasing the consistency for novel view synthesis in motion. Due to its design, our hierarchical approach is only 4% slower on average than the original Gaussian Splatting. Notably, enforcing consistency enables a reduction in the number of Gaussians by approximately half with nearly identical quality and view-consistency. Consequently, rendering performance is nearly doubled, making our approach 1.6x faster than the original Gaussian Splatting, with a 50% reduction in memory requirements.
[[arXiv](https://arxiv.org/abs/2402.00525)] [[Project](https://r4dl.github.io/StopThePop/)] [[Code](https://github.com/r4dl/StopThePop)] [[Video](https://www.youtube.com/watch?v=EmcXtHYhigk)]
**GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering**
*Abdullah Hamdi, Luke Melas-Kyriazi, Guocheng Qian, Jinjie Mai, Ruoshi Liu, Carl Vondrick, Bernard Ghanem, Andrea Vedaldi*
CVPR 2024, 15 Feb 2024
[[arXiv](https://arxiv.org/abs/2402.10128)] [[Project](https://abdullahamdi.com/ges/)] [[Code](https://github.com/ajhamdi/ges-splatting)] [[Video](https://www.youtube.com/watch?v=edSvNy3roV8&feature=youtu.be)]
**OmniGS: Omnidirectional Gaussian Splatting for Fast Radiance Field Reconstruction using Omnidirectional Images**
*Longwei Li, Huajian Huang, Sai-Kit Yeung, Hui Cheng*
arXiv preprint, 4 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.03202)]
**Hash3D: Training-free Acceleration for 3D Generation**
*Xingyi Yang, Xinchao Wang*
arXiv preprint, 9 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.06091)] [[Project](https://adamdad.github.io/hash3D/)] [[Code](https://github.com/Adamdad/hash3D)]
**I3DGS: Improve 3D Gaussian Splatting from Multiple Dimensions**
*Jinwei Lin*
arXiv preprint, 10 May 2024
[[arXiv](https://arxiv.org/abs/2405.06408)]
**RTGS: Enabling Real-TimeGaussianSplatting on Mobile Devices Using Efficiency-Guided Pruning and Foveated Rendering**
*Weikai Lin, Yu Feng, Yuhao Zhu*
arXiv preprint, 29 Jun 2024
[[arXiv](https://arxiv.org/abs/2407.00435)] [[Code](https://github.com/horizon-research/Fov-3DGS)]
**3DGS-LM: Faster Gaussian-Splatting Optimization with Levenberg-Marquardt**
*Lukas Höllein, Aljaž Božič, Michael Zollhöfer, Matthias Nießner*
arXiv preprint, 19 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.12892)] [[Project](https://lukashoel.github.io/3DGS-LM/)] [[Video](https://www.youtube.com/watch?v=tDiGuGMssg8)] [[Code](https://github.com/lukasHoel/3DGS-LM)]
**Low Latency Point Cloud Rendering with Learned Splatting**
*Yueyu Hu, Ran Gong, Qi Sun, Yao Wang*
CVPR 2024 Workshop on AIS, 24 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.16504)] [[Code](https://github.com/huzi96/gaussian-pcloud-render)]
**Sort-free Gaussian Splatting via Weighted Sum Rendering**
*Qiqi Hou, Randall Rauwendaal, Zifeng Li, Hoang Le, Farzad Farhadzadeh, Fatih Porikli, Alexei Bourd, Amir Said*
arXiv preprint, 24 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.18931)]
**Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives**
*Alex Hanson, Allen Tu, Geng Lin, Vasu Singla, Matthias Zwicker, Tom Goldstein*
30 Nov 2024
[[arXiv](https://arxiv.org/abs/2412.00578)]
**Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering**
*Cheng Sun, Jaesung Choe, Charles Loop, Wei-Chiu Ma, Yu-Chiang Frank Wang*
5 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.04459)]
**Volumetrically Consistent 3D Gaussian Rasterization**
*Chinmay Talegaonkar, Yash Belhe, Ravi Ramamoorthi, Nicholas Antipa*
4 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.03378)]
**Faster and Better 3D Splatting via Group Training**
*Chengbo Wang, Guozheng Ma, Yifei Xue, Yizhen Lao*
10 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.07608)]
**Turbo-GS: Accelerating 3D Gaussian Fitting for High-Quality Radiance Fields**
*Tao Lu, Ankit Dhiman, R Srinath, Emre Arslan, Angela Xing, Yuanbo Xiangli, R Venkatesh Babu, Srinath Sridhar*
18 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.13509)]
**Balanced 3DGS: Gaussian-wise Parallelism Rendering with Fine-Grained Tiling**
*Hao Gui, Lin Hu, Rui Chen, Mingxiao Huang, Yuxin Yin, Jin Yang, Yong Wu*
23 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.17378)]
**SG-Splatting: Accelerating 3D Gaussian Splatting with Spherical Gaussians**
*Yiwen Wang, Siyuan Chen, Ran Yi*
31 Dec 2024
[[arXiv](https://arxiv.org/abs/2501.00342)]
## 3DGS Geometry Reconstruction
**SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering**
*Antoine Guédon, Vincent Lepetit*
arXiv preprint, 21 Nov 2023
[[arXiv](https://arxiv.org/abs/2311.12775)] [[Project](https://imagine.enpc.fr/~guedona/sugar/)]
**NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance**
*Hanlin Chen, Chen Li, Gim Hee Lee*
arXiv preprint, 1 Dec, 2023
[[arXiv](https://arxiv.org/abs/2312.00846)]
**AtomGS: Atomizing Gaussian Splatting for High-Fidelity Radiance Field**
*Rong Liu, Rui Xu, Yue Hu, Meida Chen, Andrew Feng*
BMVC 2024, 20 May 2024
[[arXiv](https://arxiv.org/abs/2405.12369)] [[Project](https://rongliu-leo.github.io/AtomGS/)] [[Code](https://github.com/RongLiu-Leo/AtomGS)] [[Video](https://www.youtube.com/watch?v=1B7oga_1BqE)]
:fire:**2D Gaussian Splatting for Geometrically Accurate Radiance Fields**
*Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, Shenghua Gao*
SIGGRAPH 2024, 26 Mar 2024
Abstract
3D Gaussian Splatting (3DGS) has recently revolutionized radiance field reconstruction, achieving high quality novel view synthesis and fast rendering speed without baking. However, 3DGS fails to accurately represent surfaces due to the multi-view inconsistent nature of 3D Gaussians. We present 2D Gaussian Splatting (2DGS), a novel approach to model and reconstruct geometrically accurate radiance fields from multi-view images. Our key idea is to collapse the 3D volume into a set of 2D oriented planar Gaussian disks. Unlike 3D Gaussians, 2D Gaussians provide view-consistent geometry while modeling surfaces intrinsically. To accurately recover thin surfaces and achieve stable optimization, we introduce a perspective-correct 2D splatting process utilizing ray-splat intersection and rasterization. Additionally, we incorporate depth distortion and normal consistency terms to further enhance the quality of the reconstructions. We demonstrate that our differentiable renderer allows for noise-free and detailed geometry reconstruction while maintaining competitive appearance quality, fast training speed, and real-time rendering.
[[arXiv](https://arxiv.org/abs/2403.17888)] [[Project](https://surfsplatting.github.io/)] [[Code](https://github.com/hbb1/2d-gaussian-splatting?tab=readme-ov-file)] [[Video](https://www.youtube.com/watch?v=oaHCtB6yiKU)]
**GSDF: 3DGS Meets SDF for Improved Rendering and Reconstruction**
*Mulin Yu, Tao Lu, Linning Xu, Lihan Jiang, Yuanbo Xiangli, Bo Dai*
arXiv preprint, 25 Mar 2024
[[arXiv](https://arxiv.org/abs/2403.16964)] [[Project](https://city-super.github.io/GSDF/)] [[Code](https://github.com/city-super/GSDF)]
**Modeling uncertainty for Gaussian Splatting**
*Luca Savant, Diego Valsesia, Enrico Magli*
arXiv preprint, 27 Mar 2024
[[arXiv](https://arxiv.org/abs/2403.18476)]
**Surface Reconstruction from Gaussian Splatting via Novel Stereo Views**
*Yaniv Wolf, Amit Bracha, Ron Kimmel*
arXiv preprint, 2 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.01810)] [[Project](https://gs2mesh.github.io/)]
**Gaussian Opacity Fields: Efficient and Compact Surface Reconstruction in Unbounded Scenes**
*Zehao Yu, Torsten Sattler, Andreas Geiger*
arXiv preprint, 16 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.10772)] [[Project](https://niujinshuchong.github.io/gaussian-opacity-fields/)] [[Code](https://github.com/autonomousvision/gaussian-opacity-fields)]
**Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Monocular Videos**
*Isabella Liu, Hao Su, Xiaolong Wang*
arXiv preprint, 18 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.12379)] [[Project](https://www.liuisabella.com/DG-Mesh/)]
**Direct Learning of Mesh and Appearance via 3D Gaussian Splatting**
*Ancheng Lin, Jun Li*
arXiv preprint, 11 May 2024
[[arXiv](https://arxiv.org/abs/2405.06945)]
**TetSphere Splatting: Representing High-Quality Geometry with Lagrangian Volumetric Meshes**
*Minghao Guo, Bohan Wang, Kaiming He, Wojciech Matusik*
arXiv preprint, 30 May 2024
[[arXiv](https://arxiv.org/abs/2405.20283)]
**Tetrahedron Splatting for 3D Generation**
*Chun Gu, Zeyu Yang, Zijie Pan, Xiatian Zhu, Li Zhang*
arXiv preprint, 3 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.01579)] [[Code](https://github.com/fudan-zvg/tet-splatting)]
**RaDe-GS: Rasterizing Depth in Gaussian Splatting**
*Baowen Zhang, Chuan Fang, Rakesh Shrestha, Yixun Liang, Xiaoxiao Long, Ping Tan*
arXiv preprint, 3 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.01467)]
**Trim 3D Gaussian Splatting for Accurate Geometry Representation**
*Lue Fan, Yuxue Yang, Minxing Li, Hongsheng Li, Zhaoxiang Zhang*
arXiv preprint, 11 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.07499)] [[Project](https://trimgs.github.io/)] [[Code](https://github.com/YuxueYang1204/TrimGS)]
:fire:**PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction**
*Danpeng Chen, Hai Li, Weicai Ye, Yifan Wang, Weijian Xie, Shangjin Zhai, Nan Wang, Haomin Liu, Hujun Bao, Guofeng Zhang*
arXiv prepreint, 10 Jun 2024
Abstract
Recently, 3D Gaussian Splatting (3DGS) has attracted widespread attention due to its high-quality rendering, and ultra-fast training and rendering speed. However, due to the unstructured and irregular nature of Gaussian point clouds, it is difficult to guarantee geometric reconstruction accuracy and multi-view consistency simply by relying on image reconstruction loss. Although many studies on surface reconstruction based on 3DGS have emerged recently, the quality of their meshes is generally unsatisfactory. To address this problem, we propose a fast planar-based Gaussian splatting reconstruction representation (PGSR) to achieve high-fidelity surface reconstruction while ensuring high-quality rendering. Specifically, we first introduce an unbiased depth rendering method, which directly renders the distance from the camera origin to the Gaussian plane and the corresponding normal map based on the Gaussian distribution of the point cloud, and divides the two to obtain the unbiased depth. We then introduce single-view geometric, multi-view photometric, and geometric regularization to preserve global geometric accuracy. We also propose a camera exposure compensation model to cope with scenes with large illumination variations. Experiments on indoor and outdoor scenes show that our method achieves fast training and rendering while maintaining high-fidelity rendering and geometric reconstruction, outperforming 3DGS-based and NeRF-based methods.
[[arXiv](https://arxiv.org/abs/2406.06521)] [[Project](https://zju3dv.github.io/pgsr/)] [[Code](https://github.com/zju3dv/PGSR)]
**VCR-GauS: View Consistent Depth-Normal Regularizer for Gaussian Surface Reconstruction**
*Hanlin Chen, Fangyin Wei, Chen Li, Tianxin Huang, Yunsong Wang, Gim Hee Lee*
arXiv preprint, 9 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.05774)]
**Projecting Radiance Fields to Mesh Surfaces**
*Adrian Xuan Wei Lim, Lynnette Hui Xian Ng, Nicholas Kyger, Tomo Michigami, Faraz Baghernezhad*
SIGGRAPH Poster 2024, 17 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.11570)]
**GS-Octree: Octree-based 3D Gaussian Splatting for Robust Object-level 3D Reconstruction Under Strong Lighting**
*Jiaze Li, Zhengyu Wen, Luo Zhang, Jiangbei Hu, Fei Hou, Zhebin Zhang, Ying He*
arXiv preprint, 26 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.18199)]
**2DGH: 2D Gaussian-Hermite Splatting for High-quality Rendering and Better Geometry Reconstruction**
*Ruihan Yu, Tianyu Huang, Jingwang Ling, Feng Xu*
arXiv preprint, 30 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.16982)]
**Spurfies: Sparse Surface Reconstruction using Local Geometry Priors**
*Kevin Raj, Christopher Wewer, Raza Yunus, Eddy Ilg, Jan Eric Lenssen*
arXiv preprint, 29 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.16544)] [[Project](https://geometric-rl.mpi-inf.mpg.de/spurfies/)]
**Spiking GS: Towards High-Accuracy and Low-Cost Surface Reconstruction via Spiking Neuron-based Gaussian Splatting**
*Weixing Zhang, Zongrui Li, De Ma, Huajin Tang, Xudong Jiang, Qian Zheng, Gang Pan*
arXiv preprint, 9 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.07266)] [[Code](https://github.com/zju-bmi-lab/SpikingGS)]
**Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering**
*Meng Wei, Qianyi Wu, Jianmin Zheng, Hamid Rezatofighi, Jianfei Cai*
NeurIPS 2024, 27 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.20593)]
:fire:**GVKF: Gaussian Voxel Kernel Functions for Highly Efficient Surface Reconstruction in Open Scenes**
*Gaochao Song, Chong Cheng, Hao Wang*
NeurIPS 2024, 4 Nov 2024
Abstract
In this paper we present a novel method for efficient and effective 3D surface reconstruction in open scenes. Existing Neural Radiance Fields (NeRF) based works typically require extensive training and rendering time due to the adopted implicit representations. In contrast, 3D Gaussian splatting (3DGS) uses an explicit and discrete representation, hence the reconstructed surface is built by the huge number of Gaussian primitives, which leads to excessive memory consumption and rough surface details in sparse Gaussian areas. To address these issues, we propose Gaussian Voxel Kernel Functions (GVKF), which establish a continuous scene representation based on discrete 3DGS through kernel regression. The GVKF integrates fast 3DGS rasterization and highly effective scene implicit representations, achieving high-fidelity open scene surface reconstruction. Experiments on challenging scene datasets demonstrate the efficiency and effectiveness of our proposed GVKF, featuring with high reconstruction quality, real-time rendering speed, significant savings in storage and training memory consumption.
[[arXiv](https://arxiv.org/abs/2411.01853)]
**DyGASR: Dynamic Generalized Exponential Splatting with Surface Alignment for Accelerated 3D Mesh Reconstruction**
*Shengchao Zhao, Yundong Li*
arXiv preprint, 14 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.09156)]
**Quadratic Gaussian Splatting for Efficient and Detailed Surface Reconstruction**
*Ziyu Zhang, Binbin Huang, Hanqing Jiang, Liyang Zhou, Xiaojun Xiang, Shunhan Shen*
25 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.16392)]
**Geometry Field Splatting with Gaussian Surfels**
*Kaiwen Jiang, Venkataram Sivaram, Cheng Peng, Ravi Ramamoorthi*
26 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.17067)]
**G2SDF: Surface Reconstruction from Explicit Gaussians with Implicit SDFs**
*Kunyi Li, Michael Niemeyer, Zeyu Chen, Nassir Navab, Federico Tombari*
25 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.16898)]
**GSurf: 3D Reconstruction via Signed Distance Fields with Direct Gaussian Supervision**
*Xu Baixin, Hu Jiangbei, Li Jiaze, He Ying*
24 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.15723)] [[Code](https://github.com/xubaixinxbx/Gsurf)]
**SplatSDF: Boosting Neural Implicit SDF via Gaussian Splatting Fusion**
*Runfa Blark Li, Keito Suzuki, Bang Du, Ki Myung Brian Le, Nikolay Atanasov, Truong Nguyen*
23 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.15468)]
**HDGS: Textured 2D Gaussian Splatting for Enhanced Scene Rendering**
*Yunzhou Song, Heguang Lin, Jiahui Lei, Lingjie Liu, Kostas Daniilidis*
2 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.01823)] [[Project](https://timsong412.github.io/HDGS-ProjPage/)] [[Code])(https://github.com/TimSong412/HDGS)]
**Ref-GS: Directional Factorization for 2D Gaussian Splatting**
*Youjia Zhang, Anpei Chen, Yumin Wan, Zikai Song, Junqing Yu, Yawei Luo, Wei Yang*
1 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.00905)] [[Project](https://ref-gs.github.io/)]
**GausSurf: Geometry-Guided 3D Gaussian Splatting for Surface Reconstruction**
*Jiepeng Wang, Yuan Liu, Peng Wang, Cheng Lin, Junhui Hou, Xin Li, Taku Komura, Wenping Wang*
29 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.19454)] [[Project](https://jiepengwang.github.io/GausSurf/)] [[Code](https://github.com/jiepengwang/GausSurf)]
**3D Gaussian Splatting with Normal Information for Mesh Extraction and Improved Rendering**
*Meenakshi Krishnan, Liam Fowl, Ramani Duraiswami*
ICASSP, 14 Jan 2025
[[arXiv](https://arxiv.org/abs/2501.08370)]
**Gaussian Building Mesh (GBM): Extract a Building's 3D Mesh with Google Earth and Gaussian Splatting**
*Kyle Gao, Liangzhi Li, Hongjie He, Dening Lu, Linlin Xu, Jonathan Li*
31 Dec 2024
[[arXiv](https://arxiv.org/abs/2501.00625)]
## 3DGS+Mesh For Reconstruction
**Integrating Meshes and 3D Gaussians for Indoor Scene Reconstruction with SAM Mask Guidance**
*Jiyeop Kim, Jongwoo Lim*
arXiv preprint, 23 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.16173)]
**Enhancement of 3D Gaussian Splatting using Raw Mesh for Photorealistic Recreation of Architectures**
*Ruizhe Wang, Chunliang Hua, Tomakayev Shingys, Mengyuan Niu, Qingxin Yang, Lizhong Gao, Yi Zheng, Junyan Yang, Qiao Wang*
arXiv preprint, 22 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.15435)]
## 3DGS Based Dynamic Scene
**Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis**
*Jonathon Luiten, Georgios Kopanas, Bastian Leibe, Deva Ramanan*
arXiv preprint, 18 Aug 2023
[[arXiv](https://arxiv.org/abs/2308.09713)] [[Project](https://dynamic3dgaussians.github.io/)] [[Github](https://github.com/JonathonLuiten/Dynamic3DGaussians)]
:fire:**Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction**
*Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, Xiaogang Jin*
arXiv preprint, 22 Sep 2023
Abstract
Implicit neural representation has paved the way for new approaches to dynamic scene reconstruction and rendering. Nonetheless, cutting-edge dynamic neural rendering methods rely heavily on these implicit representations, which frequently struggle to capture the intricate details of objects in the scene. Furthermore, implicit methods have difficulty achieving real-time rendering in general dynamic scenes, limiting their use in a variety of tasks. To address the issues, we propose a deformable 3D Gaussians Splatting method that reconstructs scenes using 3D Gaussians and learns them in canonical space with a deformation field to model monocular dynamic scenes. We also introduce an annealing smoothing training mechanism with no extra overhead, which can mitigate the impact of inaccurate poses on the smoothness of time interpolation tasks in real-world datasets. Through a differential Gaussian rasterizer, the deformable 3D Gaussians not only achieve higher rendering quality but also real-time rendering speed. Experiments show that our method outperforms existing methods significantly in terms of both rendering quality and speed, making it well-suited for tasks such as novel-view synthesis, time interpolation, and real-time rendering.
[[arXiv](https://arxiv.org/abs/2309.13101)]
**4D Gaussian Splatting for Real-Time Dynamic Scene Rendering**
*Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, Xinggang Wang*
arXiv preprint, 12 Oct 2023
[[arXiv](https://arxiv.org/abs/2310.08528)] [[Project](https://guanjunwu.github.io/4dgs/)] [[Github](https://github.com/hustvl/4DGaussians)]
**Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting**
*Zeyu Yang, Hongye Yang, Zijie Pan, Xiatian Zhu, Li Zhang*
arXiv preprint, 16 Oct 2023
[[arXiv](https://arxiv.org/abs/2310.10642)]
**Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction**
*Devikalyan Das, Christopher Wewer, Raza Yunus, Eddy Ilg, Jan Eric Lenssen*
arXiv preprint, 2 Dec 2023
[[arXiv](https://arxiv.org/abs/2312.01196)]
**Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle**
*Youtian Lin, Zuozhuo Dai, Siyu Zhu, Yao Yao*
arXiv preprint, 6 Dec 2023
[[arXiv](https://arxiv.org/abs/2312.03431)]
**CoGS: Controllable Gaussian Splatting**
*Heng Yu, Joel Julin, Zoltán Á. Milacski, Koichiro Niinuma, László A. Jeni*
CVPR 2024, 9 Dec 2023
[[arXiv](https://arxiv.org/abs/2312.05664)]
**GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis**
*Yiqing Liang, Numair Khan, Zhengqin Li, Thu Nguyen-Phuoc, Douglas Lanman, James Tompkin, Lei Xiao*
arXiv preprint, 18 Dec 2023
[[arXiv](https://arxiv.org/abs/2312.11458)] [[Project](https://lynl7130.github.io/gaufre/index.html)]
:fire:**SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes**
*Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, Xiaojuan Qi*
CVPR 2024, 4 Dec 2023
Abstract
Novel view synthesis for dynamic scenes is still a challenging problem in computer vision and graphics. Recently, Gaussian splatting has emerged as a robust technique to represent static scenes and enable high-quality and real-time novel view synthesis. Building upon this technique, we propose a new representation that explicitly decomposes the motion and appearance of dynamic scenes into sparse control points and dense Gaussians, respectively. Our key idea is to use sparse control points, significantly fewer in number than the Gaussians, to learn compact 6 DoF transformation bases, which can be locally interpolated through learned interpolation weights to yield the motion field of 3D Gaussians. We employ a deformation MLP to predict time-varying 6 DoF transformations for each control point, which reduces learning complexities, enhances learning abilities, and facilitates obtaining temporal and spatial coherent motion patterns. Then, we jointly learn the 3D Gaussians, the canonical space locations of control points, and the deformation MLP to reconstruct the appearance, geometry, and dynamics of 3D scenes. During learning, the location and number of control points are adaptively adjusted to accommodate varying motion complexities in different regions, and an ARAP loss following the principle of as rigid as possible is developed to enforce spatial continuity and local rigidity of learned motions. Finally, thanks to the explicit sparse motion representation and its decomposition from appearance, our method can enable user-controlled motion editing while retaining high-fidelity appearances. Extensive experiments demonstrate that our approach outperforms existing approaches on novel view synthesis with a high rendering speed and enables novel appearance-preserved motion editing applications. Project page: this https URL
[[arXiv](https://arxiv.org/abs/2312.14937)] [[Project](https://yihua7.github.io/SC-GS-web/)] [[Code](https://github.com/yihua7/SC-GS)] [[Video](https://www.youtube.com/watch?v=CYQYX_0xi5E&feature=youtu.be)]
**Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis**
*Zhan Li, Zhang Chen, Zhong Li, Yi Xu*
CVPR 2024, 28 Dec 2023
[[arXiv](https://arxiv.org/abs/2312.16812)] [[Project](https://oppo-us-research.github.io/SpacetimeGaussians-website/)] [[Code](https://github.com/oppo-us-research/SpacetimeGaussians)] [[Video](https://www.youtube.com/watch?v=YsPPmf-E6Lg)]
**4D Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes**
*Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wenzheng Chen, Baoquan Chen*
arXiv preprint, 5 Feb 2024
[[arXiv](https://arxiv.org/abs/2402.03307)] [[Code](https://github.com/weify627/4D-Rotor-Gaussians?tab=readme-ov-file)]
**Mesh-based Gaussian Splatting for Real-time Large-scale Deformation**
*Lin Gao, Jie Yang, Bo-Tao Zhang, Jia-Mu Sun, Yu-Jie Yuan, Hongbo Fu, Yu-Kun Lai*
arXiv preprint, 7 Feb 2024
[[arXiv](https://arxiv.org/abs/2402.04796)]
**GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation**
*Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wenchao Ma, Le Chen, Danhang Tang, Ulrich Neumann*
arXiv preprint, 19 Mar 2024
[[arXiv](https://arxiv.org/abs/2403.12365)] [[Project](https://zerg-overmind.github.io/GaussianFlow.github.io/)] [[Code](https://github.com/Zerg-Overmind/GaussianFlow)] [[Video](https://www.youtube.com/watch?v=0qRcjTw7-YU&feature=youtu.be)]
**Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting**
*Jeongmin Bae, Seoha Kim, Youngsik Yun, Hahyun Lee, Gun Bang, Youngjung Uh*
arXiv preprint, 4 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.03613)] [[Project](https://jeongminb.github.io/e-d3dgs/)] [[Code](https://github.com/JeongminB/E-D3DGS)]
**3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis**
*Zhicheng Lu, Xiang Guo, Le Hui, Tianrui Chen, Min Yang, Xiao Tang, Feng Zhu, Yuchao Dai*
CVPR 2024, 9 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.06270)] [[Project](https://npucvr.github.io/GaGS/)]
**Gaussian Time Machine: A Real-Time Rendering Methodology for Time-Variant Appearances**
*Licheng Shen, Ho Ngai Chow, Lingyun Wang, Tong Zhang, Mengqiu Wang, Yuxing Han*
arXiv preprint, 22 May 2024
[[arXiv](https://arxiv.org/abs/2405.13694)]
**MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffolds**
*Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, Kostas Daniilidis*
arXiv preprint, 27 May 2024
[[arXiv](https://arxiv.org/abs/2405.17421)] [[Project](https://www.cis.upenn.edu/~leijh/projects/mosca/)] [[Video](https://www.youtube.com/watch?v=to869D5V7gQ)]
**GSDeformer: Direct Cage-based Deformation for 3D Gaussian Splatting**
*Jiajun Huang, Hongchuan Yu*
arXiv preprint, 24 May 2024
[[arXiv](https://arxiv.org/abs/2405.15491)] [[Project](https://jhuangbu.github.io/gsdeformer/)] [[Video](https://jhuangbu.github.io/gsdeformer/static/videos/gsdeformer-final-video.mp4)]
**GFlow: Recovering 4D World from Monocular Video**
*Shizun Wang, Xingyi Yang, Qiuhong Shen, Zhenxiang Jiang, Xinchao Wang*
arXiv preprint, 28 May 2024
[[arXiv](https://arxiv.org/abs/2405.18426)] [[Project](https://littlepure2333.github.io/GFlow/)]
**A Refined 3D Gaussian Representation for High-Quality Dynamic Scene Reconstruction**
*Bin Zhang, Bi Zeng, Zexin Peng*
arXiv preprint, 28 May 2024
[[arXiv](https://arxiv.org/abs/2405.17891)]
**Object-centric Reconstruction and Tracking of Dynamic Unknown Objects using 3D Gaussian Splatting**
*Kuldeep R Barad, Antoine Richard, Jan Dentler, Miguel Olivares-Mendez, Carol Martinez*
IEEE Space Robotics 2024, 30 May 2024
[[arXiv](https://arxiv.org/abs/2405.20104)]
**GaussianPrediction: Dynamic 3D Gaussian Prediction for Motion Extrapolation and Free View Synthesis**
*Boming Zhao, Yuan Li, Ziyu Sun, Lin Zeng, Yujun Shen, Rui Ma, Yinda Zhang, Hujun Bao, Zhaopeng Cui*
SIGGRAPH 2024, 30 May 2024
[[arXiv](https://arxiv.org/abs/2405.19745)] [[Project](https://zju3dv.github.io/gaussian-prediction/)]
**Reconstructing and Simulating Dynamic 3D Objects with Mesh-adsorbed Gaussian Splatting**
*Shaojie Ma, Yawei Luo, Yi Yang*
arXiv preprint, 3 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.01593)] [[Project](https://wcwac.github.io/MaGS-page/)] [[Code](https://github.com/wcwac/MaGS)]
**Self-Calibrating 4D Novel View Synthesis from Monocular Videos Using Gaussian Splatting**
*Fang Li, Hao Zhang, Narendra Ahuja*
arXiv preprint, 3 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.01042)] [[Code](https://github.com/fangli333/SC-4DGS)]
:fire:**Superpoint Gaussian Splatting for Real-Time High-Fidelity Dynamic Scene Reconstruction**
*Diwen Wan, Ruijie Lu, Gang Zeng*
ICML 2024, 6 Jun 2024
Abstract
Rendering novel view images in dynamic scenes is a crucial yet challenging task. Current methods mainly utilize NeRF-based methods to represent the static scene and an additional time-variant MLP to model scene deformations, resulting in relatively low rendering quality as well as slow inference speed. To tackle these challenges, we propose a novel framework named Superpoint Gaussian Splatting (SP-GS). Specifically, our framework first employs explicit 3D Gaussians to reconstruct the scene and then clusters Gaussians with similar properties (e.g., rotation, translation, and location) into superpoints. Empowered by these superpoints, our method manages to extend 3D Gaussian splatting to dynamic scenes with only a slight increase in computational expense. Apart from achieving state-of-the-art visual quality and real-time rendering under high resolutions, the superpoint representation provides a stronger manipulation capability. Extensive experiments demonstrate the practicality and effectiveness of our approach on both synthetic and real-world datasets. Please see our project page at this https URL.
[[arXiv](https://arxiv.org/abs/2406.03697)] [[Project](https://dnvtmf.github.io/SP_GS.github.io/)] [[Code](https://github.com/dnvtmf/SP_GS)]
**MoDGS: Dynamic Gaussian Splatting from Causually-captured Monocular Videos**
*Qingming Liu, Yuan Liu, Jiepeng Wang, Xianqiang Lv, Peng Wang, Wenping Wang, Junhui Hou*
arXiv preprint, 1 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.00434)]
**DGD: Dynamic 3D Gaussians Distillation**
*Isaac Labe, Noam Issachar, Itai Lang, Sagie Benaim*
arXiv preprint, 29 May 2024
[[arXiv](https://arxiv.org/abs/2405.19321)] [[Project](https://isaaclabe.github.io/DGD-Website/)] [[Code](https://github.com/Isaaclabe/DGD-Dynamic-3D-Gaussians-Distillation)]
**Modeling Ambient Scene Dynamics for Free-view Synthesis**
*Meng-Li Shih, Jia-Bin Huang, Changil Kim, Rajvi Shah, Johannes Kopf, Chen Gao*
SIGGRAPH 2024, 13 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.09395)] [[Project](https://ambientgaussian.github.io/)]
**Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos**
*Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, Leonidas Guibas*
arXiv preprint, 26 Jun 2024
[[arXiv](https://arxiv.org/abs/2406.18717)]
**Gaussian Splatting LK**
*Liuyue Xie, Joel Julin, Koichiro Niinuma, Laszlo A. Jeni*
arXiv preprint, 16 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.11309)]
**S4D: Streaming 4D Real-World Reconstruction with Gaussians and 3D Control Points**
*Bing He, Yunuo Chen, Guo Lu, Li Song, Wenjun Zhang*
arXiv preprint, 23 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.13036)]
**SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction**
*Marko Mihajlovic, Sergey Prokudin, Siyu Tang, Robert Maier, Federica Bogo, Tony Tung, Edmond Boyer*
ECCV 2024, 17 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.11211)]
**MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting**
*Ruijie Zhu, Yanzhe Liang, Hanzhi Chang, Jiacheng Deng, Jiahao Lu, Wenfei Yang, Tianzhu Zhang, Yongdong Zhang*
NeurIPS 2024, 10 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.07707)] [[Project](https://ruijiezhu94.github.io/MotionGS_page/)]
**DN-4DGS: Denoised Deformable Network with Temporal-Spatial Aggregation for Dynamic Scene Rendering**
*Jiahao Lu, Jiacheng Deng, Ruijie Zhu, Yanzhe Liang, Wenfei Yang, Tianzhu Zhang, Xu Zhou*
NeurIPS 2024, 17 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.13607)]
**MEGA: Memory-Efficient 4D Gaussian Splatting for Dynamic Scenes**
*Xinjie Zhang, Zhening Liu, Yifan Zhang, Xingtong Ge, Dailan He, Tongda Xu, Yan Wang, Zehong Lin, Shuicheng Yan, Jun Zhang*
arXiv preprint, 17 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.13613)]
**Fully Explicit Dynamic Gaussian Splatting**
*Junoh Lee, Chang-Yeon Won, Hyunjun Jung, Inhwan Bae, Hae-Gon Jeon*
NeurIPS 2024, 21 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.15629)]
**FreeGaussian: Guidance-free Controllable 3D Gaussian Splats with Flow Derivatives**
*Qizhi Chen, Delin Qu, Yiwen Tang, Haoming Song, Yiting Zhang, Dong Wang, Bin Zhao, Xuelong Li*
arXiv preprint, 29 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.22070)] [[Project](https://freegaussian.github.io/)] [[Code](https://github.com/freegaussian/freegaussian.github.io)]
**Grid4D: 4D Decomposed Hash Encoding for High-fidelity Dynamic Gaussian Splatting**
*Jiawei Xu, Zexin Fan, Jian Yang, Jin Xie*
NeurIPS 2024, 28 Oct 2024
[[arXiv](https://arxiv.org/abs/2410.20815)]
**HiCoM: Hierarchical Coherent Motion for Streamable Dynamic Scene with 3D Gaussian Splatting**
*Qiankun Gao, Jiarui Meng, Chengxiang Wen, Jie Chen, Jian Zhang*
NeurIPS 2024, 12 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.07541)] [[Code](https://github.com/gqk/HiCoM)]
**Adaptive and Temporally Consistent Gaussian Surfels for Multi-view Dynamic Reconstruction**
*Decai Chen, Brianne Oberson, Ingo Feldmann, Oliver Schreer, Anna Hilsmann, Peter Eisert*
arXiv preprint, 10 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.06602)] [[Project](https://fraunhoferhhi.github.io/AT-GS/)]
**4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization**
*Mijeong Kim, Jongwoo Lim, Bohyung Han*
NeurIPS 2024, 13 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.08879)]
**Sketch-guided Cage-based 3D Gaussian Splatting Deformation**
*Tianhao Xie, Noam Aigerman, Eugene Belilovsky, Tiberiu Popa*
arXiv preprint, 19 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.12168)]
**TimeFormer: Capturing Temporal Relationships of Deformable 3D Gaussians for Robust Reconstruction**
*DaDong Jiang, Zhihui Ke, Xiaobo Zhou, Zhi Hou, Xianghui Yang, Wenbo Hu, Tie Qiu, Chunchao Guo*
18 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.11941)] [[Project](https://patrickddj.github.io/TimeFormer/)]
**4D Scaffold Gaussian Splatting for Memory Efficient Dynamic Scene Reconstruction**
*Woong Oh Cho, In Cho, Seoha Kim, Jeongmin Bae, Youngjung Uh, Seon Joo Kim*
26 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.17044)]
**Event-boosted Deformable 3D Gaussians for Fast Dynamic Scene Reconstruction**
*Wenhao Xu, Wenming Weng, Yueyi Zhang, Ruikang Xu, Zhiwei Xiong*
25 Nov 2024
[[arXiv](https://arxiv.org/abs/2411.16180)]
**RelayGS: Reconstructing Dynamic Scenes with Large-Scale and Complex Motions via Relay Gaussians**
*Qiankun Gao, Yanmin Wu, Chengxiang Wen, Jiarui Meng, Luyang Tang, Jie Chen, Ronggang Wang, Jian Zhang*
3 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.02493)] [[Code](https://github.com/gqk/RelayGS)]
**Monocular Dynamic Gaussian Splatting is Fast and Brittle but Smooth Motion Helps**
*Yiqing Liang, Mikhail Okunev, Mikaela Angelina Uy, Runfeng Li, Leonidas Guibas, James Tompkin, Adam W. Harley*
5 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.04457)] [[Project](https://lynl7130.github.io/MonoDyGauBench.github.io/)] [[Code](https://github.com/lynl7130/MonoDyGauBench_code)]
**Urban4D: Semantic-Guided 4D Gaussian Splatting for Urban Scene Reconstruction**
*Ziwen Li, Jiaxin Huang, Runnan Chen, Yunlong Che, Yandong Guo, Tongliang Liu, Fakhri Karray, Mingming Gong*
4 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.03473)]
**HybridGS: Decoupling Transients and Statics with 2D and 3D Gaussian Splatting**
*Jingyu Lin, Jiaqi Gu, Lubin Fan, Bojian Wu, Yujing Lou, Renjie Chen, Ligang Liu, Jieping *
5 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.03844)] [[Project](https://gujiaqivadin.github.io/hybridgs/)] [[Code](https://github.com/Yeyuqqwx/HybridGS)]
**Template-free Articulated Gaussian Splatting for Real-time Reposable Dynamic View Synthesis**
*Diwen Wan, Yuxiang Wang, Ruijie Lu, Gang Zeng*
NeurIPS 2024, 7 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.05570)]
**4D Gaussian Splatting with Scale-aware Residual Field and Adaptive Optimization for Real-time Rendering of Temporally Complex Dynamic Scenes**
*Jinbo Yan, Rui Peng, Luyang Tang, Ronggang Wang*
9 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.06299)] [[Project](https://yjb6.github.io/SaRO-GS.github.io/)]
**Deblur4DGS: 4D Gaussian Splatting from Blurry Monocular Video**
*Renlong Wu, Zhilu Zhang, Mingyang Chen, Xiaopeng Fan, Zifei Yan, Wangmeng Zuo*
9 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.06424)] [[Code](https://github.com/ZcsrenlongZ/Deblur4DGS)]
**SplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Video**
*Jongmin Park, Minh-Quan Viet Bui, Juan Luis Gonzalez Bello, Jaeho Moon, Jihyong Oh, Munchurl Kim*
13 Dec 2024
[[arXiv](https://arxiv.org/abs/2412.09982)]
**GaussianVideo: Efficient Video Representation via Hierarchical Gaussian Splatting**
*Andrew Bond, Jui-Hsien Wang, Long Mai, Erkut Erdem, Aykut Erdem*
8 Jan 2025
[[arXiv](https://arxiv.org/abs/2501.04782)]
**GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Tracking**
*Weikang Bian, Zhaoyang Huang, Xiaoyu Shi, Yijin Li, Fu - Yun Wang, Hongsheng Li*
5 Jan 2025
[[arXiv](https://arxiv.org/abs/2501.02690)] [[Project](https://wkbian.github.io/Projects/GS-DiT/)]
## 3DGS + Depth
:fire:**DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization**
*Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning, Jun Zhou, Lin Gu*
CVPR 2024, 11 Mar 2024
Abstract
Radiance fields have demonstrated impressive performance in synthesizing novel views from sparse input views, yet prevailing methods suffer from high training costs and slow inference speed. This paper introduces DNGaussian, a depth-regularized framework based on 3D Gaussian radiance fields, offering real-time and high-quality few-shot novel view synthesis at low costs. Our motivation stems from the highly efficient representation and surprising quality of the recent 3D Gaussian Splatting, despite it will encounter a geometry degradation when input views decrease. In the Gaussian radiance fields, we find this degradation in scene geometry primarily lined to the positioning of Gaussian primitives and can be mitigated by depth constraint. Consequently, we propose a Hard and Soft Depth Regularization to restore accurate scene geometry under coarse monocular depth supervision while maintaining a fine-grained color appearance. To further refine detailed geometry reshaping, we introduce Global-Local Depth Normalization, enhancing the focus on small local depth changes. Extensive experiments on LLFF, DTU, and Blender datasets demonstrate that DNGaussian outperforms state-of-the-art methods, achieving comparable or better results with significantly reduced memory cost, a 25× reduction in training time, and over 3000× faster rendering speed.
[[arXiv](https://arxiv.org/abs/2403.06912)] [[Project](https://fictionarry.github.io/DNGaussian/)] [[Code](https://github.com/Fictionarry/DNGaussian)] [[Video](https://www.youtube.com/watch?v=xmaRI9M3g_M)]
:fire:**DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing**
*Matias Turkulainen, Xuqian Ren, Iaroslav Melekhov, Otto Seiskari, Esa Rahtu, Juho Kannala*
arXiv preprint, 26 Mar 2024
Abstract
High-fidelity 3D reconstruction of common indoor scenes is crucial for VR and AR applications. 3D Gaussian splatting, a novel differentiable rendering technique, has achieved state-of-the-art novel view synthesis results with high rendering speeds and relatively low training times. However, its performance on scenes commonly seen in indoor datasets is poor due to the lack of geometric constraints during optimization. In this work, we explore the use of readily accessible geometric cues to enhance Gaussian splatting optimization in challenging, ill-posed, and textureless scenes. We extend 3D Gaussian splatting with depth and normal cues to tackle challenging indoor datasets and showcase techniques for efficient mesh extraction. Specifically, we regularize the optimization procedure with depth information, enforce local smoothness of nearby Gaussians, and use off-the-shelf monocular networks to achieve better alignment with the true scene geometry. We propose an adaptive depth loss based on the gradient of color images, improving depth estimation and novel view synthesis results over various baselines. Our simple yet effective regularization technique enables direct mesh extraction from the Gaussian representation, yielding more physically accurate reconstructions of indoor scenes.
[[arXiv](https://arxiv.org/abs/2403.17822)]
**HoloGS: Instant Depth-based 3D Gaussian Splatting with Microsoft HoloLens 2**
*Miriam Jäger, Theodor Kapler, Michael Feßenbecker, Felix Birkelbach, Markus Hillemann, Boris Jutzi*
arXiv preprint, 3 May 2024
[[arXiv](https://arxiv.org/abs/2405.02005)]
**Self-Evolving Depth-Supervised 3D Gaussian Splatting from Rendered Stereo Pairs**
*Sadra Safadoust, Fabio Tosi, Fatma Güney, Matteo Poggi*
BMVC 2024, 11 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.07456)] [[Project](https://kuis-ai.github.io/StereoGS/)] [[Code](https://github.com/sadrasafa/StereoGS/)]
## 3DGS Based Depth Estimation
**Depth Estimation Based on 3D Gaussian Splatting Siamese Defocus**
*Jinchang Zhang, Ningning Xu, Hao Zhang, Guoyu Lu*
arXiv preprint, 18 Sep 2024
[[arXiv](https://arxiv.org/abs/2409.12323)]
## 3DGS Few-shot Reconstruction
**Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images**
*Jaeyoung Chung, Jeongtaek Oh, Kyoung Mu Lee*
arXiv preprint, 22 Nov 2023
[[arXiv](https://arxiv.org/abs/2311.13398)]
**FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting**
*Zehao Zhu, Zhiwen Fan, Yifan Jiang, Zhangyang Wang*
arXiv preprint, 1 Dec 2023
[[arXiv](https://arxiv.org/abs/2312.00451)] [[Project](https://zehaozhu.github.io/FSGS/)]
**Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers**
*Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, Song-Hai Zhang*
arXiv preprint, 14 Dec 2023
[[arXiv](https://arxiv.org/abs/2312.09147)] [[Project](https://zouzx.github.io/TriplaneGaussian/)] [[Code](https://github.com/VAST-AI-Research/TriplaneGaussian)]
**pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction**
*David Charatan, Sizhe Li, Andrea Tagliasacchi, Vincent Sitzmann*
arXiv preprint, 19 Dec 2023
[[arXiv](https://arxiv.org/abs/2312.12337)] [[Project](https://davidcharatan.com/pixelsplat/)] [[Code](https://github.com/dcharatan/pixelsplat)]
**AGG: Amortized Generative 3D Gaussians for Single Image to 3D**
*Dejia Xu, Ye Yuan, Morteza Mardani, Sifei Liu, Jiaming Song, Zhangyang Wang, Arash Vahdat*
arXiv preprint, 8 Jan 2024
[[arXiv](https://arxiv.org/abs/2401.04099)] [[Project](https://ir1d.github.io/AGG/)]
**GaussianObject: Just Taking Four Images to Get A High-Quality 3D Object with Gaussian Splatting**
*Chen Yang, Sikuang Li, Jiemin Fang, Ruofan Liang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, Qi Tian*
arXiv preprint, 15 Feb 2024
[[arXiv](https://arxiv.org/abs/2402.10259)] [[Project](https://gaussianobject.github.io/)]
**FDGaussian: Fast Gaussian Splatting from Single Image via Geometric-aware Diffusion Model**
*Qijun Feng, Zhen Xing, Zuxuan Wu, Yu-Gang Jiang*
arXiv preprint, 15 Mar 2024
[[arXiv](https://arxiv.org/abs/2403.10242)] [[Project](https://qjfeng.net/FDGaussian/)]
**Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction**
*Qiuhong Shen, Xuanyu Yi, Zike Wu, Pan Zhou, Hanwang Zhang, Shuicheng Yan, Xinchao Wang*
arXiv preprint, 27 Mar 2024
[[arXiv](https://arxiv.org/abs/2403.18795)] [[Project](https://florinshen.github.io/gamba-project/)]
:fire:**InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40 Seconds**
*Zhiwen Fan, Wenyan Cong, Kairun Wen, Kevin Wang, Jian Zhang, Xinghao Ding, Danfei Xu, Boris Ivanovic, Marco Pavone, Georgios Pavlakos, Zhangyang Wang, Yue Wang*
arXiv preprint, 29 Mar 2024
Abstract
While novel view synthesis (NVS) from a sparse set of images has advanced significantly in 3D computer vision, it relies on precise initial estimation of camera parameters using Structure-from-Motion (SfM). For instance, the recently developed Gaussian Splatting depends heavily on the accuracy of SfM-derived points and poses. However, SfM processes are time-consuming and often prove unreliable in sparse-view scenarios, where matched features are scarce, leading to accumulated errors and limited generalization capability across datasets. In this study, we introduce a novel and efficient framework to enhance robust NVS from sparse-view images. Our framework, InstantSplat, integrates multi-view stereo(MVS) predictions with point-based representations to construct 3D Gaussians of large-scale scenes from sparse-view data within seconds, addressing the aforementioned performance and efficiency issues by SfM. Specifically, InstantSplat generates densely populated surface points across all training views and determines the initial camera parameters using pixel-alignment. Nonetheless, the MVS points are not globally accurate, and the pixel-wise prediction from all views results in an excessive Gaussian number, yielding a overparameterized scene representation that compromises both training speed and accuracy. To address this issue, we employ a grid-based, confidence-aware Farthest Point Sampling to strategically position point primitives at representative locations in parallel. Next, we enhance pose accuracy and tune scene parameters through a gradient-based joint optimization framework from self-supervision. By employing this simplified framework, InstantSplat achieves a substantial reduction in training time, from hours to mere seconds, and demonstrates robust performance across various numbers of views in diverse datasets.
[[arXiv](https://arxiv.org/abs/2403.20309)] [[Project](https://instantsplat.github.io/)] [[Video](https://youtu.be/_9aQHLHHoEM)]
**CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians**
*Avinash Paliwal, Wei Ye, Jinhui Xiong, Dmytro Kotovenko, Rakesh Ranjan, Vikas Chandra, Nima Khademi Kalantari*
arXiv preprint, 28 Mar 2024
[[arXiv](https://arxiv.org/abs/2403.19495)] [[Project](https://people.engr.tamu.edu/nimak/Papers/CoherentGS/index.html)]
**Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpses**
*Inhee Lee, Byungjun Kim, Hanbyul Joo*
arXiv preprint, 22 Apr 2024
[[arXiv](https://arxiv.org/abs/2404.14410)] [[Project](https://snuvclab.github.io/gtu/)]
**GDGS: Gradient Domain Gaussian Splatting for Sparse Representation of Radiance Fields**
*Yuanhao Gong*
arXiv preprint, 8 May 2024
[[arXiv](https://arxiv.org/abs/2405.05446)]
**CoR-GS: Sparse-View 3D Gaussian Splatting via Co-Regularization**
*Jiawei Zhang, Jiahe Li, Xiaohan Yu, Lei Huang, Lin Gu, Jin Zheng, Xiao Bai*
arXiv preprint, 20 May 2024
[[arXiv](https://arxiv.org/abs/2405.12110)] [[Project](https://jiaw-z.github.io/CoR-GS/)] [[Video](https://youtu.be/O83v9Wrn3c4)]
**Sp2360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priors**
*Soumava Paul, Christopher Wewer, Bernt Schiele, Jan Eric Lenssen*
arXiv preprint, 26 May 2024
[[arXiv](https://arxiv.org/abs/2405.16517)]
**A Pixel Is Worth More Than One 3D Gaussians in Single-View 3D Reconstruction**
*Jianghao Shen, Tianfu Wu*
arXiv preprint, 30 May 2024
[[arXiv](https://arxiv.org/abs/2405.20310)]
**GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction**
*Yuxuan Mu, Xinxin Zuo, Chuan Guo, Yilin Wang, Juwei Lu, Xiaofeng Wu, Songcen Xu, Peng Dai, Youliang Yan, Li Cheng*
ECCV 2024, 5 Jul 2024
[[arXiv](https://arxiv.org/abs/2407.04237)]
**Self-augmented Gaussian Splatting with Structure-aware Masks for Sparse-view 3D Reconstruction**
*Lingbei Meng, Bi'an Du, Wei Hu*
arXiv preprint, 9 Aug 2024
[[arXiv](https://arxiv.org/abs/2408.04831)]
:fire:**ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model**
*Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang, Haowen Sun, Junliang Ye, Jun Zhang, Yueqi Duan*
arXiv preprint, 29 Aug 2024
Abstract
Advancements in 3D scene reconstruction have transformed 2D images from the real world into 3D models, producing realistic 3D results from hundreds of input photos. Despite great success in dense-view reconstruction scenarios, rendering a detailed scene from insufficient captured views is still an ill-posed optimization problem, often resulting in artifacts and distortions in unseen areas. In this paper, we propose ReconX, a novel 3D scene reconstruction paradigm that reframes the ambigu