Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/tovacinni/awesome-3d-compression

Papers and such on 3D data compression and streaming
https://github.com/tovacinni/awesome-3d-compression

List: awesome-3d-compression

Last synced: 23 days ago
JSON representation

Papers and such on 3D data compression and streaming

Awesome Lists containing this project

README 

        
# Survey of 3D Compression and Streaming



Media formats are moving beyond 1D audio and 2D images which are viewed at static viewing angles and scale

to 3D content which are viewed from dynamic viewing angles and scale. This has broad implications in 

gaming, virtual concerts, television, augmented reality, and more. Compression and streaming are at the core 

of the design of the next generation of media systems, and 3D brings new challenges which require innovations 

in algorithms as well as systems. 



This repository collects papers and talks and blog posts which broadly discuss compression and streaming for

3D content. This is an exciting field which combines computer graphics, computer vision, machine learning,

multimedia, systems, signal processing, information theory, security, and more!



This is very much work in progress and missings lots of work. The categorizations etc are very likely to change.



## Related Surveys / Position Papers / Broadly Applicable Papers



**[Compression is Again Driving the Evolution of Media](https://medium.com/making-vr-human/compression-is-again-driving-the-evolution-of-media-ac77d599d27b#.279r7yezk)**


*Philip A. Chou*



**[State-of-the-art in Compressed GPU-Based Direct Volume Rendering](https://www.ifi.uzh.ch/dam/jcr:00000000-1fe0-1a53-ffff-ffff94c9f943/CompressedDVR.pdf)**


*Marcos Balsa Rodríguez, Enrico Gobbetti, Jose Antonio Iglesias Guitian, Maxim Makhinya, Fabio Marton, Renato Pajarola, Susanne K Suter*


Eurographics State-of-the-Art Reports 2014



**[3D Mesh Compression: Survey, Comparisons, and Emerging Trends](https://dl.acm.org/doi/abs/10.1145/2693443?casa_token=KkY70WPF1oYAAAAA:qizzVOiLw7tkICfJBgdCTUvDyaTOc8Rc7-KiF5m3mR1Ady-Ec3vLOmV_5gjl52h7NiMvuqfNAKI6)**


*Adrien Maglo, Guillaume Lavoue, Florent Dupont, Celine Hudelot*


Eurographics State-of-the-Art Reports 2015



**[Nonlinear Transform Coding](https://arxiv.org/abs/2007.03034)**


*Johannes Ballé, Philip A. Chou, David Minnen, Saurabh Singh, Nick Johnston, Eirikur Agustsson, Sung Jin Hwang, George Toderici*


IEEE Journal of Selected Topics in Signal Processing 2020



**[Toward Truly Immersive Holographic-Type Communication: Challenges and Solutions](https://ieeexplor.ieee.org/document/8970173)**


*Alexander Clemm, Maria Torres Vega, Hemanth Kumar Ravuri, Tim Wauters, Filip De Turkc*


IEEE Communications Magazine 2020



**[Extremely Interactive and Low-Latency Services in 5G and Beyond Mobile Systems](https://ieeexplore.ieee.org/abstract/document/9464920)**


*Tarik Taleb, Zinelaabidine Nadir, Hannu Flinck, JaeSeung Song*


IEEE Communications Standards Magazine 2021



**[Neural Fields in Visual Computing and Beyond](https://arxiv.org/abs/2111.11426)**


*Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, Srinath Sridhar*


Eurographics State-of-the-Art Reports 2022



## Standards



**[MPEG-I: Coded Representation of Immersive Media](https://mpeg.chiariglione.org/standards/mpeg-i)**



**[Emerging MPEG Standards for Point Cloud Compression](https://ieeexplore.ieee.org/document/8571288)**


*Sebastian Schwarz, Marius Preda, Madhukar Budagavi, Pablo Cesar, Philip A. Chou, Robert A. Cohen, Maja Krivokuca, Sebastien Lasserre, Zhu Li, Joan Llach, Khaled Mammou, Rufael Mekuria, Ohji Nakagami, Ernestasia Siahaan, Ali Tabatabai, Alexis M. Tourapis, Vladyslav Zakharchenko*


IEEE JETCAS 2019



**[A Comprehensive Study and Comparison of Core Technologies for MPEG 3-D Point Cloud Compression](https://ieeexplore.ieee.org/document/8945224)**


*Hao Liu, Hui Yuan, Qi Liu, Junhui Hou, Ju Liu*


IEEE Transactions on Broadcasting 2019



**[An overview of ongoing point cloud compression standardization activities: video-based (V-PCC) and geometry-based (G-PCC)](https://www.cambridge.org/core/journals/apsipa-transactions-on-signal-and-information-processing/article/an-overview-of-ongoing-point-cloud-compression-standardization-activities-videobased-vpcc-and-geometrybased-gpcc/56FCAF660DD44348BCB1BCA9B5EC56CF)**


*D. Graziosi, O. Nakagami, S. Kuma, A. Zaghetto, T. Suzuki, A. Tabatabai*

APSIPA Transactions on Signal and Information Processing 2020



## Systems Papers



**[GigaVoxels: ray-guided streaming for efficient and detailed voxel rendering](https://dl.acm.org/doi/abs/10.1145/1507149.1507152)**


*Cyril Crassin, Fabrice Neyret, Sylvain Lefebvre, Elmar Eisemann*


I3D 2009



**[Holoportation: Virtual 3D Teleportation in Real-time](https://dl.acm.org/doi/10.1145/2984511.2984517)**


*Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, Shahram Izadi*


ACM UIST 2016



**[Project Startline: A high-fidelity telepresence system](https://research.google/pubs/pub50903/)**


*Jason Lawrence Dan B Goldman Supreeth Achar Gregory Major Blascovich Joseph G. Desloge Tommy Fortes Eric M. Gomez Sascha Häberling Hugues Hoppe Andy Huibers Claude Knaus Brian Kuschak Ricardo Martin-Brualla Harris Nover Andrew Ian Russell Steven M. Seitz Kevin Tong*


ACM Transactions on Graphics 2021



**[Nanite: A Deep Dive](https://advances.realtimerendering.com/s2021/Karis_Nanite_SIGGRAPH_Advances_2021_final.pdf)**


*Brian Karis, Rune Stubbe, Graham Wihlidal*


ACM SIGGRAPH 2021, Advances in Real-time Rendering in Games



## Continuous Point Clouds



### Compression



**[Self-similarity for accurate compression of point sampled surfaces](https://perso.liris.cnrs.fr/julie.digne/articles/pc_compression.html)**


*Julie Digne, Raphaelle Chaine, Sebastien Valette*


Eurographics 2014



**[3D point cloud compression using conventional image compression for efficient data transmission](https://ieeexplore.ieee.org/document/7340499)**


*Hamidreza Houshiar, Andreas Nuchter*


IEEE ICAT 2015



**[Compression and Rendering of Textured Point Clouds via Sparse Coding](https://diglib.eg.org/handle/10.2312/hpg20211284)**


*Kersten Schuster, Philip Trettner, Patric Schmitz, Julian Schakib, Leif Kobbelt*


HPG 2021



## Voxelized Point Clouds



We draw a slight distinction between point clouds and voxelized point clouds. The latter is quantized into

integer coordinates, while the general class of point clouds are still often quantized but not necessarily

into a regular grid.



**[A Volumetric Approach to Point Cloud Compression—Part I: Attribute Compression](https://ieeexplore.ieee.org/document/8676054)**


*Philip A. Chou, Maxim Koroteev, Maja Krivokuca*


IEEE Transactions of Image Processing



**[A Volumetric Approach to Point Cloud Compression—Part II: Geometry Compression](https://ieeexplore.ieee.org/document/8931233)**


*Maja Krivokuca, Philip A. Chou, Maxim Koroteev*


IEEE Transactions of Image Processing



### Compression



**[Point Cloud Attribute Compression with Graph Transform](https://ieeexplore.ieee.org/abstract/document/7025414)**


*Cha Zhang, Dinei Florencio, Charles Loop*


IEEE ICIP 2014



**[Compression of 3D Point Clouds Using a Region-Adaptive Hierarchical Transform](https://ieeexplore.ieee.org/document/7482691)**


*Ricardo L. de Queiroz, Philip A. Chou*


IEEE Transactions on Image Processing 2016



**[Geometry and attribute compression for voxel scenes](https://dl.acm.org/doi/10.5555/3058909.3058962)**


*Bas Dado, Timothy R. Kol, Pablo Bauszat, Jean-Marc Thiery, Elmar Eisemann*


Eurographics 2016



**[Real-time compression and streaming of 4D performances](https://dl.acm.org/doi/abs/10.1145/3272127.3275096)**


*Danhang Tang, Mingsong Dou, Peter Lincoln, Philip Davidson, Kaiwen Guo, Jonathan Taylor, Sean Fanello, Cem Keskin, Adarsh Kowdle, Sofien Bouaziz, Shahram Izadi, Andrea Tagliasacchi*


SIGGRAPH Asia 2018



**[Deep Implicit Volume Compression](https://arxiv.org/abs/2005.08877)**


*Danhang Tang, Saurabh Singh, Philip A. Chou, Christian Haene, Mingsong Dou, Sean Fanello, Jonathan Taylor, Philip Davidson, Onur G. Guleryuz, Yinda Zhang, Shahram Izadi, Andrea Tagliasacchi, Sofien Bouaziz, Cem Keskin*


CVPR 2020



**[OctSqueeze: Octree-Structured Entropy Model for LiDAR Compression](https://arxiv.org/abs/2005.07178)**


*Lila Huang, Shenlong Wang, Kelvin Wong, Jerry Liu, Raquel Urtasun*


CVPR 2020



**[MuSCLE: Multi Sweep Compression of LiDAR using Deep Entropy Models](https://arxiv.org/abs/2011.07590)**


*Sourav Biswas, Jerry Liu, Kelvin Wong, Shenlong Wang, Raquel Urtasun*


NeurIPS 2020



**[Region adaptive graph fourier transform for 3d point clouds](https://arxiv.org/abs/2003.01866)**


*Eduardo Pavez, Benjamin Girault, Antonio Ortega, Philip A. Chou*


IEEE ICIP 2020



**[VoxelContext-Net: An Octree based Framework for Point Cloud Compression](https://arxiv.org/abs/2105.02158)**


*Zizheng Que, Guo Lu, Dong Xu*


CVPR 2021



**[Moving Basis Decomposition for Precomputed Light Transport](https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.14346)**


*Ari Silvennoinen, Peter-Pike Sloan*


EGSR 2021



**[Volumetric Attribute Compression for 3D Point Clouds using Feedforward Network with Geometric Attention](https://arxiv.org/abs/2304.00335)**


*Tam Thuc Do, Philip A. Chou, Gene Cheung*


ICASSP 2023



### Post-processing



**[Deep Geometry Post-Processing for Decompressed Point Clouds](https://arxiv.org/abs/2204.13952)**


*Xiaoqing Fan, Ge Li, Dingquan Li, Yurui Ren, Wei Gao, Thomas H. Li*


ArXiv, 2022



### Adaptive Streaming



**[Rate-Utility Optimized Streaming of Volumetric Media for Augmented Reality](https://ieeexplore.ieee.org/abstract/document/8638765)**


*Jounsup Park, Philip A. Chou, Jenq-Neng Hwang*


IEEE JETCAS 2019



**[Towards 6DoF HTTP Adaptive Streaming Through Point Cloud Compression](https://dl.acm.org/doi/pdf/10.1145/3343031.3350917)**


*Jeroen van der Hooft, Tim Wauters, Filip De Turck, Christian Timmerer, Hermann Hellwagner*


ACM MM 2019



## Neural Fields



Neural fields offer a new paradigm for compression which is based on function fitting (akin to some earlier

works on compression via polynomial fitting, etc). Eventually I will write something which explains why

neural fields might be a cool alternative / complementary to traditional voxel-adjacent compression methods.



### Representation



There is a lot of work in this area. Here, we only list papers which make an experimental comparison in terms of compression (broadly defined).



**[On the Effectiveness of Weight-Encoded Neural Implicit 3D Shapes](https://arxiv.org/abs/2009.09808v3)**


*Thomas Davies, Derek Nowrouzezahrai, Alec Jacobson*


Technical Report



**[Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Surfaces](https://nv-tlabs.github.io/nglod/)**


*Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, Sanja Fidler*


CVPR 2021



**[ACORN: Adaptive Coordinate Networks for Neural Scene Representation](https://dl.acm.org/doi/abs/10.1145/3450626.3459785)**


*Julien N. P. Martel, David B. Lindell, Connor Z. Lin, Eric R. Chan, Marco Monteiro, Gordon Wetzstein*


SIGGRAPH 2021



**[BACON: Band-limited Coordinate Networks for Multiscale Scene Representation](https://arxiv.org/abs/2112.04645)**


*David B. Lindell, Dave Van Veen, Jeong Joon Park, Gordon Wetzstein*


ArXiv 2021



### Compression



**[Compressive Neural Representations of Volumetric Scalar Fields](https://arxiv.org/abs/2104.04523)**


*Yuzhe Lu, Kairong Jiang, Joshua A. Levine, Matthew Berger*


EuroVis 2021



**[3D Scene Compression through Entropy Penalized Neural Representation Functions](https://ieeexplore.ieee.org/document/9477505)**


*Thomas Bird, Johannes Balle, Saurabh Singh, Philip A. Chou*


IEEE PCS 2021



**[NeRV: Neural Representations for Videos](https://openreview.net/forum?id=BbikqBWZTGB)**


*Hao Chen, Bo He, Hanyu Wang, Yixuan Ren, Ser-Nam Lim, Abhinav Shrivastava*


NeurIPS 2021



**[COIN: COmpression with Implicit Neural Representations](https://arxiv.org/abs/2103.03123)**


*Emilien Dupont, Adam Goliński, Milad Alizadeh, Yee Whye Teh, Arnaud Doucet*


ICLR 2021 Neural Compression Workshop



**[Implicit Neural Representations for Image Compression](https://arxiv.org/abs/2112.04267)**


*Yannick Strümpler, Janis Postels, Ren Yang, Luc Van Gool, and Federico Tombari*


ECCV 2022



**[COIN++: Data Agnostic Neural Compression](https://arxiv.org/abs/2201.12904)**


*Emilien Dupont, Hrushikesh Loya, Milad Alizadeh, Adam Goliński, Yee Whye Teh, Arnaud Doucet*


ArXiv 2022



**[LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networks](https://arxiv.org/abs/2111.08988)**


*Berivan Isik, Philip A. Chou, Sung Jin Hwang, Nick Johnston, George Toderici*


ArXiv 2021



**[Implicit Neural Video Compression](https://arxiv.org/abs/2112.11312)**


*Yunfan Zhang, Ties van Rozendaal, Johann Brehmer, Markus Nagel, Taco Cohen*


ArXiv 2021



**[Neural Motion Compression with Frequency-adaptive Fourier Feature Network](https://diglib.eg.org/handle/10.2312/egs20221033)**


*Kenji Tojo, Yifei Chen, Nobuyuki Umetani*


Eurographics 2022



**[Variable Bitrate Neural Fields](https://nv-tlabs.github.io/vqad/)**


*Towaki Takikawa, Alex Evans, Jonathan Tremblay, Thomas Muller, Morgan McGuire, Alec Jacobson, Sanja Fidler*


SIGGRAPH 2022



**[Streamable Neural Fields](https://arxiv.org/abs/2207.09663)**


*Junwoo Cho, Seungtae Nam, Daniel Rho, Jong Hwan Ko, Eunbyung Park*


ECCV 2022



**[On Quantizing Implicit Neural Representations](https://openaccess.thecvf.com/content/WACV2023/html/Gordon_On_Quantizing_Implicit_Neural_Representations_WACV_2023_paper.html)**


*Cameron Gordon, Shin-Fang Chng, Lachlan MacDonald, Simon Lucey*


WACV 2023



**[Masked Wavelet Representation for Compact Neural Radiance Fields](https://arxiv.org/abs/2212.09069)**


*Daniel Rho, Byeonghyeon Lee, Seungtae Nam, Joo Chan Lee, Jong Hwan Ko, Eunbyung Park*


CVPR 2023



**[MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes](https://arxiv.org/abs/2302.12249)**


*Christian Reiser, Richard Szeliski, Dor Verbin, Pratul P. Srinivasan, Ben Mildenhall, Andreas Geiger, Jonathan T. Barron, Peter Hedman*


ArXiv 2023



**[]()**


**[]()**


**[]()**


**[]()**


**[]()**