Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/fynv/mixedresolutiongridoflightprobes


https://github.com/fynv/mixedresolutiongridoflightprobes

Last synced: 3 months ago
JSON representation

Awesome Lists containing this project

README 

          
Fei Yang



Mar. 2023



# Introduction 



This article introduces an experimental feature of the open-source project [Three.V8]( https://github.com/fynv/Three.V8), namely Mixed-resolution Grid of Light Probes (or LODProbeGrid which is used for coding).  Three.V8 is a 3D rendering engine and application framework that integrates Google's V8 JavaScript engine, so that games and applications can be coded in JavaScript, while the 3D rendering part is based on the full-featured, native graphics APIs. The LODProbeGrid is a global illumination component of Three.V8, which is evolved from Uniform Grid of Light Probes, the latter is a simple and widely used GI structure. Comparing to the uniform grid, the proposed mixed-resolution gird is more compact in storage. It maintains less number of light probes while still meeting the same requirement of probe density.



The technique mainly tackles with the issue that the requirement of probe density is often varying across the scene, because of the uneven distribution of geometry. An uniform grid of probes can be inefficient when such a requirement is present.  An uniform grid only have a fixed density everywhere, therefore, it often ends up either containing much more probes than necessary (oversampling), or use a lower density than necessary (undersampling). In view of such deficiency of an uniform grid, we propose a mixed-resolution, non-uniform grid for probe placement. We will introduce an algorithm to decide the level of subdivision for different areas of space, which will construct the structure of the grid automatically according to the geometry distribution of the scene. At the same time, because of the change to the probe grid structure, we also have to adopt a modified probe sampling scheme during rendering.



It is clear that the technique can increase the efficiency of storage and network transfer of pre-computed GI information, because of the reduced amount of data. It also reduces the time needed for GI data preparation, because of the reduced total number of probes. However, it is also observed that there is some minor performance loss at runtime on some devices, due to the increased complexity of data structure. Therefore, for performance sensitive uses, we offer an option to convert a mixed-resolution grid of light probes to an uniform grid, which can be done any time before applying it to a scene.



While the technique could potentially have more use-cases, such as using it together with GPU accelerated ray-tracing for dynamic global illumination, we are currently unable to evaluate these scenarios because of the setup of Three.V8, which is mobile targeted and doesn't contain a ray-tracing system. Here we provide the details of the technique, so that it will be easy for people to implement and test it if they are interested in other application scenarios.



# Related Techniques



## Spherical Harmonics



Spherical harmonics are ideal for representing a low-frequency spherical function like irradiance information[[3]](#3). They are extremely compact. Order 3 SH only consists of 9 spherical harmonics basis functions. The spherical harmonics basis functions are linear. When we need to interpolate the irradiance function at some spatial point, we can just calculate a linear combination of the SH coefficients.



We use spherical harmonics to approximate the irradiance function at different spatial locations. While it would have been much easier to do it using ray-traced samples, in this work, we are using a 2 step approach. First rendering to a cube-map, then convert the cube-map to SH coefficients. We use the methods described in [[4]](#4) and [[5]](#5) for the 2nd step.



## Uniform Grid of Light Probes



The simplest form of GI could be using pre-computed irradiance probes distributed in a uniform grid. It is the easiest for shaders to look up the nearest probes around a spatial point, and perform trilinear interpolation. 



Uniform grids of probes are used in NVIDIA's DDGI[[1]](#1) and related works[[2]](#2). The interpolation and weighting scheme proposed in these works, including the visibility test using distance information, has largely resolved the artifacts seen in earlier uniform grid systems, and made it robust to a large range of different environments. In this work, we derive our probe interpolation and weighting scheme based on the tricks used in DDGI for our mixed resolution grid, while restricting to precomputed GI only.



## Image MIP-mapping



When using a uniform grid of irradiance probes represented in SH form, it is easy to consider it as a 3D volumetric image with 9xRGB channels. The trilinear interpolation we use for the probes is also similar to what we do with images. Therefore, it is natural to consider to use techniques developed for images to handle the problem of probe grids.



The issue we currently have with a probe grid is the varying requirement of resolution because of the uneven distribution of geometry, which is basically a level-of-detail problem. For images, we have mip-maps to handle it. For image mip-mapping, we normally down-sample a high-resolution image to half of its dimensions each time, forming a series of images with decreasing sizes. Vertically looking, it can also be seen as a quadtree(for 2D) or an octree(for 3D) hierarchical structure. 



When there is a varying requirement of resolution, 2x2 blocks of pixels from different resolution levels can be compactly packed together, forming an image of mixed resolution. When we are sampling a continuous function, the forming of a mixed resolution grid can be done reversely. We first divide the space with a low resolution "base" grid, so we can sample at the center of each cell. Then by selectively subdividing some of the cells, we get sampling locations of a higher level of resolution, which replace the low resolution samples. This is what we are going to do with the probe grid.



## Voxelization



Voxelization techniques extract information from a 3D-scene composed of meshes into a volumetric structure. The most efficient way to do voxelization is through rasterization and random texture write operations. While it is possible to develop global illumination techniques directly based on voxelization, in this work, we are using it only for acquiring geometry distribution information, which can help us decide which probe grid cells should be sub-divided.



# The Mixed Resolution Grid



Our mixed resolution grid of probes consists of probes from a mixed levels of resolution. The base level grid is specified by user interactively. Then some of the grid cells are selectively subdivided, and each one of the corresponding low-resolution probes is replaced by 8 probes of a higher resolution level. Voxelization is used in order to decide which cells to be subdivided. A constructed mixed resolution grid can be sampled by shaders using a scheme similar to the one used in DDGI with modifications in order to adapt to the difference of the grid structure.



We first overview what the structure of a constructed mixed resolution grid looks like. Then we go into details how the grid is constructed. Finally, we explain how to sample the irradiance value from a constructed grid.



## The Grid Structure






The picture shows a mixed resolution grid in 2D form. The base(0th) level of the grid has a 3x2 division. 4 of the 6 base cells are subdivided into 16 subcells of the 1st level, then 3 of the 16 1st level cells are subdivided into 12 subcells of the 2nd level. Note that each probe is located at the center of its cell, following the same pattern of textures. With only 27 probes, the mixed resolution grid can possibly provide the same effective resolution that a 12x8 uniform grid can provide, if the presented way of subdivision is consistent with the underlying geometries.



For RAM storage, a mixed-resolution grid of light probe consists of the following data items.



* `vec3 coverage_min`: world space coordinate of the minimum position.

* `vec3 coverage_max`: world space coordinate of the maximum position.

* `ivec3 base_divisions`: divisions of the base level grid.

* `int sub_division_level`: maximum level of subdivisions. The logical resolution of the grid would be `base_divisions * (1< probe_data`: linear array of probe data. 10 x vec4 for each probe. The first vec4 stores the world-space position of the probe in its xyz channels, and the resolution level in its w channel. The other 9 vec4 stores its SH-Coefficients.

* `vector visibility_data`: 2D visibility(distance) data packed in the same way as DDGI does, which can be transferred to a GPU texture directly. (See sec.3 of [[1]](#1)) Visibility of each probe is represented using a 16x16x2x16bit texture tile. The only difference is that we store the distances and distance standard deviation in normalized integer form instead of using half-precision float.

* `vector index`: index array representing the grid structure. 



Each item of the `index` array is corresponding to an octree node, either a leaf node or not. For a leaf node at `i`, `index[i]` stores index of the probe. Corresponding probe data starts from `probe_data[index[i]*10]`. 

For a non-leaf node at `i`, `index[i]` stores the index of a subdivided node + `probe_count`, where `probe_count` is the number of probes. Index information of its 8 children can be found in `index` starting from `index[base_count + (index[i]-probe_count)*8]`, where `base_count = base_divisions.x * base_divisions.y * base_divisions.z`. 






We still use our 2D case for example. Remember we have 27 probes, so an `index[i]<27` is a leaf node, and the index points to the probe. For `index[i]>=27`, we can look for its children at `index[6 + (index[i]-27) * 4]`.



Before rendering, the `index` data is transferred to GPU buffer as is, which can be traversed by the shaders. The `probe_data` is also transferred GPU, from which we can get the probe position, level and SH data. The SH data can be linearly combined by shader before evaluating the irradiance. However it is 9 colors which is a little bit heavy. Therefore we insert an irradiance pre-sampling pre-process step to convert each SH probe to a 8x8x32bit(GL_R11G11B10F) texture tile, which is the form how DDGI stores its irradiance data (sec.3 of [[1]](#1)). 



## Constructing the Grid



The mixed-resolution grid of light probe can pre-computed during data preparation.



A crucial step of constructing a mixed-resolution grid is to decided which cells of each level should be subdivided to the next level. To that purpose, we use voxelization to figure out which subdivisions are necessary. A necessary subdivision is one that, after the subdivision, there are some pairs of sub-cells located at two different sides of some triangle faces. If all sub-cells are at the same side of the geometries, then that subdivision is not necessary to happen. Also, if a subdivision is identified as necessary, then the base-cell should exist in the first place, so its parent is necessary to be subdivided as well. 



We perform the voxelization at the highest resolution `base_divisions * (1<=0 && icoord.z



**sponza.glb using uniform grid, 40x24x24 divisions, 23040 probes, 31.7MB**






**sponza.glb using mixed-resolution grid, 5x3x3 base-divisions, 3 subdivision levels, 9488 probes, 13.3MB**






**salle_de_bain.glb using uniform grid, 16x16x16 divisions, 4096 probes, 5.62MB**






**salle_de_bain.glb using mixed-resolution grid, 2x2x2 base-divisions, 3 subdivision levels, 1800 probes, 2.56MB**






**fireplace.glb using uniform grid, 12x8x12 divisions, 1152 probes, 1.58MB**






**fireplace.glb using mixed-resolution grid, 3x2x3 base-divisions, 2 subdivision levels, 809 probes, 1.16MB**



The most noticeable difference occurs with the "salle_de_bain" data where there is large variance of lighting in the empty area, where it is under sampled in the case of a mixed-resolution grid.



# Conclusion



The technique mixed-resolution grid of light probes has some obvious advantages in storage saving and baking performance aspects. However, the potential use-cases in relation to GPU accelerated ray-tracing is unclear, which we will go on exploring as our next priority. The possible degradation in runtime performance and rendering quality in some rare cases is also in need of more tests and evaluations. We are therefore sharing the detail of the technique to people who may need it, and also hoping for more tests and evaluations from the community.



# References

[1] 

Majercik Z., Guertin J.P., Nowrouzezahrai D. AND McGuire M. 2019. Dynamic Diffuse Global Illumination with Ray-Traced Irradiance Fields. Journal of Computer Graphics Techniques.



[2] 

McGuire M., Mara M., Nowrouzezahrai D. AND Luebke D. 2017. Realtime

global illumination using precomputed light field probes. In Proceedings of the 21st

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games.



[3] 

Ramamoorthi R., Hanrahan P. 2001. An efficient representation for irradiance environment maps. SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques.



[4] 

Sloan P. P. 2008. Stupid Spherical Harmonics (SH) Tricks. GDC Lecture.



[5] 

Sloan P. P. 2013. Efficient Spherical Harmonic Evaluation. Journal of Computer Graphics Techniques.



[6] 

Cigolle, Z. H., Donow, S., Evangelakos, D., Mara, M., Mcguire, M., AND Meyer, Q. 2014. A survey of efficient representations for independent unit vectors. Journal of Computer Graphics Techniques (JCGT)



[7] 

Morgan McGuire, Computer Graphics Archive, July 2017 (https://casual-effects.com/data)