https://github.com/bloc97/fssgi
Exploratory Project on Fast Screen Space Global Illumination
https://github.com/bloc97/fssgi
computer-graphics convolutional-neural-networks global-illumination machine-learning rendering-algorithms
Last synced: 7 months ago
JSON representation
Exploratory Project on Fast Screen Space Global Illumination
- Host: GitHub
- URL: https://github.com/bloc97/fssgi
- Owner: bloc97
- License: mit
- Created: 2019-09-26T19:11:05.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2019-10-29T22:09:56.000Z (almost 6 years ago)
- Last Synced: 2025-03-18T10:12:42.215Z (7 months ago)
- Topics: computer-graphics, convolutional-neural-networks, global-illumination, machine-learning, rendering-algorithms
- Language: Jupyter Notebook
- Homepage: https://www-ens.iro.umontreal.ca/~pengbowe/ift3150/
- Size: 74.6 MB
- Stars: 35
- Watchers: 3
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Fast Screen Space Global Illumination
A screen space method of achieving global illumination using convolutional neural networks.
## Resources
[Preprint](IFT3150_E19_Fast_Screen_Space_Global_Illumination.pdf)
[Code (Jupyter Notebook)](FSSGI_Colab.ipynb)
The Jupyter Notebook should be used in [Google Colab](https://colab.research.google.com).
The dataset used in this work is located in the `Blender_Datasets` folder.
A pre-trained model for the dataset is located in the `Models` folder.
## Results
After some small changes, the new PSNR of the provided model on unseen images (validation set) is **34.3** dB, compared to 33.8 dB in the preprint.
- **Path Tracer**: 835ms
- **FSSGI**: 9ms (**92x** Speedup)
From Left to Right:
Local Illumination (**Input**) / Predicted GI (**Output**) / Ground Truth (Path Tracer @1000spp)
## Abstract
Rendering algorithms such as path tracing[1] are commonly used in practice to produce photorealistic images from a 3D scene. However, for complex scenes involving a lot of indirect lighting, path tracing is very computationally expensive and often produces a noisy image when the time budget is limited[2]. The current state of the art method for accelerating rendering time uses a deep convolutional neural network to denoise an image which has been sampled at fewer than 10 samples per pixel[3].
We present a image-translation model that removes the path tracing step entirely and only use a convolutional neural network to predict, in screen space, a visually pleasing photorealistic image using information obtained from the g-buffer. This method is similar to deferred shading, but instead of a human built shader, the 'shader' is created using a machine learning algorithm that learns to reproduce global illumination effects seen in path tracing using the g-buffer data. This method aims to reproduce the quality of path traced images in real-time for conventional rendering pipelines, and can also be used in conjunction with path tracing and denoising techniques as the image generated by the network can serve as an initial estimation which can be refined by more accurate methods.
### [Full Preprint](IFT3150_E19_Fast_Screen_Space_Global_Illumination.pdf)
## Related Work
[Deep Shading: Convolutional Neural Networks for Screen-Space Shading](http://deep-shading-datasets.mpi-inf.mpg.de/)
[Deep Illumination: Approximating Dynamic Global Illumination with Generative Adversarial Network](https://github.com/CreativeCodingLab/DeepIllumination)
[Photographic Image Synthesis with Cascaded Refinement Networks](https://cqf.io/ImageSynthesis/)
## References
[1][KAJIYA J.: The Rendering Equation, 1986](http://inst.eecs.berkeley.edu/~cs294-13/fa09/lectures/p143-kajiya.pdf)
[2][KULLA C., FAJARDO M.: Importance Sampling Techniques for Path Tracing in Participating Media, 2012](https://www.arnoldrenderer.com/research/egsr2012_volume.pdf)
[3][CHAITANYA C., KAPLANYAN A., ET AL.: Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder, 2017](https://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf)