https://github.com/nvpro-samples/vk_gltf_renderer
Rendering glTF scenes with ray tracer and raster (Vulkan)
https://github.com/nvpro-samples/vk_gltf_renderer
cubemap gltf gltf2 importance-sampling pbr raytracing tonemapping vulkan
Last synced: about 1 month ago
JSON representation
Rendering glTF scenes with ray tracer and raster (Vulkan)
- Host: GitHub
- URL: https://github.com/nvpro-samples/vk_gltf_renderer
- Owner: nvpro-samples
- License: apache-2.0
- Created: 2019-12-03T13:01:08.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-10-29T10:45:19.000Z (7 months ago)
- Last Synced: 2024-10-29T12:43:44.285Z (7 months ago)
- Topics: cubemap, gltf, gltf2, importance-sampling, pbr, raytracing, tonemapping, vulkan
- Language: C++
- Homepage:
- Size: 8.13 MB
- Stars: 152
- Watchers: 16
- Forks: 13
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING
- License: LICENSE
Awesome Lists containing this project
README
# Vulkan glTF Scene Renderer
|Pathtracer | Raster|
|:------------: | :------------: |
| ||
## Overview
This application demonstrates a dual-mode renderer for glTF 2.0 scenes, implementing both ray tracing and rasterization pipelines. It showcases the utilization of shared Vulkan resources across rendering modes, including geometry, materials, and textures.
## Key Features
- glTF 2.0 (.gltf/.glb) scene loading
- Pathtracing with global illumination
- PBR-based rasterization
- HDR environment mapping and Sun & Sky simulation
- Advanced tone mapping
- Camera control system
- Extensive debug visualization options
## Dependencies
- Vulkan SDK ([latest version](https://vulkan.lunarg.com/sdk/home))
- [Nvpro-Core](https://github.com/nvpro-samples/nvpro_core.git) framework
## Build Instructions
1. Clone the repositories
```bash
git clone https://github.com/nvpro-samples/nvpro_core.git
git clone https://github.com/nvpro-samples/vk_gltf_renderer.git
```
2. Build the project
```bash
cd vk_gltf_renderer
mkdir build
cd build
cmake ..
cmake --build . --config release
```
3. Run the application
```bash
.\bin_x64\Release\vk_gltf_renderer.exe
```
4. Install [optional] : if you want to package the application
``` bash
cmake --install .
```
### Draco Compression
To enable Draco mesh compression, you need to enable the option CMake. In the GUI interface, you will see the option `USE_DRACO`. If you are using the command line, you can add `-DUSE_DRACO=ON` to the cmake command. This will download the Draco library and it will be included in the project.
## glTF Core features
- [x] glTF 2.0 (.gltf/.glb)
- [x] images (HDR, PNG, JPEG, ...)
- [x] buffers (geometry, animation, skinning, ...)
- [x] textures (base color, normal, metallic, roughness, ...)
- [x] materials (PBR, ...)
- [x] animations
- [x] skins
- [x] morphs
- [x] cameras
- [x] lights
- [x] nodes
- [x] scenes
- [x] samplers
- [x] textures
- [x] extensions
## GLTF Extensions
Here are the list of extensions that are supported by this application
- [ ] KHR_animation_pointer
- [x] KHR_draco_mesh_compression
- [x] KHR_lights_punctual
- [x] KHR_materials_anisotropy
- [x] KHR_materials_clearcoat
- [x] KHR_materials_dispersion
- [x] KHR_materials_emissive_strength
- [x] KHR_materials_ior
- [x] KHR_materials_iridescence
- [x] KHR_materials_sheen
- [x] KHR_materials_specular
- [x] KHR_materials_transmission
- [x] KHR_materials_unlit
- [x] KHR_materials_variants
- [x] KHR_materials_volume
- [ ] KHR_mesh_quantization
- [x] KHR_texture_basisu
- [x] KHR_texture_transform
- [ ] KHR_xmp_json_ld
- [x] EXT_mesh_gpu_instancing
- [x] KHR_node_visibility
## Pathtracer
Implements a path tracer with global illumination.
The options are:
* Max Depth : number of bounces the path can do
* Max Samples: how many samples per pixel at each frame iteration
* Aperture: depth-of-field
* Debug Method: shows information like base color, metallic, roughness, and some attributes
* Choice between indirect and RTX pipeline.
* Denoiser: A-trous denoiser
## Raster
Utilizes shared Vulkan resources with the path tracer, including:
- Scene geometry
- Material data
- Textures
- Shading functions
The options are:
* Show wireframe: display wireframe on top of the geometry
* Super-Sampling: render the image 2x and blit it with linear filter.
* Debug Method: shows information like base color, metallic, roughness, and some attributes
Example with wireframe option turned on
## Features
| | |
|--|--|
| Showcase |    
   
   
 |
| Anisotropy |    
|
| Attenuation |  |
| Alpha Blend |   |
| Animation |    |
| Clear Coat |    
| Dispersion |    |
| IOR |   |
| Emissive |  |
| Iridescence |     |
| Punctual |   |
| Sheen |     |
| Transmission |    
   |
| Variant |    |
| Others |                     |
## Debug
There is also the ability to debug various out channels, such as:
|metallic|roughness|normal|base|emissive|opacity|tangent|tex coord|
|---|---|---|---|---|---|---|---|
||||| | | |  |
## Environment
### Sun & Sky
There is a built-in Sun & Sky physical shader module.
  
### HDR
Lighting of the scene can come from HDRi.
   
   
It is possible to blur HDR to various level.
   
The HDR can also be rotated to get the right illumination.
 
### Background
Background can be also solid color and if saved as PNG, the alpha channel is taking into account.
  
## Tonemapper
We could not get good results without a tone mapper. This is done with a compute shader and different settings can be made.
Multiple tonemapper are supported:
* [Filmic](http://filmicworlds.com/blog/filmic-tonemapping-operators/)
* Uncharted 2
* Clip : Simple Gamma correction (linear to sRGB)
* [ACES](https://www.oscars.org/science-technology/sci-tech-projects/aces): Academy Color Encoding System
* [AgX](https://github.com/EaryChow/AgX)
* [Khronos PBR](https://github.com/KhronosGroup/ToneMapping/blob/main/PBR_Neutral/README.md#pbr-neutral-specification) : PBR Neutral Specification
## Camera
The camera navigation follows the [Softimage](https://en.wikipedia.org/wiki/Softimage_(company)) default behavior. This means, the camera is always looking at a point of interest and orbit around it.
Here are the default navigations:
The camera information can be fine tune by editing its values.
Note: **copy** will copy in text the camera in the clipboard, and pressing the **paste** button will parse the clipboard to set the camera.
Ex: `{0.47115, 0.32620, 0.52345}, {-0.02504, -0.12452, 0.03690}, {0.00000, 1.00000, 0.00000}`
### Save and Restore Cameras
It is also possible to save and restore multiple cameras in the second tab. Press the `+` button to save a camera, the middle button to delete it. By pressing one of the saved cameras, its position, interests, orientation and FOV will be changed smoothly.
**Note**: If the glTF scene contains multiple cameras, they will be showing here.
### Other modes
Other navigation modes also exist, like fly, where the `w`, `a`, `s`, `d` keys also moves the camera.
## Depth-of-Field
Depth of field works only for ray tracing and settings can be found under the `RendererPathtracer>Depth-of-Field`
 
----
## Schema of the Program
The nvvk::Application is a class that provides a framework for creating Vulkan applications. It encapsulates the Vulkan instance, device, and surface creation, as well as window management and event handling.
When using `nvvk::Application`, you can attach `nvvkhl::IAppElement` to it and each element will be called for the different state, allowing to customize the behavior of your application. The `nvvkhl::IAppElement` class provides default implementations for these functions, so you only need to override the ones you need.
Here is a brief overview of how `nvvk::Application` works:
### Initialization:
When you create an instance of `nvvk::Application`, it sets up the Vulkan instance, device, and surface. It also creates a window and sets up event handling.
### Attaching Elements
In `main()` we are attaching many elements, like:
* `ElementCamera` : this allow to control a singleton camera
* `ElementProfiler` : allow to time the execution on the GPU
* `ElementBenchmarkParameters` : command line arguments and test purpose
* `ElementLogger` : redirect log information in a window
* `ElementNvml` : shows the status of the GPU
But the main one that interest us, and which is the main of this application is `GltfRendererElement`. This is the one that will be controlling the scene and rendering.
### Main Loop:
The `nvvk::Application` class provides a main loop that continuously processes events and updates the application state. Inside the main loop, it calls the following functions:
* **onAttach()**:
This function is called whenever the element is attached to the application. In `GltfRendererElement`, we are creating the resource needed internally.
* **onDetach()**:
This function is called when the user tries to close the window. You can override this function to handle window close events.
* **onRender(VkCommandBuffer)**:
This function is called to render the frame using the current command buffer of the frame. You can override this function to perform rendering operations using Vulkan. In `GltfRendererElement` this is where the active renderer is called.
* **onResize()**:
This function is called when the `viewport` is resized. You can override this function to handle window resize events. In `GltfRendererElement` the G-Buffer will be re-created
* **onUIRender()**:
This function is called to allow the `IAppElement` to render the UI and to query any mouse or keyboard event. In `GltfRendererElement`, we render the UI, but also the final image. The rendered image is consider a UI element, and that image covers the entire `viewport` ImGui window.
* **onUIMenu()**
Will be modifying what we see in the the window title. It will also create the menu, like `File`, `Help` and deal with some key combinations.
* **onFileDrop()**
Will receive the path of the file been dropped on. If it is a .gltf, .glb or .hdr, it will load that file.
----
## Scene Graph
The GLTF scene is loaded using tinygltf and then converted to a Vulkan version. The Vulkan version is a simplified version of the scene, where the geometry is stored in buffers, and the textures are uploaded to the GPU. The Vulkan version is used for both raster and ray tracing.
The scene is composed of nodes, where each node can have children and each node can have a mesh. The mesh is composed of primitives, where each primitive has a material. The material is composed of textures and parameters. However, none of this is directly used in the rendering, as we are using a simplified version of the scene.
Once the scene has been loaded, we proceed to parse it in order to collect the RenderNodes and RenderPrimitives. The RenderNode represents the flattened version of the tree of nodes, where the world transformation matrix and the material are stored. The RenderPrimitive, in contrast, represents the unique version of the primitive, where the index and vertex buffers are stored.
RenderNodes represent the elements to be rendered, while RenderPrimitives serve as references to the data utilized for rendering.
### Animation
If there is animation in the scene, a new section will appear under the Scene section.
It allows to play/pause, step and reset the animation, as well as changing its speed.
### Multiple Scene
If there are multiple scenes, a new section will appear under the Scene section.
It will show all the scenes and their name. Clicking on a scene name will switch to the scene.
### Material Variant
If there are multiple material variant, a new section will appear under the Scene section.
It will show all the material variant and their name. Clicking on a variant name will apply it on the models.
### Scene Graph UI
It is possible to visualize the scene hierarchy, to select node, to modify their transformation and their material, to some level.
Here's a shorter version of the text, tailored for developers on GitHub:
### Recompiling Shaders
For quick shader testing, use the `Recompile Shaders` button to hot-reload shaders. Requirements:
- Shaders in the `shaders` folder
- `shaderc_shared` and `slang` libraries in the path (copied to `bin` by default)
To force using the external shaders:
```bash
vk_gltf_reenderer.exe -forceExternalShaders
```
Note: Hot-reloading won't work without the shared libraries and shaders, but the app will still run.
## Tools
The application comes with a few tools to help debug and visualize the scene.
### Profiler
The profiler is a tool that allows to measure the time spent on the GPU. It is possible to measure the time spent on the different stages of the rendering, like the path tracing, the rasterization, the tonemapping, etc.
### Logger
The logger is a tool that allows to see the log information. It is possible to filter the log information by selecting the level of the log.
### Nvml
The Nvml is a tool that allows to see the status of the GPU. It is possible to see the temperature, the power, the memory usage, etc.
### Tangent Space
There is a tangent space tool that allows to fix or to recreate the tangent space of the model. This is useful when the normal map is not looking right or there are errors with the tangents in the scene.
## Utilities
### gltf-material-modifier.py
Modify materials in a GLTF file and optionally reorient the scene from Z-up to Y-up.
```
usage: gltf-material-modifier.py [-h] [--metallic METALLIC] [--roughness ROUGHNESS] [--override] [--reorient]
input_file output_file
```
positional arguments:
```
input_file Path to the input GLTF file.
output_file Path to save the modified GLTF file.
```
options:
```
-h, --help show this help message and exit
--metallic METALLIC Set the metallic factor (default: 0.1).
--roughness ROUGHNESS Set the roughness factor (default: 0.1).
--override Override existing material values if set.
--reorient Reorient the scene from Z-up to Y-up.
```