{"id":31842649,"url":"https://github.com/nvpro-samples/vk_denoise_dlssrr","last_synced_at":"2025-10-12T06:48:55.879Z","repository":{"id":317247728,"uuid":"968848074","full_name":"nvpro-samples/vk_denoise_dlssrr","owner":"nvpro-samples","description":"Example of integrating the DLSS-RR denoiser into a Vulkan application","archived":false,"fork":false,"pushed_at":"2025-09-29T18:07:46.000Z","size":3416,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-09-29T20:25:36.889Z","etag":null,"topics":["dlss","dlss3","pathtracing","pbr-shading","raytracing","vulkan"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nvpro-samples.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-04-18T21:07:40.000Z","updated_at":"2025-09-29T18:07:49.000Z","dependencies_parsed_at":"2025-09-29T20:25:56.288Z","dependency_job_id":"d6b75ace-3fbf-420c-9665-0d345f8e6339","html_url":"https://github.com/nvpro-samples/vk_denoise_dlssrr","commit_stats":null,"previous_names":["nvpro-samples/vk_denoise_dlssrr"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/nvpro-samples/vk_denoise_dlssrr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fvk_denoise_dlssrr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fvk_denoise_dlssrr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fvk_denoise_dlssrr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fvk_denoise_dlssrr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nvpro-samples","download_url":"https://codeload.github.com/nvpro-samples/vk_denoise_dlssrr/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fvk_denoise_dlssrr/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279010531,"owners_count":26084759,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-12T02:00:06.719Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dlss","dlss3","pathtracing","pbr-shading","raytracing","vulkan"],"created_at":"2025-10-12T06:48:50.728Z","updated_at":"2025-10-12T06:48:55.872Z","avatar_url":"https://github.com/nvpro-samples.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Integration of DLSS-RR into a Vulkan Application\n \n![A screenshot of the sample. It shows a path-traced scene, denoised using DLSS-RR_.](docs/dlss_denoiser.png)\n\nThis example demonstrates DLSS-RR (\"Deep Learning Super Sampling Ray Reconstruction\")\nin a simple path tracer rendering a glTF scene. DLSS-RR combines a denoiser, upscaler and\ntemporal antialiasing all in one package. Simplified, this means: present DLSS-RR with a series\nof noisy, jittered low resolution images and it will output a denoised, upscaled and jitter-free image.\n\n## Building \n\nFollow the general CMake process to configure and build the sample.\nDependencies will be automatically fetched, in particular the  DLSS-RR SDK will be\nautomatically pulled from [https://github.com/NVIDIA/DLSS](https://github.com/NVIDIA/DLSS)\n\n## CMake integration\n\n`CMakeLists.txt` uses nvpro_core's [`ngx.cmake`]() script to download the DLSS-RR\nSDK at configuration time. Since DLSS-RR itself does not offer direct CMake\nintegration and comes as a \"snippet\" which plugs into the NGX framework,\n`nvpro_core2/ngx.cmake` adds NGX as an imported library to the CMake build.\n\n`dlss_rr/CMakeLists.txt` then copies the DLSS shared libraries to the build and\ninstall directories:\n\n```\nlist(APPEND CMAKE_MODULE_PATH \"${CMAKE_CURRENT_LIST_DIR}/cmake\")\n\n# NGX (DLSS) dependency\nfind_package(NGX REQUIRED)\n\n...\n\ncopy_to_runtime_and_install(${PROJECT_NAME}\n  FILES ${DLSS_DLLS}\n  AUTO\n)\n```\n\n## Integration of DLSS-RR\n\nDLSS-RR can be used with DirectX and Vulkan. Its API is based on NGX, with DLSS-RR presenting itself as plugin (or\n\"Snippet\" in NGX' terms) to NGX . In this example we'll focus on the Vulkan integration. In contrast to NRD, DLSS-RR\nsupports Vulkan _directly_ - it can be fed directly with Vulkan image descriptors, it understands how to compile Vulkan\nshaders on its own and how to execute the DLSS-RR pipeline through a provided Vulkan command buffer. On top of that, it\ndoesn't require most of the texture data to be encoded in a particular way and offers some flexibility (for instance it\nsupports packed or separate normal and roughness textures). This means, there's no need to change one's own shaders\nmuch, especially if one had a denoiser already integrated. Overall, DLSS-RR's approach greatly simplifies the\nintegration with an existing Vulkan backend.\n\n### DLSS-RR wrapper classes\n\nIncluded in this sample come two classes \"NgxContext\" and \"DlssRR\".\n* \"NgxContext\" is responsible for creating a NGX context, querying the availability \nof the DLSS-RR feature and acting as factory for the DlssRR class.\n* DlssRR is used to setup and perform the upscaling-denoising.\n\nWith a little over 400 lines of code, it proves that a DLSS-RR integration \ninto Vulkan code is really simple.\n\nDlssRR wrapper is making use of the Vulkan specific NGX and DLSS-RR headers\n\n* nvsdk_ngx_vk.h\n* nvsdk_ngx_helpers_vk.h\n* nvsdk_ngx_helpers_dlssd_vk.h\n* nvsdk_ngx_defs_dlssd.h\n* nvsdk_ngx_helpers_dlssd.h\n\nBeware: there are some inconsistencies when the SDK and headers refer to DLSS-RR. Sometimes function calls are prefixed\nwith `_D` to refer to the DLSS-RR functions. The functions without the prefix actually only apply to DLSS, which is the\nolder supersampling-antialising solution. Sometimes the DLSS-RR feature is also referred to by just \"RayReconstruction\".\n\nThe two wrapper classes don't cover all functionality DLSS-RR can offer (refer to the [*DLSS-RR Integration Guide*](https://github.com/NVIDIA/DLSS/blob/af199869c51cf2d71cc64d3db5064788ff38eb02/doc/DLSS-RR%20Integration%20Guide%20API.pdf) to learn\nabout additional guide buffers and settings), but are small enough to be quickly adopted and modified for your own needs.\nBeware that the DLSS Integration Guide is thought to be addendum to the [*DLSS Programming Guide*](https://github.com/NVIDIA/DLSS/blob/af199869c51cf2d71cc64d3db5064788ff38eb02/doc/DLSS_Programming_Guide_Release.pdf)\n\nFeatures not covered in this sample\n* Providing more optional guide buffers besides the `RESOURCE_SPECULAR_HITDISTANCE`\n* Usage of 'hardware depth buffer' depth encoding in NDC coordinates ```math P.z / P.w ```\n* multi-GPU support\n\n## DLSS-RR input textures\n\nThe DlssRR class takes a number of textures ('resources' in DLSS-RR lingo) to forward to DLSS-RR later on. Here is a\ndescription of what data each resource represents. Refer to the *DLSS-RR Integration Guide* for further information on each\nresource.\n\n* `RESOURCE_COLOR_IN`: this texture contains the noisy input image to the denoiser. In this sample, its the output of the pathtracer.\n* `RESOURCE_COLOR_OUT`: this is the output texture of the denoiser, upscaled to output size\n* `RESOURCE_DIFFUSE_ALBEDO`: this texture contains the diffuse albedo\n* `RESOURCE_SPECULAR_ALBEDO`: this texture contains the specular reflectance for the current point of view, integrated over the surface's hemisphere. The DLSS-RR SDK contains a formula for computing an approximation of this term.\n* `RESOURCE_NORMALROUGHNESS`: normal and material roughness packed into a 4D vector. The normal vector should be between [-1..1] and roughness is linear.\n* `RESOURCE_MOTIONVECTOR`: this texture contains screenspace 2D vectors, pointing from the current pixel position back to \"where this pixel was in the last frame\"\n* `RESOURCE_LINEARDEPTH`: this is the linear depth of the primary hit position in camera space. For example, ```math float z = -(worldToView * hitPosition).z``` yields the desired value.\n* `RESOURCE_SPECULAR_HITDISTANCE`: this texture contains the distance from the primary hit surface to the first secondary hit. This is an optional guide buffer; complementary to specular motion vectors. Providing specular motion vectors is superior.\n\nWith the exception of `RESOURCE_COLOR_OUT`, all input buffers are presented in the (smaller) render resolution.\n`RESOURCE_COLOR_OUT` has the desired output resolution. In addition, the pathtracer will produce jittered primary\nsamples. DLSS-RR will take care of the upscaling and compensate for the jitter during the denoising process.\n\n## DLSS-RR Tips \u0026 Tricks\n\n### Interacting with the sample application\n\n* You can load a different scene or provide a different HDRI environment map by drag'n'drop of a '*.gtlf' or '*.hdr' file.\n* Some scenes have broken normal maps (vertically flipped). If thats the case the _Settings/Flip Bitangent_ setting may help\n* You can click on the guide buffers and have them shown in the main viewport instead of the normal render output. This is a nice way to see how noisy and relatively small the input signal actually is.\n* _DLSS RR/Show Buffers Scaled_ gives an idea about the upscaling in place.\n* _DLSS RR/Presets_ and _DLSS RR/Quality_ determine the (AI) model in use and quality setting\n* _DLSS RR/Input Width_ and _DLSS RR/Input Width_ lets you play with the size of the input buffers in the range the chosen quality setting allows for\n\n### Depth values\n\nPass either HW depth buffer _or_ view space (linear) depth. The HW depth range must be in [0, 1] range, while the linear depth is unbounded.\n```math \nhw_depth = clip_space.Z / clip_space.W\n```\nPass 'NVSDK_NGX_DLSS_Feature_Flags_DepthInverted' flag if smaller values of the passed depth buffer are further away.\nPrefer HW depth buffer. This is just for performance, and for matching buffers with DLSS-SR.\nPrefer 32 bit depth, but 16 bit can also work albeit might exhibit depth precision issues.\n\n### Matrices\n\nDLSS-RR expects matrices in row-major memory layout. These matrices are expected to be left-multiplied with row vectors:\n\n```math \n\\begin{bmatrix} \nx_1 \u0026 x_2 \u0026 x_3 \u0026 x_4\n\\end{bmatrix}\n\\begin{bmatrix} \na_{11} \u0026 a_{12} \u0026 a_{13} \u0026 a_{14} \\\\\na_{21} \u0026 a_{22} \u0026 a_{23} \u0026 a_{24} \\\\\na_{31} \u0026 a_{32} \u0026 a_{33} \u0026 a_{34} \\\\\na_{41} \u0026 a_{42} \u0026 a_{43} \u0026 a_{44}\n\\end{bmatrix} =\n\\begin{bmatrix}\nx_1' \u0026 x_2' \u0026 x_3' \u0026 x_4'\n\\end{bmatrix}\n````\n\nContrast this with GLM's column-major, right-multiply matrices:\n\n```math\n\\begin{bmatrix} \na_{11} \u0026 a_{12} \u0026 a_{13} \u0026 a_{14} \\\\\na_{21} \u0026 a_{22} \u0026 a_{23} \u0026 a_{24} \\\\\na_{31} \u0026 a_{32} \u0026 a_{33} \u0026 a_{34} \\\\\na_{41} \u0026 a_{42} \u0026 a_{43} \u0026 a_{44}\n\\end{bmatrix} \n\\begin{bmatrix} \nx_1 \\\\\nx_2 \\\\\nx_3 \\\\\nx_4\n\\end{bmatrix}\n=\n\\begin{bmatrix}\nx_1' \\\\\nx_2' \\\\\nx_3' \\\\\nx_4'\n\\end{bmatrix}\n```\n\nAs it happens to be, the required transpose operations to convert from one to the other results in identity; thus the sample passes the GLM matrices straight into DLSS-RR.\n\n### Sky\n\nFor the sky to not show trailing artifacts, you _must_ provide motion vectors for the sky (in particular covering the\ncamera's rotation). In this sample, we simply project the view vector as \"point on sky projected to infinity\" back into\nthe previous frame to obtain motion vectors for the sky. In addition, we bake a tonemapped color value for the sky\ninto the diffuse albedo buffer to keep the denoiser from removing desired detail in the sky.\n\n### Mirror-like surfaces\n\nMirror-like surfaces work better with a special treatment. Instead of recording the position and hit parameters at the\nsurface of the mirror, we continue following the mirrored ray until it hits something that is not a mirror - the\n_Primary Surface Replacement (PSR)_. It looks like as if we can see the mirrored objects appear \"behind\" the mirror as\nthough the mirror acts as kind of a portal into a virtual world. The G-Buffer records the data of the PSR at its\nvirtual world space, such as Normal, Roughness and ViewZ. This improves denoising reflected objects.\n\nLook into `shaders/primary__rgen.slang` for `#PSR` to find the shader code that implements\nprimary surface replacement.\n\n### MIS weighting\n\nThe implemented path tracer uses importance sampling to estimate the integral in the [rendering\nequation](https://en.wikipedia.org/wiki/Rendering_equation). Care has to be taken when the path tracer samples the same\nfunction multiple times, but uses different PDFs when doing so. This sample is built such that, at the primary hit and\neach segment of the indirect path, we sample the environment map for its direct light contribution at that point. It\nthen follows the material's BSDF to probe for indirect lighting contribution at the hit point. Thus, for any surface\npoint along the path it can happen that the environment is sampled twice: once via directly probing the environment map\n(the envmap has its own precomputed sampling distribution) and once more when following the material's BSDF at each hit\npoint. Both contributions need to be weighted accordingly. In this sample we chose the \"power heuristic\" to compute the\nMIS weights for both contributions.\n\n\n## Authors and Metadata\n\nTags:\n\n- raytracing, path-tracing, GLTF, HDR, tonemapper, picking, BLAS, TLAS, PBR material, denoising, DLSS-RR, Slang\n\nExtensions:\n\n- VK_KHR_buffer_device_address, VK_KHR_acceleration_structure,VK_KHR_ray_tracing_pipeline, \nVK_KHR_ray_query, VK_KHR_push_descriptor, VK_KHR_shader_clock, VK_KHR_create_renderpass2\n\nAuthors:\n- Mathias Heyer\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvpro-samples%2Fvk_denoise_dlssrr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvpro-samples%2Fvk_denoise_dlssrr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvpro-samples%2Fvk_denoise_dlssrr/lists"}