{"id":19817167,"url":"https://github.com/nvpro-samples/gl_vk_raytrace_interop","last_synced_at":"2025-07-27T01:05:36.692Z","repository":{"id":86083461,"uuid":"225621496","full_name":"nvpro-samples/gl_vk_raytrace_interop","owner":"nvpro-samples","description":"Adding ray traced ambient occlusion using Vulkan and OpenGL ","archived":false,"fork":false,"pushed_at":"2024-01-17T16:45:05.000Z","size":3131,"stargazers_count":33,"open_issues_count":2,"forks_count":5,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-05-01T11:36:24.277Z","etag":null,"topics":["interop","opengl","raytracing","vulkan"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nvpro-samples.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-03T13:02:47.000Z","updated_at":"2024-10-11T02:26:51.000Z","dependencies_parsed_at":"2024-11-12T10:11:54.527Z","dependency_job_id":"04d122ae-8669-4264-af5e-4c95d3302113","html_url":"https://github.com/nvpro-samples/gl_vk_raytrace_interop","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nvpro-samples/gl_vk_raytrace_interop","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fgl_vk_raytrace_interop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fgl_vk_raytrace_interop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fgl_vk_raytrace_interop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fgl_vk_raytrace_interop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nvpro-samples","download_url":"https://codeload.github.com/nvpro-samples/gl_vk_raytrace_interop/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fgl_vk_raytrace_interop/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267278631,"owners_count":24063252,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-26T02:00:08.937Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["interop","opengl","raytracing","vulkan"],"created_at":"2024-11-12T10:11:52.760Z","updated_at":"2025-07-27T01:05:36.675Z","avatar_url":"https://github.com/nvpro-samples.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OpenGL Interop - Raytracing\n\n\nBy [Martin-Karl Lefrançois](https://devblogs.nvidia.com/author/mlefrancois/)\n\n# Introduction\n\nThis blog is a step after the introduction to OpenGL and Vulkan interop blog.\nThe first example explain how to mix Vulkan and OpenGL in the same application.\nThis one includes more buffer types and instead of using a computer shader to generate an image,\nit is using raytracing to add ambient occlusion to the scene.\n\nTopics covered in the example:\n- Managing OpenGL memory from Vulkan\n- G-Buffers\n- Raytracing\n- Semaphores\n- Compositing\n\n![Screenshot](doc/screenshot.png)\n\n\n# Vulkan-OpenGL Memory Objects\n\nAs explained in the vk_gl_simple_interop example, all shared objects must be allocated by Vulkan.\nIn this example, we will use an allocator to manage the memory. This [blog](https://developer.nvidia.com/vulkan-memory-management)\ngives some explaination on how it works, but actual information is also available in the header of `allocator_dma_vk.hpp`.\n\nThe allocator we are using is call DMA, which stands for Device Memory Allocator. There is the `nvvkpp::AllocatorDma` which is\na helper for creating buffers, images and acceleration structures, and the `nvvk::DeviceMemoryAllocatorGL` which does the\nmemory allocations. The later one has a slight variation from the none-GL, as it is flagging the allocation to be exportable\nand importing the memory to OpenGL to be used.\n\nThe scene is creating all geometry meshes using a `Vertex` buffer and an `Index` buffer created with DMA/GL. We are \nallocating all geometry in the scene in Vulkan, because we want to rasterize the scene using OpenGL, but raytrace the \nscene to get ambient occlusion. For this reason, all elements of the scene are shared.\n \nIf you are familiar with other nvpro-samples examples, the allocation for Vulkan does not differ, but the OpenGL buffer will still need to be created.\n\nFor example, in the `VkGlExample::createFacetedTetrahedron()` where we are constructing the geometry of a tetrahedron,\nthe allocation is done in Vulkan.\n\n~~~~C++\n mesh.vertices.bufVk = m_alloc.createBuffer\u003cVertex\u003e(cmdBuf, vert, vk::BufferUsageFlagBits::eVertexBuffer);\n~~~~\n\nThis only creates the Vulkan buffer and the memory allocated from DMA/GL was flaged to be exported. What we need is \nto create the equivalent OpenGL buffer.\n\n~~~~C++\ninline void createBufferGL(BufferVkGL\u0026 bufGl, nvvk::DeviceMemoryAllocatorGL\u0026 memAllocGL)\n{\n  nvvk::AllocationGL allocGL = memAllocGL.getAllocationGL(bufGl.bufVk.allocation);\n\n  glCreateBuffers(1, \u0026bufGl.oglId);\n  glNamedBufferStorageMemEXT(bufGl.oglId, allocGL.size, allocGL.memoryObject, allocGL.offset);\n}\n~~~~ \n\nSee `VkGlExample::vulkanMeshToOpenGL()` for an example on how all meshes are also represented in OpenGL.\n\nFor textures, this is done is a similar fashion. The image is created and allocated in Vulkan using DMA/GL, then \nan OpenGL version is created. \n\n~~~~C++\nm_imageVkGL.texVk = m_alloc.createImage(cmdBuf, dataSize, dataImage, imgInfo, vk::ImageLayout::eShaderReadOnlyOptimal);\ncreateTextureGL(m_imageVkGL, GL_RGBA8, GL_LINEAR_MIPMAP_LINEAR, GL_LINEAR, GL_REPEAT, m_dmaAllocGL);\n~~~~\n\n~~~~C++\ninline void createTextureGL(Texture2DVkGL\u0026 texGl, int format, int minFilter, int magFilter, int wrap, nvvk::DeviceMemoryAllocatorGL\u0026 memAllocGL)\n{\n  auto allocGL = memAllocGL.getAllocationGL(texGl.texVk.allocation);\n\n  // Create a 'memory object' in OpenGL, and associate it with the memory allocated in Vulkan\n  glCreateTextures(GL_TEXTURE_2D, 1, \u0026texGl.oglId);\n  glTextureStorageMem2DEXT(texGl.oglId, texGl.mipLevels, format, texGl.imgSize.width, texGl.imgSize.height,\n                           allocGL.memoryObject, allocGL.offset);\n  glTextureParameteri(texGl.oglId, GL_TEXTURE_MIN_FILTER, minFilter);\n  glTextureParameteri(texGl.oglId, GL_TEXTURE_MAG_FILTER, magFilter);\n  glTextureParameteri(texGl.oglId, GL_TEXTURE_WRAP_S, wrap);\n  glTextureParameteri(texGl.oglId, GL_TEXTURE_WRAP_T, wrap);\n}\n~~~~\n\n\n\n\n\n\n# G-Buffers and Output Buffer\n\nThe example will render the scene in a Framebuffer Object, using 3 attached buffers.\nThose buffers will later be used by the raytracer as information to send rays for the generation\nof Ambient Occlusion. In this sample we used high-precision G-Buffers for ease of use,\nhowever in a production implementation you would want to use packed formats and reconstruct\npositions from existing depth-buffers. This would reduce bandwidth requirements greatly and\nthere is several techniques that can be used from deferred shading.\n\n\n- Position RGBA32\n- Normal RGBA32\n- Albedo RGBA8\n\n\n![Positions](doc/position.png) ![Normals](doc/normal.png) ![Albedo](doc/albedo.png)\n\n\nThe creation of the FBO and textures is done the same way as for allocating a texture image. The difference resides in\npassing nullptr to the data, as there is nothing to transfer from.\n\nThe call to `createTextureGL` is to make the image available for both OpenGL an Vulkan.\n\nThis is how we are creating the G-Buffers\n~~~~C++\nvoid VkGlExample::createGBuffers()\n{\n  m_queue.waitIdle();\n  m_device.waitIdle();\n\n  if(m_gFramebuffer != 0)\n  {\n    glDeleteFramebuffers(1, \u0026m_gFramebuffer);\n    m_gBufferColor[0].destroy(m_alloc);\n    m_gBufferColor[1].destroy(m_alloc);\n    m_gBufferColor[2].destroy(m_alloc);\n  }\n\n  glCreateFramebuffers(1, \u0026m_gFramebuffer);\n  glBindFramebuffer(GL_DRAW_FRAMEBUFFER, m_gFramebuffer);\n\n\n  vk::DeviceSize imgSize = m_size.width * m_size.height * 4 * sizeof(float);\n\n  auto samplerInfo = vk::SamplerCreateInfo();\n\n  m_gBufferColor[0].imgSize = m_size;\n  m_gBufferColor[1].imgSize = m_size;\n  m_gBufferColor[2].imgSize = m_size;\n\n  m_gBufferColor[0].texVk = m_alloc.createImage(nvvkpp::image::create2DInfo(m_size, vk::Format::eR32G32B32A32Sfloat));\n  m_gBufferColor[1].texVk = m_alloc.createImage(nvvkpp::image::create2DInfo(m_size, vk::Format::eR32G32B32A32Sfloat));\n  m_gBufferColor[2].texVk = m_alloc.createImage(nvvkpp::image::create2DInfo(m_size, vk::Format::eR8G8B8A8Unorm));\n\n  m_gBufferColor[0].texVk.descriptor =\n      nvvkpp::image::create2DDescriptor(m_device, m_gBufferColor[0].texVk.image, samplerInfo, vk::Format::eR32G32B32A32Sfloat);\n  m_gBufferColor[1].texVk.descriptor =\n      nvvkpp::image::create2DDescriptor(m_device, m_gBufferColor[1].texVk.image, samplerInfo, vk::Format::eR32G32B32A32Sfloat);\n  m_gBufferColor[2].texVk.descriptor =\n      nvvkpp::image::create2DDescriptor(m_device, m_gBufferColor[2].texVk.image, samplerInfo, vk::Format::eR8G8B8A8Unorm);\n\n  createTextureGL(m_gBufferColor[0], GL_RGBA32F, GL_NEAREST, GL_NEAREST, GL_REPEAT, m_dmaAllocGL);\n  createTextureGL(m_gBufferColor[1], GL_RGBA32F, GL_NEAREST, GL_NEAREST, GL_REPEAT, m_dmaAllocGL);\n  createTextureGL(m_gBufferColor[2], GL_RGBA8, GL_NEAREST, GL_NEAREST, GL_REPEAT, m_dmaAllocGL);\n\n  glNamedFramebufferTexture(m_gFramebuffer, GL_COLOR_ATTACHMENT0, m_gBufferColor[0].oglId, 0);\n  glNamedFramebufferTexture(m_gFramebuffer, GL_COLOR_ATTACHMENT1, m_gBufferColor[1].oglId, 0);\n  glNamedFramebufferTexture(m_gFramebuffer, GL_COLOR_ATTACHMENT2, m_gBufferColor[2].oglId, 0);\n\n  {\n    nvvkpp::ScopeCommandBuffer cmdBuf(m_device, m_graphicsQueueIndex);\n    nvvkpp::image::setImageLayout(cmdBuf, m_gBufferColor[0].texVk.image, vk::ImageLayout::eUndefined,\n                                  vk::ImageLayout::eShaderReadOnlyOptimal);\n    nvvkpp::image::setImageLayout(cmdBuf, m_gBufferColor[1].texVk.image, vk::ImageLayout::eUndefined,\n                                  vk::ImageLayout::eShaderReadOnlyOptimal);\n    nvvkpp::image::setImageLayout(cmdBuf, m_gBufferColor[2].texVk.image, vk::ImageLayout::eUndefined,\n                                  vk::ImageLayout::eShaderReadOnlyOptimal);\n  }\n\n  // - tell OpenGL which color attachments we'll use (of this framebuffer) for rendering\n  unsigned int attachments[3] = {GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1, GL_COLOR_ATTACHMENT2};\n  glDrawBuffers(3, attachments);\n\n\n  // create and attach depth buffer (render buffer)\n  unsigned int rboDepth;\n  glGenRenderbuffers(1, \u0026rboDepth);\n  glBindRenderbuffer(GL_RENDERBUFFER, rboDepth);\n  glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT, m_size.width, m_size.height);\n  glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, rboDepth);\n  // finally check if framebuffer is complete\n  if(glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE)\n  {\n    std::cout \u003c\u003c \"Framebuffer not complete!\" \u003c\u003c std::endl;\n  }\n  glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);\n}\n~~~~~~~~\n\nThe ouptput buffer used by the raytracer will be done in a very similar  way. Since we\nare writing to the image, the usage flag `eStorage` has to be present.\n\n~~~~ C+++\n  void createOutputImage(vk::Extent2D size)\n  {\n    auto usage = vk::ImageUsageFlagBits::eSampled | vk::ImageUsageFlagBits::eStorage | vk::ImageUsageFlagBits::eTransferSrc;\n    vk::DeviceSize imgSize = size.width * size.height * 4 * sizeof(float);\n    auto           format  = vk::Format::eR32Sfloat;\n    auto           layout  = vk::ImageLayout::eGeneral;\n\n    vk::SamplerCreateInfo samplerCreateInfo;  // default values\n    vk::ImageCreateInfo   imageCreateInfo = nvvkpp::image::create2DInfo(size, format, usage);\n\n    // Creating the image and the descriptor\n    m_raytracingOutputGL.imgSize = size;\n    m_raytracingOutputGL.texVk   = m_allocGL.createImage(imageCreateInfo);\n    m_raytracingOutputGL.texVk.descriptor =\n        nvvkpp::image::create2DDescriptor(m_device, m_raytracingOutputGL.texVk.image, vk::SamplerCreateInfo(), format, layout);\n\n    {\n      // Setting the layout to eGeneral\n      nvvkpp::ScopeCommandBuffer cmdBuf(m_device, m_queueIndex);\n      nvvkpp::image::setImageLayout(cmdBuf, m_raytracingOutputGL.texVk.image, vk::ImageLayout::eUndefined, layout);\n    }\n\n    // Making the OpenGL version of texture\n    createTextureGL(m_raytracingOutputGL, GL_R32F, GL_LINEAR_MIPMAP_LINEAR, GL_LINEAR, GL_REPEAT, m_dmaAllocGL);\n  }\n~~~~~~~~\n\n\n\n\n# Raytracing\n\nThe raytracing part is covered by a previous [tutorial](https://developer.nvidia.com/rtx/raytracing/vkray)\n\nThere are somehow differences in the technique used for rendering. In this example, we are rendering using\nOpenGL, the world position and normal to G-Buffers. Those buffers are used in the `rayGen` shader to\nbe the starting point to send rays.\n\n\nThere is no hit shader, just a miss shader. We assume the ray will hit, but if the short ray is not hitting\nany geometry, the miss shader will be called. With this information and sending many rays in the hemisphere\nof the hit position, we can compute the ambient occlusion.\n\n\n## Converting the Scene to Raytrace\nThe conversion of the meshes is done in `VkGlExample::meshToGeometry`, which is what the BLAS construction is expecting.\n\nThen we convert all instances to the raytracing instances to create the TLAS.\n\n\n## Extra\n\nThen we create the part which is unique to this example. We first create a descriptor set and pass the\nG-Buffer to be read, create a pipeline with our shaders and the shading binding table with the handles\nto the shaders.\n\n\n## Rendering\nOnce the acceleration structures are build and the raytracing pipeline is created. Rendering or\ntracing rays is similar to any other Vulkan call. Once finished, see section Semaphores, the\nimage with AO will be filled with the data.\n\n~~~C++\nm_commandBuffer.bindPipeline(vk::PipelineBindPoint::eRayTracingNV, m_pipeline);\nm_commandBuffer.bindDescriptorSets(vk::PipelineBindPoint::eRayTracingNV, m_pipelineLayout, 0, {m_descSet}, {});\nm_commandBuffer.pushConstants\u003cPushConstant\u003e\n(m_pipelineLayout, vk::ShaderStageFlagBits::eRaygenNV, 0, m_pushC);\n\nuint32_t progSize = m_raytracingProperties.shaderGroupHandleSize;  // Size of a program identifier\nvk::DeviceSize rayGenOffset   = 0;\nvk::DeviceSize missOffset     = progSize;\nvk::DeviceSize missStride     = progSize;\nvk::DeviceSize hitGroupOffset = progSize + progSize; // not used\nvk::DeviceSize hitGroupStride = progSize;            // not used\n\nm_commandBuffer.traceRaysNV(m_sbtBuffer.buffer, rayGenOffset, m_sbtBuffer.buffer, missOffset, missStride,\nm_sbtBuffer.buffer, hitGroupOffset, hitGroupStride, vk::Buffer{}, 0, 0,\nm_raytracingOutputGL.extent.width, m_raytracingOutputGL.extent.height, 1, m_vkctx.dynamicDispatch);\n~~~~\n\n\nThe result of the raytracer (Ambient Occlusion) will go in a result texture, which will be used when\ncompositing the final image.\n\n![ao](doc/ao.png)\n\n!!! Note\nThe raytracing code is under `raytrace_vk.hpp` and `raytrace_interop.hpp`\n\n\n\n# Semaphores\n\nDrawing with OpenGL or with Vulkan is not immediate and to make sure the information about the stage is\ncommunicated between Vulkan and OpenGL, we use semaphores.\n\nSurrounding the call to the actual raytracing, we have to signal Vulkan to wait for the `vk/gl/Complete`.\nVulkan will raytrace the image and it will signal `vk/gl/Ready`, where OpenGL wait for the signal.\nThis is important, otherwise the combination of the image will be missing parts.\n\n~~~~ C++\n// Raytracing the scene\nGLuint outRayID = m_ray.outputImage().oglId;\n{\n// Once render is complete, signal the Vulkan semaphore indicating it can start render\nGLenum dstLayout = GL_LAYOUT_SHADER_READ_ONLY_EXT;\nGLenum srcLayout = GL_LAYOUT_COLOR_ATTACHMENT_EXT;\nglSignalSemaphoreEXT(m_ray.semaphores().glComplete, 0, nullptr, 1, \u0026outRayID, \u0026dstLayout);\nm_ray.run();\n// And wait (on the GPU) for the raytraced image\nglWaitSemaphoreEXT(m_ray.semaphores().glReady, 0, nullptr, 1, \u0026outRayID, \u0026srcLayout);\n}\n~~~~\n\n\n\n# Compositing\n\nThis is the final step, where the various textures are used to compose the final image.\n\nA naive approach would be to return `Albedo * Occlusion`, but in this example we are\nalso doing a box filtering on the occlusion texture. To avoid bluring neighbor objects\na depth test is done using the `w` component of the `position` texture.\n\n![Distance](doc/distance.png)\n\n# Final Image\n![Final Image](doc/screenshot.png)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvpro-samples%2Fgl_vk_raytrace_interop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvpro-samples%2Fgl_vk_raytrace_interop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvpro-samples%2Fgl_vk_raytrace_interop/lists"}