{"id":19817156,"url":"https://github.com/nvpro-samples/gl_vk_simple_interop","last_synced_at":"2025-09-04T12:31:37.014Z","repository":{"id":39373334,"uuid":"225621686","full_name":"nvpro-samples/gl_vk_simple_interop","owner":"nvpro-samples","description":"Display an image created by Vulkan compute shader, with OpenGL","archived":false,"fork":false,"pushed_at":"2024-06-28T09:55:08.000Z","size":922,"stargazers_count":88,"open_issues_count":1,"forks_count":17,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-12-24T06:09:29.503Z","etag":null,"topics":["compute-shader","interop","opengl","vulkan"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nvpro-samples.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-03T13:03:45.000Z","updated_at":"2024-12-02T07:08:46.000Z","dependencies_parsed_at":"2024-06-28T11:20:59.145Z","dependency_job_id":null,"html_url":"https://github.com/nvpro-samples/gl_vk_simple_interop","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fgl_vk_simple_interop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fgl_vk_simple_interop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fgl_vk_simple_interop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvpro-samples%2Fgl_vk_simple_interop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nvpro-samples","download_url":"https://codeload.github.com/nvpro-samples/gl_vk_simple_interop/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231959788,"owners_count":18452127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compute-shader","interop","opengl","vulkan"],"created_at":"2024-11-12T10:11:51.364Z","updated_at":"2024-12-31T08:11:41.190Z","avatar_url":"https://github.com/nvpro-samples.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OpenGL Interop\n\n\nBy [Martin-Karl Lefrançois](https://devblogs.nvidia.com/author/mlefrancois/)\n\n\n\nThis blog is an introduction to OpenGL and Vulkan interop. The goal is to explain how to mix Vulkan and \nOpenGL in the same application. In a nutshell, to achieve this, all objects are allocated in Vulkan, \nbut rendered with OpenGL.\n\nTopics covered:\n- Managing OpenGL memory from Vulkan\n- Interoperability OGL \u003c==\u003e VK\n- Semaphores\n\n![Screenshot](doc/screenshot.png)\n\n# Interop Paradigm\n\nFor OpenGL to work with Vulkan, it is important that all memory objects (buffers) are allocated in Vulkan. \nA handle of that memory needs to be retrieved to create the OpenGL element. This new \nOpenGL object points to the exact same memory location as the Vulkan one, meaning that changes through \neither API are visible on both sides.\n\nIn the current example, we will deal with two memory objects:\n\n- Vertices: a buffer of a triangle's vertex positions\n- Image: the pixels of the image\n\nAnother important aspect is the synchronization between OpenGL and Vulkan. This topic will be discussed in detail\nin the Semaphores section.\n\n\n![Screenshot](doc/interop_api.png )\n\n# Prerequisite\n\n## Vulkan Instance and Device \n\nA Vulkan Instance and Device must be created in order to create and allocate memory buffers on a physical device. \n\nMain.cpp in the example creates a Vulkan instance and device using `nvvk::Context::init()`.  To create the Vulkan Device, we do not need a \nsurface since we will not draw anything using Vulkan.\n\n\n\n## Vulkan Extensions\n\nTo successfully export objects to other APIs, we need to enable several Vulkan extensions.\n\nSpecifically, we need these instance extensions:\n\n- **VK_KHR_EXTERNAL_MEMORY_CAPABILITIES_EXTENSION_NAME**\n- **VK_KHR_EXTERNAL_SEMAPHORE_CAPABILITIES_EXTENSION_NAME**\n\nWe specify each of these extensions using `nvvk::ContextCreateInfo::addInstanceExtension()`. If the instance doesn't support both of these, the sample will exit.\n\nIn addition, we'll need these device extensions:\n\n- **VK_KHR_EXTERNAL_MEMORY_EXTENSION_NAME**\n- **VK_KHR_EXTERNAL_SEMAPHORE_EXTENSION_NAME**\n\nAnd two platform-specific device extensions, depending on whether the app's running on Windows or Linux:\n\n**Windows:**\n\n* **VK_KHR_EXTERNAL_MEMORY_WIN32_EXTENSION_NAME**\n* **VK_KHR_EXTERNAL_SEMAPHORE_WIN32_EXTENSION_NAME**\n\n**Linux:**\n\n* **VK_KHR_EXTERNAL_MEMORY_FD_EXTENSION_NAME**\n* **VK_KHR_EXTERNAL_SEMAPHORE_FD_EXTENSION_NAME**\n\nWe specify each of these extensions using `nvvk::ContextCreateInfo::addDeviceExtension()`.  Whether a GPU supports an extension depends both on the GPU and the GPU's driver. `nvvk::Context` will choose the first GPU that supports all the extensions we need; if none exist, the sample will exit.\n\n## OpenGL\nWe use OpenGL 4.5 and need the extensions [EXT_external_objects](https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects.txt) \nand [GL_EXT_semaphore](https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects.txt).\n\nHere are the functions from these extensions we use:\n* `glCreateMemoryObjectsEXT`\n* `glImportMemoryWin32HandleEXT`\n* `glNamedBufferStorageMemEXT`\n* `glTextureStorageMem2DEXT`\n* `glSignalSemaphoreEXT`\n* `glWaitSemaphoreEXT`\n\n\n\n\n# Vulkan Allocation\n\nIn order to allocate a Vulkan buffer or image that can be exported to other APIs, we must use the [VK_KHR_external_memory](https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VK_KHR_external_memory.html) extension.\n\nIn this example, we use a simple Vulkan memory allocator, `nvvk::ExportResourceAllocatorDedicated`. This allocator uses dedicated allocation — one memory allocation per buffer. This is not the recommended way; it would be better to allocate larger memory blocks and suballocate by binding buffers to some memory sections (see [this article](https://developer.nvidia.com/vulkan-memory-management)), but it is fine for the purpose of this example. (One way of doing this using NVVK is to pass a different memory allocator, such as `DMAMemoryAllocator`, to `ExportResourceAllocator`'s constructor.)\n\nNormally, memory allocation is done like this:\n\n~~~~C++\nvirtual VkDeviceMemory AllocateMemory(const VkMemoryAllocateInfo\u0026 allocateInfo)\n{\n  VkDeviceMemory memory;\n  vkAllocateMemory(m_device, \u0026allocateInfo, nullptr, \u0026memory);\n  return memory;\n}\n~~~~\n\nBut `ExportResourceAllocator` flags all its memory allocations as exportable, like this:\n~~~C++\nVkDeviceMemory AllocateMemory(const VkMemoryAllocateInfo\u0026 allocateInfo) override\n{\n  // Enable export to either a Win32 handle or a POSIX file descriptor,\n  // depending on the OS:\n  VkExportMemoryAllocateInfo exportInfo {VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO};\n#ifdef WIN32\n  exportInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_OPAQUE_WIN32_BIT;\n#else // POSIX\n  exportInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT;\n#endif\n  VkMemoryAllocateInfo modifiedInfo = allocateInfo;\n  modifiedInfo.pNext = \u0026exportInfo;\n    \n  VkDeviceMemory memory;\n  vkAllocateMemory(m_device, \u0026modifiedInfo, nullptr, \u0026memory);\n  return memory;\n}\n~~~\n\n\n\n\n# OpenGL Handle and Memory Object\n\nTo retrieve the memory object for OpenGL, we must get the object's handle -- a `HANDLE`\n on Windows, and a POSIX file descriptor (`int`) on POSIX-compatible systems. See file: `gl_vkpp.hpp`\n\n~~~~C++\n// #VKGL Extra for Interop\nstruct BufferVkGL\n{\n  nvvk::Buffer bufVk;  // The allocated buffer\n\n#ifdef WIN32\n  HANDLE handle = nullptr;  // The Win32 handle\n#else\n  int fd = -1;\n#endif\n  GLuint memoryObject = 0;  // OpenGL memory object\n  GLuint oglId        = 0;  // OpenGL object ID\n};\n~~~~\n\n\n~~~~ C++\n// #VKGL:  Get the shared handle between Vulkan and OpenGL\n#ifdef WIN32\n  VkMemoryGetWin32HandleInfoKHR getInfo = {\n    .sType      = VK_STRUCTURE_TYPE_MEMORY_GET_WIN32_HANDLE_INFO_KHR,\n    .memory     = info.memory,\n    .handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT_KHR};\n  NVVK_CHECK(vkGetMemoryWin32HandleKHR(device, \u0026getInfo, \u0026bufGl.handle));\n#else\n  VkMemoryGetFdInfoKHR getInfo = {\n    .sType      = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR,\n    .memory     = info.memory,\n    .handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT_KHR};\n  NVVK_CHECK(vkGetMemoryFdKHR(device, \u0026getInfo, \u0026bufGl.fd));\n#endif\n~~~~\n\nUsing the handle, we can retrieve the equivalent OpenGL memory object.\n\n~~~~ C++\n  glCreateMemoryObjectsEXT(1, \u0026bufGl.memoryObject);\n#ifdef WIN32\n  glImportMemoryWin32HandleEXT(bufGl.memoryObject, req.size, GL_HANDLE_TYPE_OPAQUE_WIN32_EXT, bufGl.handle);\n#else\n  glImportMemoryFdEXT(bufGl.memoryObject, req.size, GL_HANDLE_TYPE_OPAQUE_FD_EXT, bufGl.fd);\n  // fd got consumed\n  bufGl.fd = -1;\n#endif\n~~~~\n\n\n\n# OpenGL Memory Binding\n\nTo use the retrieved OpenGL memory object, you must create the buffer then _link it_ using the \n[External Memory Object](https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects.txt) extension.\n\nIn Vulkan we bind memory to our resources, in OpenGL we can create new resources from a range within imported memory, \nor we can attach existing resources to use that memory via [NV_memory_attachment](https://www.khronos.org/registry/OpenGL/extensions/NV/NV_memory_attachment.txt).\n\n~~~~C++\n  glCreateBuffers(1, \u0026bufGl.oglId);\n  glNamedBufferStorageMemEXT(bufGl.oglId, req.size, bufGl.memoryObject, info.offset);\n~~~~\n\n`bufGl.oglId` now shares data with the buffer that was created in Vulkan.\n\n\n\n# OpenGL Images\n\nFor images, everything is done the same way as for buffers. We create an image using Vulkan, making sure to flag it as exportable using `ExportResourceAllocatorDedicated`. Then we create an OpenGL image using the same underlying memory in the `reateTextureGL()` function.\n\nWe retrieve the image's memory handle the same way we retrieved the buffer's handle:\n~~~~C++\n#ifdef WIN32\n  VkMemoryGetWin32HandleInfoKHR getInfo = {\n    .sType      = VK_STRUCTURE_TYPE_MEMORY_GET_WIN32_HANDLE_INFO_KHR,\n    .memory     = info.memory,\n    .handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT_KHR};\n  NVVK_CHECK(vkGetMemoryWin32HandleKHR(device, \u0026getInfo, \u0026texGl.handle));\n#else\n  VkMemoryGetFdInfoKHR getInfo = {\n    .sType      = VK_STRUCTURE_TYPE_MEMORY_GET_FD_INFO_KHR,\n    .memory     = info.memory,\n    .handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT_KHR};\n  NVVK_CHECK(vkGetMemoryFdKHR(device, \u0026getInfo, \u0026texGl.fd));\n#endif\n~~~~\n\nThen we import the Vulkan handle into an OpenGL memory object, like we did for the vertex buffer:\n~~~~ C++\n  // Create a 'memory object' in OpenGL, and associate it with the memory allocated in Vulkan\n  glCreateMemoryObjectsEXT(1, \u0026texGl.memoryObject);\n#ifdef WIN32\n  glImportMemoryWin32HandleEXT(texGl.memoryObject, req.size, GL_HANDLE_TYPE_OPAQUE_WIN32_EXT, texGl.handle);\n#else\n  glImportMemoryFdEXT(texGl.memoryObject, req.size, GL_HANDLE_TYPE_OPAQUE_FD_EXT, texGl.fd);\n  // fd got consumed\n  texGl.fd = -1;\n#endif\n~~~~\n\nFinally, the texture will be created using the memory object \n\n~~~~C++\n  glCreateTextures(GL_TEXTURE_2D, 1, \u0026texGl.oglId);\n  glTextureStorageMem2DEXT(texGl.oglId, texGl.mipLevels, format, texGl.imgSize.width, texGl.imgSize.height, texGl.memoryObject, info.offset);\n~~~~\n\n\n\n# Semaphores\n\nAs we are creating an image through Vulkan and displaying it with OpenGL,\nit is necessary to synchronize the two environments. We'll use a semaphore to make Vulkan wait for OpenGL to tell it when it can start rendering, and a second semaphore to make OpenGL wait for Vulkan to tell it when the image is ready.\n\n~~~~ batch\n                                                           \n  +------------+                             +------------+\n  | GL Context | signal               wait   | GL Context |\n  +------------+     |                  ^    +------------+\n                     v  +-----------+   |                  \n                   wait |Vk Context | signal               \n                        +-----------+                      \n~~~~\n\nLike images and buffers, we'll create semaphores using Vulkan, and then retrieve an OpenGL object. This part of the code is in `compute.hpp`:\n\n~~~~ C++\nstruct Semaphores\n{\n  VkSemaphore vkReady;\n  VkSemaphore vkComplete;\n  GLuint      glReady;\n  GLuint      glComplete;\n} m_semaphores;\n~~~~~\n\nExported semaphore handles use the `HANDLE` type on Windows, and the type for a POSIX file descriptor (`int`) on POSIX-compatible platforms.\n~~~~C++\n#ifdef WIN32\n  const auto handleType = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_WIN32_BIT;\n#else\n  const auto handleType = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD_BIT;\n#endif\n~~~~\n\nWhen creating semaphores, we need to flag them as exportable by adding a `VkExportSemaphoreCreateInfo` struct to the `pNext` chain of `VkSemaphoreCreateInfo`:\n~~~~C++\nVkExportSemaphoreCreateInfo esci{\n  .sType = VK_STRUCTURE_TYPE_EXPORT_SEMAPHORE_CREATE_INFO,\n  .handleTypes = handleType\n};\nVkSemaphoreCreateInfo sci{\n  .sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO,\n  .pNext = \u0026esci\n};\nNVVK_CHECK(vkCreateSemaphore(m_device, \u0026sci, nullptr, \u0026m_semaphores.vkReady));\nNVVK_CHECK(vkCreateSemaphore(m_device, \u0026sci, nullptr, \u0026m_semaphores.vkComplete));\n~~~~\n\nWe then retrieve each semaphore's handle using one of the `vkGetSemaphore*KHR` functions, and import it into OpenGL using one of the `glImportSemaphore*EXT` functions:\n~~~~C++\n// Import semaphores\n#ifdef WIN32\n  HANDLE hglReady{};\n  VkSemaphoreGetWin32HandleInfoKHR handleInfo{\n    .sType      = VK_STRUCTURE_TYPE_SEMAPHORE_GET_WIN32_HANDLE_INFO_KHR,\n    .semaphore  = m_semaphores.vkReady,\n    .handleType = handleType\n  };\n  NVVK_CHECK(vkGetSemaphoreWin32HandleKHR(m_device, \u0026handleInfo, \u0026hglReady));\n\n  HANDLE hglComplete{};\n  handleInfo.semaphore = m_semaphores.vkComplete;\n  NVVK_CHECK(vkGetSemaphoreWin32HandleKHR(m_device, \u0026handleInfo, \u0026hglComplete));\n  \n  glImportSemaphoreWin32HandleEXT(m_semaphores.glReady, GL_HANDLE_TYPE_OPAQUE_WIN32_EXT, hglReady);\n  glImportSemaphoreWin32HandleEXT(m_semaphores.glComplete, GL_HANDLE_TYPE_OPAQUE_WIN32_EXT, hglComplete);\n#else\n  int                     fdReady{};\n  VkSemaphoreGetFdInfoKHR handleInfo{\n    .sType      = VK_STRUCTURE_TYPE_SEMAPHORE_GET_FD_INFO_KHR,\n    .semaphore  = m_semaphores.vkReady,\n    .handleType = handleType\n  };\n  NVVK_CHECK(vkGetSemaphoreFdKHR(m_device, \u0026handleInfo, \u0026fdReady));\n  \n  int fdComplete{};\n  handleInfo.semaphore = m_semaphores.vkComplete;\n  NVVK_CHECK(vkGetSemaphoreFdKHR(m_device, \u0026handleInfo, \u0026fdComplete));\n  \n  glImportSemaphoreFdEXT(m_semaphores.glReady, GL_HANDLE_TYPE_OPAQUE_FD_EXT, fdReady);\n  glImportSemaphoreFdEXT(m_semaphores.glComplete, GL_HANDLE_TYPE_OPAQUE_FD_EXT, fdComplete);\n#endif\n~~~~\n\n\n\n# Animation\n\nSince the Vulkan memory for the vertex buffer was allocated using\nthe`VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT` flags, we can easily update the buffer by doing the following:\n\n~~~~C++\ng_vertexDataVK[0].pos.x = sin(t);\ng_vertexDataVK[1].pos.y = cos(t);\ng_vertexDataVK[2].pos.x = -sin(t);\nmemcpy(m_vkBuffer.mapped, g_vertexDataVK.data(), g_vertexDataVK.size() * sizeof(Vertex));\n~~~~\n\nNote that we use a host-visible buffer for the sake of simplicity, at the expense of efficiency. For best performance the geometry\nwould need to be uploaded to device-local memory through a staging buffer.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvpro-samples%2Fgl_vk_simple_interop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvpro-samples%2Fgl_vk_simple_interop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvpro-samples%2Fgl_vk_simple_interop/lists"}