{"id":18416778,"url":"https://github.com/fynv/vkinline","last_synced_at":"2025-04-07T12:32:08.065Z","repository":{"id":62587649,"uuid":"267327691","full_name":"fynv/VkInline","owner":"fynv","description":"A tool to make it easy to use Vulkan from Python. An interface for computation and off-screen rendering.","archived":false,"fork":false,"pushed_at":"2021-05-18T03:05:31.000Z","size":1127,"stargazers_count":21,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-22T18:41:08.898Z","etag":null,"topics":["gpu","offscreen-rendering","python","ray-tracing","vulkan"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fynv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-05-27T13:32:27.000Z","updated_at":"2025-02-19T12:27:40.000Z","dependencies_parsed_at":"2022-11-03T22:16:16.197Z","dependency_job_id":null,"html_url":"https://github.com/fynv/VkInline","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fynv%2FVkInline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fynv%2FVkInline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fynv%2FVkInline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fynv%2FVkInline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fynv","download_url":"https://codeload.github.com/fynv/VkInline/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247653159,"owners_count":20973775,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gpu","offscreen-rendering","python","ray-tracing","vulkan"],"created_at":"2024-11-06T04:07:15.195Z","updated_at":"2025-04-07T12:32:03.050Z","avatar_url":"https://github.com/fynv.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# VkInline\n\nTrying to develop another \"easy way\" to program GPU using a non-C++ host language.\n\nPreviously, I did [ThrustRTC](https://github.com/fynv/ThrustRTC) and [CUDAInline](https://github.com/fynv/CUDAInline),\nboth of which are based on CUDA (NVRTC). \n\nThis time, however, I'm trying to do similar things basing on Vulkan. I found it more challenging than using CUDA (NVRTC) \nbecause of less friendly lauguage feature of GLSL comparing to CUDA C, and more complicated host API struture. \nHowever, I still found it attractive to do, because:\n\n* Vulkan is neutral to GPU vendors\n* Vulkan exposes more GPU features. Computing is only a small part, there are rasterization and ray-tracing pipelines.\n\n## Progress\n\n### Computation\n\nThe computation part of VkInline has similar features as CUDAInline. You can easily launch a compute shader from Python, like:\n\n```python\n\nimport VkInline as vki\nimport numpy as np\n\nharr = np.array([1.0, 2.0, 3.0, 4.0, 5.0], dtype='float32')\ndarr = vki.device_vector_from_numpy(harr)\n\nkernel = vki.Computer(['arr_in', 'arr_out', 'k'],\n'''\nvoid main()\n{\n    uint id = gl_GlobalInvocationID.x;\n    if (id \u003e= get_size(arr_in)) return;\n    set_value(arr_out, id, get_value(arr_in, id)*k);\n}\n''')\n\ndarr_out = vki.SVVector('float', 5)\nkernel.launch(1,128, [darr, darr_out, vki.SVFloat(10.0)])\nprint (darr_out.to_host())\n\n```\nResult:\n```\n[10. 20. 30. 40. 50.]\n```\n\nGLSL lacks language features like \"struct member functions\" and \"operator overloading\". \nTherefore, array indexing doesn't look as nice as in CUDAInline.\n\nNative 2,3,4 component vector types and matrix types are supported. For example\n```python\nv = vki.device_vector_from_list([[1, -1], [2, -3], [5, 1000]], 'double')\nprint (v.name_elem_type(), v.elem_size())\nprint(v.to_host())\n```\n\nYou will get:\n```\ndvec2 16\n[[   1.   -1.]\n [   2.   -3.]\n [   5. 1000.]]\n```\n\n### Rasterization\n\nRasterization is currently very much simplified in VkInline. The limiations are:\n\n* 1 vki.Rasterizer = 1 Vulkan render-pass with 1 subpass. Multi-subpass feature of Vulkan is mostly for tiled-caching applications, which will not be implemented in VkInline.\n* Currently only vertex-shader and fragment-shader programming are supported.\n* Currently only a sub-set of pipeline options can be configured, during the construction of a DrawCall object. Those not covered are set to default value.\n* Surfaces/Swapchains/Semaphores are not exposed. This is mainly for off-screen rendering, not quite suitable for video games. \n\nExample:\n\n```python\nimport VkInline as vki\nimport numpy as np\nfrom PIL import Image\n\nVK_FORMAT_R8G8B8A8_SRGB = 43\n\nwidth = 640\nheight =  480\n\ncolorBuf = vki.Texture2D(width, height, VK_FORMAT_R8G8B8A8_SRGB)\n\npositions = np.array([ [0.0, -0.5, 0.5], [0.5, 0.5, 0.5], [-0.5, 0.5, 0.5] ], dtype = np.float32)\ngpuPos = vki.device_vector_from_numpy(positions)\n\ncolors =  np.array([ [0.0, 1.0, 0.0], [1.0, 0.0, 0.0], [0.0, 0.0, 1.0]], dtype = np.float32)\ngpuColors = vki.device_vector_from_numpy(colors)\n\nrp = vki.Rasterizer(['pos', 'col'])\n\nrp.add_draw_call(vki.DrawCall(\n'''\nlayout (location = 0) out vec3 vColor;\nvoid main() \n{\n\tgl_Position = vec4(get_value(pos, gl_VertexIndex), 1.0);\n\tvColor = get_value(col, gl_VertexIndex);\n}\n''',\n'''\nlayout (location = 0) in vec3 vColor;\nlayout (location = 0) out vec4 outColor;\n\nvoid main() \n{\n\toutColor = vec4(vColor, 1.0);\n}\n'''))\n\n\nrp.launch([3], [colorBuf], None, [0.5, 0.5, 0.5, 1.0], 1.0, [gpuPos, gpuColors])\n\nimage_out = np.empty((height, width, 4), dtype=np.uint8)\ncolorBuf.download(image_out)\n\nImage.fromarray(image_out, 'RGBA').save('output.png')\n\n```\nThe code generates the following image:\n\n\u003cimg src=\"doc/rasterization_result.png\" width=\"640px\"\u003e\n\n### Ray-tracing\n\nIn order to enable VK_KHR_ray_tracing, Vulkan 1.2, with a bunch of additional extensions are required.\nCurrently, the feature is only tested with [Nvidia Beta driver](https://developer.nvidia.com/vulkan-driver).\n\nExample:\n\n```python\nimport VkInline as vki\nimport numpy as np\nfrom PIL import Image\nimport glm\n\nwidth = 800\nheight =  400\n\naabb_unit_sphere = np.array([-1.0, -1.0, -1.0, 1.0, 1.0, 1.0], dtype = np.float32)\nd_aabb_unit_sphere = vki.device_vector_from_numpy(aabb_unit_sphere)\nblas_unit_sphere = vki.BaseLevelAS(gpuAABB = d_aabb_unit_sphere)\ntransform = glm.identity(glm.mat4)\ntransform = glm.translate(transform, glm.vec3(0.0, 0.0, -1.0))\ntransform = glm.scale(transform, glm.vec3(0.5, 0.5, 0.5))\ntlas = vki.TopLevelAS([[(blas_unit_sphere, transform)]])\n\ndarr_out = vki.SVVector('vec3', width*height)\n\nraytracer = vki.RayTracer(['arr_out', 'width', 'height'], \n'''\nstruct Payload\n{\n\tfloat t;\n\tvec3 color;\n};\n\nlayout(location = 0) rayPayloadEXT Payload payload;\n\nvoid main()\n{\n\tint x = int(gl_LaunchIDEXT.x);\n\tint y = int(gl_LaunchIDEXT.y);\n\tif (x\u003e=width || y\u003eheight) return;\n\n\tvec3 lower_left_corner = vec3(-2.0, -1.0, -1.0);\n\tvec3 horizontal = vec3(4.0, 0.0, 0.0);\n\tvec3 vertical = vec3(0.0, 2.0, 0.0);\n\tvec3 origin = vec3(0.0, 0.0, 0.0);\n\n\tfloat u = (float(x)+0.5)/float(width);\n\tfloat v = 1.0 - (float(y)+0.5)/float(height);\n\n\tvec3 direction = normalize(lower_left_corner + u * horizontal + v * vertical);\n\n\tuint rayFlags = gl_RayFlagsOpaqueEXT;\n\tuint cullMask = 0xff;\n\tfloat tmin = 0.001;\n    float tmax = 1000000.0;\n\n\ttraceRayEXT(arr_tlas[0], rayFlags, cullMask, 0, 0, 0, origin, tmin, direction, tmax, 0);\n\n\tset_value(arr_out, x+y*width, payload.color);\n}\n\n''', [\n'''\nstruct Payload\n{\n\tfloat t;\n\tvec3 color;\n};\n\nlayout(location = 0) rayPayloadInEXT Payload payload;\n\nvoid main()\n{\n\tpayload.t = -1.0;\n\tvec3 direction = gl_WorldRayDirectionEXT;\n\tfloat t = 0.5 * (direction.y + 1.0);\n\tpayload.color = (1.0 - t)*vec3(1.0, 1.0, 1.0) + t * vec3(0.5, 0.7, 1.0);\t\n}\n'''], [vki.HitShaders(\nclosest_hit = '''\nstruct Payload\n{\n\tfloat t;\n\tvec3 color;\n};\n\nlayout(location = 0) rayPayloadInEXT Payload payload;\nhitAttributeEXT vec3 hitpoint;\n\nvoid main()\n{\n\tvec3 normal = normalize(hitpoint);\n\tpayload.t = gl_HitTEXT;\n\tpayload.color = (normal+vec3(1.0, 1.0, 1.0))*0.5;\n}\n\n''',\nintersection = '''\nhitAttributeEXT vec3 hitpoint;\n\nvoid main()\n{\n\tvec3 origin = gl_ObjectRayOriginEXT;\n\tvec3 direction = gl_ObjectRayDirectionEXT;\n\tfloat tMin = gl_RayTminEXT;\n\tfloat tMax = gl_RayTmaxEXT;\n\n\tconst float a = dot(direction, direction);\n\tconst float b = dot(origin, direction);\n\tconst float c = dot(origin, origin) - 1.0;\n\tconst float discriminant = b * b - a * c;\n\n\tif (discriminant \u003e= 0)\n\t{\n\t\tconst float t1 = (-b - sqrt(discriminant)) / a;\n\t\tconst float t2 = (-b + sqrt(discriminant)) / a;\n\n\t\tif ((tMin \u003c= t1 \u0026\u0026 t1 \u003c tMax) || (tMin \u003c= t2 \u0026\u0026 t2 \u003c tMax))\n\t\t{\n\t\t\tfloat t = t1;\n\t\t\tif (tMin \u003c= t1 \u0026\u0026 t1 \u003c tMax)\n\t\t\t{\n\t\t\t\thitpoint = origin + direction * t1;\n\t\t\t}\n\t\t\telse\n\t\t\t{\n\t\t\t\tt = t2;\n\t\t\t\thitpoint = origin + direction * t2;\n\t\t\t}\n\t\t\treportIntersectionEXT(t, 0);\n\t\t}\n\t}\n\n}\n'''\n)])\n\nsvwidth = vki.SVInt32(width)\nsvheight = vki.SVInt32(height)\n\nraytracer.launch((width, height), [darr_out, svwidth, svheight], [tlas])\n\nout = darr_out.to_host()\nout = out.reshape((height,width,3))*255.0\nout = out.astype(np.uint8)\nImage.fromarray(out, 'RGB').save('output.png')\n```\n\nThe code generates the following image:\n\n\u003cimg src=\"doc/raytrace_result.png\" width=\"800px\"\u003e\n\n\n## Installation\n\n### Install from Source Code\n\nSource code of VkInline is available at:\nhttps://github.com/fynv/VkInline\n\nAt build time, you will need:\n* UnQLite source code, as submodule: thirdparty/unqlite\n* glslang, as submodule: thirdparty/glslang\n* SPIRV-Cross, as submodule: thirdparty/SPIRV-Cross \n* Vulkan-Headers, as submodule: thirdparty/Vulkan-Headers\n* volk, as submodule: thirdparty/volk\n* CMake 3.x\n\nAfter cloning the repo from github and resolving the submodules, you can build it\nwith CMake.\n\n```\n$ mkdir build\n$ cd build\n$ cmake .. -DBUILD_PYTHON_BINDINGS=true -DVKINLINE_BUILD_TESTS=true -DVKINLINE_INCLUDE_PYTESTS=true\n$ make\n$ make install\n```\nYou will get the library headers, binaries and examples in the \"install\" directory.\n\n### Install PyVkInline from PyPi\n\nBuilds for Win64/Linux64 + Python 3.x are available from Pypi. If your\nenvironment matches, you can try:\n\n```\n$ pip3 install VkInline\n```\n\n## Runtime Dependencies\n\nA Vulkan-capable GPU and a recent driver is needed at run-time.\nFor ray-tracing, a [Nvidia Beta driver](https://developer.nvidia.com/vulkan-driver) might be needed.\n\nYou may also need Vulkan SDK at runtime for some platforms.\n\nAt Python side, VkInline depends on:\n* Python 3\n* cffi\n* numpy\n* pyglm\n\n## License \n\nI've decided to license this project under ['\"Anti 996\" License'](https://github.com/996icu/996.ICU/blob/master/LICENSE)\n\nBasically, you can use the code any way you like unless you are working for a 996 company.\n\n[![996.icu](https://img.shields.io/badge/link-996.icu-red.svg)](https://996.icu)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffynv%2Fvkinline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffynv%2Fvkinline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffynv%2Fvkinline/lists"}