{"id":34300398,"url":"https://github.com/juliaattic/cudart.jl","last_synced_at":"2025-12-17T06:03:19.738Z","repository":{"id":15518405,"uuid":"18252732","full_name":"JuliaAttic/CUDArt.jl","owner":"JuliaAttic","description":"Julia wrapper for CUDA runtime API","archived":false,"fork":false,"pushed_at":"2020-02-08T14:52:47.000Z","size":333,"stargazers_count":79,"open_issues_count":19,"forks_count":29,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-11-29T13:39:46.202Z","etag":null,"topics":["cuda","gpu","julia"],"latest_commit_sha":null,"homepage":null,"language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JuliaAttic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-03-29T22:54:51.000Z","updated_at":"2024-05-29T16:02:21.000Z","dependencies_parsed_at":"2022-09-16T04:54:22.478Z","dependency_job_id":null,"html_url":"https://github.com/JuliaAttic/CUDArt.jl","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/JuliaAttic/CUDArt.jl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaAttic%2FCUDArt.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaAttic%2FCUDArt.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaAttic%2FCUDArt.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaAttic%2FCUDArt.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JuliaAttic","download_url":"https://codeload.github.com/JuliaAttic/CUDArt.jl/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaAttic%2FCUDArt.jl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27778406,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-17T02:00:08.291Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cuda","gpu","julia"],"created_at":"2025-12-17T06:03:18.430Z","updated_at":"2025-12-17T06:03:19.732Z","avatar_url":"https://github.com/JuliaAttic.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CUDArt\n\n**IMPORTANT NOTE**: this package is not actively developed, please use [CUDAdrv](https://github.com/JuliaGPU/CUDAdrv.jl) instead!\n\n**Build status**: [![][buildbot-julia05-img]][buildbot-julia05-url] [![][buildbot-julia06-img]][buildbot-julia06-url]\n\n**Code coverage**: [![][coverage-img]][coverage-url]\n\n[buildbot-julia05-img]: http://ci.maleadt.net/shields/build.php?builder=CUDArt-julia05-x86-64bit\u0026name=julia%200.5\n[buildbot-julia05-url]: http://ci.maleadt.net/shields/url.php?builder=CUDArt-julia05-x86-64bit\n[buildbot-julia06-img]: http://ci.maleadt.net/shields/build.php?builder=CUDArt-julia06-x86-64bit\u0026name=julia%200.6\n[buildbot-julia06-url]: http://ci.maleadt.net/shields/url.php?builder=CUDArt-julia06-x86-64bit\n\n[coverage-img]: https://codecov.io/gh/JuliaGPU/CUDArt.jl/coverage.svg\n[coverage-url]: https://codecov.io/gh/JuliaGPU/CUDArt.jl\n\nThis package wraps the [CUDA runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/).\nFor a wrapper of the [driver API](http://docs.nvidia.com/cuda/cuda-driver-api/), see\n[CUDAdrv](https://github.com/JuliaGPU/CUDAdrv.jl).\n\nCUDAdrv.jl is the preferred way to program a GPU from Julia; Only use CUDArt.jl if you _really_ require the runtime API.\n\n\n\n## Platform support\n\nThis has been tested on Linux, OSX, and Windows. With Windows, at least Visual Studio\n2010/2012/2013/2015 are supported.\n\n\n\n## Installation\n\nFirst, you need to have an NVIDIA GPU device in your computer (one that is available for\ncomputation, i.e., most likely not your graphics card), and the CUDA library installed. You\nhave to perform these steps manually. **Choose either 32-bit or 64-bit versions to match\nyour Julia installation.**\n\nInstall the Julia package using:\n```julia\nPkg.add(\"CUDArt\")\n```\n\nDuring installation, it should compile a couple of files in the `deps/` directory. These\nfiles provide utility functions necessary for certain functionality in this package. If the\nbuild step fails, try fixing the problems and running `Pkg.build(\"CUDArt\")` manually.\n\nAfter installation, it's probably a good idea to run the `test/runtests.jl` script to find\nout whether everything is working on your system, or just say `Pkg.test(\"CUDArt\")`.\n\nIn case of errors, one thing to check is your CUDA installation itself. For example, examine\nwhether the `*.ptx` files are present in `deps/` and `test/`; look at those files and make\nsure they seem appropriate. (E.g., if your computer is 64-bit, are they compiled for\n64-bit?)\n\n\n\n## Usage\n\nStart by saying `using CUDArt`, or `import CUDArt` if you prefer to qualify everything with\nthe module name. For most use cases, you'll also need to install and import the `CUDAdrv`\npackage, which among other things provides functionality to launch kernels.\n\n### GPU initialization\n\nOne or more GPUs can be initialized, used for computations, and freed for other uses. There\nare some complexities in this process due to the interaction with Julia's garbage\ncollection---a CUDA array object allocated in one \"session\" should not be usable if you\nclose the device and then open a new \"session.\" Fortunately, CUDArt should make the process\ntransparent, and as a user you shouldn't have to think about this at all.\n\nThe easiest way to ensure that you get full functionality, with proper cleanup of resources,\nis by using the `do` block syntax:\n\n```julia\nresult = devices(dev-\u003etrue) do devlist\n    # Code that does GPU computations\nend\n```\n\nThe argument to `devices` is a function that accepts an integer input (the integer\nrepresenting the CUDA device, starting with 1) and returns `true` or `false`, indicating\nwhether the device should or should not be used, respectively. `dev-\u003etrue` means that very\ndevice will be used. The `devlist` variable will be defined inside the block, and is a\n`Vector{Int}` of the available devices.\n\nIf you need to make sure that only devices with sufficient capabilities are used, then use a\nconstruct like this:\n\n```julia\nresult = devices(dev-\u003ecapability(dev)[1]\u003e=2) do devlist\n    # Code that does GPU computations\nend\n```\n\nThis will select all devices that have a major capability of 2 or higher. You can query any\nof the properties of your device; see the `device_properties` and `attribute` functions and\nthe list of\n[fields](http://docs.nvidia.com/cuda/cuda-runtime-api/structcudaDeviceProp.html#structcudaDeviceProp).\nIf you want to restrict your computations to just one device (perhaps leaving other devices\nfor other users), use the `nmax` keyword:\n\n```julia\nresult = devices(func, nmax=1) do devlist\n    # Code that does GPU computations\nend\n```\n\nFinally, you can request only those devices that are not busy with other tasks using:\n\n```julia\nresult = devices(func, status=:free) do devlist\n    # Code that does GPU computations\nend\n```\n\nYou can wait for specific devices to become available with `wait_free(devlist)`.\n\n\nThe `do` block syntax initializes the devices and loads some utility functions (defined in\n`deps/utils.cu`) onto each GPU; it also ensures proper freeing of memory and unloading of\ncode when the `do` block finishes. Should you want to initialize the utilities manually, you\ncan do so by calling `CUDArt.init(devlist)` and `CUDArt.close(devlist)` where `devlist` is\nan integer device number or a list of them, e.g. `0` or `[0,1]`. This can be handy in case\nof trouble, because unfortunately the `do` syntax does not usually result in ideal\nbacktraces.\n\nIf your work doesn't require any of the utility functions, you can manually manage the\ndevice:\n\n```julia\ndevice(dev)\n# Code that does GPU computations\ndevice_reset(dev)\n```\n\nwhere `dev` is the integer device number.\n\n#### Choosing/querying the active device\n\nAt any point in your code, the command `device(dev)` makes `dev` the active device. For\nexample, commands that allocate device memory will be executed on whichever device is\ncurrently active.\n\nCalling `dev = device()` will return the currently-active device\n\n\n### Arrays\n\n#### Device arrays\n\nCUDArt supports two main types of device arrays: `CudaArray`s and `CudaPitchedArray`s.\nThese correspond to contiguous memory blocks and \"pitched pointers\", respectively.\n\nTo declare an uninitialized array on the device, use:\n\n\n```julia\nd_A = CudaArray(Float64, (200,300))\nd_B = CudaPitchedArray(Int32, (15, 40, 27))\n```\nThe `d_` is a conventional way of reminding yourself that the array is allocated on the device.\nTo copy a host array to the device, use any of\n```julia\nd_A = CudaArray(A)\nd_AP = CudaPitchedArray(A)\ncopy!(d_A, A)\ncopy!(d_AP, A)\n```\nTo copy a device array back to the host, use either of\n```julia\nA = to_host(d_A)\ncopy!(A, d_A)\n```\n\nMost of the typical Julia functions, like `size`, `ndims`, `reinterpret`, `eltype`, `fill!`,\netc.,  work on CUDA array types. One noteworthy omission is that you can't directly index a\nCUDA array: `d_A[2,4]` will fail. This is not supported because host/device memory transfers\nare relatively slow, and you don't want to write code that (on the host side) makes use of\nindividual elements in a device array. If you want to inspect the values in a device array,\nfirst use `to_host` to copy it to host memory.\n\nYou can find out which device is storing an array using:\n\n```julia\ndev = device(d_A)\n```\n\n#### Host arrays\n\nAnother important array type is the `HostArray`, which is allocated by the CUDA library\nusing pinned memory:\n\n```julia\nh_A = HostArray(Float32, (1000,1200))\n```\n\nThere are circumstances where using a `HostArray` may improve the speed of memory transfers,\nor allow asynchronous operations using `Stream`s.\n\n**Warning: using a `HostArray` in conjunction with a large memory-mapped file has been\nobserved to cause segfaults; at the present time there is no known workaround.**\n\n\n### Modules and custom kernels\n\nThis will not teach you about CUDA programming; for that, please refer to the CUDA\ndocumentation and other online sources. You can find an example file in `deps/utils.cu`.\n\n#### Compiling your own modules\n\nYou can write and use your own custom kernels, first writing a `.cu` file and compiling it\nas a `ptx` module. On Linux, compilation would look something like this:\n\n```\nnvcc -ptx mycudamodule.cu\n```\n\nYou can specify that the code should be compiled for compute capability 2.0 devices or\nhigher using:\n\n```\nnvcc -ptx -gencode=arch=compute_20,code=sm_20 mycudamodule.cu\n```\n\nIf you want to write code that will support multiple datatypes (e.g., `Float32` and\n`Float64`), it's recommended that you use C++ and write your code using templates. Then use\n`extern C` to instantiate bindings for each datatype. For example:\n\n```\ntemplate \u003ctypename T\u003e\n__device__ void kernel_function1(T *data) {\n    // Code goes here\n}\ntemplate \u003ctypename T1, typename T2\u003e\n__device__ void kernel_function2(T1 *data1, T2 *data2) {\n    // Code goes here\n}\n\nextern \"C\"\n{\n    void __global__ kernel_function1_float(float *data) {kernel_function1(data);}\n    void __global__ kernel_function1_double(double *data) {kernel_function1(data);}\n    void __global__ kernel_function2_int_float(int *data1, float *data2) {kernel_function2(data1,data2);}\n}\n```\n\n#### Initializing and freeing PTX modules\n\nTo easily make your kernels available, the recommended approach is to define something\nanalogous to the following for each `ptx` module (this example uses the kernels described in\nthe previous section):\n\n```julia\nmodule MyCudaModule\n\nimport CUDAdrv: CuModule, CuModuleFile, CuFunction, cudacall\nusing CUDArt\n\nexport function1\n\nconst ptxdict = Dict()\nconst mdlist = Array{CuModule}(0)\n\nfunction mdinit(devlist)\n    global ptxdict\n    global mdlist\n    isempty(mdlist) || error(\"mdlist is not empty\")\n    for dev in devlist\n        device(dev)\n        md = CuModuleFile(\"mycudamodule.ptx\")\n        ptxdict[(dev, \"function1\", Float32)] = CuFunction(md, \"kernel_function1_float\")\n        ptxdict[(dev, \"function1\", Float64)] = CuFunction(md, \"kernel_function1_double\")\n        ptxdict[(dev, \"function2\", Int32, Float32)] = CuFunction(md, \"kernel_function2_int_float\")\n\n        push!(mdlist, md)\n    end\nend\n\nmdclose() = (empty!(mdlist); empty!(ptxdict))\n\nfunction init(f::Function, devlist)\n    local ret\n    mdinit(devlist)\n    try\n        ret = f(devlist)\n    finally\n        mdclose()\n    end\n    ret\nend\n\nfunction function1{T}(data::CudaArray{T})\n    dev = device(data)\n    cufunction1 = ptxdict[(dev, \"function1\", T)]\n    # Set up grid and block, see below\n    cudacall(cufunction1, grid, block, (Ptr{T},), data)\nend\n\n...\n\nend  # MyCudaModule\n```\n\nUsage will look something like the following:\n\n```julia\nusing CUDArt, MyCudaModule\n\nA = rand(10,5)\n\nresult = devices(dev-\u003ecapability(dev)[1]\u003e=2) do devlist\n    MyCudaModule.init(devlist) do dev\n        device(dev)\n        function1(CudaArray(A))\n    end\nend\n```\n\n#### Grid and block dimensions\n\nTo be written.\n\n\n### Streams\n\nOne can use streams to manage or synchronize computations between the CPU \u0026 GPU, or using\nmultiple CUDA devices.  Using [Julia's `@sync` and `@async`\nmacros](http://docs.julialang.org/en/latest/manual/parallel-computing/), here is a short\ndemonstration that activates processing on multiple devices:\n\n```julia\nmeasured_sleep_time = CUDArt.devices(dev-\u003etrue, nmax=2) do devlist\n    sleeptime = 0.5\n    results = Array{Float64}(3*length(devlist))\n    streams = [(device(dev); Stream()) for dev in devlist]\n    # Force one run to precompile\n    cudasleep(sleeptime; dev=devlist[1], stream=streams[1])\n    wait(streams[1])\n    i = 1\n    nextidx() = (idx=i; i+=1; idx)\n    @sync begin\n        for idev = 1:length(devlist)\n            @async begin\n                while true\n                    idx = nextidx()\n                    if idx \u003e length(results)\n                        break\n                    end\n                    tstart = time()\n                    dev = devlist[idev]\n                    stream = streams[idev]\n                    cudasleep(sleeptime; dev=dev, stream=stream)\n                    wait(stream)\n                    tstop = time()\n                    results[idx] = tstop-tstart\n                end\n            end\n        end\n    end\n    results\nend\n```\n\nIn a more realistic version of this demonstration, you would \"feed\" work and collect the\nresults from all of your CUDA devices using a single Julia process to organize the efforts.\n\n\n\n## Random notes\n\n### Notes on memory\n\nJulia convention is that matrices are stored in column-major order, whereas C (and CUDA) use\nrow-major. For efficiency this wrapper avoids reordering memory, so that the linear sequence\nof addresses is the same between main memory and the GPU. For most usages, this is probably\nwhat you want.\n\nHowever, for the purposes of linear algebra, this effectively means that one is storing the\ntranspose of matrices on the GPU. (TODO: create `CudaMatrix` and `CudaPitchedMatrix` types\nthat will automatically take the transpose when copying between main and GPU memory. This\nwill be useful for cuBLAS.)\n\nNote that the size of a CudaArray/CudaPitchedArray is represented as the size _of the\ncorresponding main-memory object_; thus, an array's dimensions (as reported by Julia) will\nnot change when you copy it between main and GPU memory.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliaattic%2Fcudart.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuliaattic%2Fcudart.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliaattic%2Fcudart.jl/lists"}