{"id":24373304,"url":"https://github.com/j3soon/go-nvml-mig-create-instance","last_synced_at":"2026-04-19T23:34:52.513Z","repository":{"id":272822219,"uuid":"917866052","full_name":"j3soon/go-nvml-mig-create-instance","owner":"j3soon","description":"Unofficial example on creating Multi-Instance GPU (MIG) instances with NVIDIA Management Library (NVML) Go bindings.","archived":false,"fork":false,"pushed_at":"2025-09-05T05:40:14.000Z","size":5,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-04-04T05:41:46.130Z","etag":null,"topics":["docker","golang","multi-instance-gpu","nvidia-management-library","nvml"],"latest_commit_sha":null,"homepage":"https://tutorial.j3soon.com/hpc/extras/go-nvml-mig-create-instance/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/j3soon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-16T19:42:33.000Z","updated_at":"2025-09-07T07:29:14.000Z","dependencies_parsed_at":"2025-01-16T20:53:31.725Z","dependency_job_id":"07379eff-492c-4412-814d-c894792b3949","html_url":"https://github.com/j3soon/go-nvml-mig-create-instance","commit_stats":null,"previous_names":["j3soon/go-nvml-mig-create-instance"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/j3soon/go-nvml-mig-create-instance","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j3soon%2Fgo-nvml-mig-create-instance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j3soon%2Fgo-nvml-mig-create-instance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j3soon%2Fgo-nvml-mig-create-instance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j3soon%2Fgo-nvml-mig-create-instance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/j3soon","download_url":"https://codeload.github.com/j3soon/go-nvml-mig-create-instance/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j3soon%2Fgo-nvml-mig-create-instance/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32026768,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-18T20:23:30.271Z","status":"online","status_checked_at":"2026-04-19T02:00:07.110Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","golang","multi-instance-gpu","nvidia-management-library","nvml"],"created_at":"2025-01-19T05:10:36.446Z","updated_at":"2026-04-19T23:34:52.485Z","avatar_url":"https://github.com/j3soon.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Create MIG Instances with NVML Go Bindings\n\nUnofficial example on creating Multi-Instance GPU (MIG) instances with NVIDIA Management Library (NVML) Go bindings.\n\nPrerequisites:\n\n- [NVIDIA Driver](https://ubuntu.com/server/docs/nvidia-drivers-installation)\n- [Docker](https://docs.docker.com/engine/install/ubuntu/)\n- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)\n- [Enable MIG Mode](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/#enable-mig-mode)\n\nTake A30 as an example:\n\n1. Clone this repo and `cd` into it:\n   ```sh\n   git clone https://github.com/j3soon/go-nvml-mig-create-instance.git\n   cd go-nvml-mig-create-instance\n   ```\n2. Launch docker container for Go:  \n   ```sh\n   docker run --rm -it --gpus all \\\n       -v $(pwd):/workspace \\\n       --cap-add=SYS_ADMIN \\\n       -e NVIDIA_MIG_CONFIG_DEVICES=all \\\n       golang\n   # in the container\n   cd /workspace\n   ```\n   Note: `--runtime=nvidia`, `-e NVIDIA_VISIBLE_DEVICES=all`, and `-e NVIDIA_DRIVER_CAPABILITIES=all` may be required depending on your environment and use cases.  \n   Alternatively, you can install Go on your host machine and skip this step.\n3. Run the example and observe results:\n   ```sh\n   go run main.go\n   # List the available CIs and GIs\n   nvidia-smi mig -lgi; nvidia-smi mig -lci;\n   # Destroy all the CIs and GIs\n   nvidia-smi mig -dci; nvidia-smi mig -dgi;\n   ```\n\nThis should also work on A100/H100/H200 by substituting the MIG profile to [a supported one](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/#supported-mig-profiles).\n\n## Description\n\nTo create MIG GIs and CIs, we should get instance profile information and then create instances based on the profile.\n\n### Example Code Snippet\n\n```go\n// Assuming the 0-th device is MIG-enabled\ndevice, ret := nvml.DeviceGetHandleByIndex(0)\n// Create GPU Instance\ngiProfileInfo, ret := device.GetGpuInstanceProfileInfo(nvml.GPU_INSTANCE_PROFILE_4_SLICE)\ngi, ret := device.CreateGpuInstance(\u0026giProfileInfo)\n// Create Compute Instance\nciProfileInfo, ret := gi.GetComputeInstanceProfileInfo(nvml.COMPUTE_INSTANCE_PROFILE_2_SLICE, nvml.COMPUTE_INSTANCE_ENGINE_PROFILE_SHARED)\n_, ret = gi.CreateComputeInstance(\u0026ciProfileInfo)\n```\n\n\u003e The following source code references are based on `go-nvml v0.12.4-1`\n\n### Creating GPU Instances (GIs)\n\n- After getting the device handle of the 0-th GPU, we want to create a GI based on `CreateGpuInstance`. Take a look at its Go binding (ref: [cpp](https://docs.nvidia.com/deploy/nvml-api/group__nvmlMultiInstanceGPU.html#group__nvmlMultiInstanceGPU_1ge2fc1ce20e869dbf249304e1abedd52f), [go](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml#Device.CreateGpuInstance), [src](https://github.com/NVIDIA/go-nvml/blob/8a7f6b796317a8c7cd5347a25be164108967e7b4/gen/nvml/nvml.h#L10106-L10129)):\n\n  ```\n  CreateGpuInstance(*GpuInstanceProfileInfo) (GpuInstance, Return)\n  ```\n\n- We can see that it takes the reference of `GpuInstanceProfileInfo` as the argument. Take a look at its source (ref: [cpp](https://docs.nvidia.com/deploy/nvml-api/structnvmlGpuInstanceProfileInfo__t.html#structnvmlGpuInstanceProfileInfo__t), [go](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml#GpuInstanceProfileInfo), [src](https://github.com/NVIDIA/go-nvml/blob/8a7f6b796317a8c7cd5347a25be164108967e7b4/gen/nvml/nvml.h#L9752-L9768)):\n\n  ```\n  /**\n   * GPU instance profile information.\n   */\n  typedef struct nvmlGpuInstanceProfileInfo_st\n  {\n      unsigned int id;                  //!\u003c Unique profile ID within the device\n      unsigned int isP2pSupported;      //!\u003c Peer-to-Peer support\n      unsigned int sliceCount;          //!\u003c GPU Slice count\n      unsigned int instanceCount;       //!\u003c GPU instance count\n      unsigned int multiprocessorCount; //!\u003c Streaming Multiprocessor count\n      unsigned int copyEngineCount;     //!\u003c Copy Engine count\n      unsigned int decoderCount;        //!\u003c Decoder Engine count\n      unsigned int encoderCount;        //!\u003c Encoder Engine count\n      unsigned int jpegCount;           //!\u003c JPEG Engine count\n      unsigned int ofaCount;            //!\u003c OFA Engine count\n      unsigned long long memorySizeMB;  //!\u003c Memory size in MBytes\n  } nvmlGpuInstanceProfileInfo_t;\n  ```\n\n- We suspect that these information isn't meant to be filled by hand. We should check the source for using the `GetGpuInstanceProfileInfo` API to retrieve these information (ref: [cpp](https://docs.nvidia.com/deploy/nvml-api/group__nvmlMultiInstanceGPU.html#group__nvmlMultiInstanceGPU_1g3f1c9b8bbbe2188d9c0c4f8bdb021a4d), [go](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml#Device.GetGpuInstanceProfileInfo), [src](https://github.com/NVIDIA/go-nvml/blob/8a7f6b796317a8c7cd5347a25be164108967e7b4/gen/nvml/nvml.h#L10002-L10022)):\n\n  ```\n  /**\n   * Get GPU instance profile information\n   *\n   * Information provided by this API is immutable throughout the lifetime of a MIG mode.\n   *\n   * For Ampere \u0026tm; or newer fully supported devices.\n   * Supported on Linux only.\n   *\n   * @param device                               The identifier of the target device\n   * @param profile                              One of the NVML_GPU_INSTANCE_PROFILE_*\n   * @param info                                 Returns detailed profile information\n   *\n   * @return\n   *         - \\ref NVML_SUCCESS                 Upon success\n   *         - \\ref NVML_ERROR_UNINITIALIZED     If library has not been successfully initialized\n   *         - \\ref NVML_ERROR_INVALID_ARGUMENT  If \\a device, \\a profile or \\a info are invalid\n   *         - \\ref NVML_ERROR_NOT_SUPPORTED     If \\a device doesn't support MIG or \\a profile isn't supported\n   *         - \\ref NVML_ERROR_NO_PERMISSION     If user doesn't have permission to perform the operation\n   */\n  nvmlReturn_t DECLDIR nvmlDeviceGetGpuInstanceProfileInfo(nvmlDevice_t device, unsigned int profile,\n                                                           nvmlGpuInstanceProfileInfo_t *info);\n  ```\n\n- Seems like we need to pass a `NVML_GPU_INSTANCE_PROFILE_*` as the `profile` argument. Let's view the source (ref: [cpp](https://docs.nvidia.com/deploy/nvml-api/group__nvmlMultiInstanceGPU.html#group__nvmlMultiInstanceGPU_1g099e78d565c10a3a0300d26adebe9f81), [go](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml#GPU_INSTANCE_PROFILE_1_SLICE), [src](https://github.com/NVIDIA/go-nvml/blob/8a7f6b796317a8c7cd5347a25be164108967e7b4/gen/nvml/nvml.h#L9712-L9728)):\n\n  ```\n  /**\n   * GPU instance profiles.\n   *\n   * These macros should be passed to \\ref nvmlDeviceGetGpuInstanceProfileInfo to retrieve the\n   * detailed information about a GPU instance such as profile ID, engine counts.\n   */\n  #define NVML_GPU_INSTANCE_PROFILE_1_SLICE      0x0\n  #define NVML_GPU_INSTANCE_PROFILE_2_SLICE      0x1\n  #define NVML_GPU_INSTANCE_PROFILE_3_SLICE      0x2\n  #define NVML_GPU_INSTANCE_PROFILE_4_SLICE      0x3\n  #define NVML_GPU_INSTANCE_PROFILE_7_SLICE      0x4\n  #define NVML_GPU_INSTANCE_PROFILE_8_SLICE      0x5\n  #define NVML_GPU_INSTANCE_PROFILE_6_SLICE      0x6\n  #define NVML_GPU_INSTANCE_PROFILE_1_SLICE_REV1 0x7\n  #define NVML_GPU_INSTANCE_PROFILE_2_SLICE_REV1 0x8\n  #define NVML_GPU_INSTANCE_PROFILE_1_SLICE_REV2 0x9\n  #define NVML_GPU_INSTANCE_PROFILE_COUNT        0xA\n  ```\n\n  \u003e Please note that the `NVML_GPU_INSTANCE_PROFILE_COUNT` here is only a trick to get the number of profiles. It is not meant to be used as a profile.\n\n- We can see that our hypothesis is correct based on the comments. We use `NVML_GPU_INSTANCE_PROFILE_4_SLICE` in our example.\n\n### Creating Compute Instances (CIs)\n\n- After creating a GI, we want to create a CI based on `CreateComputeInstance`. Take a look at its Go binding (ref: [cpp](https://docs.nvidia.com/deploy/nvml-api/group__nvmlMultiInstanceGPU.html#group__nvmlMultiInstanceGPU_1ga79996bd9a91d34b2f486112653d0042), [go](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml#Interface.GpuInstanceCreateComputeInstance), [src](https://github.com/NVIDIA/go-nvml/blob/8a7f6b796317a8c7cd5347a25be164108967e7b4/gen/nvml/nvml.h#L10352-L10378)):\n\n  ```\n  GpuInstanceCreateComputeInstance(GpuInstance, *ComputeInstanceProfileInfo) (ComputeInstance, Return)\n  ```\n\n- Similar to the case in creating GIs, we'll need a `ComputeInstanceProfileInfo`. Let's look at its source (ref: [cpp](https://docs.nvidia.com/deploy/nvml-api/structnvmlComputeInstanceProfileInfo__t.html#structnvmlComputeInstanceProfileInfo__t), [go](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml#ComputeInstanceProfileInfo), [src](https://github.com/NVIDIA/go-nvml/blob/8a7f6b796317a8c7cd5347a25be164108967e7b4/gen/nvml/nvml.h#L9863-L9877)):\n\n  ```\n  /**\n   * Compute instance profile information.\n   */\n  typedef struct nvmlComputeInstanceProfileInfo_st\n  {\n      unsigned int id;                    //!\u003c Unique profile ID within the GPU instance\n      unsigned int sliceCount;            //!\u003c GPU Slice count\n      unsigned int instanceCount;         //!\u003c Compute instance count\n      unsigned int multiprocessorCount;   //!\u003c Streaming Multiprocessor count\n      unsigned int sharedCopyEngineCount; //!\u003c Shared Copy Engine count\n      unsigned int sharedDecoderCount;    //!\u003c Shared Decoder Engine count\n      unsigned int sharedEncoderCount;    //!\u003c Shared Encoder Engine count\n      unsigned int sharedJpegCount;       //!\u003c Shared JPEG Engine count\n      unsigned int sharedOfaCount;        //!\u003c Shared OFA Engine count\n  } nvmlComputeInstanceProfileInfo_t;\n  ```\n\n- Similarly, let's check the source for `GetComputeInstanceProfileInfo` API (ref: [cpp](https://docs.nvidia.com/deploy/nvml-api/group__nvmlMultiInstanceGPU.html#group__nvmlMultiInstanceGPU_1ge5ed2a23a041a3395886a324a0d2947e), [go](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml#GpuInstance.GetComputeInstanceProfileInfo), [src](https://github.com/NVIDIA/go-nvml/blob/8a7f6b796317a8c7cd5347a25be164108967e7b4/gen/nvml/nvml.h#L10241-L10263)):\n\n  ```\n  /**\n   * Get compute instance profile information.\n   *\n   * Information provided by this API is immutable throughout the lifetime of a MIG mode.\n   *\n   * For Ampere \u0026tm; or newer fully supported devices.\n   * Supported on Linux only.\n   *\n   * @param gpuInstance                          The identifier of the target GPU instance\n   * @param profile                              One of the NVML_COMPUTE_INSTANCE_PROFILE_*\n   * @param engProfile                           One of the NVML_COMPUTE_INSTANCE_ENGINE_PROFILE_*\n   * @param info                                 Returns detailed profile information\n   *\n   * @return\n   *         - \\ref NVML_SUCCESS                 Upon success\n   *         - \\ref NVML_ERROR_UNINITIALIZED     If library has not been successfully initialized\n   *         - \\ref NVML_ERROR_INVALID_ARGUMENT  If \\a gpuInstance, \\a profile, \\a engProfile or \\a info are invalid\n   *         - \\ref NVML_ERROR_NOT_SUPPORTED     If \\a profile isn't supported\n   *         - \\ref NVML_ERROR_NO_PERMISSION     If user doesn't have permission to perform the operation\n   */\n  nvmlReturn_t DECLDIR nvmlGpuInstanceGetComputeInstanceProfileInfo(nvmlGpuInstance_t gpuInstance, unsigned int profile,\n                                                                    unsigned int engProfile,\n                                                                    nvmlComputeInstanceProfileInfo_t *info);\n  ```\n\n- We should pass a `NVML_COMPUTE_INSTANCE_PROFILE_*` as the first (`profile`) argument. Let's view the source (ref: [cpp](https://docs.nvidia.com/deploy/nvml-api/group__nvmlMultiInstanceGPU.html#group__nvmlMultiInstanceGPU_1g9f17acf87c071aaaeb79b4d3c9bccebb), [go](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml#COMPUTE_INSTANCE_PROFILE_1_SLICE), [src](https://github.com/NVIDIA/go-nvml/blob/8a7f6b796317a8c7cd5347a25be164108967e7b4/gen/nvml/nvml.h#L9838-L9852)):\n\n  ```\n  /**\n   * Compute instance profiles.\n   *\n   * These macros should be passed to \\ref nvmlGpuInstanceGetComputeInstanceProfileInfo to retrieve the\n   * detailed information about a compute instance such as profile ID, engine counts\n   */\n  #define NVML_COMPUTE_INSTANCE_PROFILE_1_SLICE       0x0\n  #define NVML_COMPUTE_INSTANCE_PROFILE_2_SLICE       0x1\n  #define NVML_COMPUTE_INSTANCE_PROFILE_3_SLICE       0x2\n  #define NVML_COMPUTE_INSTANCE_PROFILE_4_SLICE       0x3\n  #define NVML_COMPUTE_INSTANCE_PROFILE_7_SLICE       0x4\n  #define NVML_COMPUTE_INSTANCE_PROFILE_8_SLICE       0x5\n  #define NVML_COMPUTE_INSTANCE_PROFILE_6_SLICE       0x6\n  #define NVML_COMPUTE_INSTANCE_PROFILE_1_SLICE_REV1  0x7\n  #define NVML_COMPUTE_INSTANCE_PROFILE_COUNT         0x8\n  ```\n\n- We use `COMPUTE_INSTANCE_PROFILE_2_SLICE` for the first argument in our example. As for the second argument (`engProfile`), let's also look at the source (ref: [cpp](https://docs.nvidia.com/deploy/nvml-api/group__nvmlMultiInstanceGPU.html#group__nvmlMultiInstanceGPU_1gb080d08cc8508ffc3ae348d7a4e37aa5), [go](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml#COMPUTE_INSTANCE_ENGINE_PROFILE_SHARED), [src](https://github.com/NVIDIA/go-nvml/blob/8a7f6b796317a8c7cd5347a25be164108967e7b4/gen/nvml/nvml.h#L9854-L9855)):\n\n  ```\n  #define NVML_COMPUTE_INSTANCE_ENGINE_PROFILE_SHARED 0x0 //!\u003c All the engines except multiprocessors would be shared\n  #define NVML_COMPUTE_INSTANCE_ENGINE_PROFILE_COUNT  0x1\n  ```\n\n  We can only use `COMPUTE_INSTANCE_ENGINE_PROFILE_SHARED` for the second argument in our example.\n\n  \u003e Although we currently only have the ability to share GPU engines (Copy Engine (CE), NVENC, NVDEC, NVJPEG, Optical Flow Accelerator (OFA), etc.) between CIs within the same GI, this struct may be extended to support isolating these engines for each CI within the same GI in the future.\n\n## References\n\nSome references I found useful during the investigation.\n\n- [NVIDIA/go-nvml](https://github.com/NVIDIA/go-nvml)\n  - [Quick Start](https://github.com/NVIDIA/go-nvml?tab=readme-ov-file#quick-start)\n  - [CreateComputeInstance() shows “Not Supported” - Issue #65](https://github.com/NVIDIA/go-nvml/issues/65)\n- [(Archived) NVML in NVIDIA/gpu-monitoring-tools](https://github.com/NVIDIA/gpu-monitoring-tools/blob/master/bindings/go/nvml/nvml.h)\n- [Docker MIG Manager](https://github.com/j3soon/docker-mig-manager)\n\nAPI References (Useful for searching API definitions):\n\n- [NVML API Reference Guide](https://docs.nvidia.com/deploy/nvml-api/index.html)\n  - [4.28. Multi Instance GPU Management](https://docs.nvidia.com/deploy/nvml-api/group__nvmlMultiInstanceGPU.html)\n- [NVML Go Bindings](https://pkg.go.dev/github.com/NVIDIA/go-nvml/pkg/nvml)\n- [NVML Go Bindings Source Code Definitions](https://github.com/NVIDIA/go-nvml/blob/v0.12.4-1/gen/nvml/nvml.h)\n\n## Acknowledgement\n\nThanks [@Irene-Ting](https://github.com/Irene-Ting) for discussions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fj3soon%2Fgo-nvml-mig-create-instance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fj3soon%2Fgo-nvml-mig-create-instance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fj3soon%2Fgo-nvml-mig-create-instance/lists"}