{"id":13579463,"url":"https://github.com/bryansteiner/gpu-passthrough-tutorial","last_synced_at":"2025-05-14T14:09:41.104Z","repository":{"id":41963015,"uuid":"232950386","full_name":"bryansteiner/gpu-passthrough-tutorial","owner":"bryansteiner","description":null,"archived":false,"fork":false,"pushed_at":"2024-11-17T17:31:49.000Z","size":2444,"stargazers_count":1620,"open_issues_count":4,"forks_count":101,"subscribers_count":30,"default_branch":"master","last_synced_at":"2025-04-14T01:49:13.235Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bryansteiner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-10T02:39:58.000Z","updated_at":"2025-04-13T20:21:18.000Z","dependencies_parsed_at":"2024-01-14T16:10:10.698Z","dependency_job_id":"42fd3be4-ffdc-48d5-b55b-e09934d241f2","html_url":"https://github.com/bryansteiner/gpu-passthrough-tutorial","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bryansteiner%2Fgpu-passthrough-tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bryansteiner%2Fgpu-passthrough-tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bryansteiner%2Fgpu-passthrough-tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bryansteiner%2Fgpu-passthrough-tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bryansteiner","download_url":"https://codeload.github.com/bryansteiner/gpu-passthrough-tutorial/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254160442,"owners_count":22024569,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:01:39.681Z","updated_at":"2025-05-14T14:09:36.086Z","avatar_url":"https://github.com/bryansteiner.png","language":"Shell","readme":"\u003ch2\u003e\n    Table of Contents\n\u003c/h2\u003e\n\n* [Introduction](#introduction)\n    * [Considerations](#considerations)\n    * [Hardware Requirements](#hardware_requirements)\n    * [Hardware Setup](#hardware_setup)\n* [Tutorial](#tutorial)\n    * [Part 1: Prerequisites](#part1)\n    * [Part 2: VM Logistics](#part2)\n    * [Part 3: Creating the VM](#part3)\n    * [Part 4: Improving VM Performance](#part4)\n    * [Part 5: Benchmarks](#part5)\n    * [Part 6: Software Licensing Considerations](#part6)\n* [Credits \u0026 Resources](#credits)\n* [Footnotes](#footnotes)\n\n\u003ch2 name=\"introduction\"\u003e\n    Introduction\n\u003c/h2\u003e\n\nIn this post, I will be giving detailed instructions on how to run a KVM setup with GPU passthrough. This setup uses a Linux host installed with [Pop!\\_OS 20.10](https://system76.com/pop) (kernel v5.8.0) and a guest VM running Windows 10.\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/kvm_architecture.jpg\" width=\"500\"\u003e\n    \u003cp\u003e\n        Source: \u003ca href=\"https://events.static.linuxfound.org/sites/events/files/slides/KVM,%20OpenStack,%20and%20the%20Open%20Cloud%20-%20ANJ%20MK%20%20-%2013Oct14.pdf\"\u003eOpen Virtualization Alliance (Jollans, IBM , Kadera, Intel)\u003c/a\u003e\n    \u003c/p\u003e\n\u003c/div\u003e\n\n\u003ch3 name=\"considerations\"\u003e\n    Considerations\n\u003c/h3\u003e\n\nThe main reason I wanted to get this setup working was because I found myself tired of using a dual-boot setup. I wanted to launch a Windows VM specifically for gaming while still being able to use my Linux host for development work (simultaneously).\n\nAt this point, you might be wondering... Why not just game on Linux? This is definitely an option for many people, but not one that suited my particular needs. Gaming on Linux requires the use of tools like [Wine](https://en.wikipedia.org/wiki/Wine_(software)) which act as a compatabilty layer for translating Windows system calls to Linux system calls. On the other hand, a GPU passthrough setup utilizes [KVM](https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine) as a hypervisor to launch individual VMs with specific hardware attached to them. Performance wise, there are pros and cons to each approach.\u003cspan name=\"return1\"\u003e\u003csup\u003e[1](#footnote1)\u003c/sup\u003e\u003c/span\u003e\n\nIn this tutorial, I will create a GPU passthrough setup. Specifically, I will be passing through an NVIDIA GPU to my guest VM while using an AMD GPU for my host. You could easily substitute an iGPU for the host but I chose to use a dGPU for performance reasons.\u003cspan name=\"return2\"\u003e\u003csup\u003e[2](#footnote2)\u003c/sup\u003e\u003c/span\u003e\n\n\u003ch3 name=\"hardware_requirements\"\u003e Hardware Requirements \u003c/h3\u003e\n\nYou're going to need the following to achieve a high-performance VM:\n- Two graphics cards.\n- [Hardware that supports IOMMU](https://en.wikipedia.org/wiki/List_of_IOMMU-supporting_hardware).\n- A monitor with two inputs\u003cspan name=\"return3\"\u003e\u003csup\u003e[3](#footnote3)\u003c/sup\u003e\u003c/span\u003e *or* multiple monitors.\n\nIf you haven't built a PC yet but want it to be KVM/VFIO-focused, check out [this list](https://passthroughpo.st/vfio-increments/) of parts suggested by The Passthrough Post.\n\n\u003ch3 name=\"hardware_setup\"\u003e\n    Hardware Setup\n\u003c/h3\u003e\n\n- CPU:\n    - AMD Ryzen 9 3900X\n- Motherboard:\n    - Gigabyte X570 Aorus Pro Wifi\n- GPUs:\n    - NVIDIA RTX 3080\n    - AMD RX 5700\n- Memory:\n    - Corsair Vengeance LPX DDR4 3200 MHz 32GB (2x16)\n- Disk:\n    - Samsung 970 EVO Plus SSD 500GB - M.2 NVMe (host)\n    - Samsung 970 EVO Plus SSD 1TB - M.2 NVMe (guest)\n\n\u003ch2 name=\"tutorial\"\u003e\n    Tutorial\n\u003c/h2\u003e\n\n\u003ch3 name=\"part1\"\u003e\n    Part 1: Prerequisites\n\u003c/h3\u003e\n\nBefore we begin, let's install some necessary packages:\n\n```\n$ sudo apt install libvirt-daemon-system libvirt-clients qemu-kvm qemu-utils virt-manager ovmf\n```\n\nRestart your machine and boot into BIOS. Enable a feature called `IOMMU`. You'll also need to enable CPU virtualization. For Intel processors, look for something called `VT-d`. For AMD, look for something called `AMD-Vi`. My motherboard is unique so I had to enable a feature called `SVM Mode`. Save any changes and restart the machine.\n\nOnce you've booted into the host, make sure that IOMMU is enabled:\n`$ dmesg | grep IOMMU`\n\nAlso check that CPU virtualization is enabled:\u003cbr\u003e\u003cbr\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; For Intel: `$ dmesg | grep VT-d` \u003cbr\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; For AMD: `$ dmesg | grep AMD-Vi`\n\nNow you're going to need to pass the hardware-enabled IOMMU functionality into the kernel as a [kernel parameter](https://wiki.archlinux.org/index.php/kernel_parameters). For our purposes, it makes the most sense to enable this feature at boot-time. Depending on your boot-loader (i.e. grub, systemd, rEFInd), you'll have to modify a specific configuration file. Since my machine uses systemd and these configuration files are often overwritten on updates, I will be using a tool called [kernelstub](https://github.com/pop-os/kernelstub):\n\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; For Intel: `$ sudo kernelstub --add-options \"intel_iommu=on\"`\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; For AMD: `$ sudo kernelstub --add-options \"amd_iommu=on\"`\n\nSimilarly, if your system is configured with [GRUB2](https://help.ubuntu.com/community/Grub2), you can achieve the same result by editing the `/etc/default/grub` file with sudo permissions and including the kernel parameter as follows:\n\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; For Intel: `GRUB_CMDLINE_LINUX_DEFAULT=\"quiet splash intel_iommu=on\"`\u003cbr/\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; For AMD: `GRUB_CMDLINE_LINUX_DEFAULT=\"quiet splash amd_iommu=on\"`\n\nWhen planning my GPU passthrough setup, I discovered that many tutorials at this point will go ahead and have you blacklist the nvidia/amd drivers. The logic stems from the fact that since the native drivers can't attach to the GPU at boot-time, the GPU will be freed-up and available to bind to the vfio drivers instead. Most tutorials will have you add a kernel parameter called `pci-stub` with the [PCI bus ID](https://wiki.debian.org/HowToIdentifyADevice/PCI) of your GPU to achieve this. I found that this solution wasn't suitable for me. I prefer to dynamically unbind the nvidia/amd drivers and bind the vfio drivers right before the VM starts and subsequently reversing these actions when the VM stops ([see Part 2](#part2)). That way, whenever the VM isn't in use, the GPU is available to the host machine to do work on its native drivers.\u003cspan name=\"return4\"\u003e\u003csup\u003e[4](#footnote4)\u003c/sup\u003e\u003c/span\u003e\n\nNext, we need to determine the IOMMU groups of the graphics card we want to pass through to the VM. For those of you who don't already know, [IOMMU](https://en.wikipedia.org/wiki/Input%E2%80%93output_memory_management_unit) refers to the chipset device that maps virtual addresses to physical addresses on your I/O devices (i.e. GPU, disk, etc.). Its function is analogous to the memory management unit ([MMU](https://en.wikipedia.org/wiki/Memory_management_unit)) that maps virtual addresses to physical addresses on your CPU.\n\nWe want to make sure that our system has an appropriate IOMMU grouping scheme. Essentially, we need to remember that devices residing within the same IOMMU group need to be passed through to the VM (they can't be separated). To determine your IOMMU grouping, use the following script:\n\n`iommu.sh`:\n```\n#!/bin/bash\nfor d in /sys/kernel/iommu_groups/*/devices/*; do\n  n=${d#*/iommu_groups/*}; n=${n%%/*}\n  printf 'IOMMU Group %s ' \"$n\"\n  lspci -nns \"${d##*/}\"\ndone\n```\n\nFor Intel systems, here's some sample output:\n\n```\n...\nIOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)\nIOMMU Group 1 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) [8086:1905] (rev 07)\n...\nIOMMU Group 30 0d:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2206] (rev a1)\nIOMMU Group 30 0d:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aef] (rev a1)\nIOMMU Group 30 0c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:731f] (rev c4)\nIOMMU Group 31 0c:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 HDMI Audio [1002:ab38]\n...\n```\n\nHere we see that both the NVIDIA and AMD GPUs reside in IOMMU group 30. This presents a problem. If you want to use the AMD GPU for the host machine while passing through the NVIDIA GPU to the guest VM, you need to figure out a way to separate their IOMMU groups.\n\n1. One possible solution is to switch the PCI slot to which the AMD graphics card is attached. This may or may not produce the desired solution.\n2. An alternative solution is something called the [ACS Override Patch](https://queuecumber.gitlab.io/linux-acs-override/). For an in-depth discussion, it's definitely worth checking out this post from [Alex Williamson](https://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html). Make sure to consider the risks.\u003cspan name=\"return5\"\u003e\u003csup\u003e[5](#footnote5)\u003c/sup\u003e\u003c/span\u003e\n\nFor my system, I was lucky\u003cspan name=\"return6\"\u003e\u003csup\u003e[6](#footnote6)\u003c/sup\u003e\u003c/span\u003e because the NVIDIA and AMD GPUs resided in different IOMMU groups:\n\n```\n...\nIOMMU Group 30 0c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:731f] (rev c4)\nIOMMU Group 31 0c:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 HDMI Audio [1002:ab38]\nIOMMU Group 32 0d:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2206] (rev a1)\nIOMMU Group 32 0d:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aef] (rev a1)\n...\n```\n\nIf your setup is like mine\u003cspan name=\"return7\"\u003e\u003csup\u003e[7](#footnote7)\u003c/sup\u003e\u003c/span\u003e and you had isolated IOMMU groups, feel free [to skip the following section](#part1.2). Otherwise, please continue reading...\n\n\u003ch4 name=\"part 1.1\"\u003e\n    ACS Override Patch (Optional):\n\u003c/h4\u003e\n\nFor most linux distributions, the ACS Override Patch requires you to download the kernel source code, manually insert the ACS patch, compile + install the kernel, and then boot directly from the newly patched kernel.\u003cspan name=\"return8\"\u003e\u003csup\u003e[8](#footnote8)\u003c/sup\u003e\u003c/span\u003e\n\nSince I'm running a Debian-based distribution, I can use one of the [pre-compiled kernels](https://queuecumber.gitlab.io/linux-acs-override/) with the ACS patch already applied. After extracting the package contents, install the kernel and headers:\n\n```\n$ sudo dpkg -i linux-headers-5.3.0-acso_5.3.0-acso-1_amd64.deb\n$ sudo dpkg -i linux-image-5.3.0-acso_5.3.0-acso-1_amd64.deb\n$ sudo dpkg -i linux-libc-dev_5.3.0-acso-1_amd64.deb\n```\n\nNavigate to /boot and verify that you see the new initrd.img and vmlinuz:\n\n```\n$ ls\nconfig-5.3.0-7625-generic    initrd.img-5.3.0-7625-generic  vmlinuz\nconfig-5.3.0-acso            initrd.img-5.3.0-acso          vmlinuz-5.3.0-7625-generic\nefi                          initrd.img.old                 vmlinuz-5.3.0-acso\ninitrd.img                   System.map-5.3.0-7625-generic  vmlinuz.old\ninitrd.img-5.3.0-24-generic  System.map-5.3.0-acso\n```\n\nWe still have to copy the current kernel and initramfs image onto the ESP so that they are automatically loaded by EFI. We check the current configuration with [kernelstub](https://github.com/pop-os/kernelstub):\n\n```\n$ sudo kernelstub --print-config\nkernelstub.Config    : INFO     Looking for configuration...\nkernelstub           : INFO     System information:\n\n    OS:..................Pop!_OS 19.10\n    Root partition:....../dev/dm-1\n    Root FS UUID:........2105a9ac-da30-41ba-87a9-75437bae74c6\n    ESP Path:............/boot/efi\n    ESP Partition:......./dev/nvme0n1p1\n    ESP Partition #:.....1alt=\"virtman_3\"\n    NVRAM entry #:.......-1\n    Boot Variable #:.....0000\n    Kernel Boot Options:.quiet loglevel=0 systemd.show_status=false splash amd_iommu=on\n    Kernel Image Path:.../boot/vmlinuz\n    Initrd Image Path:.../boot/initrd.img\n    Force-overwrite:.....False\n\nkernelstub           : INFO     Configuration details:\n\n   ESP Location:................../boot/efi\n   Management Mode:...............True\n   Install Loader configuration:..True\n   Configuration version:.........3\n```\n\nYou can see that the \"Kernel Image Path\" and the \"Initrd Image Path\" are symbolic links that point to the old kernel and initrd.\n\n```\n$ ls -l /boot\ntotal 235488\n-rw-r--r-- 1 root root   235833 Dec 19 11:56 config-5.3.0-7625-generic\n-rw-r--r-- 1 root root   234967 Sep 16 04:31 config-5.3.0-acso\ndrwx------ 6 root root     4096 Dec 31  1969 efi\nlrwxrwxrwx 1 root root       29 Dec 20 11:28 initrd.img -\u003e initrd.img-5.3.0-7625-generic\n-rw-r--r-- 1 root root 21197115 Dec 20 11:54 initrd.img-5.3.0-24-generic\n-rw-r--r-- 1 root root 95775016 Jan 17 00:33 initrd.img-5.3.0-7625-generic\n-rw-r--r-- 1 root root 94051072 Jan 18 19:57 initrd.img-5.3.0-acso\nlrwxrwxrwx 1 root root       29 Dec 20 11:28 initrd.img.old -\u003e initrd.img-5.3.0-7625-generic\n-rw------- 1 root root  4707483 Dec 19 11:56 System.map-5.3.0-7625-generic\n-rw-r--r-- 1 root root  4458808 Sep 16 04:31 System.map-5.3.0-acso\nlrwxrwxrwx 1 root root       26 Dec 20 11:28 vmlinuz -\u003e vmlinuz-5.3.0-7625-generic\n-rw------- 1 root root 11398016 Dec 19 11:56 vmlinuz-5.3.0-7625-generic\n-rw-r--r-- 1 root root  9054592 Sep 16 04:31 vmlinuz-5.3.0-acso\nlrwxrwxrwx 1 root root       26 Dec 20 11:28 vmlinuz.old -\u003e vmlinuz-5.3.0-7625-generic\n```\n\nLet's change that:\n\n```\n$ sudo rm /boot/vmlinuz\n$ sudo ln -s /boot/vmlinuz-5.3.0-acso /boot/vmlinuz\n$ sudo rm /boot/initrd.img\n$ sudo ln -s /boot/initrd.img-5.3.0-acso /boot/initrd.img\n```\n\nVerify that the symbolic links now point to the correct kernel and initrd images:\n\n```\n$ ls -l /boot\ntotal 235488\n-rw-r--r-- 1 root root   235833 Dec 19 11:56 config-5.3.0-7625-generic\n-rw-r--r-- 1 root root   234967 Sep 16 04:31 config-5.3.0-acso\ndrwx------ 6 root root     4096 Dec 31  1969 efi\nlrwxrwxrwx 1 root root       27 Jan 18 20:02 initrd.img -\u003e /boot/initrd.img-5.3.0-acso\n-rw-r--r-- 1 root root 21197115 Dec 20 11:54 initrd.img-5.3.0-24-generic\n-rw-r--r-- 1 root root 95775016 Jan 17 00:33 initrd.img-5.3.0-7625-generic\n-rw-r--r-- 1 root root 94051072 Jan 18 19:57 initrd.img-5.3.0-acso\nlrwxrwxrwx 1 root root       29 Dec 20 11:28 initrd.img.old -\u003e initrd.img-5.3.0-7625-generic\n-rw------- 1 root root  4707483 Dec 19 11:56 System.map-5.3.0-7625-generic\n-rw-r--r-- 1 root root  4458808 Sep 16 04:31 System.map-5.3.0-acso\nlrwxrwxrwx 1 root root       24 Jan 18 20:02 vmlinuz -\u003e /boot/vmlinuz-5.3.0-acso\n-rw------- 1 root root 11398016 Dec 19 11:56 vmlinuz-5.3.0-7625-generic\n-rw-r--r-- 1 root root  9054592 Sep 16 04:31 vmlinuz-5.3.0-acso\nlrwxrwxrwx 1 root root       26 Dec 20 11:28 vmlinuz.old -\u003e vmlinuz-5.3.0-7625-generic\n```\n\nFinally, add the ACS Override Patch to your list of kernel parameter options:\n\n```\n$ sudo kernelstub --add-options \"pcie_acs_override=downstream\"\n```\n\nReboot and verify that the IOMMU groups for your graphics cards are different:\n\n```\n...\nIOMMU Group 30 0c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:731f] (rev c4)\nIOMMU Group 31 0c:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 HDMI Audio [1002:ab38]\nIOMMU Group 32 0d:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2206] (rev a1)\nIOMMU Group 32 0d:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aef] (rev a1)\n...\n```\n\n\u003ch4 name=\"part1.2\"\u003e\n    Download ISO files (Mandatory):\n\u003c/h4\u003e\n\nSince we're building a Windows VM, we're going to need to download and use the virtIO drivers. [virtIO](https://www.linux-kvm.org/page/Virtio) is a virtualization standard for network and disk device drivers. Adding the virtIO drivers can be done by attaching its relevant ISO to the Windows VM during creation. Fedora provides the virtIO drivers for [direct download](https://docs.fedoraproject.org/en-US/quick-docs/creating-windows-virtual-machines-using-virtio-drivers/#virtio-win-direct-downloads).\n\nSince I am passing through an entire NVMe SSD (1TB), I won't need to install any 3rd party drivers on top of the virtIO driver. Passing through the SSD as a PCI device lets Windows deal with it as a native NVMe device and therefore *should* offer better performance. If you choose to use a raw disk image instead, things are going to be a little different... Make sure to follow the instructions in [this guide](https://frdmtoplay.com/virtualizing-windows-7-or-linux-on-a-nvme-drive-with-vfio/#builddriveriso). The guide will show you how to add 3rd party drivers on top of the existing virtIO drivers by rebuilding the ISO.\n\nFor the final step, we're going to need to download the Windows 10 ISO from Microsoft which you can find [here](https://www.microsoft.com/en-us/software-download/windows10ISO).\n\n\u003ch3 name=\"part2\"\u003e\n    Part 2: VM Logistics\n\u003c/h3\u003e\n\nAs mentioned earlier, we are going to dynamically bind the vfio drivers before the VM starts and unbind these drivers after the VM terminates. To achieve this, we're going to use [libvirt hooks](https://libvirt.org/hooks.html). Libvirt has a hook system that allows you to run commands on startup or shutdown of a VM. All relevant scripts are located within the following directory: `/etc/libvirt/hooks`. If the directory doesn't exist, go ahead and create it. Lucky for us, The Passthrough POST has a [hook helper tool](https://passthroughpo.st/simple-per-vm-libvirt-hooks-with-the-vfio-tools-hook-helper/) to make our lives easier. Run the following commands to install the hook manager and make it executable:\n\n```\n$ sudo wget 'https://raw.githubusercontent.com/PassthroughPOST/VFIO-Tools/master/libvirt_hooks/qemu' \\\n     -O /etc/libvirt/hooks/qemu\n$ sudo chmod +x /etc/libvirt/hooks/qemu\n```\n\nGo ahead and restart libvirt to use the newly installed hook helper:\n\n```\n$ sudo service libvirtd restart\n```\n\nLet's look at the most important hooks:\n\n```\n# Before a VM is started, before resources are allocated:\n/etc/libvirt/hooks/qemu.d/$vmname/prepare/begin/*\n\n# Before a VM is started, after resources are allocated:\n/etc/libvirt/hooks/qemu.d/$vmname/start/begin/*\n\n# After a VM has started up:\n/etc/libvirt/hooks/qemu.d/$vmname/started/begin/*\n\n# After a VM has shut down, before releasing its resources:\n/etc/libvirt/hooks/qemu.d/$vmname/stopped/end/*\n\n# After a VM has shut down, after resources are released:\n/etc/libvirt/hooks/qemu.d/$vmname/release/end/*\n```\n\nIf we place an executable script in one of these directories, the hook manager will take care of everything else. I've chosen to name my VM \"win10\" so I set up my directory structure like this:\n\n```\n$ tree /etc/libvirt/hooks/\n/etc/libvirt/hooks/\n├── qemu\n└── qemu.d\n    └── win10\n        ├── prepare\n        │   └── begin\n        └── release\n            └── end\n```\n\nIt's time to get our hands dirty... Create a file named `kvm.conf` and place it under `/etc/libvirt/hooks/`. Add the following entries to the file:\n\n```\n## Virsh devices\nVIRSH_GPU_VIDEO=pci_0000_0a_00_0\nVIRSH_GPU_AUDIO=pci_0000_0a_00_1\nVIRSH_GPU_USB=pci_0000_0a_00_2\nVIRSH_GPU_SERIAL=pci_0000_0a_00_3\nVIRSH_NVME_SSD=pci_0000_04_00_0\n```\n\nMake sure to substitute the correct bus addresses for the devices you'd like to passthrough to your VM (in my case a GPU and SSD). Just in case it's still unclear, you get the virsh PCI device IDs from the `iommu.sh` script's output. Translate the address for each device as follows: `IOMMU Group 1 01:00.0 ...` --\u003e `VIRSH_...=pci_0000_01_00_0`. Now create two bash scripts:\n\n`bind_vfio.sh`:\n```\n#!/bin/bash\n\n## Load the config file\nsource \"/etc/libvirt/hooks/kvm.conf\"\n\n## Load vfio\nmodprobe vfio\nmodprobe vfio_iommu_type1\nmodprobe vfio_pci\n\n## Unbind gpu from nvidia and bind to vfio\nvirsh nodedev-detach $VIRSH_GPU_VIDEO\nvirsh nodedev-detach $VIRSH_GPU_AUDIO\nvirsh nodedev-detach $VIRSH_GPU_USB\nvirsh nodedev-detach $VIRSH_GPU_SERIAL\n## Unbind ssd from nvme and bind to vfio\nvirsh nodedev-detach $VIRSH_NVME_SSD\n```\n\n`unbind_vfio.sh`:\n```\n#!/bin/bash\n\n## Load the config file\nsource \"/etc/libvirt/hooks/kvm.conf\"\n\n## Unbind gpu from vfio and bind to nvidia\nvirsh nodedev-reattach $VIRSH_GPU_VIDEO\nvirsh nodedev-reattach $VIRSH_GPU_AUDIO\nvirsh nodedev-reattach $VIRSH_GPU_USB\nvirsh nodedev-reattach $VIRSH_GPU_SERIAL\n## Unbind ssd from vfio and bind to nvme\nvirsh nodedev-reattach $VIRSH_NVME_SSD\n\n## Unload vfio\nmodprobe -r vfio_pci\nmodprobe -r vfio_iommu_type1\nmodprobe -r vfio\n```\n\nDon't forget to make these scripts executable with `chmod +x \u003cscript_name\u003e`. Then place these scripts so that your directory structure looks like this:\n\n```\n$ tree /etc/libvirt/hooks/\n/etc/libvirt/hooks/\n├── kvm.conf\n├── qemu\n└── qemu.d\n    └── win10\n        ├── prepare\n        │   └── begin\n        │       └── bind_vfio.sh\n        └── release\n            └── end\n                └── unbind_vfio.sh\n```\n\nWe've succesfully created libvirt hook scripts to dynamically bind the vfio drivers before the VM starts and unbind these drivers after the VM terminates. At the moment, we're done messing around with libvirt hooks. We'll revisit this topic later on when we make performance tweaks to our VM ([see Part 4](#part4)).\n\n\u003ch3 name=\"part3\"\u003e\n    Part 3: Creating the VM\n\u003c/h3\u003e\n\nWe're ready to begin creating our VM. There are basically two options for how to achieve this: **(1)** If you prefer a GUI approach, then follow the rest of this tutorial. **(2)** If you prefer bash scripts, take a look at YuriAlek's series of [GPU passthrough scripts](https://gitlab.com/YuriAlek/vfio) and customize them to fit your needs. The main difference between these two methods lies with the fact that the scripting approach uses [bare QEMU](https://www.mankier.com/1/qemu) commands\u003cspan name=\"return9\"\u003e\u003csup\u003e[9](#footnote9)\u003c/sup\u003e\u003c/span\u003e, while the GUI approach uses [virt-manager](https://virt-manager.org/). Virt-manager essentially builds on-top of the QEMU base-layer and adds other features/complexity.\u003cspan name=\"return10\"\u003e\u003csup\u003e[10](#footnote10)\u003c/sup\u003e\u003c/span\u003e\n\nGo ahead and start virt-manager from your list of applications. Select the button on the top left of the GUI to create a new VM:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_1.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nSelect the \"Local install media\" option. My ISOs are stored in my home directory `/home/user/.iso`, so I'll create a new pool and select the Windows 10 ISO from there:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_2.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nConfigure some custom RAM and CPU settings for your VM:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_3.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nNext, the GUI asks us whether we want to enable storage for the VM. As already mentioned, my setup will be using SSD passthrough so I chose not to enable virtual storage. However, you still have the option to enable storage and create a RAW disk image which will be stored under the default path of `/var/lib/libvirt/images`:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_4.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nOn the last step, review your settings and select a name for your VM. Make sure to select the checkbox \"Customize configuration before installation\" and click Finish:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_5.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nA new window should appear with more advanced configuration options. You can alter these options through the GUI or the associated libvirt XML settings. Make sure that on the Overview page under Firmware you select `UEFI x86_64: /usr/share/OVMF/OVMF_CODE.fd`:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_6.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nGo to the CPUs page and remove the check next to `Copy host CPU configuration` and under Model type `host-passthrough`. Also make sure to check the option for `Enable available CPU security flaw mitigations` to prevent against Spectre/Meltdown vulnerabilities.\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_7.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nI've chosen to remove several of the menu options that won't be useful to my setup (feel free to keep them if you'd like):\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_8.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nLet's add the \u003cspan name=\"virtio-iso\"\u003evirtIO drivers\u003c/span\u003e. Click 'Add Hardware' and under 'Storage', create a custom storage device of type `CDROM`. Make sure to locate the ISO image for the virtIO drivers from earlier:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_9.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nUnder the NIC menu, change the device model to `virtIO` for improved networking performance:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_10.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nNow it's time to configure our passthrough devices! Click 'Add Hardware' and under 'PCI Host Device', select the Bus IDs corresponding to your GPU.\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_11.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nMake sure to repeat this step for all the devices associated with your GPU in the same IOMMU group (usually VGA, audio controller, etc.):\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_12.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nSince I'm passing through an entire disk to my VM, I selected the Bus ID corresponding to the 1TB Samsung NVMe SSD which has Windows 10 (and my games) installed on it.\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_13.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nThen under the 'Boot Options' menu, I added a check next to `Enable boot menu` and reorganized the devices so that I could boot directly from the 1TB SSD:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_14.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nYou can now go ahead and select the USB Host Devices you'd like to passthrough to your guest VM (usually a keyboard, mouse, etc.). Please note that these devices will be held by the guest VM from the moment it's created until it's stopped and will be unavailable to the host.\u003cspan name=\"return11\"\u003e\u003csup\u003e[11](#footnote11)\u003c/sup\u003e\u003c/span\u003e\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_15.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nUnfortunately, not everything we need can be accomplished within the virt-manager GUI. For the rest of this section, we'll have to do some fine-tuning by directly editing the XML (make sure to \"Enable XML settings\" under Edit -\u003e Preferences -\u003e General or use `$ sudo virsh edit win10` for a command-line approach):\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/virtman_16.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nIf you're like me and you're passing through an NVIDIA GPU to your VM, then you might run into the following common roadblock. [Error 43](https://passthroughpo.st/apply-error-43-workaround/) occurs because NVIDIA intentionally disables virtualization features on its GeForce line of cards. The way to deal with this is to have the hypervisor hide its existence. Inside the `hyperv` section, add a tag for `vendor_id` such that `state=\"on\"` and `value` is any string up to 12 characters long:\n\n```\n\u003cfeatures\u003e\n    ...\n    \u003chyperv\u003e\n        \u003crelaxed state=\"on\"/\u003e\n        \u003cvapic state=\"on\"/\u003e\n        \u003cspinlocks state=\"on\" retries=\"8191\"/\u003e\n        \u003cvendor_id state=\"on\" value=\"kvm hyperv\"/\u003e\n    \u003c/hyperv\u003e\n    ...\n\u003c/features\u003e\n```\n\nIn addition, instruct the kvm to hide its state by adding the following code directly below the `hyperv` section:\n\n```\n\u003cfeatures\u003e\n    ...\n    \u003chyperv\u003e\n        ...\n    \u003c/hyperv\u003e\n    \u003ckvm\u003e\n      \u003chidden state=\"on\"/\u003e\n    \u003c/kvm\u003e\n    ...\n\u003c/features\u003e\n```\n\nFinally, if you're using QEMU 4.0 with the q35 chipset you also need to add the following code at the end of `\u003cfeatures\u003e`:\n\n```\n\u003cfeatures\u003e\n    ...\n    \u003cioapic driver=\"kvm\"/\u003e\n\u003c/features\u003e\n```\n\nNow you should have no issues with regards to the NVIDIA Error 43. Later on, we will be making more changes to the XML to achieve better performance ([see Part 4](#part4)). At this point however, you can apply the changes and select \"Begin Installation\" at the top left of the GUI. Please be aware that this may take several minutes to complete.\n\n\u003ch3 name=\"part4\"\u003e\n    Part 4: Improving VM Performance\n\u003c/h3\u003e\n\nNone of the following performance optimizations are necessary to get a working GPU passthrough system. However, these tweaks will make a difference if you're at all concerned about reaching buttery-smooth gaming performance. Though some of these changes are more difficult than others, I highly advise you to at least consider them.\n\n\u003ch4\u003e\n    Hugepages\n\u003c/h4\u003e\n\nMemory (RAM) is divided up into basic segments called *pages*. By default, the x86 architecture has a page size of 4KB. CPUs utilize pages within the built in memory management unit ([MMU](https://en.wikipedia.org/wiki/Memory_management_unit)). Although the standard page size is suitable for many tasks, *hugepages* are a mechanism that allow the Linux kernel to take advantage of large amounts of memory with reduced overhead. Hugepages can vary in size anywhere from 2MB to 1GB. Hugepages are enabled by default but if they aren't, make sure to download the package: `$ sudo apt install libhugetlbfs-bin`.\u003cspan name=\"return12\"\u003e\u003csup\u003e[12](#footnote12)\u003c/sup\u003e\u003c/span\u003e\n\nGo back to your VM's XML settings by either using the virt-man GUI or the command: `$ sudo virsh edit {vm-name}`. Insert the `memoryBacking` lines so that your configuration looks like this:\n\n```\n\u003cmemory unit=\"KiB\"\u003e16777216\u003c/memory\u003e\n\u003ccurrentMemory unit=\"KiB\"\u003e16777216\u003c/currentMemory\u003e\n\u003cmemoryBacking\u003e\n    \u003chugepages/\u003e\n\u003c/memoryBacking\u003e\n```\n\nMany tutorials will have you reserve hugepages for your guest VM at host boot-time. There's a significant downside to this approach: a portion of RAM will be unavailable to your host even when the VM is inactive. In my setup, I've chose to allocate hugepages before the VM starts and deallocate those pages on VM shutdown through the use of two additional executable scripts\u003cspan name=\"return13\"\u003e\u003csup\u003e[13](#footnote13)\u003c/sup\u003e\u003c/span\u003e inside libvirt hooks ([see Part 2](#part2)):\n\n```\n$ tree /etc/libvirt/hooks/\n/etc/libvirt/hooks/\n├── kvm.conf\n├── qemu\n└── qemu.d\n    └── win10\n        ├── prepare\n        │   └── begin\n        │       ├── ...\n        │       └── alloc_hugepages.sh\n        └── release\n            └── end\n                ├── ...\n                └── dealloc_hugepages.sh\n```\n\n`alloc_hugepages.sh`:\n```\n#!/bin/bash\n\n## Load the config file\nsource \"/etc/libvirt/hooks/kvm.conf\"\n\n## Calculate number of hugepages to allocate from memory (in MB)\nHUGEPAGES=\"$(($MEMORY/$(($(grep Hugepagesize /proc/meminfo | awk '{print $2}')/1024))))\"\n\necho \"Allocating hugepages...\"\necho $HUGEPAGES \u003e /proc/sys/vm/nr_hugepages\nALLOC_PAGES=$(cat /proc/sys/vm/nr_hugepages)\n\nTRIES=0\nwhile (( $ALLOC_PAGES != $HUGEPAGES \u0026\u0026 $TRIES \u003c 1000 ))\ndo\n    echo 1 \u003e /proc/sys/vm/compact_memory            ## defrag ram\n    echo $HUGEPAGES \u003e /proc/sys/vm/nr_hugepages\n    ALLOC_PAGES=$(cat /proc/sys/vm/nr_hugepages)\n    echo \"Succesfully allocated $ALLOC_PAGES / $HUGEPAGES\"\n    let TRIES+=1\ndone\n\nif [ \"$ALLOC_PAGES\" -ne \"$HUGEPAGES\" ]\nthen\n    echo \"Not able to allocate all hugepages. Reverting...\"\n    echo 0 \u003e /proc/sys/vm/nr_hugepages\n    exit 1\nfi\n```\n\n`dealloc_hugepages.sh`\n```\n#!/bin/bash\n\n## Load the config file\nsource \"/etc/libvirt/hooks/kvm.conf\"\n\necho 0 \u003e /proc/sys/vm/nr_hugepages\n```\n\nBe sure to update your kvm.conf to include `MEMORY=\u003csize in mb\u003e` as the `alloc_hugepages.sh` script requires a value here. To calculate your hugepages, you can use the following commands to see your total memory and your hugepage size:\n\n```\n$ grep MemTotal /proc/meminfo\nMemTotal: 132151496 kB\n$ grep Hugepagesize /proc/meminfo\nHugepagesize:       2048 kB\n```\n\nYou will want to set `MEMORY=` to a multiple of your hugepage size in MB. From the example above: size = 2048 kB and we want roughly 16GB of memory, so we can choose `MEMORY=16384` (2048 * 8000 / 1000 = 16384).\n\n\u003ch4\u003e\n    CPU Governor\n\u003c/h4\u003e\n\nThis performance tweak\u003cspan name=\"return14\"\u003e\u003csup\u003e[14](#footnote14)\u003c/sup\u003e\u003c/span\u003e takes advantage of the [CPU frequency scaling governor](https://wiki.archlinux.org/index.php/CPU_frequency_scaling#Scaling_governors) in Linux. It's a feature that is often ofterlooked in many passthrough tutorials, but we include it here because it's recommended. Once again, we'll be utilizing libvirt's hook system ([see Part 2](#part2)):\n\n```\n$ tree /etc/libvirt/hooks/\n/etc/libvirt/hooks/\n├── kvm.conf\n├── qemu\n└── qemu.d\n    └── win10\n        ├── prepare\n        │   └── begin\n        │       ├── ...\n        │       └── cpu_mode_performance.sh\n        └── release\n            └── end\n                ├── ...\n                └── cpu_mode_ondemand.sh\n```\n\n`cpu_mode_performance.sh`:\n```\n#!/bin/bash\n\n## Load the config file\nsource \"/etc/libvirt/hooks/kvm.conf\"\n\n## Enable CPU governor performance mode\ncat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor\nfor file in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do echo \"performance\" \u003e $file; done\ncat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor\n\n```\n\n`cpu_mode_ondemand.sh`:\n```\n#!/bin/bash\n\n## Load the config file\nsource \"/etc/libvirt/hooks/kvm.conf\"\n\n## Enable CPU governor on-demand mode\ncat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor\nfor file in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do echo \"ondemand\" \u003e $file; done\ncat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor\n```\n\n\u003ch4\u003e\n    CPU Pinning\n\u003c/h4\u003e\n\nThis performance tweak applies *only* to those of you whose processors are [multithreaded](https://en.wikipedia.org/wiki/Multithreading_(computer_architecture)). My setup has an AMD Ryzen 9 3900X which has 12 physical cores and 24 threads (i.e. logical cores).\n\nVMs are unable to distinguish between these physical and logical cores. From the guest's perspective, virt-manager sees that there are 24 virtual CPUs (vCPUs) available. From the host's perspective however, two virtual cores map to a single physical core on the CPU die.\n\nIt's **very important** that when we passthrough a core, we include its sibling. To get a sense of your cpu topology, use the command `$ lscpu -e\"`. A matching core id (i.e. \"CORE\" column) means that the associated threads (i.e. \"CPU\" column) run on the same physical core.\u003cspan name=\"return15\"\u003e\u003csup\u003e[15](#footnote15)\u003c/sup\u003e\u003c/span\u003e\n\n```\nCPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE    MAXMHZ    MINMHZ\n  0    0      0    0 0:0:0:0          yes 3800.0000 2200.0000\n  1    0      0    1 1:1:1:0          yes 3800.0000 2200.0000\n  2    0      0    2 2:2:2:0          yes 3800.0000 2200.0000\n  3    0      0    3 3:3:3:1          yes 3800.0000 2200.0000\n  4    0      0    4 4:4:4:1          yes 3800.0000 2200.0000\n  5    0      0    5 5:5:5:1          yes 3800.0000 2200.0000\n  6    0      0    6 6:6:6:2          yes 3800.0000 2200.0000\n  7    0      0    7 7:7:7:2          yes 3800.0000 2200.0000\n  8    0      0    8 8:8:8:2          yes 3800.0000 2200.0000\n  9    0      0    9 9:9:9:3          yes 3800.0000 2200.0000\n 10    0      0   10 10:10:10:3       yes 3800.0000 2200.0000\n 11    0      0   11 11:11:11:3       yes 3800.0000 2200.0000\n 12    0      0    0 0:0:0:0          yes 3800.0000 2200.0000\n 13    0      0    1 1:1:1:0          yes 3800.0000 2200.0000\n 14    0      0    2 2:2:2:0          yes 3800.0000 2200.0000\n 15    0      0    3 3:3:3:1          yes 3800.0000 2200.0000\n 16    0      0    4 4:4:4:1          yes 3800.0000 2200.0000\n 17    0      0    5 5:5:5:1          yes 3800.0000 2200.0000\n 18    0      0    6 6:6:6:2          yes 3800.0000 2200.0000\n 19    0      0    7 7:7:7:2          yes 3800.0000 2200.0000\n 20    0      0    8 8:8:8:2          yes 3800.0000 2200.0000\n 21    0      0    9 9:9:9:3          yes 3800.0000 2200.0000\n 22    0      0   10 10:10:10:3       yes 3800.0000 2200.0000\n 23    0      0   11 11:11:11:3       yes 3800.0000 2200.0000\n\n```\n\nIf you're more of a visual learner, perhaps a diagram of your CPU architecture will help you visualize what's going on. Download the `hwloc` package with `$ sudo apt install hwloc`. Then simply type the command `$ lstopo`:\n\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"./img/lstopo.png\" width=\"450\"\u003e\n\u003c/div\u003e\u003cbr\u003e\n\nIt's time to edit the XML configuration of our VM. I've added the following lines of code to pass physical cores #6-11 to the guest and leave physical cores #0-5 with the host (customize for your processor):\n\n```\n\u003cvcpu placement=\"static\"\u003e12\u003c/vcpu\u003e\n\u003ccputune\u003e\n    \u003cvcpupin vcpu=\"0\" cpuset=\"6\"/\u003e\n    \u003cvcpupin vcpu=\"1\" cpuset=\"18\"/\u003e\n    \u003cvcpupin vcpu=\"2\" cpuset=\"7\"/\u003e\n    \u003cvcpupin vcpu=\"3\" cpuset=\"19\"/\u003e\n    \u003cvcpupin vcpu=\"4\" cpuset=\"8\"/\u003e\n    \u003cvcpupin vcpu=\"5\" cpuset=\"20\"/\u003e\n    \u003cvcpupin vcpu=\"6\" cpuset=\"9\"/\u003e\n    \u003cvcpupin vcpu=\"7\" cpuset=\"21\"/\u003e\n    \u003cvcpupin vcpu=\"8\" cpuset=\"10\"/\u003e\n    \u003cvcpupin vcpu=\"9\" cpuset=\"22\"/\u003e\n    \u003cvcpupin vcpu=\"10\" cpuset=\"11\"/\u003e\n    \u003cvcpupin vcpu=\"11\" cpuset=\"23\"/\u003e\n    \u003cemulatorpin cpuset=\"0-3\"/\u003e\n    \u003ciothreadpin iothread='1' cpuset='4-5,12-17'/\u003e\n\u003c/cputune\u003e\n```\n\nIf you're wondering why I tuned my CPU configuration this way, I'll refer you to [this section](https://libvirt.org/formatdomain.html#elementsCPUTuning) of the Libvirt domain XML format.\u003cspan name=\"return16\"\u003e\u003csup\u003e[16](#footnote16)\u003c/sup\u003e\u003c/span\u003e More specifically, consider the `cputune` element and its underlying `vcpupin`, `emulatorpin`, and `iothreadpin` elements. The Arch Wiki recommends to pin the emulator and iothreads to host cores (if available) rather than the VCPUs assigned to the guest. In the example above, 12 out of my 24 threads are assigned as vCPUs to the guest and from the remaining 12 threads on the host, 4 are assigned to the emulator and 8 are assigned to an iothread [see below](#disk).\n\nGo ahead and edit `\u003ccpu\u003e` to formally define the CPU topography of your VM. In my case, I'm allocating 1 socket with 6 physical cores and 2 threads per core:\n\n```\n\u003ccpu mode=\"host-passthrough\" check=\"none\"\u003e\n  \u003ctopology sockets=\"1\" cores=\"6\" threads=\"2\"/\u003e\n  \u003ccache mode='passthrough'/\u003e\n  \u003cfeature policy='require' name='topoext'/\u003e\n\u003c/cpu\u003e\n```\n\n\u003ch4 id=\"disk\"\u003e\n    Disk Tuning\n\u003c/h4\u003e\n\nAs you may or may not remember, my setup passes control of an SSD device controller to the VM. This bypasses any need or concern I'd have with improving virtualized disk performance (I/O reads + writes). If this is not the case for your setup, then you probably allocated a virtual storage disk on your host device. For the rest of this section, let's assume my setup uses a RAW virtual disk image stored at `/var/lib/libvirt/images/pool/win10.img` on which I'd like to improve I/O performance.\n\nKVM and QEMU provide two paravirtualized storage backends: the older virtio-blk (default) and the more modern virtio-scsi. Although it's beyond the scope of this tutorial to discuss their differences, [this post](https://mpolednik.github.io/2017/01/23/virtio-blk-vs-virtio-scsi/) highlights the main architectural difference between the two:\n\n`virtio-blk`:\n```\nguest: app -\u003e Block Layer -\u003e virtio-blk\nhost: QEMU -\u003e Block Layer -\u003e Block Device Driver -\u003e Hardware\n```\n\n`virtio-scsi`:\n```\nguest: app -\u003e Block Layer -\u003e SCSI Layer -\u003e scsi_mod\nhost: QEMU -\u003e Block Layer -\u003e SCSI Layer -\u003e Block Device Driver -\u003e Hardware\n```\n\nIn essence, virtio-scsi adds an additional complexity layer that provides it with more features and flexibility than virtio-blk.\u003cspan name=\"return17\"\u003e\u003csup\u003e[17](#footnote17)\u003c/sup\u003e\u003c/span\u003e Whichever paravirtualized storage type you decide to go with is entirely up to you; I suggest you run performance tests on both. Make sure that in your CPU configuration, you've assigned an [IOThread](https://libvirt.org/formatdomain.html#elementsIOThreadsAllocation):\n\n```\n\u003cvcpu placement=\"static\"\u003e12\u003c/vcpu\u003e\n\u003ciothreads\u003e1\u003c/iothreads\u003e\n\u003ccputune\u003e\n    ...\n    \u003cemulatorpin cpuset=\"0-3\"/\u003e\n    \u003ciothreadpin iothread='1' cpuset='4-5,12-17'/\u003e\n\u003c/cputune\u003e\n```\n\nHere you can see that I've included an `iothreads` element with a value of 1. I've also included the `iothreadpin` element to define the number of CPU pins applied to the single iothread. I highly recommend reviewing [this section](https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Virtio_disk) of the Arch Wiki to decide on your CPU pinning strategy. Ultimately, it's up to you on how you want to divide the CPU pins among the emulator and iothreads.\n\n\nThe final step is to either: **(1)** create the virtio-scsi controller and attach our disk or **(2)** make sure our disk is defined correctly for virtio-blk (default). Note that you can *only* have one iothread per disk controller.\n\n`virtio-scsi`:\n```\n\u003cdomain type=\"kvm\"\u003e\n    ...\n    \u003cdevices\u003e\n        ...\n        \u003cdisk type='file' device='disk'\u003e\n            \u003cdriver name='qemu' type='raw' cache='none' io='threads' discard='unmap' queues='8'/\u003e\n            \u003csource dev='/var/lib/libvirt/images/pool/win10.img'/\u003e\n            \u003ctarget dev='sdc' bus='scsi'/\u003e\n            \u003caddress type='drive' controller='0' bus='0' target='0' unit='2'/\u003e\n        \u003c/disk\u003e\n        ...\n        \u003ccontroller type='scsi' index='0' model='virtio-scsi'\u003e\n            \u003cdriver iothread='1' queues='8'/\u003e\n            \u003caddress type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/\u003e\n        \u003c/controller\u003e       \n        ...\n      \u003c/devices\u003e\n\u003c/domain\u003e\n```\n\n`virtio-blk`:\n```\n\u003cdomain type=\"kvm\"\u003e\n    ...\n    \u003cdevices\u003e\n        ...\n        \u003cdisk type='file' device='disk'\u003e\n            \u003cdriver name='qemu' type='raw' cache='none' io='native' discard='unmap' iothread='1' queues='8'/\u003e\n            \u003csource dev='/var/lib/libvirt/images/pool/win10.img'/\u003e\n            \u003ctarget dev='vdc' bus='virtio'/\u003e\n        \u003c/disk\u003e\n        ...\n    \u003c/devices\u003e\n    ...\n\u003c/domain\u003e\n```\n\nThe final thing to remember is that during the windows installation on your virtual disk, you need to include the virtio-iso as the second CDROM to load the drivers (we've already completed this in a [previous section](#virtio-iso)).\n\n\u003ch4\u003e\n    Hyper-V Enlightenments\n\u003c/h4\u003e\n\nHyper-V enlightenments help the guest VM handle virtualization tasks. [Libvirt](https://libvirt.org/formatdomain.html#elementsFeatures) has a detailed breakdown of these features. I've chosen to go with the set of features recommended in [this tutorial](https://mathiashueber.com/performance-tweaks-gaming-on-virtual-machines/) due to hardware similarities:\n\n```\n\u003cfeatures\u003e\n    ...\n    \u003chyperv\u003e\n      \u003crelaxed state=\"on\"/\u003e\n      \u003cvapic state=\"on\"/\u003e\n      \u003cspinlocks state=\"on\" retries=\"8191\"/\u003e\n      \u003cvendor_id state=\"on\" value=\"kvm hyperv\"/\u003e\n      \u003cvpindex state='on'/\u003e\n      \u003csynic state='on'/\u003e\n      \u003cstimer state='on'/\u003e\n      \u003creset state='on'/\u003e\n      \u003cfrequencies state='on'/\u003e\n    \u003c/hyperv\u003e\n    ...\n\u003c/features\u003e\n```\n\n\u003ch3 name=\"part5\"\u003e\n    Part 5: Benchmarks\n\u003c/h3\u003e\n\nCongrats! You've finished setting up your Windows gaming VM! But now comes the most important part... Let's compare the bare-metal performance of Windows against our KVM. If everything goes according to plan, we can expect somewhat close to native performance on the VM. In order to test this theory, I used the following benchmark software: [UserBenchmark](https://www.userbenchmark.com/). Check out the results\u003cspan name=\"return18\"\u003e\u003csup\u003e[18](#footnote18)\u003c/sup\u003e\u003c/span\u003e for yourself:\n\n* [Windows Native](https://www.userbenchmark.com/UserRun/25008533)\n* [Windows KVM](https://www.userbenchmark.com/UserRun/25008992)\n\nHopefully your results are as good as mine, if not better!\n\n\u003ch3 name=\"part6\"\u003e\n    Part 6: Software Licensing Considerations\n\u003c/h3\u003e\n\nWhen running in the qemu environment, as described above, unique system identifiers are set by the virtual environment. These identifiers are often used to tie a software license to a physical machine. Because the virtual machine is merely\nduplicating the physical machine, one can copy the physical system system identifiers into the virtual machine. If one is also using a dedicated physical device for the virtual machine storage, this allows booting the Windows installation\nas a virtual machine or natively with dual-boot.\n\nTo do this, one needs to modify the XML of the virtual machine to replicate their system. An example with some valid values is below:\n\n```\n\u003csysinfo type=\"smbios\"\u003e\n    \u003cbios\u003e\n      \u003centry name=\"vendor\"\u003eAmerican Megatrends, Inc.\u003c/entry\u003e\n      \u003centry name=\"version\"\u003e0812\u003c/entry\u003e\n      \u003centry name=\"date\"\u003e02/24/2023\u003c/entry\u003e\n      \u003centry name=\"release\"\u003e8.12\u003c/entry\u003e\n    \u003c/bios\u003e\n    \u003csystem\u003e\n      \u003centry name=\"manufacturer\"\u003eASUS\u003c/entry\u003e\n      \u003centry name=\"product\"\u003eSystem Product Name\u003c/entry\u003e\n      \u003centry name=\"version\"\u003eSystem Version\u003c/entry\u003e\n      \u003centry name=\"serial\"\u003eSystem Serial Number\u003c/entry\u003e\n      \u003centry name=\"uuid\"\u003eUNIQUE_UUID\u003c/entry\u003e\n      \u003centry name=\"sku\"\u003eSKU\u003c/entry\u003e\n      \u003centry name=\"family\"\u003eTo be filled by O.E.M.\u003c/entry\u003e\n    \u003c/system\u003e\n    \u003cbaseBoard\u003e\n      \u003centry name=\"manufacturer\"\u003eASUSTeK COMPUTER INC.\u003c/entry\u003e\n      \u003centry name=\"product\"\u003ePRIME Z790-P WIFI\u003c/entry\u003e\n      \u003centry name=\"version\"\u003eRev 1.xx\u003c/entry\u003e\n      \u003centry name=\"serial\"\u003eUNIQUE_SERIAL_NUMBER\u003c/entry\u003e\n      \u003centry name=\"asset\"\u003eDefault string\u003c/entry\u003e\n    \u003c/baseBoard\u003e\n  \u003c/sysinfo\u003e\n```\n\nAcquiring the system values involves using `dmidecode`. Root privileges are required. An example invocation is `dmidecode -s bios-vendor`; the full translation to the XML above is:\n\n```\n\u003csysinfo type=\"smbios\"\u003e\n    \u003cbios\u003e\n      \u003centry name=\"vendor\"\u003edmidecode -s bios-vendor\u003c/entry\u003e\n      \u003centry name=\"version\"\u003edmidecode -s bios-vendor\u003c/entry\u003e\n      \u003centry name=\"date\"\u003edmidecode -s bios-release-date\u003c/entry\u003e\n      \u003centry name=\"release\"\u003edmidecode -s bios-version\u003c/entry\u003e\n    \u003c/bios\u003e\n    \u003csystem\u003e\n      \u003centry name=\"manufacturer\"\u003edmidecode -s system-manufacturer\u003c/entry\u003e\n      \u003centry name=\"product\"\u003edmidecode -s system-product-name\u003c/entry\u003e\n      \u003centry name=\"version\"\u003edmidecode -s system-version\u003c/entry\u003e\n      \u003centry name=\"serial\"\u003edmidecode -s system-serial-number\u003c/entry\u003e\n      \u003centry name=\"uuid\"\u003edmidecode -s system-uuid\u003c/entry\u003e\n      \u003centry name=\"sku\"\u003edmidecode -s system-sku-number\u003c/entry\u003e\n      \u003centry name=\"family\"\u003edmidecode -s system-family\u003c/entry\u003e\n    \u003c/system\u003e\n    \u003cbaseBoard\u003e\n      \u003centry name=\"manufacturer\"\u003edmidecode -s baseboard-manufacturer\u003c/entry\u003e\n      \u003centry name=\"product\"\u003edmidecode -s baseboard-product-name\u003c/entry\u003e\n      \u003centry name=\"version\"\u003edmidecode -s baseboard-version\u003c/entry\u003e\n      \u003centry name=\"serial\"\u003edmidecode -s baseboard-serial-number\u003c/entry\u003e\n      \u003centry name=\"asset\"\u003edmidecode -s baseboard-asset-tag\u003c/entry\u003e\n    \u003c/baseBoard\u003e\n  \u003c/sysinfo\u003e\n```\n\nLastly, by default Linux systems store the physical hardware clock as UTC. When dual-booting; this conflicts with the relative clock used by default in Windows or the virtual environment setup. This clock delta can cause the time to change\non the system in unhealthy ways during reboots; specifically voiding certificates and causing havoc with software licensing tools. To rectify this, modify the clock section of the virtual machine XML, to utilize UTC instead of localtime.\n\n```\n\u003cclock offset=\"utc\"\u003e\n    \u003ctimer name=\"rtc\" tickpolicy=\"catchup\"/\u003e\n    \u003ctimer name=\"pit\" tickpolicy=\"delay\"/\u003e\n    \u003ctimer name=\"hpet\" present=\"no\"/\u003e\n    \u003ctimer name=\"hypervclock\" present=\"yes\"/\u003e\n  \u003c/clock\u003e\n```\n\nIn addition, within the virtual machine edit the registry to add a new dword(`\"RealTimeIsUniversal\"=dword:00000001`) to `HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Control\\TimeZoneInformation`.\n\nPower-off and power-on the VM to verify the virtual machine is reporting the correct time. At this point one could natively boot into the operating system and use many hardware-locked license protected software offerings.\n\n\u003ch2 name=\"credits\"\u003e\n    Credits \u0026 Resources\n\u003c/h2\u003e\n\n- Docs\n    - ArchWiki\n        - [QEMU](https://wiki.archlinux.org/index.php/QEMU)\n        - [KVM](https://wiki.archlinux.org/index.php/KVM)\n        - [Libvirt](https://wiki.archlinux.org/index.php/Libvirt)\n        - [PCI Passthrough](https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF)\n        - [Kernel Parameters](https://wiki.archlinux.org/index.php/Kernel_parameters)\n    - Libvirt\n        - [VM Lifecycle](https://wiki.libvirt.org/page/VM_lifecycle)\n        - [Domain XML](https://libvirt.org/formatdomain.html)\n        - [Hooks](https://libvirt.org/hooks.html)\n        - [libvirtd](https://libvirt.org/manpages/libvirtd.html)\n        - [virsh](https://libvirt.org/manpages/virsh.html)\n        - [virtIO](https://wiki.libvirt.org/page/Virtio)\n        - [virtio-blk vs. virtio-scsi](https://mpolednik.github.io/2017/01/23/virtio-blk-vs-virtio-scsi/)\n    - Linux Kernel\n        - [KVM](https://www.kernel.org/doc/html/latest/virt/kvm/index.html)\n        - [VFIO](https://www.kernel.org/doc/html/latest/driver-api/vfio.html?highlight=vfio%20pci)\n- Tutorials\n    - Heiko Sieger - [Running Windows 10 on Linux using KVM with VGA Passthrough](https://heiko-sieger.info/running-windows-10-on-linux-using-kvm-with-vga-passthrough)\n    - Alex Williamson - VFIO GPU How To series\n        - [Part 1 - The hardware](https://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-1-hardware.html)\n        - [Part 2 - Expectations](https://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-2.html)\n        - [Part 3 - Host configuration](https://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-3-host.html)\n        - [Part 4 - Our first VM](https://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-4-our-first.html)\n        - [Part 5 - A VGA-mode, SeaBIOS VM](https://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-5-vga-mode.html)\n    - David Yates - [GPU passthrough: gaming on Windows on Linux](https://davidyat.es/2016/09/08/gpu-passthrough/)\n    - Wendell - [VFIO in 2019 – Pop!_OS How-To](https://forum.level1techs.com/t/vfio-in-2019-pop-os-how-to-general-guide-though-draft/142287)\n        - Wendell is from [Level1Techs](https://level1techs.com/). He has contributed to the FOSS community with a cool application called [Looking Glass](https://looking-glass.hostfission.com/). I recommend you check out this [video](https://www.youtube.com/watch?v=okMGtwfiXMo) for more information.\n        - Wendell has even collaborated with Linus from [Linus Tech Tips](https://www.youtube.com/user/LinusTechTips) and put out [this video](https://www.youtube.com/watch?v=SsgI1mkx6iw).\n    - Yuri Alek - [Single GPU passthrough](https://gitlab.com/YuriAlek/vfio)\n    - Jack Ford - [Ubuntu 18.04 - KVM/QEMU Windows 10 GPU Passthrough](https://blog.zerosector.io/2018/07/28/kvm-qemu-windows-10-gpu-passthrough/)\n    - Bsilvereagle - [Virtualizing Windows 7 (or Linux) on a NVMe drive with VFIO](https://frdmtoplay.com/virtualizing-windows-7-or-linux-on-a-nvme-drive-with-vfio/)\n    - Mathias Hauber\n        - [Windows virtual machine GPU passthrough Ubuntu](https://mathiashueber.com/windows-virtual-machine-gpu-passthrough-ubuntu/)\n        - [Performance tweaks gaming on virtual machines](https://mathiashueber.com/performance-tweaks-gaming-on-virtual-machines/)\n        - [Configuring Hugepages to use in a virtual machine](https://mathiashueber.com/configuring-hugepages-use-virtual-machine/)\n        - [QEMU Troubleshooting errors-gpu-passthrough-vm](https://mathiashueber.com/qemu-troubleshooting-errors-gpu-passthrough-vm/)\n    - Arseniy Shestakov - [How To: pass GPU to VM and back without X restart](https://arseniyshestakov.com/2016/03/31/how-to-pass-gpu-to-vm-and-back-without-x-restart/)\n    - Rokas Kupstys - [Performance of your gaming VM](https://rokups.github.io/#!pages/gaming-vm-performance.md)\n- Videos\n    - GrayWolfTech - [Play games in Windows on Linux! PCI passthrough quick guide](https://www.youtube.com/watch?v=dsDUtzMkxFk)\n    - Raven Repair Co. - [How to create a KVM gaming virtual machine in under 30 minutes!](https://www.youtube.com/watch?v=HXgQVAl4JB4\u0026list=PLoQO63DBLOOM1zf4Fm2HG3QhzlJK7PTAI)\n    - Level1Linux - [GPU Passthrough for Virtualization with Ryzen: Now Working](https://www.youtube.com/watch?v=aLeWg11ZBn0\u0026t=1595s)\n- Blogs\n    - The Passthrough Post\n        - [VFIO PC Builds](https://passthroughpo.st/vfio-increments/)\n        - [Howto: Libvirt Automation Using VFIO-Tools Hook Helper](https://passthroughpo.st/simple-per-vm-libvirt-hooks-with-the-vfio-tools-hook-helper/)\n        - [How to Apply the Error 43 Workaround](https://passthroughpo.st/apply-error-43-workaround/)\n    - Heiko Sieger\n        - [Glossary of Virtualization Terms](https://heiko-sieger.info/glossary-of-virtualization-terms/)\n        - [IOMMU Groups – What You Need to Consider](https://heiko-sieger.info/iommu-groups-what-you-need-to-consider/)\n        - [Virtualization Hardware Accessories](https://heiko-sieger.info/virtualization-hardware-accessories/)\n        - [Windows 10 Virtual Machine Benchmarks](https://heiko-sieger.info/windows-10-virtual-machine-benchmarks/)\n        - [Windows 10 Benchmarks (Virtual Machine)](https://heiko-sieger.info/benchmarks/)\n- Lectures\n    - Alex Williamson - Red Hat\n        - [An Introduction to PCI Device Assignment with VFIO](https://www.youtube.com/watch?v=WFkdTFTOTpA\u0026feature=emb_title)\n        - [VFIO Device Assignment Quirks, How to use Them and How to Avoid Them](https://www.youtube.com/watch?v=A9rV2_3yIOk\u0026t=790s)\n    - Martin Polednik - Red Hat\n        - [Helping Users Maximize VM Performance](https://www.youtube.com/watch?v=_SlUlQRcnQg)\n    - Neo Jia \u0026 Kirti Wankhede - NVIDIA\n        - [vGPU on KVM - A VFIO Based Framework](https://www.youtube.com/watch?v=Xs0TJU_sIPc\u0026t=164s)\n- Communities\n    - [Reddit /r/vfio](https://www.reddit.com/r/VFIO/wiki/index)\n    - [Level1Techs](https://forum.level1techs.com/)\n    - [Red Hat vfio-users](https://www.redhat.com/archives/vfio-users/index.html)\n    - [KVM Forum](https://www.youtube.com/channel/UCRCSQmAOh7yzgheq-emy1xA)\n\n\u003ch2 name=\"footnotes\"\u003e\n    Footnotes\n\u003c/h2\u003e\n\n\u003col\u003e\n    \u003cli name=\"footnote1\"\u003e\n        Check out \u003ca href=\"https://news.ycombinator.com/item?id=18328323\"\u003ethis thread\u003c/a\u003e from Hacker News for more information.\n        \u003ca href=\"#return1\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote2\"\u003e\n        I'll be using the term \u003ci\u003eiGPU\u003c/i\u003e to refer to Intel's line of integrated GPUs that usually come built into their processors, and the term \u003ci\u003edGPU\u003c/i\u003e to refer to dedicated GPUs which are much better performance-wise and meant for gaming or video editing (NVIDIA/AMD).\n        \u003ca href=\"#return2\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote3\"\u003e\n        Make sure that the monitor input used for your gaming VM supports FreeSync/G-Sync technology. In my case, I reserved the displayport 1.2 input for my gaming VM since G-Sync is not supported across HDMI (which was instead used for host graphics).\n        \u003ca href=\"#return3\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote4\"\u003e\n        I specifically wanted my Linux host to be able to perform \u003ca href=\"https://developer.nvidia.com/cuda-downloads\"\u003eCUDA\u003c/a\u003e work on the attached NVIDIA GPU. Just because my graphics card wasn't attached to a display didn't stop me from wanting to use \u003ca href=\"https://developer.nvidia.com/cudnn\"\u003ecuDNN\u003c/a\u003e for ML/AI applications.\n        \u003ca href=\"#return4\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote5\"\u003e\n        Applying the ACS Override Patch \u003cb\u003emay compromise system security\u003c/b\u003e. Check out \u003ca href=\"https://www.reddit.com/r/VFIO/comments/bvif8d/official_reason_why_acs_override_patch_is_not_in/\"\u003ethis post\u003c/a\u003e to see why the ACS patch will probably never make its way upstream to the mainline kernel.\n        \u003ca href=\"#return5\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote6\"\u003e\n        I'm actually being a bit disingenuous here... I deliberately purchased hardware that I knew would provide ACS implementation (and hence good IOMMU isolation). After flashing the most recent version of my motherboard's BIOS, I made sure to enable the following features under the \"AMD CBS\" menu: \u003ccode\u003eACS Enable\u003c/code\u003e, \u003ccode\u003eAER CAP\u003c/code\u003e, \u003ccode\u003eARI Support\u003c/code\u003e.\n        \u003ca href=\"#return6\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote7\"\u003e\n        AMD CPUs/motherboards/chipsets tend to provide better ACS support than their Intel counterparts. The Intel Xeon family of processors is a notable exception. Xeon is mainly targeted at non-consumer workstations and thus are an excellent choice for PCI/VGA passthrough. Be aware that they do demand a hefty price tag.\n        \u003ca href=\"#return7\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote8\"\u003e\n        Credit to the solution presented in \u003ca href=\"https://forum.level1techs.com/t/how-to-apply-acs-override-patch-kubuntu-18-10-kernel-4-18-16/134204\"\u003ethis post\u003c/a\u003e.\n        \u003ca href=\"#return8\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote9\"\u003e\n        If you decide to use bash scripts to launch your VM, I've included a file in the repository called \u003ccode\u003eqemu.sh\u003c/code\u003e. Make sure to fill out the \u003ccode\u003e#TODO\u003c/code\u003e section of the code with your custom version of the command \u003ccode\u003eqemu-system-x86-64\u003c/code\u003e.\n        \u003ca href=\"#return9\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote10\"\u003e\n        See \u003ca href=\"https://www.stratoscale.com/blog/compute/using-bare-qemu-kvm-vs-libvirt-virt-install-virt-manager/\"\u003ethis link\u003c/a\u003e for more details and a comparison between QEMU and virt-manager.\n        \u003ca href=\"#return10\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote11\"\u003e\n        See \u003ca href=\"https://heiko-sieger.info/running-windows-10-on-linux-using-kvm-with-vga-passthrough/#About_keyboard_and_mouse\"\u003ethis link\u003c/a\u003e and \u003ca href=\"https://github.com/debauchee/barrier\"\u003e this\u003c/a\u003e for software/hardware solutions that share your keyboard and mouse across your host and guest.\n        \u003ca href=\"#return11\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote12\"\u003e\n        For more information on hugepages, refer to \u003ca href=\"https://help.ubuntu.com/community/KVM%20-%20Using%20Hugepages\"\u003ethis link.\n        \u003ca href=\"#return12\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote13\"\u003e\n        Credit to the comment from /u/tholin in \u003ca href=\"https://www.reddit.com/r/VFIO/comments/dmie86/setting_up_hugepages/\"\u003ethis post\u003c/a\u003e.\n        \u003ca href=\"#return13\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote14\"\u003e\n        Credit to Mathias Hueber in \u003ca href=\"https://mathiashueber.com/performance-tweaks-gaming-on-virtual-machines/\"\u003ethis post\u003c/a\u003e.\n        \u003ca href=\"#return14\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote15\"\u003e\n        See a similar discussion here from Rokas Kupstys in \u003ca href=\"https://rokups.github.io/#!pages/gaming-vm-performance.md\"\u003ethis post.\n        \u003ca href=\"#return15\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote16\"\u003e\n        If you're curious about the best CPU pinning strategy for optimizing the latency vs. performance tradeoff, I recommend you check out \u003ca href=\"https://www.redhat.com/archives/vfio-users/2017-February/msg00010.html\"\u003ethis discussion\u003c/a\u003e.\n        \u003ca href=\"#return16\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote17\"\u003e\n        Although the overall performance between virtio-blk and virtio-scsi is similar, passing a single virtio-scsi controller can handle a multitude of PCI devices, whereas virtio-blk exposes one PCI device per controller. \u003ca href=\"https://www.reddit.com/r/redhat/comments/6cuydw/in_rhv_what_is_the_difference_between_virtio_and/\"\u003eThis comment\u003c/a\u003e on Reddit from a RedHat employee provides some good context and resources.\n        \u003ca href=\"#return17\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli name=\"footnote18\"\u003e\n        For the sake of fairness, I chose to passthrough all 12-cores/24 threads to the KVM. That way, the bare-metal installation won't have an unfair advantage over the KVM when it comes to multi-core processes. Unfortunately, I couldn't passthrough all 32GB of RAM to the KVM since the host naturally reserves some of its own. In order to mitigate this as much as possible, I passed the remaining 29GB of RAM to the KVM. Due to its nature, a surplus of RAM doesn't really improve performance so much as it prevents bottlenecking.\n        \u003ca href=\"#return18\"\u003e\u003csup\u003e\u0026#x21ba;\u003c/sup\u003e\u003c/a\u003e\n    \u003c/li\u003e\n\u003c/ol\u003e\n","funding_links":[],"categories":["Shell","others","HarmonyOS"],"sub_categories":["Windows Manager"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbryansteiner%2Fgpu-passthrough-tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbryansteiner%2Fgpu-passthrough-tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbryansteiner%2Fgpu-passthrough-tutorial/lists"}