https://github.com/portellam/guest-machine-guide
Guide for setup of a Virtual Machine (VM) on the Linux desktop. Includes general overview, optimizations for Windows VMs and hardware-passthrough (VFIO), and more.
https://github.com/portellam/guest-machine-guide
documentation guest guide hugepages hyperv kvm libvirt passthrough pci-passthrough qemu vfio vfio-passthrough vfio-pci vfio-setup virtual-machine virtualization
Last synced: about 2 months ago
JSON representation
Guide for setup of a Virtual Machine (VM) on the Linux desktop. Includes general overview, optimizations for Windows VMs and hardware-passthrough (VFIO), and more.
- Host: GitHub
- URL: https://github.com/portellam/guest-machine-guide
- Owner: portellam
- License: gpl-3.0
- Created: 2024-04-17T19:26:09.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2024-06-23T04:25:35.000Z (10 months ago)
- Last Synced: 2025-01-11T23:42:41.021Z (4 months ago)
- Topics: documentation, guest, guide, hugepages, hyperv, kvm, libvirt, passthrough, pci-passthrough, qemu, vfio, vfio-passthrough, vfio-pci, vfio-setup, virtual-machine, virtualization
- Homepage:
- Size: 86.9 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Guest Machine Guide
### Draft Rev. 1
Guide for setup of a guest Libvirt/QEMU Virtual Machine (VM). Includes general
overview and references, and optimizations for Windows guests and
hardware-passthrough (VFIO).
## Status: Work-In-Progress
## Table of Contents
- [1. Why?](#1-why)
- [2. Related Projects](#2-related-projects)
- [3. Documentation](#3-documentation)
- [4. Host Optimizations](#4-host-optimizations)
- [5. Guest Optimizations](#5-guest-optimizations)
- [6. Guest XML Layout](#6-guest-xml-layout)
- [6.1. Syntax](#61-syntax)
- [6.2. First Lines in XML](#62-first-lines-in-xml)
- [6.3. Memory](#63-memory)
- [6.4. CPU Topology (1 / 2)](#64-cpu-topology-1--2)
- [6.5. System Information Spoofing](#65-system-information-spoofing)
- [6.6. Features](#66-features)
- [6.7. CPU Topology (2 / 2)](#67-cpu-topology-2--2)
- [6.8. Power Management](#68-power-management)
- [6.9. Devices](#69-devices)
- [6.10. QEMU Command Line](#610-qemu-command-line)
- [6.11. QEMU Overrides](#611-qemu-overrides)
- [7. Benchmarking Guest Performance](#7-benchmarking-guest-performance)
- [8. Contact](#8-contact)
- [9. References](#9-references)
## Contents
### 1. Why?
The purpose of this document is to inform a new or returning user how to
optimize a Guest machine, without demanding greater research and time.
This document does not serve to replace existing knowledge-bases. If you have
any unexpected questions, wish to fact-check, or want to expand your knowledge,
then please visit these places!
Copy and paste what you need from here and/or any example XML files, to your
Guest XML file.
### 2. Related Projects
| Project | Codeberg | GitHub |
| :--- | :---: | :---: |
| Deploy VFIO | [link][codeberg1] | [link][github1] |
| Auto X.Org | [link][codeberg2] | [link][github2] |
| Generate Evdev | [link][codeberg3] | [link][github3] |
| **Guest Machine Guide** | [link][codeberg4] | [link][github4] |
| Libvirt Hooks | [link][codeberg5] | [link][github5] |
| Power State Virtual Machine Manager | [link][codeberg6] | [link][github6] |
[codeberg1]: https://codeberg.org/portellam/deploy-VFIO
[github1]: https://github.com/portellam/deploy-VFIO
[codeberg2]: https://codeberg.org/portellam/auto-xorg
[github2]: https://github.com/portellam/auto-xorg
[codeberg3]: https://codeberg.org/portellam/generate-evdev
[github3]: https://github.com/portellam/generate-evdev
[codeberg4]: https://codeberg.org/portellam/guest-machine-guide
[github4]: https://github.com/portellam/guest-machine-guide
[codeberg5]: https://codeberg.org/portellam/libvirt-hooks
[github5]: https://github.com/portellam/libvirt-hooks
[codeberg6]: https://codeberg.org/portellam/powerstate-virtmanager
[github6]: https://github.com/portellam/powerstate-virtmanager
### 3. Documentation
- [What is VFIO?](#12)
- [VFIO Discussion and Support](#13)
- [Hardware-Passthrough Guide](#9)
- [Virtual Machine XML Format Guide](#16)
### 4. Host Optimizations
TODO: add here.
### 5. Guest Optimizations
TODO: add here.
### 6. Guest XML Layout
Below is an *incomplete* layout for building a Guest machine. The lines include
additional features, of which are absent when creating a Guest XML (with the
`virsh` CLI command or `virt-manager` GUI application).
#### 6.1. Syntax
```xml
child_tag_value
```
#### 6.2. First Lines in XML
| `` Tag | Attribute | Value | Description |
| ------------------ | ------------ | ---------------------------------------------- | -------------------------------------------------- |
| | `xmlns:qemu` | `"http://libvirt.org/schemas/domain/qemu/1.0"` | Enable QEMU [command lines](#qemu-command-line) and [overrides](#qemu-overrides). |
| | `type` | `"kvm"` | Enable QEMU [command lines](#qemu-command-line) and [overrides](#qemu-overrides). |
| ``[1](#21a-name-best-practice) | none | text | Name of the Guest. |
##### 2.a. `` Best practice:
**Note:** The following formatting examples are a personal preference of the
Author.
**Format:** `purpose`**_**`vendor`\***_**`operating system`**_**`architecture`**_**`chipset`**_**`firmware`**_**`topology`
**\*** Optional, if Host machine contains two (2) or more video devices
(GPU/VGA).
- Example systems and names:
- Modern gaming machine: `game_nvidia_win10_x64_q35_uefi_6c12t`
- Older 2000s gaming machine: `retro_amd_winxp_x86_i440fx_bios_2c4t`
- Retro 1990s gaming machine: `retro_3dfx_win98_x86_i440fx_bios_1c1t`
- Intel MacOS workstation: `work_macos_amd_x64_q35_uefi_6c12t`
- Purpose of the Guest (and suggested names):
- Gaming PC: `game`
- Legacy/Retro PC: `retro`
- Server: `server`
- Workstation PC: `work`
- etc.
- Vendor name of the Video device:
- AMD: `amd`
- Intel: `intel`
- NVIDIA: `nvidia`
- emulated: `virtgpu`
- non-mainstream or legacy:
- 3DFX: `3dfx`.
- Short name of the Operating System (OS):
- Apple Macintosh: `macos`
- Linux: `arch`, `debian`, `redhat`, `ubuntu`
- Microsoft Windows: `win98`, `winxp`, `win10`
- etc.
- Short name of the CPU architecture:
- AMD/Intel 32-bit: [`x86`](#15)
- AMD/Intel 64-bit: [`x64`](#14)
- ARM 32-bit: [`aarch32`](#2)
- ARM 64-bit: [`aarch64`](#3)
- etc.
- Virtualized chipset:
- I440FX: [`i440fx`](#6)
- Emulated, older chipset by Intel.
- Does support legacy Guests (example: Windows NT 5 and before, XP/2000, 9x).
- PCI bus only; expansion devices will exist on a PCI topology.
- Will accept PCIe devices.
- Q35: [`q35`](#10)
- Virtual, newer platform.
- Native PCIe bus; expansion devices will exist on a PCIe topology.
- Does not support legacy Guests.
- Firmware:
- BIOS: [`bios`](#4)
- UEFI: [`uefi`](#11)
- Short-hand of Core topology: `4c8t`
- Given the amount of physical cores (example: 4).
- Given the amount of logical threads per core (2 * 4 = 8).
#### 6.3. Memory
To gather information of system memory, execute: `free --kibi --total --wide`
For more information on this topic, click [here](#1).
| `` Tag | Attribute | Value | Description |
| -------------- | --------- | -------- | ----------------------------------------------------------------- |
| | parent | a number | The total amount of memory. |
| | `unit` | `"KiB"` | Amount of memory in Kibibytes (1024 bytes, Kilo is 1000 bytes). |
| `` Tag | Attribute | Value | Description |
| --------------------- | --------- | -------- | ----------------------------------------------------------------- |
| | parent | a number | The current amount of memory. |
| | `unit` | `"KiB"` | Amount of memory in Kibibytes (1024 bytes, Kilo is 1000 bytes). |
| `` Tag | Attribute | Value | Description |
| --------------------- | --------- | ----------- | ----------------------------------------------------------------- |
| `` | `mode` | `immediate` | Specifies how memory allocation is performed. |
| `` | none | none | TODO: define, what is this? |
| [``](#5) | none | none | Enable Huge memory pages. |
| `` | none | none | Prevents the Host from merging the same memory used among Guests. |
##### 3.a. ``
- Static allocation of *Host* memory pages into *Guest* memory pages.
- **Huge:** Memory page size greater than 4K bytes (2M or 1G bytes). The greater
the size, the lower the Host overhead.
- Dynamic *Host* memory page allocation is more flexible, but will require
defragmentation before use as *Guest* memory pages (before a Guest machine may start).
- **Warning:** If the specified *Guest* memory pages exceeds the allocated
*Host* memory pages, then the Guest machine will fail to start.
For more information on this topic, click [here](#5).
#### 6.4. CPU Topology (1 / 2)
To gather information about your CPU, execute: `lscpu | grep --extended-regexp --ignore-case "per core|per socket|socket"`
| `` Tag | Attribute | Value | Description |
| ------------ | ----------- | ---------- | --------------------------------------------------------------- |
| | parent | a number | Specify number of Host threads for Guest (same as `` tag). |
| | `placement` | `"static"` | Static allocation of Guest CPU threads. |
| `` Tag | Attribute | Value | Description |
| ----------------- | --------- | ----------- | -------------------------------------------------------------------------------------------- |
| | none | a number | Specify number of Host threads to manage storage block devices (see as `` tag). |
| `` Tag | Attribute | Value | Description |
| --------------- | ---------- | ----------- | -------------------------------------------- |
| `` | `vcpu` | a number | Guest CPU: the Guest thread ID number.[1](#1-vcpupin-vcpu) |
| `` | `cpuset` | a number | Guest CPU: the Host thread ID number.[2](#2-vcpupin-cpuset) |
| `` | `cpuset` | a number | Guest IRQ: the Guest IRQ thread ID number.[3](#3-emulatorpin-cpuset) |
| `` | `iothread` | a number | Guest IO: the Guest IO thread ID number. |
| `` | `cpuset` | a number | Guest IO: the Host thread ID numbers.[4](#4-iothreadpin-cpuset) |
##### 4.a. ``
- Count does not exceed value as defined in ``.
##### 4.b. ``
- Threads should not overlap Host process threads.
##### 4.c. ``
- Emulator threads handle Interrupt Requests for Guest hardware emulation.
- Threads should not overlap Guest CPU threads as defined in `vcpupin cpuset`.
##### 4.d. ``
- IO threads handle IO processes for Guest virtual drives/disks.
- Threads should not overlap Guest CPU threads as defined in `vcpupin cpuset`.
##### 4.e. Example XML:
```xml
4
1
```
#### 6.5. System Information Spoofing
To gather information about your BIOS, execute:
`sudo dmidecode --type bios | grep --extended-regexp --ignore-case "vendor|version|release date"`
To gather information about your system, execute:
`sudo dmidecode --type system | grep --extended-regexp --ignore-case "manufacturer|product name|version|serial number|sku number|family"`
```xml
American Megatrends Inc.
version_of_bios_firmware
MM/DD/YYYY
vendor_of_motherboard
product_name
Default string
Default string
Default string
Default string
```
#### 6.6. Features
TODO: make the following inline XML into chart, describe each feature.
```xml
```
#### 6.7. CPU Topology (2 / 2)
To gather information about your CPU, execute: `lscpu | grep --extended-regexp --ignore-case "per core|per socket|socket"`
##### 7.a. Example output:
```
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
```
TODO: make the following inline XML into chart, describe each feature.
```xml
```
| `` Tag | Attribute | Value | Description |
| ------------ | ------------ | -------------------- | -------------------------------------------------------------- |
| parent | `mode` | `"host-passthrough"` | Spoof the Guest CPU info, with the actual Host CPU info. |
| | `check` | `"none"` | TODO: add here. |
| | `migratable` | `"on"` | TODO: add here. |
| `` | `sockets` | a number | The number of CPU sockets, or maxmium number of dies. |
| | `dies` | a number | The number of CPU dies (typically one). |
| | `cores` | a number | The number of CPU cores. |
| | `threads` | a number | The number of CPU threads per core (typically a factor of two). |
| `` | `mode` | `"passthrough"` | Passthrough the Host CPU cache. |
| `` | `policy` | `"disable"` | Disables policy. |
| | `name` | `"hypervisor"` | TODO: add here. |
| `` | `policy` | `"disable"` | Disables policy. |
| | `present` | `"yes"` | TODO: add here. |
| `` | `name` | `"tsc"` | TODO: add here. |
| | `present` | `"yes"` | TODO: add here. |
| | `mode` | `"native"` | TODO: add here. |
#### 6.8. Power Management
```xml
```
#### 6.9. Devices
For more information on this topic, click [here](#8).
TODO: make the following inline XML into chart, describe each feature???
```xml
...
```
##### 9.a. Emulated Devices
TODO: make this inline XML?
- QEMU
- RedHat
- Other legacy devices
- SoftGPU https://github.com/JHRobotics/softgpu
lorem ipsum
##### 9.b. Real/Passthrough Hardware Devices
TODO: make this inline XML?
- AMD
- Integrated
- Dedicated
- Reset bug
- Intel
- Integrated
- Dedicated
- **Note:** Not much research done regarding this topic. Please refer to any mentioned guides, the Reddit forum, or use an Internet search engine with the keywords `intel gpu vfio`.
- NVIDIA
- Problems:
- Boot bug (Solution: Use known-good copy of given GPU's video BIOS or VBIOS).
- TODO: add references, programs used, instructions, and XML here.
##### 9.c. Memory Devices
TODO: make this inline XML?
- Looking Glass
- Scream
- ???
#### 6.10. QEMU Command Line
TODO: make the following inline XML into chart, describe each feature.
- Evdev
```xml
...
```
#### 6.11. QEMU Overrides
TODO: make the following inline XML into chart, describe each feature.
```xml
...
```
### 7. Benchmarking Guest Performance
TODO: add here.
### 8. Contact
Do you need help? Please visit the **Issues page** ([Codeberg][codeberg-issues],
[GitHub][github-issues]).
[codeberg-issues]: https://codeberg.org/portellam/auto-xorg/issues
[github-issues]: https://github.com/portellam/auto-xorg/issues
### 9. References
#### 1.
**8.2. Memory Tuning on Virtual Machines Red Hat Enterprise Linux 7**. Red Hat
Customer Portal. Accessed June 15, 2024.
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/sect-virtualization_tuning_optimization_guide-memory-tuning.
#### 2.
**AARCH64**. Wikipedia, May 30, 2024.
https://en.wikipedia.org/wiki/AArch64.
#### 3.
**ARM Architecture Family**. Wikipedia, May 31, 2024.
https://en.wikipedia.org/wiki/ARM_architecture_family#AArch32.
#### 4.
**BIOS**. Wikipedia, June 7, 2024.
https://en.wikipedia.org/wiki/BIOS.
#### 5.
**Hugepages**. Hugepages - Debian Wiki. Accessed June 15, 2024.
https://wiki.debian.org/Hugepages.
#### 6.
**I440fx PC (PC-I440fx, PC)**. i440fx PC (pc-i440fx, pc) - QEMU documentation.
Accessed June 15, 2024.
https://www.qemu.org/docs/master/system/i386/pc.html.
#### 7.
**Intel 440FX**. Wikipedia, August 20, 2023.
https://en.wikipedia.org/wiki/Intel_440FX.
#### 8.
**PCI Overview**. QEMU Wiki. Accessed June 15, 2024.
https://wiki.qemu.org/images/f/f6/PCIvsPCIe.pdf.
#### 9.
**PCI passthrough via OVMF**. ArchWiki. Accessed June 14, 2024.
https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF.
#### 10.
**Q35**. QEMU Wiki. Accessed June 15, 2024.
https://wiki.qemu.org/images/4/4e/Q35.pdf.
#### 11.
**UEFI**. Wikipedia, June 7, 2024.
https://en.wikipedia.org/wiki/UEFI.
#### 12.
**VFIO - ‘Virtual Function I/O’ - The Linux Kernel Documentation**.
The linux kernel. Accessed June 14, 2024.
https://www.kernel.org/doc/html/latest/driver-api/vfio.html.
#### 13.
**VFIO Discussion and Support**. Reddit. Accessed June 14, 2024.
https://www.reddit.com/r/VFIO/.
#### 14.
**X64**. Wikipedia, July 27, 2023.
https://en.wikipedia.org/wiki/X64.
#### 15.
**X86**. Wikipedia, May 18, 2024.
https://en.wikipedia.org/wiki/X86.
#### 16.
**XML Design Format**. GitHub - libvirt/libvirt. Accessed June 18, 2024.
https://github.com/libvirt/libvirt/blob/master/docs/formatdomain.rst.
## TODO:
- [x] add About.
- [ ] add Documentation for:
- [x] guest XML layout
- [x] name syntax
- [x] introduction
- [ ] memory
- [ ] cpu topology (1/2)
- [ ] system information spoofing
- [ ] features
- [ ] cpu topology (2/2)
- [ ] power management
- [ ] qemu command line
- [ ] evdev macro
- [ ] gpu stuff?
- [ ] qemu overrides
- [ ] host optimizations
- [ ] guest optimizations
- [ ] benchmarking performance
- [x] add References.
- [ ] use collapsable dropdowns (https://gist.github.com/pierrejoubert73/902cc94d79424356a8d20be2b382e1ab).
- [ ] add Guest XML file with all lines referenced in "Guest XML Layout".
- [ ] add pictures of virt-manager.
- [ ] flesh out "Guest XML Layout".
- [ ] how to setup virtio disks (virtio, virtio-scsi).