{"id":23738058,"url":"https://github.com/csmart/virt-infra-ansible","last_synced_at":"2025-09-04T13:32:19.994Z","repository":{"id":46408119,"uuid":"241861300","full_name":"csmart/virt-infra-ansible","owner":"csmart","description":"An example Ansible playbook for my virt-infra Ansible role.","archived":false,"fork":false,"pushed_at":"2022-06-20T08:20:12.000Z","size":269,"stargazers_count":45,"open_issues_count":0,"forks_count":32,"subscribers_count":5,"default_branch":"master","last_synced_at":"2023-04-13T11:10:35.460Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://github.com/csmart/ansible-role-virt-infra","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/csmart.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-20T10:56:33.000Z","updated_at":"2023-04-13T11:10:35.460Z","dependencies_parsed_at":"2022-07-22T13:19:17.041Z","dependency_job_id":null,"html_url":"https://github.com/csmart/virt-infra-ansible","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csmart%2Fvirt-infra-ansible","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csmart%2Fvirt-infra-ansible/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csmart%2Fvirt-infra-ansible/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csmart%2Fvirt-infra-ansible/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/csmart","download_url":"https://codeload.github.com/csmart/virt-infra-ansible/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231965415,"owners_count":18453071,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-31T08:52:29.537Z","updated_at":"2024-12-31T08:52:30.133Z","avatar_url":"https://github.com/csmart.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- vim-markdown-toc GFM --\u003e\n\n* [Manage KVM networks and guests with Ansible](#manage-kvm-networks-and-guests-with-ansible)\n* [Too long; didn't read](#too-long-didnt-read)\n* [Too long; but gonna read anyway](#too-long-but-gonna-read-anyway)\n\t* [Requirements](#requirements)\n\t\t* [Get the code](#get-the-code)\n\t\t* [Ansible](#ansible)\n\t\t* [KVM](#kvm)\n\t\t* [Other tools](#other-tools)\n\t\t* [SSH keys](#ssh-keys)\n\t\t* [Guest image](#guest-image)\n\t\t* [Setup KVM host](#setup-kvm-host)\n\t\t\t* [Fedora](#fedora)\n\t\t\t* [CentOS 7](#centos-7)\n\t\t\t* [CentOS 8](#centos-8)\n\t\t\t* [Debian](#debian)\n\t\t\t* [Ubuntu](#ubuntu)\n\t\t\t* [openSUSE](#opensuse)\n\t\t\t* [Using routed networks](#using-routed-networks)\n\t\t\t* [Configuring bridges with NetworkManager](#configuring-bridges-with-networkmanager)\n\t\t\t\t* [Linux bridge](#linux-bridge)\n\t\t\t\t* [Open vSwitch (OVS) bridge](#open-vswitch-ovs-bridge)\n\t* [Inventory](#inventory)\n\t\t* [Defaults](#defaults)\n\t\t* [KVM host](#kvm-host)\n\t\t\t* [Multiple KVM hosts](#multiple-kvm-hosts)\n\t\t\t* [libvirt networks](#libvirt-networks)\n\t\t\t\t* [Using ovs-bridge in inventory](#using-ovs-bridge-in-inventory)\n\t\t* [Guests](#guests)\n\t\t\t* [Running custom shell commands on guest disk](#running-custom-shell-commands-on-guest-disk)\n\t\t\t* [Using Linux bridge in inventory](#using-linux-bridge-in-inventory)\n\t\t\t* [Using ovs-bridge in inventory](#using-ovs-bridge-in-inventory-1)\n\t\t\t* [Static network configuration with cloud-init](#static-network-configuration-with-cloud-init)\n\t\t\t\t* [Network device names and MAC addresses](#network-device-names-and-mac-addresses)\n\t* [Cloud images](#cloud-images)\n\t* [Running the playbook](#running-the-playbook)\n\t\t* [Cleanup](#cleanup)\n\t* [Post setup configuration](#post-setup-configuration)\n\n\u003c!-- vim-markdown-toc --\u003e\n\n# Manage KVM networks and guests with Ansible\n\nThis is an example Ansible playbook for my [Virtual Infrastructure Ansible\nrole](https://github.com/csmart/ansible-role-virt-infra).\n\nIt uses separate YAML Ansible [inventory files](#inventory) to define and\nmanage networks and guests on KVM hosts. Ansible's `--limit` option lets you\nmanage them individually or as a group.\n\nIt is really designed for dev work, where the KVM host is your local machine,\nyou have `sudo` and talk to `libvirtd` at `qemu:///system` however it also\nworks on remote hosts.\n\nTo test this out, maybe spin up a supported distro as a guest on a host that\nsupports nested virtualisation (CPU passthrough).\n\n\u003cimg src=\"virt-infra-ansible.png\" alt=\"Virtual Infrastructure with Ansible\"\u003e\n\nAn [SVG demo is included](demo.svg), if you want to see it in action.\n\n# Too long; didn't read\n\nSpin up three CentOS 8 guests from _simple_ Ansible hostgroup on localhost,\nusing defaults.\n\n```bash\ncurl -O https://cloud.centos.org/centos/8-stream/x86_64/images/CentOS-Stream-GenericCloud-8-20220125.1.x86_64.qcow2\nsudo mkdir -p /var/lib/libvirt/images\nsudo mv -iv CentOS-Stream-GenericCloud-8-20220125.1.x86_64.qcow2 /var/lib/libvirt/images/\n\ngit clone --recursive https://github.com/csmart/virt-infra-ansible.git\ncd virt-infra-ansible\n\n./run.sh --limit kvmhost,simple\n```\n\n# Too long; but gonna read anyway\n\nSetting guest states to _running_, _shutdown_, _destroyed_ or _undefined_ (to\ndelete and clean up) are supported.\n\nYou can set whatever memory, CPU, disks and network cards you want for your\nguests, either via hostgroups or individually. A mixture of multiple disks is\nsupported, including _scsi_, _sata_, _virtio_ and even _nvme_ (on supported\ndistros).\n\nYou can create private NAT libvirt networks on the KVM host and then put VMs on\nany number of them. Guests can use those libvirt networks or _existing_ Linux\nbridge devices (e.g. `br0`) and Open vSwitch (OVS) bridge on the KVM host (this\nwon't create bridges on the host, but it will check that the bridge interface\nexists). You can specify the model of network card as well as the MAC for each\ninterface if you require, however it defaults to an idempotent address based on\nthe hostname.\n\nYou can also create routed libvirt networks on the KVM host and then put VMs on\nany number of them. In this case, a new bridge is created with the name you\nspecify (e.g. `br1`), wired to an _existing_ interface (e.g. `eth0`). You can\nspecify the MAC for each interface if you require.\n\nThis supports various distros and uses their qcow2 [cloud\nimages](#guest-cloud-images) for convenience (although you could use your own\nimages). I've tested CentOS Stream, Debian, Fedora, openSUSE, RHEL and Ubuntu.\n\nFor RHEL, set the `virt_infra_sm_creds` variable (perhaps from a vault) in\norder to temporarily register the machine to Red Hat portal when preparing the\ndisk. This follows the [format from\nvirt-builder](https://libguestfs.org/virt-builder.1.html#subscription-manager),\nsuch as:\n\n```yaml\n- virt_infra_sm_creds: MYUSER:password:MYPASSWORD\n```\n\nThe qcow2 cloud base images to use for guests are specified as variables in the\ninventory and should exist under libvirt images directory (default is\n`/var/lib/libvirt/images/`). That is to say, this won't download the images for\nyou automatically.\n\nGuest qcow2 boot images are created from those base images. By default these\nuse the cloud image as a backing file, however it also supports cloning\ninstead. You can create additional disks as you like. You can also choose to\nkeep any disk image rather than deleting it when a VM is undefined. The\ncloud-init ISOs are created automatically and attached to the guest to\nconfigure it on boot.\n\nThe timezone will be set to match the KVM host by default and the ansible user\nwill be used for the guest, along with your public SSH keys on the KVM host\n(you can override that). Host entries are added to `/etc/hosts` on the KVM host\nand it also modifies ansible user's SSH config and adds the fingerprint to\n`known_hosts` so that you can SSH straight in (which it tests as part of the\n\n## Requirements\n\nAll that's really needed is a Linux host capable of running KVM with `git`\ninstalled, some guest images and a basic inventory. The Ansible will do the\nrest (on supported distros).\n\nFor supported distros, the `run.sh` script will install Ansible if it is not\nfound. The plays will also install KVM, libvirtd and other required packages\nand also make sure the libvirtd is running.\n\n### Get the code\n\nOnce you've installed `git` on your Linux box, _recursively_ clone this repo\nwhich will pull in the role.\n\n```bash\ngit clone --recursive https://github.com/csmart/virt-infra-ansible.git\ncd virt-infra-ansible\n```\n\n### Ansible\n\nYou'll probably need Ansible and Jinja \u003e= 2.8 because this uses multiple\ninventory files and things like 'equalto' comparisons.\n\n### KVM\n\nA working x86_64 KVM host where the user running Ansible can communicate with\nlibvirtd via sudo. I have tested this on CentOS 8, Fedora 31+, Debian 10, Ubuntu\nBionic/Eoan and openSUSE 15 hosts, but other Linux machines probably work.\n\nIt expects hardware support for KVM in the CPU so that we an create accelerated\nguests and pass the CPU through (supports nested virtualisation).\n\n### Other tools\n\nSeveral user space tools are also required on the KVM host.\n\n* qemu-img\n* osinfo-query\n* virsh\n* virt-customize\n* virt-sysprep\n\n### SSH keys\n\nAt least one SSH key pair on your KVM host (run `ssh-keygen`).\n\n### Guest image\n\nDownload the guest images you want to use ([this is what I\ndownloaded](#cloud-images)) and put them in libvirt images path (usually\n/var/lib/libvirt/images). This will check that the images you specified exist\nand error if they are not found.\n\n### Setup KVM host\n\nThis is how you can manually configure your KVM host (this is all done\nautomatically on supported distros).\n\n#### Fedora\n\n```bash\n# Create SSH key if you don't have one\nssh-keygen\n\n# libvirtd\nsudo dnf install -y @virtualization\nsudo systemctl enable --now libvirtd\n\n# Ansible\nsudo dnf install -y ansible\n\n# Other deps\nsudo dnf install -y \\\ngit \\\ngenisoimage \\\nlibguestfs-tools-c \\\nlibosinfo \\\npython3-libvirt \\\npython3-lxml \\\nqemu-img \\\nvirt-install\n```\n\n#### CentOS 7\n\nCentOS 7 won't work until we have `libselinux-python3` package, which is coming in 7.8...\n\n * https://bugzilla.redhat.com/show_bug.cgi?id=1719978\n * https://bugzilla.redhat.com/show_bug.cgi?id=1756015\n\nBut here are (hopefully) the rest of the steps for when it is available.\n\n```bash\n# Create SSH key if you don't have one\nssh-keygen\n\n# libvirtd\nsudo yum groupinstall -y \"Virtualization Host\"\nsudo systemctl enable --now libvirtd\n\n# Ansible\nsudo yum install -y epel-release\nsudo yum install -y python36\npip3 install --user ansible\n\n# Other deps\nsudo yum install -y \\\ngit \\\ngenisoimage \\\nlibguestfs-tools-c \\\nlibosinfo \\\npython36-libvirt \\\npython36-lxml \\\nlibselinux-python3 \\\nqemu-img \\\nvirt-install\n```\n\n#### CentOS 8\n\n```bash\n# Create SSH key if you don't have one\nssh-keygen\n\n# libvirtd\nsudo dnf groupinstall -y \"Virtualization Host\"\nsudo systemctl enable --now libvirtd\n\n# Ansible\nsudo dnf install -y epel-release\nsudo dnf install -y ansible\n\n# Other deps\nsudo dnf install -y \\\ngit \\\ngenisoimage \\\nlibguestfs-tools-c \\\nlibosinfo \\\npython3 \\\npython3-libvirt \\\npython3-lxml \\\nqemu-img \\\nvirt-install\n```\n\n#### Debian\n\n```bash\n# Create SSH key if you don't have one\nssh-keygen\n\n# libvirtd\nsudo apt update\nsudo apt install -y --no-install-recommends qemu-kvm libvirt-clients libvirt-daemon-system\nsudo systemctl enable --now libvirtd\n\n# Ansible\nsudo apt install -y gnupg2\necho 'deb http://ppa.launchpad.net/ansible/ansible/ubuntu trusty main' | sudo tee -a /etc/apt/sources.list\nsudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 93C4A3FD7BB9C367\nsudo apt update\nsudo apt install -y ansible\n\n# Other deps\nsudo apt install -y --no-install-recommends \\\ncloud-image-utils \\\ndnsmasq \\\ngit \\\ngenisoimage \\\nlibguestfs-tools \\\nlibosinfo-bin \\\npython3-libvirt \\\npython3-lxml \\\nqemu-utils \\\nvirtinst\n```\n\n#### Ubuntu\n\n```bash\n Create SSH key if you don't have one\nssh-keygen\n\n# libvirtd\nsudo apt update\nsudo apt install -y --no-install-recommends libvirt-clients libvirt-daemon-system qemu-kvm\nsudo systemctl enable --now libvirtd\n\n# Ansible\nsudo apt install -y software-properties-common\nsudo apt-add-repository --yes --update ppa:ansible/ansible\nsudo apt install -y ansible\n\n# Other deps\nsudo apt install -y --no-install-recommends \\\ndnsmasq \\\ngit \\\ngenisoimage \\\nlibguestfs-tools \\\nlibosinfo-bin \\\npython3-libvirt \\\npython3-lxml \\\nqemu-utils \\\nvirtinst\n```\n\n#### openSUSE\n\nIf you're running JeOS, we need to change the kernel to `kernel-default` as\n`kernel-default-base` which comes with JeOS is missing KVM modules.\n\n```bash\n# Create SSH key if you don't have one\nssh-keygen\n\n# Install suitable kernel\nsudo zypper install kernel-default\nsudo reboot\n```\n\nContinue after reboot.\n\n```bash\n# libvirtd\nsudo zypper install -yt pattern kvm_server kvm_tools\nsudo systemctl enable --now libvirtd\n\n# Ansible\nsudo zypper install -y ansible\n\n# Other deps\nsudo zypper install -y \\\ngit \\\nguestfs-tools \\\nlibosinfo \\\nmkisofs \\\npython3-libvirt-python \\\npython3-lxml \\\nqemu-tools \\\nvirt-install\n```\n\n#### Using routed networks\n\nYou can route traffic into a newly created bridge by specifying forward *type: route*.\nThis code supports automatic creation of a new bridge named *bridge_dev* which will be\nwired onto an existing interface in the host, specified by parameter *host_dev*.\n\nThe example below shows how a bridge can be created, supporting both IPv4 and IPv6:\n\n```yaml\nkvmhost:\n  hosts:\n    localhost:\n      ansible_connection: local\n      ansible_python_interpreter: /usr/bin/python3\n      virt_infra_host_libvirt_url: qemu:///system\n  vars:\n    virt_infra_host_networks:\n      present:\n        - name: example\n          domain: f901.example.com\n          type: route\n          host_dev: eth0\n          bridge_dev: virbr1\n          bridge_stp: on\n          bridge_delay: 0\n          mac: 52:54:00:f9:01:00\n          ip_address: 10.249.1.1\n          ip_netmask: 255.255.255.0\n          dhcp_start: 10.249.1.11\n          dhcp_end: 10.249.1.254\n          ip6_address: 2001:0db8::f901:1\n          ip6_prefix: 64\n          dhcp6_start: 2001:0db8::f901:0000\n          dhcp6_end: 2001:0db8::f901:00ff\n```\n\nNotes:\n\n1. The IPv6 block 2001:0db8/32 as shown above is provided for the sake of documentation\n   purposes only. You will have to substitute that by your own delegated /48 block\n   (in general) given to you by your IPv6 provider or by a IPv6 over IPv4 tunnelling\n   solution such as [Hurricane Electric's tunnel broker service](http://tunnelbroker.net/).\n\n2. It's highly recommended that you stick with *ip6_prefix: 64*, since it is the\n   recommended setting in Libvirt documentation.\n\n#### Configuring bridges with NetworkManager\n\nThis code supports connecting VMs to both Linux and Open vSwitch bridges, but\nthey must already exist on the KVM host.\n\nHere is how to convert an existing ethernet device into a bridge. Be careful if\ndoing this on a remote machine with only one connection! Make sure you have\nsome other way to log in (e.g. console), or maybe add additional interfaces\ninstead.\n\nFirst, export the the device you want to convert so we can easily reference it\nlater (e.g.  `eth1`).\n\n```bash\nexport NET_DEV=\"eth1\"\n```\n\nNow list the current NetworkManager connections for your device exported above\nso we know what to disable later.\n\n```bash\nsudo nmcli con |egrep -w \"${NET_DEV}\"\n```\n\nThis might be something like `System eth1` or `Wired connection 1`, let's export\nit too for later reference.\n\n```bash\nexport NM_NAME=\"Wired connection 1\"\n```\n\n##### Linux bridge\n\nHere is an example of creating a persistent Linux bridge with NetworkManager.\nIt will take a device such as `eth1` (substitute as appropriate) and convert it\ninto a bridge.\n\nRemember your device's existing NetworkManager connection name from above, you\nwill use it below (e.g. `Wired connection 1`).\n\n```bash\nexport NET_DEV=eth1\nexport NM_NAME=\"Wired connection 1\"\nsudo nmcli con add ifname br0 type bridge con-name br0\nsudo nmcli con add type bridge-slave ifname \"${NET_DEV}\" master br0\n```\n\nOK now you have your bridge device! Note the bridge will have a different MAC\naddress to the underlying device, so if you're expecting it to get a specific\naddress, you'll need to update your DHCP static lease.\n\n```bash\nsudo ip link show dev br0\n```\n\nDisable the current NetworkManager config for the device so that it doesn't\nconflict with the bridge (don't delete it yet, you may lose connection if\nyou're using it for SSH).\n\n```\nsudo nmcli con modify id \"${NM_NAME}\" ipv4.method disabled ipv6.method disabled\n```\n\nNow you can either simply `reboot`, or stop the current interface and bring up\nthe bridge in one command. Remember that the bridge will have a new MAC address\nso it will get a new IP, unless you've updated your DHCP static leases!\n\n```bash\nsudo nmcli con down \"${NM_NAME}\" ; sudo nmcli con up br0\n```\n\nAs mentioned above, by default the Linux bridge will get an address via DHCP.\nIf you don't want it to be on the network (you might have another dedicated\ninterface) then disable DHCP on it.\n\n```bash\nsudo nmcli con modify id br0 ipv4.method disabled ipv6.method disabled\n```\n\n##### Open vSwitch (OVS) bridge\n\nHere is an example of creating a persistent OVS bridge with NetworkManager. It\nwill take a device such as `eth1` (substitute as appropriate) and convert it\ninto an ovs-bridge.\n\nYou will need openvswitch installed as well as the OVS NetworkManager plugin\n(substitute for your distro).\n\n```bash\nsudo dnf install -y NetworkManager-ovs openvswitch\nsudo systemctl enable --now openvswitch\nsudo systemctl restart NetworkManager\n```\n\nNow we can create the OVS bridge (assumes your device is `eth1` and existing\nNetworkManager config is `Wired connection 1`, substitute as appropriate).\n\n```bash\nexport NET_DEV=eth1\nexport NM_NAME=\"Wired connection 1\"\nsudo nmcli con add type ovs-bridge conn.interface ovs-bridge con-name ovs-bridge\nsudo nmcli con add type ovs-port conn.interface port-ovs-bridge master ovs-bridge\nsudo nmcli con add type ovs-interface slave-type ovs-port conn.interface ovs-bridge master port-ovs-bridge\nsudo nmcli con add type ovs-port conn.interface ovs-port-eth master ovs-bridge con-name ovs-port-eth\nsudo nmcli con add type ethernet conn.interface \"${NET_DEV}\" master ovs-port-eth con-name ovs-int-eth\n```\n\nDisable the current NetworkManager config for the device so that it doesn't\nconflict with the bridge (don't delete it yet, you may lose connection if\nyou're using it for SSH).\n\n```\nsudo nmcli con modify id \"${NM_NAME}\" ipv4.method disabled ipv6.method disabled\n```\n\nNow you can either simply `reboot`, or stop the current interface and bring up\nthe bridge in one command.\n\n```bash\nsudo nmcli con down \"${NM_NAME}\" ; sudo nmcli con up ovs-slave-ovs-bridge\n```\n\nBy default the OVS bridge will get an address via DHCP. If you don't want it to\nbe on the network (you might have another dedicated interface) then disable\nDHCP on it.\n\n```bash\nsudo nmcli con modify id ovs-slave-ovs-bridge ipv4.method disabled ipv6.method disabled\n```\n\nShow the switch config and bridge with OVS tools.\n\n```bash\nsudo ovs-vsctl show\n```\n\n## Inventory\n\nThe inventories are split into multiple files for ease of management, under the\ninventories directory in this Git repo.\n\nThis includes the required core inventory for kvmhost:\n\n* kvmhost.yml (vars for the KVM host)\n\nAnd also includes two sample inventories:\n\n* simple.yml (hostgroup for CentOS guests, using defaults)\n* example.yml (hostgroup for guests using multiple distros and custom vars)\n\nThe role contains defaults so that it works mostly out of the box. You probably\njust need to download the CentOS [cloud image](#cloud-images) (see [Too long;\ndidn't read](#too-long-didnt-read)).\n\nCustom settings can be provided for each host or group of hosts in the\ninventory.\n\nTo create a new group of guests to manage, create a new yml file under the\ninventory directory. For example, if you wanted a set of guests for\nOpenStack, you could create an openstack.yml file and populate it as required.\n\nTo manage specific hosts or groups, simply use Ansible's _--limit_ option to\nspecify the hosts or hostgroups (must also include _kvmhost_ group). This way\nyou can use the one inventory for lots of different guests and manage them\nseparately.\n\n### Defaults\n\nAll the defaults are set in the role, which you can see at:\n\n* roles/ansible-role-virt-infra/defaults/main.yml\n\nThe defaults should be something like this.\n\n```yaml\n---\n# Defaults for virt-infra Ansible role\n# Values which are commented out are optional\n\n## Guest related\n\n# Valid guest states are: running, shutdown, destroyed or undefined\nvirt_infra_state: \"running\"\n\n# Guests are not autostarted on boot\nvirt_infra_autostart: \"no\"\n\n# Guest user, by default this will be set to the same user as KVM host user\nvirt_infra_user: \"{{ hostvars[kvmhost].ansible_env.USER }}\"\n\n# Password of default user (consider a vault if you need secure passwords)\n# No root password by default\nvirt_infra_password: \"password\"\n#virt_infra_root_password:\n\n# VM specs for guests\n# See virt-install manpage for supported values\nvirt_infra_ram: \"1024\"\nvirt_infra_ram_max: \"{{ virt_infra_ram }}\"\nvirt_infra_cpus: \"1\"\nvirt_infra_cpus_max: \"{{ virt_infra_cpus }}\"\nvirt_infra_cpu_model: \"host-passthrough\"\nvirt_infra_machine_type: \"q35\"\n\n# SSH keys are a list, you can add more than one\n# If not specified, we default to all public keys on KVM host\nvirt_infra_ssh_keys: []\n\n# If no SSH keys are specified or found on the KVM host, we create one with this\nvirt_infra_ssh_key_size: \"2048\"\nvirt_infra_ssh_key_type: \"rsa\"\n\n# Whether to enable SSH password auth\nvirt_infra_ssh_pwauth: true\n\n# Whether to use cloud-init to configure networking on guest\nvirt_infra_network_config: false\n\n# Networks are a list, you can add more than one\n# \"type\" is optional, both \"nat\" and \"bridge\" are supported\n#  - \"nat\" is default type and should be a libvirt network\n#  - \"bridge\" type requires the bridge interface as the name (e.g. name: \"br0\") which also must already be setup on KVM host\n# \"model\" is also optional\nvirt_infra_networks:\n  - name: \"default\"\n    type: \"nat\"\n    model: \"virtio\"\n\n# Disks, support various libvirt options\n# We generally don't set them though and leave it to hypervisor default\n# See virt-install manpage for supported values\nvirt_infra_disk_size: \"20\"\nvirt_infra_disk_bus: \"scsi\"\nvirt_infra_disk_io: \"threads\"\nvirt_infra_disk_cache: \"writeback\"\n\n# Disks are a list, you can add more than one\n# If you override this, you must still include 'boot' device first in the list\n# Only 'name' is required, others are optional (default size is 20GB)\n# All guests require at least a boot drive (which is the default)\nvirt_infra_disks:\n  - name: \"boot\"\n    size: \"{{ virt_infra_disk_size }}\"\n    bus: \"{{ virt_infra_disk_bus }}\"\n#   io: \"{{ virt_infra_disk_io }}\"\n#   cache: \"{{ virt_infra_disk_cache }}\"\n\n# Default distro is CentOS 8, override in guests or groups\nvirt_infra_distro_image: \"CentOS-Stream-GenericCloud-8-20210603.0.x86_64.qcow2\"\n\n# Determine supported variants on your KVM host with command, \"osinfo-query os\"\n# This doesn't really make much difference to the guest, maybe slightly different bus\n# You could probably just set this as \"centos7.0\" for all distros, if you wanted to\n#virt_infra_variant: \"centos7.0\"\n\n# These distro vars are here for reference and convenience\nvirt_infra_distro: \"centos\"\nvirt_infra_distro_release: \"7\"\nvirt_infra_distro_image_url: \"https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2\"\nvirt_infra_distro_image_checksum_url: \"https://cloud.centos.org/centos/7/images/sha256sum.txt\"\n\n## KVM host related\n\n# Connect to system libvirt instance\nvirt_infra_host_libvirt_url: \"qemu:///system\"\n\n# Path where disk images are kept\nvirt_infra_host_image_path: \"/var/lib/libvirt/images\"\n\n# Disable qemu security driver by default\n# This is overridden in distro specific vars\nvirt_infra_security_driver: \"none\"\n\n# Virtual BMC is disabled by default\nvirt_infra_vbmc: false\n\n# By default we install with pip, but if you prefer to do it manually, set this to false\nvirt_infra_vbmc_pip: true\n\n# Default vbmc service, override if something else on your distro\nvirt_infra_vbmc_service: vbmcd\n\n# Networks on kvmhost are a list, you can add more than one\n# You can create and remove NAT networks on kvmhost (creating bridges not supported)\n# The 'default' network is the standard one shipped with libvirt\n# By default we don't remove any networks (empty absent list)\nvirt_infra_host_networks:\n  absent: []\n  present:\n    - name: \"default\"\n      type: \"nat\"\n      ip_address: \"192.168.122.1\"\n      subnet: \"255.255.255.0\"\n      dhcp_start: \"192.168.122.2\"\n      dhcp_end: \"192.168.122.254\"\n\n# Command for creating ISO images\nvirt_infra_mkiso_cmd: genisoimage\n\n# List of binaries to check for on KVM Host\nvirt_infra_host_deps:\n  - qemu-img\n  - osinfo-query\n  - virsh\n  - virt-customize\n  - virt-sysprep\n\n# Comma separated list of packages to install into guest disks\nvirt_infra_guest_deps:\n  - cloud-init\n  - qemu-guest-agent\n```\n\nVarious other distro specific vars are sourced, based on the host, mostly\naround dependencies to install.\n\n### KVM host\n\nThe KVM host is defined in the kvmhost.yml inventory file as *localhost* in the\n*kvmhost* hostgroup. The Ansible tasks refer to the first entry in this\nhostgroup when performing tasks on the KVM host.\n\n```yaml\n---\n# YAML based inventory, see:\n# https://docs.ansible.com/ansible/latest/plugins/inventory/yaml.html\n\nkvmhost:\n  hosts:\n    localhost:\n      ansible_connection: local\n```\n\n#### Multiple KVM hosts\n\nYou can also specify multiple KVM hosts.\n\n```yaml\n---\nkvmhost:\n  hosts:\n    kvmhost1:\n    kvmhost2:\n    kvmhost3:\n  vars:\n    ansible_python_interpreter: /usr/bin/python3\n    virt_infra_host_networks:\n      absent: []\n      present:\n        - name: \"default\"\n          ip_address: \"192.168.112.1\"\n          subnet: \"255.255.255.0\"\n          dhcp_start: \"192.168.112.2\"\n          dhcp_end: \"192.168.112.254\"\n\n```\n\nTo have a VM land on a specific KVM host, you must add the variable `kvmhost`\nwith a string that matches a KVM host from the `kvmhost` group.\n\nFor example, six CentOS hosts across three KVM hosts:\n\n```yaml\n---\nsimple:\n  hosts:\n    simple-centos-[1:2]:\n      kvmhost: kvmhost1\n    simple-centos-[3:4]:\n      kvmhost: kvmhost2\n    simple-centos-[5:6]:\n      kvmhost: kvmhost3\n  vars:\n    ansible_python_interpreter: /usr/libexec/platform-python\n    virt_infra_distro_image: \"CentOS-Stream-GenericCloud-8-20210603.0.x86_64.qcow2\"\n```\n\nIf no kvmhost is specified for a VM it will default to the first KVM host in\nthe `kvmhost` group (i.e. kvmhost[0]) which matches the original behaviour for\nthe role.\n\nValidation checks have been updated to make sure that all of the KVM hosts are\nvalid and that any specified KVM host for a VM is in the `kvmhost` group.\n\nTo group VMs on certain KVM hosts, consider making child groups and specify\n\n#### libvirt networks\n\nThe KVM host is where the libvirt networks are created and therefore specified\nas vars under that hostgroup.\n\nHere is an example which makes sure three networks are created (two are libvirt\nmanaged the other is ovs) and one has been deleted.\n\n```yaml\n---\n# YAML based inventory, see:\n# https://docs.ansible.com/ansible/latest/plugins/inventory/yaml.html\n\nkvmhost:\n  hosts:\n    localhost:\n      ansible_connection: local\n  vars:\n    virt_infra_host_networks:\n      absent:\n        - name: \"example-removed\"\n      present:\n        - name: \"example\"\n          ip_address: \"172.31.255.1\"\n          subnet: \"255.255.255.0\"\n          dhcp_start: \"172.31.255.2\"\n          dhcp_end: \"172.31.255.254\"\n          mtu: 9000\n        - name: \"example2\"\n          ip_address: \"10.255.255.1\"\n          subnet: \"255.255.255.0\"\n          dhcp_start: \"10.255.255.2\"\n          dhcp_end: \"10.255.255.254\"\n          mtu: 9000\n        - name: ovs-bridge\n          bridge_dev: ovs-bridge\n          type: ovs\n          portgroup:\n            - name: ovs-trunk\n              trunk: true\n              native_vlan: 1\n              vlan:\n                - 1\n                - 2\n            - name: home\n              native_vlan: 1\n              vlan:\n                - 1\n            - name: guest\n              native_vlan: 2\n              vlan:\n                - 2\n```\n\n##### Using ovs-bridge in inventory\n\nAfter configuring [Open vSwitch (OVS) bridge](#open-vswitch-ovs-bridge) on your\nhost you can use `ovs-bridge` as the `device` of an `ovs` bridge in your\n`kvmhost` inventory `virt_infra_host_networks` entry and it will create the OVS\nlibvirt networks for you. You can set up multiple VLANs and set one as default\nnative (if required).\n\n```yaml\n      virt_infra_host_networks:\n        present:\n          - name: ovs-bridge\n            bridge_dev: ovs-bridge\n            type: ovs\n            portgroup:\n              # This is a portgroup with multiple VLANs\n              # It is native VLAN 1 and also allows traffic tagged with VLAN 99\n              - name: ovs-trunk\n                trunk: true\n                native_vlan: 1\n                vlan:\n                  - 1\n                  - 99\n              # This is portgroup just for native VLAN 1\n              - name: default\n                native_vlan: 1\n                vlan:\n                  - 1\n              # This is portgroup just for native VLAN 99\n              - name: other\n                native_vlan: 99\n                vlan:\n                  - 99\n```\n\n### Guests\n\nGuests are defined in their own yaml files under the inventories directory. Two\nsamples are included by default, simple.yml and example.yml.\n\nHere's an example hostgroup called _simple_ with three VMs, using defaults\n(apart from the Python interpreter). Note that these will be CentOS 7 guests\nbecause that's the default in default.yml.\n\n```yaml\n---\n## YAML based inventory, see:\n## https://docs.ansible.com/ansible/latest/plugins/inventory/yaml.html\n\nsimple:\n  hosts:\n    centos-simple-[0:2]:\n      ansible_python_interpreter: /usr/bin/python\n```\n\nIf you want a group of VMs to all be the same, set the vars at the hostgroup\nlevel. You can still override hostgroup vars with individual vars for specific\nhosts, if required.\n\nHere's an example setting various hostgroup and individual host vars.\n\n```yaml\n---\n## YAML based inventory, see:\n## https://docs.ansible.com/ansible/latest/plugins/inventory/yaml.html\n\nexample:\n  hosts:\n    centos-7-example:\n      virt_infra_state: shutdown\n      virt_infra_timezone: \"Australia/Melbourne\"\n      ansible_python_interpreter: /usr/bin/python\n      virt_infra_networks:\n        - name: \"br0\"\n          type: bridge\n          mtu: 9000\n        - name: \"extra_network\"\n          type: nat\n          model: e1000\n        - name: ovs-bridge\n          portgroup: guest\n          type: ovs\n      virt_infra_disks:\n        - name: \"boot\"\n        - name: \"nvme\"\n          size: \"100\"\n          bus: \"nvme\"\n    centos-8-example:\n      virt_infra_timezone: \"Australia/Melbourne\"\n      ansible_python_interpreter: /usr/libexec/platform-python\n    opensuse-15-example:\n      virt_infra_distro: opensuse\n      virt_infra_distro_image: openSUSE-Leap-15.1-JeOS.x86_64-15.1.0-OpenStack-Cloud-Current.qcow2\n      virt_infra_variant: opensuse15.1\n      virt_infra_disks:\n        - name: \"boot\"\n          bus: \"scsi\"\n    ubuntu-eoan-example:\n      virt_infra_cpu: 2\n      virt_infra_distro: ubuntu\n      virt_infra_distro_image: eoan-server-cloudimg-amd64.img\n      virt_infra_variant: ubuntu18.04\n  vars:\n    virt_infra_ram: 1024\n    virt_infra_disks:\n      - name: \"boot\"\n      - name: \"data\"\n        bus: \"sata\"\n        keep: true\n    virt_infra_networks:\n      - \"example\"\n```\n\n#### Running custom shell commands on guest disk\n\nUsing the variable `virt_infra_disk_cmd`, you can create a list of shell\ncommands that you want to be run on the guest disk, before the VM is created.\nThese can be any shell command, and used for simple things you need to do\nbefore the VM is created, otherwise you should do it with Ansible afterwards.\n\nFor example, setting a mirror on a Fedora guest:\n\n```yaml\n      virt_infra_disk_cmd:\n        - sed -i s/^metalink/#metalink/g /etc/yum.repos.d/*repo\n        - sed -i s/^#baseurl/baseurl/g /etc/yum.repos.d/*repo\n        - dnf config-manager --save --setopt baseurl=http://path-to-mirror fedora\n        - dnf config-manager --save --setopt baseurl=http://path-to-mirror-updates updates\n```\n\nOr finding the fastest mirror on Debian:\n\n```yaml\n      virt_infra_disk_cmd:\n        - apt-get update\n        - apt-get install -y netselect-apt\n        - netselect-apt -n -o /etc/apt/sources.list\n```\n\n#### Using Linux bridge in inventory\n\nAfter configuring a [Linux bridge](#linux-bridge) on your host, there's nothing\nto do on the `kvmhost` side of the inventory.\n\nFor any guests you want to connect to the bridge, simply specify it in their\ninventory. Use `br0` as the `name` of a network under `virt_infra_networks`\nwith type `bridge`.\n\n```yaml\n      virt_infra_networks:\n        - name: br0\n          type: bridge\n```\n\n#### Using ovs-bridge in inventory\n\nFirst, configure an [Open vSwitch (OVS) bridge](#open-vswitch-ovs-bridge) on\nyour host and also configuring the [kvmhost\ninventory](#using-ovs-bridge-in-inventory) to create the required networks.\n\nThen you can then specify your VMs to be on specific portgroups and libvirt\nwill automatically set up the ports for your VMs and you.\n\n```yaml\n      virt_infra_networks:\n        - name: ovs-bridge\n          portgroup: default\n          type: ovs\n```\n\nOnce your VMs are running, you can see their OVS ports with `sudo ovs-vsctl\nshow` on the KVM host.\n\n#### Static network configuration with cloud-init\n\nGuest network interfaces can be configured statically using network\nconfiguration v2 support in cloud-init. Currently only ethernet devices are\nsupported (not bonds, bridges or VLANs).\n\nFor details, see the [cloud-init\ndocumentation](https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v2.html).\n\nSome distros might not support this, so in order to activate network config\nplease enable the following variable for your guests:\n\n```yaml\nvirt_infra_network_config: true\n```\n\nThe network config file for cloud-init ISO is then populated with new optional\nnetwork configuration options for each VM's network interfaces, including:\n\n - device (_string_ for device name, defaults to generated `eth0`, `eth1`, etc)\n - addresses (_list_)\n - dhcp4 (_boolean_, disabled if addresses is defined, otherwise true)\n - dhcp6 (_boolean_, disabled if addresses is defined, otherwise true)\n - gateway4 (_string_ for IPv4 default gateway)\n - gateway6 (_string_ for IPv6 default gateway)\n - nameservers (_dict_)\n   - search (_list_ of DNS domain search)\n   - addresses (_list_ of DNS servers)\n - routes (_list_ of _dicts_)\n   - to (_string_ for subnet to route to)\n   - via (_string_ for gateway address/subnet)\n   - metric (_int_)\n\n**NOTE:** Due to the way cloud-init works (see documentation link above), if\nDHCP is enabled then some manual settings (including gateway and nameservers)\nare ignored. Manual routes are kept.\n\nIt is worth noting that _none_ of the options above are actually required (will\ndefault to `eth0` and `dhcp`), but here is an example:\n\n```yaml\n  virt_infra_networks:\n    - name: default\n      device: eth0\n      addresses:\n        - 192.168.0.123/24\n      gateway4: 192.168.0.1\n      nameservers:\n        search: [foo.local, bar.local]\n        addresses: [8.8.8.8]\n      routes:\n        - to: 10.0.0.0/24\n          via: 192.168.0.254\n          metric: 100\n```\n\n##### Network device names and MAC addresses\n\nThe network config format requires specifying a network device to configure,\nwhich needs to match the actual interface to work properly. Unfortunately,\nthere's no way to always know that, as some distros will enable persistent\nnames and others won't.\n\nFor example, if we say \"eth0\" but the interfaces are actually \"ens0\" instead,\nthen the config will still be written to `eth0` and therefore not actually\napplied to `ens0`.\n\nTo make this work, we need to rename the interface so that it will match the\nconfiguration, however in order to ensure we pick the exact interface we must\nmatch on MAC address.\n\nTherefore, if MAC addresses are not specified, each interface in the network\nlist will have an idempotent one generated based on the inventory hostname and\nthe interface's order in the list.\n\n## Cloud images\n\nThis is designed to use standard cloud images provided by various distros\n(OpenStack [provides some\nsuggestions](https://docs.openstack.org/image-guide/obtain-images.html)).\n\nMake sure the Image you're specifying for your guests already exists under your\nlibvirt storage dir (by default this is _/var/lib/libvirt/images/_).\n\nI have tested the following guests successfully:\n\n* CentOS 7\n  * https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2\n* CentOS 8\n  * https://cloud.centos.org/centos/8/x86_64/images/CentOS-8-GenericCloud-8.1.1911-20200113.3.x86_64.qcow2\n* Fedora 33\n  * https://download.fedoraproject.org/pub/fedora/linux/releases/33/Cloud/x86_64/images/Fedora-Cloud-Base-33-1.2.x86_64.qcow2\n* Fedora 34\n  * https://download.fedoraproject.org/pub/fedora/linux/releases/34/Cloud/x86_64/images/Fedora-Cloud-Base-34-1.2.x86_64.qcow2\n* Debian 10\n  * http://cdimage.debian.org/cdimage/openstack/current-10/debian-10-openstack-amd64.qcow2\n* Ubuntu 18.04 LTS\n  * http://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img\n* Ubuntu 20.04 LTS\n  * http://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img\n* openSUSE 15.3 JeOS\n  * http://download.opensuse.org/distribution/leap/15.3/appliances/openSUSE-Leap-15.3-JeOS.x86_64-15.3-OpenStack-Cloud-Current.qcow2\n\nSo that we can configure the guest and get its IP, both `cloud-init` and\n`qemu-guest-agent` will be installed into you guest's image, just in case.\n\nThis can be changed or overridden using the `virt_infra_guest_deps` variable,\nwhich is a list.\n\nSysprep is also run on the guest image to make sure it's clean of things like\nold MAC addresses.\n\n## Running the playbook\n\nI've tried to keep the Ansible as a simple, logical set of steps and not get\ntoo tricky.\n\nHaving said that, the playbook is quite specific.\n\nThere are some tasks which can only be run on the KVM host and others in a\nspecific order.\n\nThe reason I've done this is to make the tasks more clear, so that the task\nonly runs for the host(s) it's designed for. Instead, I could have run all\ntasks against all hosts, but then every single task would be a statement to\ninclude or exclude the kvmhost.\n\nIt will also help by running a bunch of validation checks on the KVM host and\nfor your guest configs to try to catch anything that's not right.\n\nTo deploy, run the Ansible playbook against the kvmhost and any set of VMs you\nwant to manage.\n\nFor example, to run and set up the guests in the example group, run this\n(note that it includes _kvmhost_ in --limit option).\n\n```bash\nansible-playbook \\\n--limit kvmhost,example \\\n./virt-infra.yml\n```\n\nYou can also run the included shell script (which will also install Ansible on\nsupported distros).\n\n```bash\n./run.sh --limit kvmhost,example\n```\n\nYou can also override a number of guest settings on the command line.\n\n```bash\nansible-playbook \\\n./virt-infra.yml \\\n--limit kvmhost,example \\\n-e virt_infra_root_password=password \\\n-e virt_infra_disk_size=100 \\\n-e virt_infra_ram=4096 \\\n-e virt_infra_ram_max=8192 \\\n-e virt_infra_cpus=8 \\\n-e virt_infra_cpus_max=16 \\\n-e '{ \"virt_infra_networks\": [{ \"name\": \"br0\", \"type\": \"bridge\" }] }' \\\n-e virt_infra_state=running\n```\n\nTo keep the command simple, the included ansible.cfg file is already configured\nto look for the inventory under the inventory directory and to prompt for your\nbecome password. If you don't want to use that ansible.cfg, then also pass in\n_--ask-become-pass_ and _--inventory_ options, as required.\n\n### Cleanup\n\nTo remove a bunch of guests, you could specify them (or the hostgroup) with\n--limit and pass in _virt_infra_state=undefined_ as a command line extra arg.\n\nThis will override the guest state to undefined and if they exist, they will be\ndeleted.\n\n```bash\nansible-playbook \\\n--limit kvmhost,example \\\n--extra-vars virt_infra_state=undefined \\\n./virt-infra.yml\n```\n\n## Post setup configuration\n\nOnce you have set up your infra, you could run another playbook against your\nsame inventory to do whatever you wanted with those machines...\n\nHere's an example playbook called `update-hosts.yml` which uses the same\ninventory to update and reboot all nodes.\n\n```yaml\n- hosts: all,!kvmhost\n  tasks:\n    - name: Upgrade all packages\n      package:\n        name: '*'\n        state: latest\n      become: true\n      register: result_package_update\n      retries: 30\n      delay: 10\n      until: result_package_update is succeeded\n\n    - name: Install packages\n      package:\n        name:\n          - git\n          - tmux\n          - vim\n        state: present\n      become: true\n      register: result_package_install\n      retries: 30\n      delay: 10\n      until: result_package_install is succeeded\n\n    - name: Reboot nodes after updates\n      reboot:\n      become: true\n```\n\nYou could run it like this:\n\n```bash\nansible-playbook \\\n--ask-become-pass \\\n--inventory ./inventory.d \\\n./ansible/update-vms.yml\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsmart%2Fvirt-infra-ansible","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcsmart%2Fvirt-infra-ansible","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsmart%2Fvirt-infra-ansible/lists"}