Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/chromakode/linux-vm-tools

Tools I use for developing with VMs.
https://github.com/chromakode/linux-vm-tools

Last synced: 23 days ago
JSON representation

Tools I use for developing with VMs.

Awesome Lists containing this project

README 

        
# libvirt / KVM tools + HOWTO



## Background



For the past few years, I've used [KVM](http://www.linux-kvm.org) and

[libvirt](http://libvirt.org) to isolate my development environment when

working on reddit and other projects. The modern qemu/KVM/libvirt stack on

Ubuntu has a lot of nice features relevant to my purporses:



  * Isolated execution from the host, with autogenerated AppArmor profiles

    providing additional access control.

  * Excellent performance ([within 2% of bare metal CPU performance] [1]).

  * Diverse command line management tools, and a good quality optional GUI.

  * Fast snapshotting and state restoration.

  * Passthrough filesystem access.



In practice, I've found working with VMs to provide many advantages in my

workflow. It allows me to keep projects separate and contained from each other.

When using a laptop, it's nice to be able to start, stop, and pause entire

project process hierarchies at once. Also, a side benefit I've noticed is that

treating projects as distinct hosts helps me think about deployment; I can run

roughly the same services and ports as in production without juggling

configuration files for each project. As expected, using virtual machines also

makes it convenient to use different OSes and experiment with unfamiliar

software packages in a sandboxed environment.



  [1]: https://major.io/2014/06/22/performance-benchmarks-kvm-vs-xen/



## Overview



For each project I develop on, I typically clone an Ubuntu Server VM specific

to the project, with minimal customizations:



  * To access dev files, I create a passthrough filesystem set to read-only (so

    that VMs cannot manipulate the state of the host machine). However, in

    practice, I've found that some software expects to be able to write to its

    code directory. To allow this, I use a little filesystem magic to overlay a

    writable local directory on top (thanks to @spladug for suggesting

    [overlayfs] [2]).  This allows writes to the passthrough directory which

    are only visible inside the VM.

  * I install `avahi-daemon` to access the VM via `ssh hostname.local`.



In the past, I did most of the configuration and setup gruntwork for each VM

manually, but this doesn't scale well as projects multiply. This repository

contains some tools I've created to automate the process.



  [2]: https://git.kernel.org/cgit/linux/kernel/git/mszeredi/vfs.git/tree/Documentation/filesystems/overlayfs.txt?h=overlayfs.current



## Using makevm.py



`makevm.py` is a simple tool to rapidly create and bootstrap development VMs by

cloning a base install.



For example, given a base VM called "ubuntu-14.04-server", to create a new VM

with read-only access to /home/chromakode/dev/project:



    ./makevm.py ubuntu-14.04-server my-new-vm -d /home/chromakode/dev/project



To create the base OS image, I run a standard Ubuntu Server install, with one

customization: `roottyS0.conf` is placed in `/etc/init`. This upstart job

starts a root serial TTY console used on first boot to run bootstrapping

commands.



### Technical details



First `makevm.py` creates a clone of the base VM, generating a new UUID and MAC

address for the network interface. Clones use the qcow2 disk image format's

backing feature to store only incremental changes to the base OS image; this

reduces the disk usage of VMs created from the same base. Then, `sysprep.sh` is

run, which sets up the passthrough filesystem and basic ssh access. To run

commands on the new VM, `sysprep.sh` accesses a serial TTY console running a

root shell (via `roottyS0.conf`) on the VM. `sysprep.sh` updates the hostname,

installs packages, and configures the passthrough filesystem. Finally,

`roottyS0.conf` is removed to disable the root shell.