Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/tatsuya6502/bhyve-scripts

A collection of shell scripts to run bhyve VMs on my FreeBSD 12.x desktop machine
https://github.com/tatsuya6502/bhyve-scripts

bhyve bhyve-vms freebsd

Last synced: 23 days ago
JSON representation

A collection of shell scripts to run bhyve VMs on my FreeBSD 12.x desktop machine

Host: GitHub
URL: https://github.com/tatsuya6502/bhyve-scripts
Owner: tatsuya6502
License: mit
Created: 2015-03-16T05:17:47.000Z (over 9 years ago)
Default Branch: master
Last Pushed: 2019-09-14T00:07:38.000Z (about 5 years ago)
Last Synced: 2023-03-11T06:18:40.469Z (over 1 year ago)
Topics: bhyve, bhyve-vms, freebsd
Language: Shell
Homepage:
Size: 44.9 KB
Stars: 13
Watchers: 4
Forks: 1
Open Issues: 0
Metadata Files:
- Readme: README.adoc
- License: LICENSE

Awesome Lists containing this project

README

== bhyve-scripts

A collection of shell scripts to run bhyve VMs on my FreeBSD desktop
and server machines.

== Environment

- *Host Machines*
* *FreeBSD 10.2-RELEASE*
* ZFS zvols as the virtual disks for bhyve VMs
* Bridged tap devices for network access
- *Guest Machines*
* *Linux Guests:* CentOS 6, 7, RHEL 7 and Ubuntu 14.04 LTS
* *BSD Guests:* OpenBSD 5.8

== ZFS zvols

These +disk1+ are thin-provisioned zvols.

----
$ zfs list -r zroot/bhyve
NAME USED AVAIL REFER MOUNTPOINT
zroot/bhyve 107G 364G 144K none
zroot/bhyve/centos6-vertica 49.7G 364G 96K none
zroot/bhyve/centos6-vertica/disk1 49.7G 364G 41.2G -
zroot/bhyve/centos7-docker1 38.2G 364G 144K none
zroot/bhyve/centos7-docker1/disk1 38.2G 364G 32.3G -
zroot/bhyve/openbsd 969M 364G 96K none
zroot/bhyve/openbsd/disk1 969M 364G 969M -
zroot/bhyve/rhel7-ost-icehouse 7.55G 364G 144K none
zroot/bhyve/rhel7-ost-icehouse/disk1 7.55G 364G 8.27G -
zroot/bhyve/rhel7-ost-juno 5.20G 364G 144K none
zroot/bhyve/rhel7-ost-juno/disk1 5.20G 364G 4.76G -
zroot/bhyve/ubuntu-14.04 5.00G 364G 144K none
zroot/bhyve/ubuntu-14.04/disk1 5.00G 364G 5.00G -
----

----
$ sudo zfs create zroot/bhyve/centos7-docker1
$ sudo zfs create -V 48G -s zroot/bhyve/centos7-docker1/disk1
----

== Prerequisites

(*TODO*) Install grub-bhyve etc.

== Setup

The following script loads the kernel modules for bhyve and creates
tap devices for the VMs.

----
$ sudo ./setup.sh
----

== Installing Linux -- Booting a VM from installer ISO Image

Edit +device.map+.

.bhyve-scripts/centos7/device.map
----
(hd0) /dev/zvol/zroot/bhyve/centos7-docker1/disk1
(cd0) /home/tatsuya/installers/centos/CentOS-7.0-1406-x86_64-DVD.iso
----

Run +grub-bhyve+ as root.

----
$ cd centos7
$ sudo grub-bhyve -m device.map -r cd0 -M 4096M centos7-docker1
grup> ls
(hd0) (hd0,msdos2) (hd0,msdos1) (cd0) (cd0,msdos2) (host) (lvm/centos-root) (lvm/centos-swap)
grub> ls (cd0)/
CentOS_BuildTag EFI/ EULA GPL images/ isolinux/ LiveOS/ Packages/
repodata/ RPM-GPG-KEY-CentOS-Testing-7 RPM-GPG-KEY-CentOS-7 TRANS.TBL
grup> ls (cd0)/isolinux
boot.cat boot.msg grub.conf initrd.img isolinux.bin isolinux.cfg
memtest splash.png TRANS.TBL upgrade.img vesamenu.c32 vmlinuz

grub> linux (cd0)/isolinux/vmlinuz
grub> initrd (cd0)/isolinux/initrd.img
grub> boot
----

*NOTE*:

For CentOS, it's recommended to perform VNC-based graphical
installation rather than console-based installation because graphical
one gives you more customization options (e.g. custom disk layout)
To do that, add the following options to +linux+ command.

----
grub> linux (cd0)/isolinux/vmlinuz vnc headless ip=dhcp ksdevice=eth0 lang=en_US keymap=us
----

Once the installer started, it will give tell you VNC server address
and port.

For example,
----
Please manually connect your vnc client to 10.0.0.117:1 to begin the install.
----

Run +bhyve+ as root. Note that the VM has the installer DVD ISO
attached.

----
$ sudo bhyve -c 4 -m 4096M -H -P -A -l com1,stdio \
-s 0:0,hostbridge -s 1:0,lpc \
-s 2:0,virtio-net,tap1 \
-s 3,ahci-cd,/home/tatsuya/installers/centos/CentOS-7.0-1406-x86_64-DVD.iso \
-s 4,virtio-blk,/dev/zvol/zroot/bhyve/centos7-docker1/disk1 \
centos7-docker1
----

When installation is completed, press the "Restart" button on the
installer screen. This will shutdown the VM and drop you into a shell
prompt (instead of restarting). Proceed to next chapter to boot the VM
from the disk image.

== Booting a Linux VM from Disk Image

----
$ cd centos7
----

Edit +grub.in+. You might want to take a look at grub menu for kernel
parameters.

Run the VM.

----
$ sudo ./run.sh
----

== Installing OpenBSD -- Booting a VM from installer ISO Image

Edit +device.map+.

.bhyve-scripts/openbsd/device.map
----
(hd0) /dev/zvol/zroot/bhyve/openbsd/disk1
(cd0) /home/tatsuya/installers/openbsd/openbsd-5.7/install57.iso
----

Run +grub-bhyve+ as root. Ensure to boot with the OpenBSD install
kernel (+bsd.rd+).

----
$ cd openbsd
$ sudo grub-bhyve -m device.map -r cd0 -M 256M openbsd
grup> ls
(hd0) (cd0) (host)
grub> ls (cd0)/
5.7/ etc/ TRANS.TBL
grup> ls (cd0)/5.7
amd64/ TRANS.TBL
grub> ls (cd0)/5.7/amd64/
base57.tgz boot.catalog bsd bsd.mp bsd.rd cdboot cdbr comp57.tgz
game57.tgz INSTALL.amd64 man57.tgz SHA256 TRANS.TBL xbase57.tgz
xfont57.tgz xserv57.tgz xshare57.tgz

grub> kopenbsd -h com0 (cd0)/5.7/amd64/bsd.rd
grub> boot
----

*NOTE*: https://forums.freebsd.org/threads/howto-bhyve-using-openbsd-as-main-firewall-in-freebsd.50470/#post-282880[Example]

Run +bhyve+ as root. Note that the VM has the installer ISO
attached. Also +-W+ option seems required. It forces virtio to use
single-vector MSI.

----
$ sudo bhyve -W -m 256M -H -P -A -l com1,stdio \
-s 0:0,hostbridge \
-s 1:0,lpc \
-s 2:0,virtio-net,tap4 \
-s 3,ahci-cd,/home/tatsuya/installers/openbsd/openbsd-5.7/install57.iso \
-s 4,virtio-blk,/dev/zvol/zroot/bhyve/openbsd/disk1 \
openbsd
----

=== IMPORTANT: Adjust the System Clock

Login to the VM and *ensure the system clock is correct*. By some
reason it's usually advance 3 days, so `ntpd` won't be able to
adjust the clock. Run +ntpdate+ to fix it.2

----
[docker1] /home/tatsuya% sudo systemctl stop ntpd.service
[docker1] /home/tatsuya% sudo ntpdate -b pool.ntp.org
16 Mar 13:46:27 ntpdate[2046]: step time server 202.112.29.82 offset -277199.180289 sec
[docker1] /home/tatsuya% sudo systemctl start ntpd.service
----

*NOTE*: This might be fixed in 10.2-RELEASE
(https://www.quernus.co.uk/2015/07/27/openbsd-as-freebsd-router/#comment-2163062052[Reference])

== When Linux Kernel in a VM is Upgraded

Edit +grub.in+ so that it will boot from the latest Linux Kernel.

.bhyve-scripts/centos7/grub.in
----
linux (hd0,msdos1)/vmlinuz-3.10.0-123.20.1.el7.x86_64 root=/dev/mapper/centos-root
initrd (hd0,msdos1)/initramfs-3.10.0-123.20.1.el7.x86_64.img
boot
----

== License

bhyve-script is open-sourced under the MIT license. See the
link:LICENSE[LICENSE] file for details.