An open API service indexing awesome lists of open source software.

https://github.com/nalajala4naresh/ch-vmm

Run Cloud Hypervisor VM's in Kubernetes
https://github.com/nalajala4naresh/ch-vmm

cloud-hypervisor golang kubernetes microvm virtualization

Last synced: 7 days ago
JSON representation

Run Cloud Hypervisor VM's in Kubernetes

Awesome Lists containing this project

README

          

# ch-vmm: Lightweight Virtualization Add-on for Kubernetes

[![build](https://github.com/nalajala4naresh/ch-vmm/actions/workflows/build.yml/badge.svg)](https://github.com/nalajala4naresh/ch-vmm/actions/workflows/build.yml)
[![Go Report Card](https://goreportcard.com/badge/github.com/nalajala4naresh/ch-vmm)](https://goreportcard.com/report/github.com/nalajala4naresh/ch-vmm)
[![codecov](https://codecov.io/gh/nalajala4naresh/ch-vmm/branch/main/graph/badge.svg?token=6GXYM2BFLT)](https://codecov.io/gh/nalajala4naresh/ch-vmm)

ch-vmm is a [Kubernetes](https://github.com/kubernetes/kubernetes) add-on for running [Cloud Hypervisor](https://github.com/cloud-hypervisor/cloud-hypervisor) virtual machines. By using Cloud Hypervisor as the underlying hypervisor, ch-vmm enables a lightweight and secure way to run fully virtualized workloads in a canonical Kubernetes cluster.

Compared to [KubeVirt](https://github.com/kubevirt/kubevirt), ch-vmm:

- does not use libvirt or QEMU. By leveraging Cloud Hypervisor, VMs has lower memory (≈30MB) footprints, higher performance and smaller attack surface.
- does not require a long-running per-Pod launcher process, which further reduces runtime memory overhead (≈80MB).

Compared to [VirtInk](https://github.com/smartxworks/virtink), ch-vmm:
- does support snapshot and restore features
- Supports newer version k8s controller-runtime and k8s versions & cloud-hypervisor v52.0
- `VMPool` and `VMSet` to manage fleet of VM's, checkout docs folder for examples.

ch-vmm consists of 3 components:

- `ch-vmm-controller` is the cluster-wide controller, responsible for creating Pods to run Cloud Hypervisor VMs.
- `ch-daemon` is the per-Node daemon, responsible for further controlling Cloud Hypervisor VMs on Node bases.
- `virt-prerunner` is the per-Pod pre-runner, responsible for preparing VM networks and building Cloud Hypervisor VM configuration.

**NOTE**: ch-vmm is still a work in progress, its API may change without prior notice.

## Installation

### Requirements

A few requirements need to be met before you can begin:

- Kubernetes Version v1.35+ (In-place vertical scaling of VM with release v1.2.0)
- Kubernetes Version < v1.35 , please use release v1.1.0
- Kubernetes apiserver must have `--allow-privileged=true` in order to run ch-vmm's privileged DaemonSet. It's usually set by default.
- [cert-manager](https://cert-manager.io/) v1.16 installed in Kubernetes cluster. You can install it with `kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.yaml`.
- Deploy ch-vmm onto k8s cluster with ``` kubectl apply -f https://github.com/nalajala4naresh/ch-vmm/releases/latest/download/ch-vmm.yaml```
- Deploy CDI operator to manage DataVolume objects as disks to the VM's

```bash
$ export VERSION=$(curl -s https://api.github.com/repos/kubevirt/containerized-data-importer/releases/latest | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/')
$ kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-operator.yaml
$ kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-cr.yaml
```

- Deploy external snapshotter for disk management features
``` bash

$ kubectl -n kube-system kustomize deploy/kubernetes/snapshot-controller | kubectl create -f -
```
#### Container Runtime Support

ch-vmm currently supports the following container runtimes:

- Docker
- containerd
- cloud-hypervisor v52.0 is supported.

Other container runtimes, which do not use virtualization features, should work too. However, they are not tested officially.

#### Hardware Virtualization Support

Hardware with virtualization support is required. You should check if `/dev/kvm` exists on each Kubernetes nodes.

#### Host Kernel Version

- Minimum: v4.11

## Getting Started

### Create a VM

Apply the following manifest to Kubernetes. Note it uses a [container rootfs](samples/Dockerfile.container-rootfs-ubuntu) and as such doesn’t persist data.

```bash
cat <