https://github.com/containercraft/ccio-simplestreams
LXD SimpleStreams Image Service
https://github.com/containercraft/ccio-simplestreams
Last synced: 10 days ago
JSON representation
LXD SimpleStreams Image Service
- Host: GitHub
- URL: https://github.com/containercraft/ccio-simplestreams
- Owner: ContainerCraft
- License: apache-2.0
- Created: 2020-03-28T19:20:31.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2020-03-29T05:15:47.000Z (about 6 years ago)
- Last Synced: 2025-03-05T15:51:51.016Z (over 1 year ago)
- Language: Python
- Size: 5.28 MB
- Stars: 0
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Lxd-image-server
================
[Docker Hub Image](https://hub.docker.com/r/containercraft/ccio-simplestreams)
[Application Repo](https://github.com/containercraft/simplestreams-service)
Creates and manages a simplestreams lxd image server on top of nginx.
If installed as a debian package, a new service is created and it monitors
if there are any changes in the image directory and updates json files.
Requirements
------------
* Python: Version 3.5.2 or higher.
* Nginx
* OpenSSL
Building the debian package
---------------------------
### Building the package ###
To build lxd-image-server, first, install the build dependecies:
```bash
# debhelper >= 9
# dh-virtualenv >= 9
apt-get install debhelper dh-exec python3 python-dev dh-virtualenv
```
Then build the package:
```bash
dpkg-buildpackage -us -uc -b
```
### Building the package in a Docker container ###
To build lxd-image-server itself in a Docker container, call docker build:
```bash
docker build --tag lxd-image-server-builder .
```
This will build the DEB package for Ubuntu Bionic by default. Add e.g.
--build-arg distro=ubuntu:xenial to build for Ubuntu Xenial.
The resulting files must be copied out of the build container, using these
commands:
```bash
mkdir -p dist && docker run --rm lxd-image-server-builder tar -C /dpkg -c . | tar -C dist -xv
```
Installation
------------
### From debian package (recommended) ###
The debian package will automatically copy the source files, create the user lxdadm
to upload the files and setup the nginx server with its configuration (included a
self signed ssl certificate).
* **Install from repository**:
```sh
apt-get install lxd-image-server
```
* **Install using dpkg**:
```sh
dpkg -i lxd-image-server_0.0.1~xenial_all.deb
```
After the installation of the package, a rsa key has to be generated at
/home/lxdadm/.ssh to control the upload of images:
```sh
ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): # use /home/lxdadm/.ssh/id_rsa
...
```
The generated id_rsa key will be used to upload files to the server.
### From source code ###
Clone the repository and run:
```sh
python setup.py install
```
The subcommand init generates all the default directories, ssl keys and links nginx
configuration (when using default configuration it is recommended to use debian installation)
Usage
-----
Lxd-image-server is not just a new remote for your lxd, it also allows you to
distribute your images to different mirrors. Clients will update new images to
the master server and the master will mirror the image on the mirrors defined
on the configuration file.
The following picture describes, the master server which replicates images to
other two servers. Clients can get images from any of them but they can only
upload new images to the master.
The configuration file would be:
```yaml
[mirrors]
[mirror1]
user = "lxdadm"
url = "https://mirror1.xxxxxxx.com:8443"
key_path = "/etc/lxd-image-server/lxdhub.key"
[mirror2]
user = "lxdadm"
url = "https://mirror2.xxxxxxx.com:8443"
key_path = "/etc/lxd-image-server/lxdhub.key"
```
Configuration files are 'toml' files and are read from (/etc,~/.config)/lxd-image-server/(config.toml,conf.d/*)
Code should automatically reload them if they change.
The installed service on the master will automatically monitor the image
directory and update all the required metadata. No further commands are needed.
This is the structure the simplestreams server needs to have:
```
- /var/www # document root
`- simplestreams
|- images # images folder
| `- iats # environment|product
| `- xenial # release
| `- amd64 # architecture
| `- default # box type
| `- 20180716_12:00 # version 1 (mandatory format: 'YYYYMMDD_HH:MM')
| |- lxd.tar.xz # index and templates
| `- rootfs.squashfs # rootfs of container
`- streams
`- v1
|- index.json # index of products
`- images.json # info with versions of products
```
Your product will be named by concatenating all the directories names. As seen above, name will be
"iats:xenial:amd64:default"
Your product definition, inside images.json file will have these fields:
- arch = the name of the architecture directory (amd64, in the example)
- os = the environment|product name (iats, in the example)
- release = the release directory name (xenial, in the example)
- release_title = the release directory name (xenial, in the example)
- aliases: a **single** alias with the name of the product (iats:xenial:amd64:default, in the example)
You can change these values by uploading also a 'metadata.json' file (which will not appear in the images.json file),
and which can contain the following fields:
- os
- release_title
- aliases = string of aliases, joined by ','. Name of product alias will be added automatically.
Preferably you should upload this 'metadata.json' at the beginning rather than after the big actual image files.
The command `lxd-image-server` can be used to manage the server manually:
```sh
Usage: lxd-image-server [OPTIONS] COMMAND [ARGS]...
Options:
--log-file TEXT [default: ./lxd-image-server.log]
use 'STDOUT' or 'STDERR' as file names, if needed
--verbose Sets log level to debug
--help Show this message and exit.
Commands:
init
update
watch
Default paths:
- Index files: /var/www/simplestreams/streams/v1
- Image dirs: /var/www/simplestreams/images
```
### Subcommands ###
#### init ####
Init creates default configuration needed for the server. See Installation
section for more info.
#### Update ####
Update recreates all the metadata from scratch, and recalculate the
sha256 info for all the images. This option is only intended as a safeguard or in case
the service is not running.
#### Watch ####
Watch will start the monitoring of the directory. It is intended to be
used only if the service is not running.
How to use my new server?
-------------------------
### How to add server as lxd remote ###
Once your own image server is running, you can add it as new remote on lxc:
```bash
lxc remote add --protocol=simplestreams
```
Remember to add the certificate:
If behind a SSL terminating proxy, skip this step, and you can run 'init' step with '--ssl_skip'
```bash
openssl s_client -showcerts -connect :443 /dev/null | openssl x509 -outform PEM > my-lxd-image-server.cert
cp my-lxc-image-server.cert /user/local/share/ca-certificates
update-ca-certificates
systemctl restart lxd
```
Also, you can use https://letsencrypt.org/ to make your server easier to set up.
### Publish a new image ###
Create a image container and publish:
```bash
lxc launch lxc:ubuntu/bionic/amd64 n1
lxc exec n1 -- apt-get -y install vim
lxc stop n1
lxc publish --public n1 --alias=bionic-vim
lxc image copy bionic-vim
```
Now, you can use your image in a new container
```bash
lxc launch bionic-vim ntest
lxc exec ntest -- vim -c "smile"
```