Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/stackstorm/st2-packages
StackStorm deb/rpm packages (automated docker build pipeline)
https://github.com/stackstorm/st2-packages
circleci deb docker docker-compose rhel8 rhel9 rpm st2 stackstorm ubuntu2004 ubuntu2204
Last synced: 6 days ago
JSON representation
StackStorm deb/rpm packages (automated docker build pipeline)
- Host: GitHub
- URL: https://github.com/stackstorm/st2-packages
- Owner: StackStorm
- Created: 2015-06-26T13:20:51.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2024-11-13T16:27:33.000Z (2 months ago)
- Last Synced: 2025-01-08T09:09:11.994Z (14 days ago)
- Topics: circleci, deb, docker, docker-compose, rhel8, rhel9, rpm, st2, stackstorm, ubuntu2004, ubuntu2204
- Language: Shell
- Homepage: https://stackstorm.com/
- Size: 2.26 MB
- Stars: 28
- Watchers: 21
- Forks: 58
- Open Issues: 35
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
Awesome Lists containing this project
README
# Stackstorm packages build environment
[](https://circleci.com/gh/StackStorm/st2-packages)
[](https://hub.docker.com/r/stackstorm/)
[](https://packagecloud.io/StackStorm/)
## Highlights
- **Docker based**. Leveraging docker it's possible to deliver packages for any OS distro in a fast and reliable way. *Use the latest Docker version with a Docker Compose plugin that supports V2 syntax.*
- [Rake](https://github.com/ruby/rake) + [sshkit](https://github.com/capistrano/sshkit)-based execution enables easy configuration via **simple DSL** and brings **parallel task processing** out of the box.
- **Test-driven workflow**. Artifacts built are not only available for any enabled OS distro but at the same time tested on a bunch of platforms, providing feedback such as can be installed, services can start up, operations can be executed etc.
## Overview
Packages build environment is a _multi-container docker_ application defined and managed with [docker-compose](https://github.com/docker/compose). It consists of four types of containers:
- **Packaging runner** (https://quay.io/stackstorm/packagingrunner) - the main entry point, package build and test processing controller container.
- **Packaging build** (https://hub.docker.com/r/stackstorm/packagingbuild/) - container where actual `.deb`/`.rpm` artifacts build takes place. It's used to bring up the build environment specific for OS distro. This means that different containers are available such as _packagingbuild:rocky8_, _packagingbuild:focal_, _packagingbuild:jammy_, correspondingly for RockyLinux 8, Ubuntu Focal, and Ubuntu Jammy.
- **Packaging test** (https://hub.docker.com/r/stackstorm/packagingtest/) - containers where built artifacts are tested, i.e. _artifacts are installed, configuration is written and tests are performed_.
- **Services** - these are different containers required for testing such as _rabbitmq and mongodb_
`Dockerfiles` sources are available at [StackStorm/st2-dockerfiles](https://github.com/stackstorm/st2-dockerfiles).
The Packages build environment compose application brings a self-sufficient pipeline to deliver ready to use packages.
# Usage
It's very simple to invoke the whole build-test pipeline. First just make sure that [docker-compose.yml](docker-compose.yml) has your distro specification, after that issue the following commands:
```shell
# (Optional) First clean out previous build containers
docker compose kill
docker compose rm -f
# To build packages for ubuntu focal (--rm will wipe packaging runner container. All others will remain active).
docker compose run --rm focal
```
Execution takes a while, so grab a cup of tea or coffee and wait until it finishes. When build and test processes succeed, you'll find the StackStorm packages in `/tmp/st2-packages` on your host machine:
```shell
ls -l1 | grep ".deb$"
-rw-r--r-- 1 root root 30872652 Feb 9 18:32 st2_1.4dev-1_amd64.deb
```
## Manual testing inside the docker environment
After the build and test stages are finished all docker containers remain active, so you are welcome to do more in-depth testing if desired. To do so simply run:
```shell
docker ps
# Find the required testing container
# In our case it will be st2packages_jammytest_1
# Simply exec to docker
docker exec -it st2packages_jammytest_1 bash
```
Once done, you are inside the testing environment where all services are up and running. Don't forget to do (after exec):
```shell
export TERM=xterm
```
At this point you can do any manual testing which is required.
# Vagrant based build and test
In order to build, package, install and test ST2 in an isolated Vagrant VM, run the following:
```shell
vagrant up $TARGET
```
Where `$TARGET` is one of `focal`, `jammy`, `el8` or `el9`. If you are using `el8`, comment
out the `vm_config.vm.provision :docker` line in the Vagrantfile. There is logic in `setup-vagrant.sh`
to install docker in `el8`.
The following steps are run while provisioning the Vagrant VM:
1. Install `docker` that includes `docker compose` V2.
2. Run `docker compose run --rm $TARGET` to build, test and package ST2 as described in prior
sections.
3. Install the packages built in step 2, unless the host `$ST2_INSTALL` environment variable is set to
a value other than `yes`.
4. Execute the `st2-self-check` script, unless the host `$ST2_VERIFY` environment variable is set to
a value other than `yes`.
As currently implemented, it is not possible to bypass steps 1 and 2. In the future, we may want to
consider allowing the host to provide existing ST2 packages, and install/self-check those in the
Vagrant VM.
To specify the ST2 source URL and REV (i.e., branch), use `ST2_GITURL` and `ST2_GITREV` environment
variables on the host prior to provisioning the VM.
Prior to running `st2-self-check`, the required auth token is generated using `st2 auth`. If necessary,
you can change the default username and password passed to `st2 auth`. To do this, set the `ST2USER`
and `ST2PASSWORD` environment variables on the host prior to provisioning the VM. The default values
are `st2admin` and `Ch@ngeMe` respectively.
# Installation
Current community packages are hosted on https://packagecloud.io/StackStorm. For detailed instructions how install st2 and perform basic configuration follow these instructions:
- [Ubuntu/Debian](https://docs.stackstorm.com/install/deb.html)
- [RHEL8/RockyLinux8](https://docs.stackstorm.com/install/rhel8.html)
## Adding Support For a New Distribution
If you are adding support for a new distribution for which `packagingbuild` and `packagingtest`
images are not yet published to Docker Hub and you want to test the build pipeline locally, you
need to update `docker-compose.yml` file to use locally built Docker images.
For example:
```yaml
services:
...
jammy:
...
image: quay.io/stackstorm/packagingrunner
...
...
jammybuild:
...
image: jammybuild
...
...
jammytest:
...
image: jammytest
...
```
NOTE: Main ``distro`` definition (e.g. ``focal``, ``el8`` etc.) needs to use packaging runner image.
As you can see, `image` attribute references local image tagged `jammybuild` instead of a
remote image (e.g. `stackstorm/packagingbuild:jammy` or similar).
Before that will work, you of course also need to build those images locally.
For example:
```bash
cd ~/st2packaging-dockerfiles/packagingbuild/jammy
docker build -t jammybuild .
cd ~/st2packaging-dockerfiles/packagingtest/jammy/systemd
docker build -t jammytest .
```
# License and Authors
- Author:: StackStorm (st2-dev) ()
- Author:: Denis Baryshev ()