https://github.com/ad-freiburg/gantry
Gantry is a pipeline management tool using containers as its building blocks
https://github.com/ad-freiburg/gantry
docker-compose pipeline
Last synced: 2 months ago
JSON representation
Gantry is a pipeline management tool using containers as its building blocks
- Host: GitHub
- URL: https://github.com/ad-freiburg/gantry
- Owner: ad-freiburg
- License: apache-2.0
- Created: 2019-02-07T14:46:31.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2023-04-04T16:56:38.000Z (about 3 years ago)
- Last Synced: 2025-08-14T16:50:00.372Z (11 months ago)
- Topics: docker-compose, pipeline
- Language: Go
- Homepage:
- Size: 263 KB
- Stars: 1
- Watchers: 3
- Forks: 2
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Gantry (Container Crane)
[](https://godoc.org/github.com/ad-freiburg/gantry)
[](./LICENSE)
[](https://travis-ci.org/ad-freiburg/gantry)
[](https://goreportcard.com/report/github.com/ad-freiburg/gantry)
[](https://github.com/ad-freiburg/gantry/releases)
[](https://aur.archlinux.org/packages/gantry)
Gantry is a pipeline management tool using containers for all relevant steps.
It supports a basic `docker-compose` subset allowing `docker-compose` like
deployments with [wharfer](https://github.com/ad-freiburg/wharfer). If `wharfer`
is not installed `docker` will be used directly.
## Services and Steps
Services define docker containers which provide a continued service to other
tasks in and outside of the pipeline. They directly resemble the service
concept from `docker-compose` and once started run concurrently to the rest of
the pipeline including anything depending on the service.
Steps on the other hand run to completion and only then are their dependents
executed. Note however, that independent steps are executed concurrently with
each other. Steps are often used for tasks that produce a result that is
needed by their dependents such as a download, creation of a database index
or training of a machine learning model.
The End-to-End tests of QLever [example](./examples/qlever_e2e) demonstrates
the usage and interaction of both container types.
## Installation
### Download a prebuild Release
Binary releases are provided as
[github releases](https://github.com/ad-freiburg/gantry/releases).
cd /tmp
rm gantry_$(uname -m).tar.bz2
wget https://github.com/ad-freiburg/gantry/releases/download/v0.5.0/gantry_$(uname -m).tar.bz2
tar -xavf gantry_$(uname -m).tar.bz2
sudo mv gantry_$(uname -m)/gantry /usr/local/bin/gantry
gantry --version
### Arch Linux
Gantry is available in the [AUR](https://aur.archlinux.org/) as
[gantry](https://aur.archlinux.org/packages/gantry) and
[gantry-git](https://aur.archlinux.org/packages/gantry-git)
### From source
To install gantry into the users `~/go/bin` path it is enough to just run
go install -ldflags="-X github.com/ad-freiburg/gantry.Version=$(git describe --always --long --dirty)" ./...
To install gantry globally copy it from `~/go/bin` path or use
go build -ldflags="-X github.com/ad-freiburg/gantry.Version=$(git describe --always --long --dirty)" cmd/gantry/gantry.go
sudo mv gantry /usr/local/bin/
gantry --version
### Using go get
Gantry is go getable through
go get github.com/ad-freiburg/gantry/cmd/gantry
This will result in a binary without a set version.
## Building a (new) Release
To build a release version first make sure everything works.
This includes checking the codestyle
gofmt -d .
and running the tests
go test ./...
The complete test coverage can be viewed with
go test -coverprofile=coverage.out ./... && go tool cover -html=coverage.out
When all is correct edit the [Download a prebuild release](#download-a-prebuild-release)
section of this Readme so the download link points to the future version.
*Only after committing this final change tag the release*
git tag -a vX.Y.Z -m
Then build with `-ldflags` such that the version is added to the binary
go build -ldflags="-X github.com/ad-freiburg/gantry.Version=$(git describe --always --long --dirty)" cmd/gantry/gantry.go
Finally use the GitHub Releases mechanism to release a new version.