Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/akshaymankar/dhall-concourse
Library to type check concourse configurations in dhall
https://github.com/akshaymankar/dhall-concourse
concourse dhall dhall-lang
Last synced: 24 days ago
JSON representation
Library to type check concourse configurations in dhall
- Host: GitHub
- URL: https://github.com/akshaymankar/dhall-concourse
- Owner: akshaymankar
- Created: 2018-11-29T17:17:24.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2021-07-08T14:41:37.000Z (over 3 years ago)
- Last Synced: 2023-03-12T06:03:08.456Z (over 1 year ago)
- Topics: concourse, dhall, dhall-lang
- Language: Dhall
- Homepage:
- Size: 209 KB
- Stars: 19
- Watchers: 2
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Dhall Concourse
Concourse types and helpers in dhall.
Dhall Concourse provides Dhall bindings for Concourse, so you can generate concourse pipelines from Dhall expressions. This lets the pipelines be easily typechecked, templated and modularized.
## Why do I need this?
There are a lot of issues one could face while building any non-trivial pipeline. Few of them could be:
1. Pipeline yaml becomes very big and unmanageable
2. Same set of jobs are required to be run in different environments
3. Same set of hooks but with slight changes in all jobs. E.g. slack notifications, releasing resources on failure, etc.
Most common ways to deal with these have been to use a templating language like erb or tools like spruce. But this gets very messy very fast. We can do a lot better with a typed total language. I'll let [dhall speak for itself](https://dhall-lang.org/).
## Usage
### Using dhall-fly
One way to translate pipelines written in dhall-concourse into yaml is using [dhall-fly](https://github.com/akshaymankar/dhall-fly#installation).
### Using dhall-to-json and jq
#### Jobs without Groups
To use native rendering to render a list of jobs in a file called `jobs.dhall`, you'd have to write a dhall expression like this:
```dhall
let Concourse =
https://raw.githubusercontent.com/akshaymankar/dhall-concourse/0.6.0/package.dhall
let jobs = ./jobs.dhall
in Concourse.render.pipeline jobs
```
Now you can render this using dhall-to-json and jq like this:
```bash
dhall-to-json <<< './pipeline.dhall' \
| jq '.resources = (.resources|unique)' \
| jq '.resource_types = (.resource_types|unique)'
```
#### Jobs with groups
Similarly, to render a list of `GroupedJob`s in a filed called `grouped-jobs.dhall`, this would be the expression to render:
```dhall
let Concourse =
https://raw.githubusercontent.com/akshaymankar/dhall-concourse/0.6.0/package.dhall
let groupedJobs = ./grouped-jobs.dhall
in Concourse.render.groupedJobs groupedJobs
```
Now you can render this using dhall-to-json and jq like this:
```bash
dhall-to-json <<< './pipeline.dhall' \
| jq '.resources = (.resources|unique)' \
| jq '.resource_types = (.resource_types|unique)' \
| jq '.groups = (.groups | group_by(.name) | map({name: .[0].name, jobs: (map(.jobs) | flatten) }))'
```
## Defining a pipeline
### Example 1: Hello World
This dhall expression will create a pipeline with one job, which would have one task. The task would run in a busybox container and echo "Hello Dhall".
```dhall
let Concourse =
https://raw.githubusercontent.com/akshaymankar/dhall-concourse/0.5.0/package.dhall
let Prelude =
https://prelude.dhall-lang.org/v11.1.0/package.dhall sha256:99462c205117931c0919f155a6046aec140c70fb8876d208c7c77027ab19c2fa
let busyboxImage =
Concourse.schemas.ImageResource::{
, type = "docker-image"
, source = Some (toMap { repository = Prelude.JSON.string "busybox" })
}
let job =
Concourse.schemas.Job::{
, name = "hello"
, plan =
[ Concourse.helpers.taskStep
Concourse.schemas.TaskStep::{
, task = "hello"
, config =
Concourse.Types.TaskSpec.Config
Concourse.schemas.TaskConfig::{
, image_resource = Some busyboxImage
, run =
Concourse.schemas.TaskRunConfig::{
, path = "bash"
, args = Some [ "-c", "echo Hello Dhall" ]
}
}
}
]
}
in [ job ]
```
To set the pipeline, run this command:
```
fly -t set-pipeline -p hello-dhall -c <(dhall-fly set-pipeline -p hello-dhall -c <(dhall-fly --pipeline-type grouped-jobs