https://github.com/digtux/laminar
automated (GitOps) tool for docker tag promotion in git
https://github.com/digtux/laminar
docker ecr fluxcd gcr git github gitops google-artifact-registry
Last synced: 6 days ago
JSON representation
automated (GitOps) tool for docker tag promotion in git
- Host: GitHub
- URL: https://github.com/digtux/laminar
- Owner: digtux
- License: apache-2.0
- Created: 2020-07-06T10:35:09.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2023-04-18T15:52:11.000Z (almost 3 years ago)
- Last Synced: 2023-09-05T03:30:30.049Z (over 2 years ago)
- Topics: docker, ecr, fluxcd, gcr, git, github, gitops, google-artifact-registry
- Language: Go
- Homepage:
- Size: 166 KB
- Stars: 8
- Watchers: 4
- Forks: 4
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Laminar
## Don't let your deployments be turbulent!
## What is this?
Laminar is a reasonably simple `GitOps` tool, inspired by things like [flux](https://github.com/fluxcd/flux) but with a very different architecture.
The TL;DR of what it does is this:
Step 1, you tell it about your docker registry(s):
```yaml
dockerRegistries:
- reg: 1122334455.dkr.ecr.eu-west-2.amazonaws.com/acmecorp
name: acmecorp-aws
```
Step 2, you give it (git+ssh) access to your git repo:
```yaml
git:
- name: manifests
url: gitoperations@github.com:acmecorp/k8s-manifests.gitoperations
branch: master
key: ~/example_ssh_id_rsa # path to the SSH key (needed for gitoperations)
pollFreq: 120 # How often to sync..
updates: [] # list of updates (see next step)
```
Step 3, you tell laminar which files you want it to operate on (in the same git repo)
```yaml
git:
- name: manifests
updates:
- pattern: "glob:develop-*" # will match docker tags such as "develop-1.2" or "develop-short_sha"
# where to look in your gitoperations repo
files:
- dev/ # laminar will search in directory "dev"
```
Step 4, run laminar
```shell
AWS_PROFILE=myprofile ./laminar
```
Laminar will then do the following:
- clone the git repo `k8s-manifests` and checkout branch `master`
- search for files under `dev/` inside your git repo
- inside the files search for strings such as: `1122334455.dkr.ecr.eu-west-2.amazonaws.com/acmecorp/app-name:develop-52af76b8` (any tag that matches `develop-*`)
- index the tags available from `1122334455.dkr.ecr.eu-west-2.amazonaws.com/acmecorp/app-name`
- check if there are any tags (matching `develop-*`) which are more recent than `develop-52af76b8`
- a. if there is a more recent tag in ECR (eg: `develop-b2ee56fd`) update the file from `develop-52af76b8` -> `develop-b2ee56fd`
- b: git commit the updated file(s), then git push
- sleep until the end of `pollFreq`
- repeat
### Main loop sequence
```mermaid
sequenceDiagram
participant laminar as Laminar
participant git as Git
participant dr as Docker Registry
loop every interval
loop for each git repo
laminar->>git: clone "k8s-manifest"
laminar->>laminar: checkout "branch"
laminar->>laminar: search files for string matching "pattern"
end
loop for each docker registry
laminar->>dr: get available images tags
end
loop for each git repo
alt new tag found
laminar->>laminar: replace tag
laminar->>laminar: git commit
laminar->>git: push
end
end
end
```
# Reasoning
We love weave flux.. but it makes working with templated manifests challenging. If you're running 10x kubernetes clusters it also makes very little sense to have each one polling your docker registries.
- With this pattern you only need one tool to automate your GitOps.
- This is also compatible with docker-compose or practically any text file really.
- easier to co-ordinate many changes at the same time.
- Lets assume my manifests are templated in my git repo.. flux would want to patch the file that is the output of my templating. With laminar you can update both, or just one. This opens the door to using `kapitan`, `tanka`, `cuelang` or any templating really.
# Features
- [x] checkout your git repo(s)
- [x] search yaml files looking for obvious docker images
- [x] check if there are more recent docker tags are in your registry and ready to be deployed
- [x] update your git repo with the more recent tags
- [x] dynamically load a list of files and image:tag patterns from the remote git repos (`.laminar.yaml`)
- [x] add `exec` action so commands can be run after modifying git (and before the `git commit`)
- [x] docker image + deployment manifest
- [ ] prometheus metrics
# issues/todo
- [ ] exclude list (users can blacklist promoting specific image+tag patterns).. feature untested
- [ ] after initialCheckout(), if the (remote) git repo is reverted with a `--force` push we should handle that and re-clone
- [ ] more tests, do this when refactoring the logic
- [ ] the main loop is currently (MVP) and simply just a `time.Sleep()`. There is no concurrnecy/`time.Tick()` yet.
- [ ] occasional errors from registry polling not surfacing correctly. probably needs some attention.
- [ ] tidy up (specifically the business logic around change requests and add maybe add some concurrency)
- [ ] quick start/tutorial/example docs!
- [ ] example: PrometheusAlerts
- [ ] example: grafana dashboard
## optional behaviours
- [x] glob filters on tags (eg `master-*` )
- [x] only operate on specific files or directories in your git repo
- [x] multiple git repos (not tested well)
- [ ] built in "post sync" `actions` such as: "slack alert", "github PR"
- [x] user configurable `actions` such as running a shell script to re-render charts
- [ ] user adjustable file exclude list
- [ ] an api endpoint that can trigger a sync (so your CI can hit it after pushing a new image)
- [ ] a simple gui with some info about tags and images
- [ ] individual auth configuration available for registries (allowing support for multiple GCR and ECR)
- [x] other tag matching patterns, specifically: `regex`
- [ ] other tag matching patterns, specifically: `semver`