Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/srfrnk/jabos
Just Another Boring Ops System - Jabos attempts to be a fully automated K8s GitOps framework.
https://github.com/srfrnk/jabos
ci-cd continuous-delivery docker-image github gitops jsonnet k8s kubernetes kubernetes-operator
Last synced: about 2 months ago
JSON representation
Just Another Boring Ops System - Jabos attempts to be a fully automated K8s GitOps framework.
- Host: GitHub
- URL: https://github.com/srfrnk/jabos
- Owner: srfrnk
- License: mit
- Created: 2021-09-25T09:10:42.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2024-10-13T23:35:33.000Z (2 months ago)
- Last Synced: 2024-10-15T23:28:19.002Z (2 months ago)
- Topics: ci-cd, continuous-delivery, docker-image, github, gitops, jsonnet, k8s, kubernetes, kubernetes-operator
- Language: TypeScript
- Homepage:
- Size: 73.6 MB
- Stars: 3
- Watchers: 3
- Forks: 1
- Open Issues: 46
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Security: SECURITY.md
Awesome Lists containing this project
README
# jabos
Jabos attempts to be a fully automated K8s GitOps framework.
**This is WIP** - any comments, requests or issues would be welcome! please use this link
## TL;DR - What does that mean?
### What you need to do?
1. Installing Jabos into your K8s cluster using
1. Setting up the K8s objects for your
- Git Repository
- Docker images
- Manifest folder
### What happens next?
1. Any new commits would be picked up from Git automatically
1. Docker images would get build from new commits and pushed automatically
1. New manifest versions would be deployed automatically
## Goals
- Automate all steps to deploy from Git repositories into a K8s cluster
- No GUI requiring manual human intervention
- Git as a single source of truth
- Pull only model from within runtime environments.
- Isolation of build environment from runtime environment
- Idempotent builds
- Preview environment + Pre-deploy integration testing
## Guidelines
- Minimal set of tools/technologies as pre-requisites/installs
- Stay as tech-stack agnostic as possible
- Minimal steps to install or setup development environment. Automated as possible.
## Installation
Video version:
1. Minikube setup
1. Jabos and prerequisites
Instructions:
1. Make sure Metacontroller is installed on your cluster. Find instructions here
1. Optionally install kube-prometheus-stack to expose metrics from `jabos`
1. Optionally install grafana-dashboard-operator to setup grafana dashboards for `jabos`
1. Create a namespace for jabos to use. E.g. `kubectl create namespace jabos`. Use the same namespace with the next command.
1. Run `kubectl apply -n -f https://github.com/srfrnk/jabos/releases/latest/download/jabos-manifests.yaml`
## Usage
Jabos uses CRDs in order for users to define a codebase and how to build and deploy that.
See API Docs here
Video version:
1. Configure CRDs
1. Push changes and trigger builds
### Resource Status
All resources show current status using `Status Sub-resource` and `Event Resources`.
These can be viewed as with any K8s resource. i.e. `kubectl describe git-repositories.jabos.io`
`GitRepository` resources have a `Syncing` condition in the status.
If it becomes `False` an `Event` will describe the error.
They also have a `Latest Commit`(`latestCommit`) status containing the latest `git` commit id found.
`DockerImage` and `***Manifest` resources have a `Synced` condition in the status.
If it becomes `False` an `Event` will describe the error.
They also have a `Latest Commit`(`latestCommit`) status containing the latest `git` commit id found
and a `Built Commit`(`builtCommit`) status containing the `git` commit id last built.
### Jsonnet example
Create a file `example.jsonnet`:
```jsonnet
function(latestCommitId) {
apiVersion: 'apps/v1',
kind: 'Deployment',
metadata: {
name: 'test-deployment',
labels: {
app: 'test-deployment',
},
},
spec: {
replicas: 1,
selector: {
matchLabels: {
app: 'test-deployment',
},
},
template: {
metadata: {
labels: {
app: 'test-deployment',
},
},
spec: {
containers: [
{
name: 'test-deployment',
image: 'registry.kube-system:80/example-image:' + latestCommitId,
},
],
},
},
},
}
```
### Git Repository Authentication
#### GitHub
##### Using SSH Keys
1. Create an SSH key and add it to GitHub - **Optionally skip the "adding it to the ssh-agent" section.**
1. Create a secret with the passphrase and key created in the previous step. (i.e. `kubectl create secret generic -n example-env first-repo-private --from-file=git_ssh_passphrase=./build/passphrase --from-file=git_ssh_key=./build/key`)
1. Add an `ssh` property to each applicable `GitRepository` object to point to the secret.
### Image Registry Authentication
#### Docker Hub
1. Obtain your Docker Hub username.
1. Obtain your Docker Hub password or access token.
1. Create secret with the credentials. (i.e. `kubectl create secret generic -n example-env docker-hub --from-file=docker_hub_username=./build/docker_hub_username --from-file=docker_hub_password=./build/docker_hub_password`)
1. Add a `dockerHub` property to any applicable `DockerImage` object to point to the secret.
#### GCP (GCR and Artifact Registry)
1. Obtain a Service Account with the required permissions.
1. Obtain the Service Account JSON key.
1. Create secret with the JSON key. (i.e. `kubectl create secret generic -n example-env gcp --from-file=gcp_service_account.json=./build/gcp_service_account.json`)
1. Add a `gcp` property to any applicable `DockerImage` object to point to the secret.
#### AWS (ECR)
1. Obtain an Access Key with the required permissions.
1. Obtain the `Access key ID` and `Secret Access Key`.
1. Create secret with these credentials. (i.e. `kubectl create secret generic -n example-env aws --from-file=aws_access_key_id=./build/aws_access_key_id --from-file=aws_secret_access_key=./build/aws_secret_access_key`)
1. Add a `aws` property to any applicable `DockerImage` object to point to the secret.
**Note**: You can use instance roles instead when pushing to ECR from a EC2 instance or from EKS, by [configuring the instance role permissions](https://docs.aws.amazon.com/AmazonECR/latest/userguide/ECR_on_EKS.html).
#### Metrics
All metrics are exported into `Prometheus` using the `ServiceMonitor` API by `kube-prometheus-stack`.
To otherwise configure `Prometheus` to collect the metrics you need to point it to 'OPERATOR_POD_IP:3000/metrics'.
All metrics exported are prefixed with `jabos_operator_`.
Numerous metrics are exported most of them describe `nodsjs` and `expresjs` operations. [**Removed due to security audit fails**](#34)
Important metrics for the operation of Jabos are:
```yaml
# HELP jabos_operator_latest_commit_changed new "latest commit" detected for git repository
# TYPE jabos_operator_latest_commit_changed counter
# HELP jabos_operator_docker_image_build_trigger new build triggered for a docker image
# TYPE jabos_operator_docker_image_build_trigger counter
# HELP jabos_operator_jsonnet_manifests_build_trigger new build triggered for jsonnet manifests
# TYPE jabos_operator_jsonnet_manifests_build_trigger counter
# HELP jabos_operator_git_repository_updater_start GitRepositoryUpdater start
# TYPE jabos_operator_git_repository_updater_start counter
# HELP jabos_operator_git_repository_updater_end GitRepositoryUpdater end
# TYPE jabos_operator_git_repository_updater_end counter
# HELP jabos_operator_git_repository_updater_duration GitRepositoryUpdater duration
# TYPE jabos_operator_git_repository_updater_duration gauge
# HELP jabos_operator_docker_image_builder_start DockerImageBuilder start
# TYPE jabos_operator_docker_image_builder_start counter
# HELP jabos_operator_docker_image_builder_end DockerImageBuilder end
# TYPE jabos_operator_docker_image_builder_end counter
# HELP jabos_operator_docker_image_builder_duration DockerImageBuilder duration
# TYPE jabos_operator_docker_image_builder_duration gauge
# HELP jabos_operator_jsonnet_manifests_builder_start JsonnetManifestsBuilder start
# TYPE jabos_operator_jsonnet_manifests_builder_start counter
# HELP jabos_operator_jsonnet_manifests_builder_end JsonnetManifestsBuilder end
# TYPE jabos_operator_jsonnet_manifests_builder_end counter
# HELP jabos_operator_jsonnet_manifests_builder_duration JsonnetManifestsBuilder duration
# TYPE jabos_operator_jsonnet_manifests_builder_duration gauge
```
## Use Cases
Diagrams for supported and future planned use-cases [are here](https://miro.com/app/board/uXjVOY5CIn0=)
### Image Reuse
Build images in DEV/QA only and reuse when commit is promoted to other environments.
To mark a `DockerImage` for reuse of an image built in another environnement add `build: false` to the spec.
## Security
`Jabos` images and manifest are being scanned by [`CodeQL`](https://codeql.github.com/) and [`Snyk`](https://snyk.io/) as part of the release process using GitHub Actions.
`Jabos` makes no attempt at protecting applications, networks, disks from malicious access. It is the responsibility of the user to put in place such measures.
`Jabos` should always be contained inside a dedicated namespace to reduce risk to other systems.
**Special attention** must be given to the `Jabos` docker image builder pods which use `Kaniko`. At this time it is required for `Kaniko` to be executed with `root` user and with a writable file system. This known limitation is a low risk as these pods have a very short life span... however it does pose a risk especially when the code pulled from a `Git` repository may contain vulnerabilities.
**It is advisable to always scan all code which is pulled by `Jabos` from `Git`!**
**It is advisable to use `NetworkPolicy` and other methods to ensure any egress from docker image builder pods is limited to what is required by your images to build!**
### Security Overview
The scan results can be found [here](https://github.com/srfrnk/jabos/security)
As of version 1.x there are no known vulnerabilities.
### Reporting a Vulnerability
Create an issue [here](https://github.com/srfrnk/jabos/issues).
Please add a `security` label for quicker response.
## Development
### Prerequisites
1. `make` installed (Depending on your OS - start here)
1. `docker` installed (To install see here)
1. `minikube` installed (To install minikube see this)
1. `NodeJS` installed (To install NodeJS see this)
1. `Typescript` development tools installed `npm install -g ts-node typescript '@types/node'`
1. `GNU Parallel` installed for your OS. For Debian based you can use `sudo apt-get install parallel`.
1. `K9s` installed (To install see here). For automated port forwarding set [K9s configuration](~/.config/k9s/config.yml) with `scanForAutoPf: true`. Make sure K9s version supports the feature (https://github.com/derailed/k9s/pull/1498).
### Environment Setup
1. Clone repo: `git clone [email protected]:srfrnk/jabos.git` (or using HTTPS/GitHub CLI - see instructions here)
1. CD into folder
1. Start a minikube cluster `minikube start`
1. Run `make setup` once
1. Run `make build` after each code change
1. To deploy examples
1. Locally clone jabos-examples repo
1. Follow instructions from the README file in the cloned folder
1. The examples would be deployed into namespace `example-env`
## Credits
- Jabos uses the kaniko project to build docker images inside the kubernetes cluster.
- Jabos uses yq to parse and update yaml and json data.
- Jabos uses jsonnet to process jsonnet templates and create K8s manifests.
- Jabos uses minikube for local development
- Jabos uses metacontroller to control K8s operators.
- Jabos uses efk-stack-helm for local centralized logging.
- Jabos uses kube-prometheus-stack for local monitoring and alerting.
- Jabos uses GNU Parallel for local port-forwarding to multiple services
- Jabos uses expressjs as the web server to run the operator
- [**Removed due to security audit fails**](#34) - Jabos uses express-prometheus-middleware to export basic metrics to prometheus