{"id":17170074,"url":"https://github.com/srfrnk/jabos","last_synced_at":"2025-04-13T12:27:20.085Z","repository":{"id":40274057,"uuid":"410226522","full_name":"srfrnk/jabos","owner":"srfrnk","description":"Just Another Boring Ops System - Jabos attempts to be a fully automated K8s GitOps framework.","archived":false,"fork":false,"pushed_at":"2024-12-19T14:53:48.000Z","size":77309,"stargazers_count":3,"open_issues_count":47,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-13T12:27:11.996Z","etag":null,"topics":["ci-cd","continuous-delivery","docker-image","github","gitops","jsonnet","k8s","kubernetes","kubernetes-operator"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/srfrnk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-25T09:10:42.000Z","updated_at":"2023-10-17T07:25:38.000Z","dependencies_parsed_at":"2024-03-27T04:37:26.766Z","dependency_job_id":"eec267cf-13eb-4bd5-9526-e533d786ffd7","html_url":"https://github.com/srfrnk/jabos","commit_stats":null,"previous_names":[],"tags_count":83,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srfrnk%2Fjabos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srfrnk%2Fjabos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srfrnk%2Fjabos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/srfrnk%2Fjabos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/srfrnk","download_url":"https://codeload.github.com/srfrnk/jabos/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248713403,"owners_count":21149678,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ci-cd","continuous-delivery","docker-image","github","gitops","jsonnet","k8s","kubernetes","kubernetes-operator"],"created_at":"2024-10-14T23:28:29.547Z","updated_at":"2025-04-13T12:27:20.022Z","avatar_url":"https://github.com/srfrnk.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# jabos\n\n![build](https://github.com/srfrnk/jabos/actions/workflows/push.yml/badge.svg?branch=main)\n![GitHub release](https://img.shields.io/github/v/release/srfrnk/jabos?label=latest%20release\u0026style=plastic)\n![License](https://img.shields.io/badge/License-MTA-blue.svg)\n\nJabos attempts to be a fully automated K8s GitOps framework.\n\n**This is WIP** - any comments, requests or issues would be welcome! please use \u003ca href=\"https://github.com/srfrnk/jabos/issues\" target=\"_blank\"\u003ethis link\u003c/a\u003e\n\n## TL;DR - What does that mean?\n\n### What you need to do?\n\n1. Installing Jabos into your K8s cluster using\n1. Setting up the K8s objects for your\n   - Git Repository\n   - Docker images\n   - Manifest folder\n\n### What happens next?\n\n1. Any new commits would be picked up from Git automatically\n1. Docker images would get build from new commits and pushed automatically\n1. New manifest versions would be deployed automatically\n\n## Goals\n\n- Automate all steps to deploy from Git repositories into a K8s cluster\n- No GUI requiring manual human intervention\n- Git as a single source of truth\n- Pull only model from within runtime environments.\n- Isolation of build environment from runtime environment\n- Idempotent builds\n- Preview environment + Pre-deploy integration testing\n\n## Guidelines\n\n- Minimal set of tools/technologies as pre-requisites/installs\n- Stay as tech-stack agnostic as possible\n- Minimal steps to install or setup development environment. Automated as possible.\n\n## Installation\n\nVideo version:\n\n1. \u003ca href=\"https://youtu.be/616aMiKHtks\" target=\"_blank\"\u003eMinikube setup\u003c/a\u003e\n1. \u003ca href=\"https://youtu.be/Ex5hi3GOkjg\" target=\"_blank\"\u003eJabos and prerequisites\u003c/a\u003e\n\nInstructions:\n\n1. Make sure Metacontroller is installed on your cluster. Find instructions \u003ca href=\"https://metacontroller.github.io/metacontroller/guide/install.html\" target=\"_blank\"\u003ehere\u003c/a\u003e\n1. Optionally install \u003ca href=\"https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack\" target=\"_blank\"\u003ekube-prometheus-stack\u003c/a\u003e to expose metrics from `jabos`\n1. Optionally install \u003ca href=\"https://github.com/srfrnk/grafana-dashboard-operator/\" target=\"_blank\"\u003egrafana-dashboard-operator\u003c/a\u003e to setup grafana dashboards for `jabos`\n1. Create a namespace for jabos to use. E.g. `kubectl create namespace jabos`. Use the same namespace with the next command.\n1. Run `kubectl apply -n \u003cNAMESPACE\u003e -f https://github.com/srfrnk/jabos/releases/latest/download/jabos-manifests.yaml`\n\n## Usage\n\nJabos uses \u003ca href=\"https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/\" target=\"_blank\"\u003eCRDs\u003c/a\u003e in order for users to define a codebase and how to build and deploy that.\n\nSee \u003ca href=\"https://srfrnk.github.io/jabos\" target=\"_blank\"\u003eAPI Docs here\u003c/a\u003e\n\nVideo version:\n\n1. \u003ca href=\"https://youtu.be/PqMUliEHx60\" target=\"_blank\"\u003eConfigure CRDs\u003c/a\u003e\n1. \u003ca href=\"https://youtu.be/OlB6qybsqng\" target=\"_blank\"\u003ePush changes and trigger builds\u003c/a\u003e\n\n### Resource Status\n\nAll resources show current status using `Status Sub-resource` and `Event Resources`.\nThese can be viewed as with any K8s resource. i.e. `kubectl describe git-repositories.jabos.io`\n\n`GitRepository` resources have a `Syncing` condition in the status.\nIf it becomes `False` an `Event` will describe the error.\nThey also have a `Latest Commit`(`latestCommit`) status containing the latest `git` commit id found.\n\n`DockerImage` and `***Manifest` resources have a `Synced` condition in the status.\nIf it becomes `False` an `Event` will describe the error.\nThey also have a `Latest Commit`(`latestCommit`) status containing the latest `git` commit id found\nand a `Built Commit`(`builtCommit`) status containing the `git` commit id last built.\n\n### Jsonnet example\n\nCreate a file `example.jsonnet`:\n\n```jsonnet\nfunction(latestCommitId) {\n  apiVersion: 'apps/v1',\n  kind: 'Deployment',\n  metadata: {\n    name: 'test-deployment',\n    labels: {\n      app: 'test-deployment',\n    },\n  },\n  spec: {\n    replicas: 1,\n    selector: {\n      matchLabels: {\n        app: 'test-deployment',\n      },\n    },\n    template: {\n      metadata: {\n        labels: {\n          app: 'test-deployment',\n        },\n      },\n      spec: {\n        containers: [\n          {\n            name: 'test-deployment',\n            image: 'registry.kube-system:80/example-image:' + latestCommitId,\n          },\n        ],\n      },\n    },\n  },\n}\n```\n\n### Git Repository Authentication\n\n#### GitHub\n\n##### Using SSH Keys\n\n1. \u003ca href=\"https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account\" target=\"_blank\"\u003eCreate an SSH key and add it to GitHub\u003c/a\u003e - **Optionally skip the \"adding it to the ssh-agent\" section.**\n1. Create a secret with the passphrase and key created in the previous step. (i.e. `kubectl create secret generic -n example-env first-repo-private --from-file=git_ssh_passphrase=./build/passphrase --from-file=git_ssh_key=./build/key`)\n1. Add an `ssh` property to each applicable `GitRepository` object to point to the secret.\n\n### Image Registry Authentication\n\n#### Docker Hub\n\n1. Obtain your Docker Hub username.\n1. Obtain your Docker Hub password or \u003ca href=\"https://docs.docker.com/docker-hub/access-tokens/\" target=\"_blank\"\u003eaccess token\u003c/a\u003e.\n1. Create secret with the credentials. (i.e. `kubectl create secret generic -n example-env docker-hub --from-file=docker_hub_username=./build/docker_hub_username --from-file=docker_hub_password=./build/docker_hub_password`)\n1. Add a `dockerHub` property to any applicable `DockerImage` object to point to the secret.\n\n#### GCP (GCR and Artifact Registry)\n\n1. Obtain a Service Account with the required permissions.\n1. Obtain the Service Account JSON key.\n1. Create secret with the JSON key. (i.e. `kubectl create secret generic -n example-env gcp --from-file=gcp_service_account.json=./build/gcp_service_account.json`)\n1. Add a `gcp` property to any applicable `DockerImage` object to point to the secret.\n\n#### AWS (ECR)\n\n1. Obtain an \u003ca href=\"https://aws.amazon.com/premiumsupport/knowledge-center/create-access-key/\" target=\"_blank\"\u003eAccess Key\u003c/a\u003e with the required permissions.\n1. Obtain the `Access key ID` and `Secret Access Key`.\n1. Create secret with these credentials. (i.e. `kubectl create secret generic -n example-env aws --from-file=aws_access_key_id=./build/aws_access_key_id --from-file=aws_secret_access_key=./build/aws_secret_access_key`)\n1. Add a `aws` property to any applicable `DockerImage` object to point to the secret.\n\n**Note**:  You can use instance roles instead when pushing to ECR from a EC2 instance or from EKS, by [configuring the instance role permissions](https://docs.aws.amazon.com/AmazonECR/latest/userguide/ECR_on_EKS.html).\n\n#### Metrics\n\nAll metrics are exported into `Prometheus` using the `ServiceMonitor` API by `kube-prometheus-stack`.\nTo otherwise configure `Prometheus` to collect the metrics you need to point it to 'OPERATOR_POD_IP:3000/metrics'.\n\nAll metrics exported are prefixed with `jabos_operator_`.\nNumerous metrics are exported most of them describe `nodsjs` and `expresjs` operations. [**Removed due to security audit fails**](#34)\n\nImportant metrics for the operation of Jabos are:\n\n```yaml\n# HELP jabos_operator_latest_commit_changed new \"latest commit\" detected for git repository\n# TYPE jabos_operator_latest_commit_changed counter\n\n# HELP jabos_operator_docker_image_build_trigger new build triggered for a docker image\n# TYPE jabos_operator_docker_image_build_trigger counter\n\n# HELP jabos_operator_jsonnet_manifests_build_trigger new build triggered for jsonnet manifests\n# TYPE jabos_operator_jsonnet_manifests_build_trigger counter\n\n# HELP jabos_operator_git_repository_updater_start GitRepositoryUpdater start\n# TYPE jabos_operator_git_repository_updater_start counter\n\n# HELP jabos_operator_git_repository_updater_end GitRepositoryUpdater end\n# TYPE jabos_operator_git_repository_updater_end counter\n\n# HELP jabos_operator_git_repository_updater_duration GitRepositoryUpdater duration\n# TYPE jabos_operator_git_repository_updater_duration gauge\n\n# HELP jabos_operator_docker_image_builder_start DockerImageBuilder start\n# TYPE jabos_operator_docker_image_builder_start counter\n\n# HELP jabos_operator_docker_image_builder_end DockerImageBuilder end\n# TYPE jabos_operator_docker_image_builder_end counter\n\n# HELP jabos_operator_docker_image_builder_duration DockerImageBuilder duration\n# TYPE jabos_operator_docker_image_builder_duration gauge\n\n# HELP jabos_operator_jsonnet_manifests_builder_start JsonnetManifestsBuilder start\n# TYPE jabos_operator_jsonnet_manifests_builder_start counter\n\n# HELP jabos_operator_jsonnet_manifests_builder_end JsonnetManifestsBuilder end\n# TYPE jabos_operator_jsonnet_manifests_builder_end counter\n\n# HELP jabos_operator_jsonnet_manifests_builder_duration JsonnetManifestsBuilder duration\n# TYPE jabos_operator_jsonnet_manifests_builder_duration gauge\n```\n\n## Use Cases\n\nDiagrams for supported and future planned use-cases [are here](https://miro.com/app/board/uXjVOY5CIn0=)\n\n### Image Reuse\n\nBuild images in DEV/QA only and reuse when commit is promoted to other environments.\nTo mark a `DockerImage` for reuse of an image built in another environnement add `build: false` to the spec.\n\n## Security\n\n`Jabos` images and manifest are being scanned by [`CodeQL`](https://codeql.github.com/) and [`Snyk`](https://snyk.io/) as part of the release process using GitHub Actions.\n\n`Jabos` makes no attempt at protecting applications, networks, disks from malicious access. It is the responsibility of the user to put in place such measures.\n\n`Jabos` should always be contained inside a dedicated namespace to reduce risk to other systems.\n\n**Special attention** must be given to the `Jabos` docker image builder pods which use `Kaniko`. At this time it is required for `Kaniko` to be executed with `root` user and with a writable file system. This known limitation is a low risk as these pods have a very short life span... however it does pose a risk especially when the code pulled from a `Git` repository may contain vulnerabilities.\n\n**It is advisable to always scan all code which is pulled by `Jabos` from `Git`!**\n\n**It is advisable to use `NetworkPolicy` and other methods to ensure any egress from docker image builder pods is limited to what is required by your images to build!**\n\n### Security Overview\n\nThe scan results can be found [here](https://github.com/srfrnk/jabos/security)\n\nAs of version 1.x there are no known vulnerabilities.\n\n### Reporting a Vulnerability\n\nCreate an issue [here](https://github.com/srfrnk/jabos/issues).\nPlease add a `security` label for quicker response.\n\n## Development\n\n### Prerequisites\n\n1. `make` installed (Depending on your OS - start \u003ca href=\"https://www.gnu.org/software/make/\" target=\"_blank\"\u003ehere\u003c/a\u003e)\n1. `docker` installed (To install see \u003ca href=\"https://www.docker.com/get-started\" target=\"_blank\"\u003ehere\u003c/a\u003e)\n1. `minikube` installed (To install minikube see \u003ca href=\"https://minikube.sigs.k8s.io/docs/start/\" target=\"_blank\"\u003ethis\u003c/a\u003e)\n1. `NodeJS` installed (To install NodeJS see \u003ca href=\"https://nodejs.org\" target=\"_blank\"\u003ethis\u003c/a\u003e)\n1. `Typescript` development tools installed `npm install -g ts-node typescript '@types/node'`\n1. `GNU Parallel` installed for \u003ca href=\"https://www.gnu.org/software/parallel/\" target=\"_blank\"\u003eyour OS\u003c/a\u003e. For Debian based you can use `sudo apt-get install parallel`.\n1. `K9s` installed (To install see \u003ca href=\"https://k9scli.io/topics/install/\" target=\"_blank\"\u003ehere\u003c/a\u003e). For automated port forwarding set  [K9s configuration](~/.config/k9s/config.yml) with `scanForAutoPf: true`. Make sure K9s version supports the feature (https://github.com/derailed/k9s/pull/1498).\n\n### Environment Setup\n\n1. Clone repo: `git clone git@github.com:srfrnk/jabos.git` (or using HTTPS/GitHub CLI - see instructions \u003ca href=\"https://github.com/srfrnk/jabos\" target=\"_blank\"\u003ehere\u003c/a\u003e)\n1. CD into folder\n1. Start a minikube cluster `minikube start`\n1. Run `make setup` once\n1. Run `make build` after each code change\n1. To deploy examples\n   1. Locally clone \u003ca href=\"https://github.com/srfrnk/jabos-examples\" target=\"_blank\"\u003ejabos-examples repo\u003c/a\u003e\n   1. Follow instructions from the README file in the cloned folder\n   1. The examples would be deployed into namespace `example-env`\n\n## Credits\n\n- Jabos uses \u003ca href=\"https://github.com/GoogleContainerTools/kaniko\" target=\"_blank\"\u003ethe kaniko project\u003c/a\u003e to build docker images inside the kubernetes cluster.\n- Jabos uses \u003ca href=\"https://github.com/mikefarah/yq\" target=\"_blank\"\u003eyq\u003c/a\u003e to parse and update yaml and json data.\n- Jabos uses \u003ca href=\"github.com/google/go-jsonnet\" target=\"_blank\"\u003ejsonnet\u003c/a\u003e to process jsonnet templates and create K8s manifests.\n- Jabos uses \u003ca href=\"https://github.com/kubernetes/minikube\" target=\"_blank\"\u003eminikube\u003c/a\u003e for local development\n- Jabos uses \u003ca href=\"https://github.com/metacontroller/metacontroller\" target=\"_blank\"\u003emetacontroller\u003c/a\u003e to control K8s operators.\n- Jabos uses \u003ca href=\"https://github.com/srfrnk/efk-stack-helm\" target=\"_blank\"\u003eefk-stack-helm\u003c/a\u003e for local centralized logging.\n- Jabos uses \u003ca href=\"https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack\" target=\"_blank\"\u003ekube-prometheus-stack\u003c/a\u003e for local monitoring and alerting.\n- Jabos uses \u003ca href=\"https://www.gnu.org/software/parallel/\" target=\"_blank\"\u003eGNU Parallel\u003c/a\u003e for local port-forwarding to multiple services\n- Jabos uses \u003ca href=\"https://github.com/expressjs/express\" target=\"_blank\"\u003eexpressjs\u003c/a\u003e as the web server to run the operator\n- [**Removed due to security audit fails**](#34) - Jabos uses \u003ca href=\"https://github.com/joao-fontenele/express-prometheus-middleware\" target=\"_blank\"\u003eexpress-prometheus-middleware\u003c/a\u003e to export basic metrics to prometheus\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrfrnk%2Fjabos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsrfrnk%2Fjabos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrfrnk%2Fjabos/lists"}