{"id":13741213,"url":"https://github.com/KohlsTechnology/eunomia","last_synced_at":"2025-05-08T21:32:55.962Z","repository":{"id":35876518,"uuid":"189272386","full_name":"KohlsTechnology/eunomia","owner":"KohlsTechnology","description":"A GitOps Operator for Kubernetes","archived":true,"fork":false,"pushed_at":"2022-03-10T17:50:03.000Z","size":7674,"stargazers_count":148,"open_issues_count":0,"forks_count":37,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-11-15T01:39:50.315Z","etag":null,"topics":["gitops","golang","k8s","kubernetes","kubernetes-operator","operator","operator-sdk"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KohlsTechnology.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-29T17:46:27.000Z","updated_at":"2024-08-13T03:42:17.000Z","dependencies_parsed_at":"2022-08-08T12:15:19.930Z","dependency_job_id":null,"html_url":"https://github.com/KohlsTechnology/eunomia","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KohlsTechnology%2Feunomia","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KohlsTechnology%2Feunomia/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KohlsTechnology%2Feunomia/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KohlsTechnology%2Feunomia/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KohlsTechnology","download_url":"https://codeload.github.com/KohlsTechnology/eunomia/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224717817,"owners_count":17357914,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gitops","golang","k8s","kubernetes","kubernetes-operator","operator","operator-sdk"],"created_at":"2024-08-03T04:00:56.874Z","updated_at":"2024-11-15T11:31:02.129Z","avatar_url":"https://github.com/KohlsTechnology.png","language":"Go","funding_links":[],"categories":["Kubernetes Operators"],"sub_categories":[],"readme":"![eunomia-logo-small.png](./media/eunomia-logo-small.png)\n\n# Eunomia - a GitOps Operator for Kubernetes\n\n[![Join the chat at https://gitter.im/KohlsTechnology/eunomia](https://badges.gitter.im/KohlsTechnology/eunomia.svg)](https://gitter.im/KohlsTechnology/eunomia?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n[![Build Status](https://travis-ci.com/KohlsTechnology/eunomia.svg?branch=master)](https://travis-ci.com/KohlsTechnology/eunomia)\n[![Docker Repository on Quay](https://quay.io/repository/kohlstechnology/eunomia-operator/status \"Docker Repository on Quay\")](https://quay.io/repository/kohlstechnology/eunomia-operator)\n[![Go Report Card](https://goreportcard.com/badge/github.com/KohlsTechnology/eunomia)](https://goreportcard.com/report/github.com/KohlsTechnology/eunomia)\n[![codecov](https://codecov.io/gh/KohlsTechnology/eunomia/branch/master/graph/badge.svg)](https://codecov.io/gh/KohlsTechnology/eunomia)\n\n## Project Deprication and Archival\n\n**As of March 9, 2022 this project is no longer maintained. This repository is being archived(marked as read only).**\n\n## Who is Eunomia\n\nAccording to Wikipedia:\n\n\u003eEunomia (Greek: Εὐνομία) was a minor Greek goddess of law and legislation (her name can be translated as \"good order\", \"governance according to good laws\"), as well as the spring-time goddess of green pastures (eû means \"well, good\" in Greek, and νόμος, nómos, means \"law\", while pasturelands are called nomia).\n\n## How to Pronounce it\n\nEunomia is pronounced: yuˈnoʊ mi ə. The English phonetic respelling is yoo-noh-mee-uh.\n\nYou can listen to it [here](https://www.howtopronounce.com/eunomia) and [here](https://www.youtube.com/watch?v=59SItXQe1aM).\n\n## What is GitOps\n\nGitOps is all about turning day 2 operations into code! Not just that, it means you start thinking about day 2 on day 1. This is a dream come true for any Operations team!\nGitOps leverages the strength of automation and combines it with the power of git based workflows. It is a natural evolution beyond infrastructure-as-code and builds on top of DevOps best practices.\n\n### Next Generation Change Management\n\nEspecially in large Enterprises, Change Management is usually a painful experience. GitOps allows to take a lot of that pain out and streamline the process itself. It does so by still providing what the process tries to accomplish (and thus still meet audit requirements), but does so in a way that is much faster, much more secure, and much more reliable.\n\nYour changes now all of a sudden provide:\n\n- Version Control\n- Peer Reviews\n- Approvals\n- Audit Trail\n- Reproducibility\n- Consistency\n- Reliability\n\nWhat's your backout plan for your change? How about simply moving back to the previous commit and getting EXACTLY what you had before?\n\n## Purpose\n\nThe Eunomia provides the ability to implement these git-based flows for any resources in Kubernetes. Eunomia does not care if you have a plain Kubernetes, a cloud based Kubernetes (like GKE), or a complete PaaS platform based on Kubernetes (like OpenShift). Eunomia also does not care how you want to structure your data, how many repos you want to use, or which templating engine is your favourite.\n\nEunomia can handle straight-up (static) yaml files with the complete definition or create dynamic ones based on your templating engine of choice. Eunomia already supports *Helm Charts*, *OpenShift Templates*, and *Jinja2 Templates*, but can easily be extended to support others.\n\nThese templates will be merged and processed with a set of environment-specific parameters to get a list of resource manifests. Then these manifest can be created/updated/deleted in Kubernetes.\n\n## Getting started\n\nIf you want to deploy Eunomia from scratch to a local Minikube cluster, begin with the [Getting started](./GETTING_STARTED.md) document. It will guide your through the setup step-by-step.\n\n## Vision\n\nWhile this controller can certainly be used to directly populate an individual namespace with a configuration stored in git, the vision is that a hierarchy of controllers will be used to populate multiple namespaces. Ideally this approach will be used to bring a newly created cluster to a desired configured state. Only the initial seeding CR ([Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)) should have cluster-level permissions. Any sub-CRs should have progressively less access assigned to their service accounts.\n\nHere is a very simple example of how the hierarchy might look like:\n\n![Gitops-hierarchy.png](./media/Gitops-hierarchy.png)\n\nThe main sections of the GitOpsConfig CRD ([Custom Resource Definition](https://docs.okd.io/latest/admin_guide/custom_resource_definitions.html#crd_admin-guide-custom-resources)) are described below.\n\n## Example\n\nThe configuration is described in the GitOpsConfig CRD, here is an example:\n\n```yaml\napiVersion: eunomia.kohls.io/v1alpha1\nkind: GitOpsConfig\nmetadata:\n  name: simple-test\nspec:\n  # Add fields here\n  templateProcessorArgs: \"-e cluster_name=my_cluster_name\"\n  templateSource:\n    uri: https://github.com/KohlsTechnology/eunomia\n    ref: master\n    contextDir: simple/templates\n    secretRef: template-gitconfig\n  parameterSource:\n    contextDir: simple/parameters\n    secretRef: parameter-gitconfig\n  triggers:\n  - type: Change\n  - type: Webhook\n  - type: Periodic\n    cron: \"0 * * * *\"\n  serviceAccountRef:      \"mysvcaccount\"\n  templateProcessorImage: \"quay.io/kohlstechnology/eunomia-base:latest\"\n  resourceDeletionMode:   \"Delete\"\n  resourceHandlingMode:   \"Apply\"\n```\n\n## TemplateSource and ParameterSource and TemplateProcessorArgs\n\nThe `TemplateSource` and `ParameterSource` specify where the templates and the parameters are stored. The exact contents of these locations depend on the templating engine that has been selected.\n\nThe `TemplateProcessorArgs`  can be used to pass arguments/flags to the template processor. They can be accessed by the template processor in an environment variable named `TEMPLATE_PROCESSOR_ARGS`.\n\nThe fields of this section are:\n\n```yaml\n  templateProcessorArgs: \"-e cluster_name=my_cluster_name\"\n  templateSource:\n    uri: https://github.com/KohlsTechnology/eunomia\n    ref: master\n    contextDir: simple/templates\n    httpProxy: \u003chttp proxy\u003e\n    httpsProxy: \u003chttps proxy\u003e\n    noProxy: \u003cno Proxy\u003e\n    SecretRef: \u003cgitconfig and credentials secret\u003e\n\n  parameterSource:\n    uri: https://github.com/KohlsTechnology/eunomia\n    ref: master\n    contextDir: seed/parameters\n    httpProxy: \u003chttp proxy\u003e\n    httpsProxy: \u003chttps proxy\u003e\n    noProxy: \u003cno Proxy\u003e\n    SecretRef: \u003cgitconfig and credentials secret\u003e\n```\n\nThese are the mandatory constraints and default behaviors of the fields:\n\n| field name  | mandatory  | default  |\n|:---|:---:|:---|\n| uri  | yes  | N/A  |\n| ref  | no  | `master`  |\n| contextDir  | no  | `.`  |\n| httpProxy  | no  |   |\n| httpsProxy  | no  |   |\n| noProxy  | no  |   |\n| SecretRef  | no  |   |\n\nIf a secret is provided, then it is assumed that the connection to Git requires authentication. See the [Git Authentication](#git\\ authentication) section below for more details.\n\nIf the `uri` is not specified in the `parameterSource` section, then it will default to the `uri` specified under `templateSource`.\n\n### Git Submodules\n\nSome helm charts might require the configuration to be part of the chart itself (you can't read files from outside the chart). Loading files into a configmap is one example of this.\n\nSeparating the charts (templateSource) from the actual configuration (parameterSource) is a best practice. This allows you to separate your code (templates) from your configuration, which helps tremendously with change management.\n\nOne way to go about this is to use the config repo as a submodule and point to the master branch. During development you can of course point against another branch, just make sure you correct it in `.gitmodules` before the PR gets merged.\n\n#### Add submodule to track master branch\n\n```\ngit submodule add -b master \u003crepo-url\u003e\n```\n\n#### Checking out a repo with submodules\n\n```\ngit clone \u003crepo-url\u003e\ncd \u003crepo\u003e\ngit submodule init\ngit submodule update --recursive --remote\n```\n\n### parameterSource processing\nEunomia uses the [yq command](http://mikefarah.github.io/yq/) to merge all yaml files in the specified folder. You have to be careful, if you have the same variable name in multiple files. Dictionaries will merge, lists will get overwritten.\n\n#### Variable Hierarchy\nYou can provide a file `hierarchy.lst`, to allow a variable hierarchy. This will allow you to specify a default value and overwrite it on an environment level if necessary. This will greatly simplify your configuration and allows for deduplication of data, making your operational life a lot easier.\n\nThe contents of the file are simply relative path names, with the base being `contextDir`.\n\nExample `hierarchy.lst`:\n```\n../defaults    #this is first ... lowest priority\n../marketing   #this is second\n../development #this is third ... highest priority\n```\n\nIn this case it will load all yaml files from `../defaults`, then merge it with everything in `../marketing`, and lastly merges it with everything in `../development`.\nYou can also use the relative path `./`, which means it'll also load the variables defined in `contextDir` directly (same folder that as `hierarchy.lst`). You can insert `./` in whatever order you want in the `hierarchy.lst` - it will determine its priority.\n\n#### Upcoming features\nOnce [issue #4](https://github.com/KohlsTechnology/eunomia/issues/4) is resolved, you will be able to specify variable names to dynamically determine the correct folder. This will allow you to only have one `hierarchy.lst`. (Technically, it is actually already possible to use environment variables, but without #4, there are just none set that would be of any practical use in hierarchy.lst.)\n\n### Git Authentication\n\nSpecifying a `SecretRef` will automatically turn on git authentication. The secrets for the template and parameter repos will be mounted respectively in the `/template-gitconfig` and `/parameter-gitconfig` of the job pod.\nThe referenced secrets must be available and how they are provisioned is beyond the scope of this operator. See the [Vision](#vision) paragraph on how to build a hierarchical structure, where the resources needed to run a given GitOpsConfig are configured by a predecessor GitpOpsConfig instance.\n\nThis secret will be linked from `~/` of the used running the pod. The secret *must* contain a `.gitconfig` file and may contain other files. The passed `.gitconfig` will be used during the git operations. It is advised to reference any additional files via the absolute path.\n\n#### Username and password authentication\n\nFor username and password based authentication create the following `.gitconfig`:\n\n```ini\n[http]\n    sslCAInfo = /template-gitconfig/ca.crt\n\n[user]\n    name = gitconfig\n\n[credential]\n    helper = store\n```\n\nWhen the credential helper is of type `store`, credentials are by default retrieved from the `~/.git-credentials` file. This file should also be added to the secret and has the following format:\n\n```text\nhttps://\u003cusername\u003e:\u003cpassword\u003e@\u003cgit server fqdn\u003e\n```\n\nDon't forget to provide the `ca.crt` file to the secret.\n\n#### Certificate based authentication\n\nFor certificate based authentication, create the following `.gitconfig`:\n\n```ini\n[core]\n    sshCommand = \"ssh -i /template-gitconfig/mykey.rsa -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no\"\n```\n\nand add the `mykey.rsa` file to the secret.\n\n## Job templates\n\nFor Eunomia to work properly there is a need for a specific [`Job`](https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/) or a [`CronJob`](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/).\n\nA [`Job default template`](./build/job-templates/job.yaml) and a [`CronJob default template`](./build/job-templates/cronjob.yaml) are built into the Dockerfile.\n\nIf you want to provide your own job templates, set the env variables `JOB_TEMPLATE` and `CRONJOB_TEMPLATE`. Their values should be set to paths, where appropriate yaml files can be found.\nThe files themselves have to be accessible in the pod. To achieve this, you can for instance [`add ConfigMap data to a Volume`](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#add-configmap-data-to-a-volume).\n\n## Triggers\n\nYou can enable one or multiple triggers.\n\n| Name  | Description  |\n|:---|:---|\n|`Change` | This triggers every time the CR is changed, including when it is created.|\n|`Periodic` | Periodically apply the configuration. This can be used to either schedule changes for a specific time, use it for drift management to revert any changes, or as a safeguard in case webhooks were missed. It uses a cron-style expression.\n|`Webhook` | This triggers when something on git changes. You have to configure the webhook yourself. For branches use just branch name in GitOpsConfig CR `ref`, but if you want webhook working for git tag, use refs/tags/[tag_name].\n\n### GitHub webhook configuration\n\nTo set up GitHub webhook follow this [GitHub documentation](https://developer.github.com/webhooks/creating/).\nCreate route on port 8080 to eunomia-service and use this route as GitHub webhook `Payload URL` with added webhook/ endpoint at the end.\n\nContent type needs to be set to `application/json`.\n\nChoose `Just the push event` to trigger webhook.\n\n## Template Engine\n\nWhen it's time to apply a configuration, the GitOps controller runs a job pod. The image of the job pod can be specified in the `templateProcessorImage` field.\nThis is the plugin mechanism to support multiple template engines.\nA base image is provided that can be inherited to simplify the process of adding support for a new templating engine.\nThe base image provides the following workflow:\n\n1. `gitClone.sh` : This will clone the template and parameter repos. It is expected that there will be no need to customize this. Any required changes are most likely worthy of a pull-request upstream.\n2. `discoverEnvironment.sh` : This will create a set of environment variables that are specific to the target Kubernetes environment. Currently the following variables are supported:\n\n    | Name  | Description  |\n    |:---|:---|\n    | `CA_BUNDLE`  | Path to the [platform-level CA bundle](https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/#accessing-the-api-from-a-pod)  |\n    | `SERVICE_CA_BUNDLE`  | Path to the [service-level CA bundle](https://docs.openshift.com/container-platform/3.11/dev_guide/secrets.html#service-serving-certificate-secrets)  |\n    | `NAMESPACE`  | Current namespace  |\n\n3. [`hierarchy`](https://github.com/KohlsTechnology/hierarchy) : processes all the parameter files and generates `/tmp/eunomia_values_processed.yaml`. It currently supports the following features:\n    - Merging of all existing yaml files in the `CLONED_PARAMETER_GIT_DIR` location, into a single file for processing by the templating engine.\n    - Substitution of variables with environment variables within the hierarchy.\n    - Substitution of variables with environment variables within the final yaml file.\n\n4. `processTemplate.sh` : This file needs to be overwritten in order to support a different templating engine. The contract is the following:\n\n    - Templates are available at the location specified by the variable: `CLONED_TEMPLATE_GIT_DIR`\n    - Parameters are available at the location specified by the variable: `CLONED_PARAMETER_GIT_DIR`\n    - After the template processing completes, the processed manifests should be stored at the location of this variable: `MANIFEST_DIR`\n\n5. `resourceManager.sh` :  Processes the resources in `MANIFEST_DIR`. One or more files can be present, and all will be processed.\n\nCurrently the following templating engines are supported (follow the link to see examples of how new template processors can be added):\n\n- [Raw Manifests](./template-processors/base)\n- [OpenShift Templates](./template-processors/ocp-template)\n- [Helm Charts](./template-processors/helm)\n- [Jinja Templates](./template-processors/jinja)\n- [OpenShift Applier](./template-processors/applier)\n\n## serviceAccountRef\n\nThis is the service account used by the job pod that will process the resources.\nThe service account must be present in the same namespace as the one where the GitOpsConfig CR is and must have enough permission to manage the resources.\nIt is out of scope of this controller how that service account is provisioned, although you can use a different GitOpsConfig CR to provision it (seeding CR).\n\nIn the Helm deployment, the `ClusterRole` `eunomia-cluster-list` provides `list` access to all resources, and will be provisioned if you set `.Values.eunomia.operator.deployment.clusterViewer` to `true`.\nThis `ClusterRole` is intended to be used in a `ClusterRoleBinding` with \"job runner\" service accounts so they can find all of the resources that it owns.\nWithout the `ClusterRoleBinding`, the jobs can still successfully run, however there will be error logs stating it can not find any cluster scoped resources.\n\n## Resource Handling Mode\n\nThis field specifies how resources should be handled, once the templates are processed. The following modes are currently supported:\n\n1. `Apply`, which is roughly equivalent to `kubectl apply`. Additionally, auto-detection of resources removed from git is performed, and they're deleted from the cluster. This is done by marking all the resources with a custom label, and removing resources for which the label was not touched by `kubectl apply`.\n2. `Patch`. Patch requires objects to already exists and will patch them. It's useful when customizing objects that are provided through other means.\n3. `Create`, equivalent to `kubectl create`. Template processors which take over the resource handling phase are not required to support this mode.\n4. `Replace`, equivalent to `kubectl replace`. Template processors which take over the resource handling phase are not required to support this mode.\n5. `Delete`, equivalent to `kubectl delete`. Template processors which take over the resource handling phase are not required to support this mode.\n6. `None`. In some cases there may be template processors or automation frameworks where the processing of templates and handling of generated resources are a single step. In that case, Eunomia can be configured to skip the built-in resource handling step.\n\n## Resource Deletion Mode\n\nThis field specifies how to handle resources when the GitOpsConfig object is deleted. Two options are available:\n\n1. `Retain`, resources previously created are left intact.\n2. `Delete`, resources are delete with the `cascade` option.\n3. `None`, resource deletion is not handled at all.\n\n## Installing Eunomia\n\n### Installing on Kubernetes Using Helm\n\nSee the [helm README](deploy/helm/eunomia-operator/README.md) for details.\n\n### Installing on Kubernetes Using Operator Hub\n\nSee the [getting started document](GETTING_STARTED.md#install-eunomia-from-operatorhub) for details.\n\n## Examples / Demos\n\nWe've created several examples for you to test out Eunomia. See [EXAMPLES](examples/README.md) for details.\n\n## Monitoring\n\n### Monitoring with Prometheus\n\n[Prometheus](https://prometheus.io/) is an open-source systems monitoring and alerting toolkit.\n\nPrometheus collects metrics from monitored targets by scraping metrics HTTP endpoints.\n\n- [configuring-prometheus](https://prometheus.io/docs/introduction/first_steps/#configuring-prometheus)\n\n- `scrape_configs` controls what resources Prometheus monitors.\n\n- `kubernetes_sd_configs` Kubernetes SD configurations allow retrieving scrape targets. Please see [kubernetes_sd_configs](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#endpoints) for details.\n\n- Additionally, `relabel_configs` allow advanced modifications to any target and its labels before scraping.\n\nBy default, the metrics in Operator SDK are exposed on `0.0.0.0:8383/metrics`\n\nFor more information, see [Metrics in Operator SDK](https://github.com/operator-framework/operator-sdk/blob/v0.17.1/doc/user/metrics/README.md)\n\n#### Usage:\n\n```\nscrape_configs:\n  - job_name: 'kubernetes-service-endpoints'\n    kubernetes_sd_configs:\n    - role: endpoints\n    relabel_configs:\n      - source_labels: [__meta_kubernetes_namespace]\n        action: keep\n        regex: test-eunomia-operator\n```\nYou can find additional examples on their [GitHub page](https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus-kubernetes.yml).\n\n#### Verify metrics port:\nkubectl exec `POD-NAME` curl localhost:8383/metrics  -n `NAMESPACE`\n\n(e.g. `kubectl exec eunomia-operator-5b9b664cfc-6rdrh curl localhost:8383/metrics  -n test-eunomia-operator`)\n\n### Kubernetes Events\n\nEunomia emits the following events in the namespace of the GitOpsConfig CR:\n\n  - JobSuccessful - when a Job applying the CR finished successfully\n  - JobFailed - when a Job applying the CR has finished with a failure (after all retries have failed)\n\n## Development\n\nPlease see our [development documentation](DEVELOPMENT.md) for details.\n\n## Troubleshooting\n\nPlease see our [troubleshooting guide](TROUBLESHOOTING.md) for details.\n\n## License\n\nSee [LICENSE](LICENSE) for details.\n\n## Code of Conduct\n\nSee [CODE_OF_CONDUCT.md](.github/CODE_OF_CONDUCT.md)\nfor details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKohlsTechnology%2Feunomia","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FKohlsTechnology%2Feunomia","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKohlsTechnology%2Feunomia/lists"}