{"id":13818004,"url":"https://github.com/redhat-cop/global-load-balancer-operator","last_synced_at":"2025-04-09T22:01:59.531Z","repository":{"id":45656198,"uuid":"279486056","full_name":"redhat-cop/global-load-balancer-operator","owner":"redhat-cop","description":"A global load balancer operator for OpenShift","archived":false,"fork":false,"pushed_at":"2024-04-06T23:44:55.000Z","size":710,"stargazers_count":54,"open_issues_count":11,"forks_count":15,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-04-09T22:01:29.414Z","etag":null,"topics":["container-cop","k8s-operator"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/redhat-cop.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-14T04:56:07.000Z","updated_at":"2025-02-09T22:09:45.000Z","dependencies_parsed_at":"2024-01-25T03:23:48.341Z","dependency_job_id":"a17e348b-0a5a-49d3-a541-9b657eb6bb53","html_url":"https://github.com/redhat-cop/global-load-balancer-operator","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-cop%2Fglobal-load-balancer-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-cop%2Fglobal-load-balancer-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-cop%2Fglobal-load-balancer-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-cop%2Fglobal-load-balancer-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/redhat-cop","download_url":"https://codeload.github.com/redhat-cop/global-load-balancer-operator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248119296,"owners_count":21050755,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["container-cop","k8s-operator"],"created_at":"2024-08-04T07:00:26.659Z","updated_at":"2025-04-09T22:01:59.486Z","avatar_url":"https://github.com/redhat-cop.png","language":"Go","funding_links":[],"categories":["Networking"],"sub_categories":[],"readme":"\n# Global Load Balancer Operator\n\n![build status](https://github.com/redhat-cop/global-load-balancer-operator/workflows/push/badge.svg)\n[![Go Report Card](https://goreportcard.com/badge/github.com/redhat-cop/global-load-balancer-operator)](https://goreportcard.com/report/github.com/redhat-cop/global-load-balancer-operator)\n![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/redhat-cop/global-load-balancer-operator)\n[![CRD Docs](https://img.shields.io/badge/CRD-Docs-brightgreen)](https://doc.crds.dev/github.com/redhat-cop/global-load-balancer-operator)\n\nThe global-load-balancer-operator implements automation to program a global load balancer for applications deployed to multiple OpenShift clusters.\nThis operator is designed to be deployed to a control cluster which will watch the load balanced clusters (controlled clusters).\nA global load balancer can be implemented as a DNS or a global IP anycast configuration.\nThere are two main concepts (APIs) provided by this operator:\n\n1. GlobalDNSZone: represents the global load balancer provider and DNS zone in which the global record will be created.\n2. GlobalDNSRecord: represent a globally load balancer application with a DNS record and a set of end points to be load balanced.\n\n## GlobalDNSZone\n\nThe `GlobalDNSZone` CR allows you to configure a zone which will contain global load balanced records and the provider used to populate it.\nHere is an example of GlobalDNSZone:\n\n```yaml\napiVersion: redhatcop.redhat.io/v1alpha1\nkind: GlobalDNSZone\nmetadata:\n  name: external-dns-zone\nspec:\n  # Add fields here\n  domain: global.myzone.io\n  provider:\n    externalDNS:\n      annotations:\n        type: global\n```\n\nHere is a table summarizing the supported DNS-based load balancer providers and their capabilities:\n\n| Provider  | Supports Health Checks  | Supports Multivalue LB | Supports Latency LB  | Supports GeoProximity LB  |\n|:--:|:--:|:---:|:---:|:---:|\n| External-dns  | no  | yes | no  | no  |\n| Route53  | yes | yes(*)  | yes(*)  | yes(*)  |\n| Azure Traffic Manager (**) | yes | yes | yes | yes |\n\n(*) only if all controlled clusters run on AWS.\n(**) currently all controlled clusters must be on Azure.\n\nHere is a table summarizing the supported Anycast-based load balancer providers and their capabilities:\n\n| Provider  | Supports Health Checks  | Supports TCP-session affinity | Supports Latency LB  | Supports GeoProximity LB  |\n|:--:|:--:|:---:|:---:|:---:|\n| Google Global Load Balancer (***) | yes | yes | client connection are routed to the closer endpoint in terms of network hops up to a maximum of 100 connections per VM | N/A |\n\n(***) currently all controlled clusters must be on Google Cloud.\n\n## GlobalDNSRecord\n\nThe `GlobalDNSRecord` CR allows you to specify the intention to create a global dns record. Here is an example\n\n```yaml\napiVersion: redhatcop.redhat.io/v1alpha1\nkind: GlobalDNSRecord\nmetadata:\n  name: hello-global-record\nspec:\n  name: hello.global.myzone.io\n  endpoints:\n  - clusterName: cluster1\n    clusterCredentialRef:\n      name: kubeconfig\n      namespace: cluster1\n    loadBalancerServiceRef:\n      name: router-default\n      namespace: openshift-ingress\n  - clusterName: cluster2\n    clusterCredentialRef:\n      name: kubeconfig\n      namespace: cluster2\n    loadBalancerServiceRef:\n      name: router-default\n      namespace: openshift-ingress\n  - clusterName: cluster3\n    clusterCredentialRef:\n      name: kubeconfig\n      namespace: cluster3\n    loadBalancerServiceRef:\n      name: router-default\n      namespace: openshift-ingress\n  ttl: 60\n  loadBalancingPolicy: Multivalue\n  globalZoneRef:\n    name: external-dns-zone\n  healthCheck:\n    failureThreshold: 3\n    httpGet:\n      host: hello.global.myzone.io\n      httpHeaders:\n        - name: ciao\n          value: hello\n      path: /readyz\n      port: 80\n      scheme: HTTP\n    periodSeconds: 10\n    successThreshold: 1\n    timeoutSeconds: 1\n```\n\nThe list of endpoints identifies the set of `LoadBalancer` type of services to watch on the remote clusters (at the moment only LoadBalancer services are supported). These are the `LoadBalancer` services created by the [ingress controller](https://docs.openshift.com/container-platform/4.5/networking/ingress-operator.html) on which the routers rely. Each entry of this list requires a reference to the loadbalancer service on the remote cluster and a reference to a local secret containing the credential to connect to the remote cluster.\n\nThis secret must contain one entry called `kubeconfig` with a kubeconfig file with a default context pointing to the remote cluster. The account used by the kubeconfig file (presumably a service account) will require at a minimum cluster-level permissions as described in this [cluster role](./docs/scripts/remote-account-cluster-role.yaml).\n\nThe `globalZoneRef` field refers to a local (same namespace) `GlobalZone` CR. The DNS record represented by this `GlobalDNSRecord`, will be created in the referenced zone.\n\n`ttl` is the TTL of the crated DNS record.\n\n`loadBalancingPolicy` is the load balancing policy for this global record. It must match one of the policy supported by the provider of the referenced `GlobalZone`.\n\nFinally, `healthcheck` represent a [probe](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#probe-v1-core) to be used to test the health of a record. This field will be ignored if the provider does not support health checks.\n\n## External DNS Provider\n\nThe [external-dns](https://github.com/kubernetes-sigs/external-dns) provider delegates to external-dns the creation of the actual DNS records by creating a `DNSEndpoint` object.\nThe `DNSEndpoint` object will be created in the same namespace as the `GlobalDNSRecord` and will be owned by it.\nThe `DNSEdnpoint` object will have the same labels as the `GlobalDNSRecord` and the annotations specified in the GlobalDNSZone configuration.\nExternal-dns should be configured to watch for DNSEnpoints at the cluster level and to point to the desired provider.\nDetails on configuration can be found at the external-dns git repository.\nThe External-dns should be used as a fall back option when other options are not available as it does not support health checks and advanced load balancing policies.\n\n## AWS Route53 provider\n\nAWS Route53 provider uses the Route53 service as a global loadbalancer and offers advanced routing capabilities via [route53 traffic policies](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/traffic-flow.html) (note that traffic policies will trigger an expense).\nThe following routing polices are currently supported:\n\n1. [Multivalue](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.html#routing-policy-multivalue).\n2. [Geoproximity](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.html#routing-policy-geoproximity).\n3. [Latency](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.html#routing-policy-latency).\n\nAWS Route53 provider at the moment requires that all the controlled clusters run in AWS.\n\nIf health checks are defined, a route53 health check originating from every region (you have to ensure connectivity) will be created for each of the endpoint. Because the endpoint represents a shared ELB (shared with other apps, that is) and the health check is app specific, we cannot use the ELB health check, so the route53 endpoint is created with one of the two IP exposed by the ELB. This is suboptimal, but it works in most situations.\n\n## Azure Traffic Manager provider\n\nAzure traffic manager provider uses (Azure Traffic Manager)[] to create global load balancer configuration.\nThe following routing policies are supported:\n\n1. [Multivalue](https://docs.microsoft.com/en-us/azure/traffic-manager/traffic-manager-routing-methods#multivalue).\n2. [Geographic](https://docs.microsoft.com/en-us/azure/traffic-manager/traffic-manager-routing-methods#geographic).\n3. [Performance](https://docs.microsoft.com/en-us/azure/traffic-manager/traffic-manager-routing-methods#performance).\n\nThis is an initial implementation which requires all of the controlled clusters to be on Azure and to be publicly exposed.\n\nHealth checks are supported by this provider.\n\n## Google Global Load Balancer provider\n\nGoogle Global Load Balancer provider creates a global IP that will load balance directly to the nodes and node port of the service in the GlobalDNSRecord. It sets a maximum of 100 connections per node (currently hard coded). It supports TCP-session affinity.\nThe `loadBalancingPolicy` field will be ignored by this provider.\n\nHealth checks are supported by this provider and are mandatory.\n\n## Global Route Auto Discovery\n\nThe above examples showed how to create global DNS records. This can be good in some situations, but most of the times in an openshift deployment global DNS records will point to routes that are intended to be global.\nThe global-load-balancer operator can auto-discover these routes and automatically create the corresponding `GloablDNSRecord`. The `GlobalRouteDiscovery` CRD is used to configure the discovery process, here is an example:\n\n```yaml\napiVersion: redhatcop.redhat.io/v1alpha1\nkind: GlobalRouteDiscovery\nmetadata:\n  name: route53-globalroutediscovery\nspec:\n  clusters:\n  - clusterName: cluster1\n    clusterCredentialRef:\n      name: ${cluster1_secret_name}\n      namespace: cluster1\n...\n  routeSelector:\n    matchLabels:\n      route-type: global\n  defaultLoadBalancingPolicy: Multivalue\n  defaultTTL: 30\n  globalZoneRef:\n    name: route53-global-dns-zone\n```\n\nThis global discovery route will discover routes in the provided list of cluster. Only the routes that match the route selector will be considered global.\nThe default load balancing policy and default TTL can be expressed in the `GlobalRouteDiscovery` CR. However with the following annotations, it's possible to configure route-specific values:\n\n* `global-load-balancer-operator.redhat-cop.io/load-balancing-policy` to set the load balancing policy\n* `global-load-balancer-operator.redhat-cop.io/ttl` to set the TTL\n* `global-load-balancer-operator.redhat-cop.io/health-check` to set the health check. This must be a v1core.Probe object in json format.\n\nAs an example here is what a v1core.Probe object looks like in json format:\n\n```json\n'{\"httpGet\":{\"path\":\"health?ready=1\",\"port\":443,\"scheme\":\"HTTPS\"},\"timeoutSeconds\":1,\"periodSeconds\":10,\"successThreshold\":1,\"failureThreshold\":3}'\n```\n\nThe globalZoneRef refers to the global zone to be used for the created `GlobalDNSRecords`.\n\nIf the health check annotation is not provided, the controller will try to automatically discover the health check. If the pods behind the route expose a readiness check of `httpGet` kind, that configuration will be used to create the GlobalDNSRecord health check.\nWhen more than one container is present in the pod, by default the first container will be examined for health check. This behavior can be overridden with the this annotation on the route: `global-load-balancer-operator.redhat-cop.io/container-probe` where the value will container the name of the container with teh correct readiness probe.\n\nIf routes with the same namespace and name exists in multiple cluster, the following conditions must be met:\n\n* all host names must be the same\n* all load balancing policy must be the same\n* all TTLs must be the same\n* all discovered readiness checks must be the same\n\n## Examples\n\nThese examples are intended to help you setting up working configuration with each of the providers\n\n### Cluster Set up\n\nTwo approaches for cluster setup are provided\n\n1. [One cluster, three ingress-gateways.](./docs/one-cluster-three-ingresses.md) This approach is intended for development purposes and has the objective to keep resource consumption at the minimum.\n2. [Control cluster and three controlled clusters in different regions.](./docs/three-clusters.md). This approach represents a more realistic set-up albeit it consumes more resources.\n\nYou can also set up the cluster on your own, at the end the following conditions must be met:\n\nThree namespace `cluster1` `cluster2` `cluster3` are created.\nthe following environment variables are initialized for each cluster:\n\n1. `\u003ccluster\u003e_secret_name`. Pointing to a secret in each of the cluster namespaces containing a valid kubeconfig fot that cluster\n2. `\u003ccluster\u003e_service_name`.  Pointing to the name of the loadbalancer service to be used for that cluster.\n3. `\u003ccluster\u003e_service_namespace`. Pointing to the namespace of the loadbalancer service to be used for that cluster.\n\nHere are examples for the supported provider:\n\n1. [Setting up external-dns as provider](./docs/external-dns-provider.md)\n2. [Setting up route53 as a provider](./docs/aws-route53-provider.md)\n3. [Setting up Azure Traffic Manager as a provider](./docs/azure-trafficmanager-provider.md)\n4. [Setting up Google Global Load Balancer as a provider](./docs/GCPGLB-provider.md)\n\n## Deploying the Operator\n\nThis is a cluster-level operator that you can deploy in any namespace, `global-load-balancer-operator` is recommended.\n\nIt is recommended to deploy this operator via [`OperatorHub`](https://operatorhub.io/), but you can also deploy it using [`Helm`](https://helm.sh/).\n\n### Multiarch Support\n\n| Arch  | Support  |\n|:-:|:-:|\n| amd64  | ✅ |\n| arm64  | ✅  |\n| ppc64le  | ✅  |\n| s390x  | ❌  |\n\n### Deploying from OperatorHub\n\n\u003e **Note**: This operator supports being installed disconnected environments\n\nIf you want to utilize the Operator Lifecycle Manager (OLM) to install this operator, you can do so in two ways: from the UI or the CLI.\n\n#### Deploying from OperatorHub UI\n\n* If you would like to launch this operator from the UI, you'll need to navigate to the OperatorHub tab in the console.Before starting, make sure you've created the namespace that you want to install this operator to with the following:\n\n```shell\noc new-project global-load-balancer-operator\n```\n\n* Once there, you can search for this operator by name: `global loadbalancer`. This will then return an item for our operator and you can select it to get started. Once you've arrived here, you'll be presented with an option to install, which will begin the process.\n* After clicking the install button, you can then select the namespace that you would like to install this to as well as the installation strategy you would like to proceed with (`Automatic` or `Manual`).\n* Once you've made your selection, you can select `Subscribe` and the installation will begin. After a few moments you can go ahead and check your namespace and you should see the operator running.\n\n![Global Load Balancer Operator](./media/global-load-balancer-operator.png)\n\n#### Deploying from OperatorHub using CLI\n\nIf you'd like to launch this operator from the command line, you can use the manifests contained in this repository by running the following:\n\n```shell\noc new-project global-load-balancer-operator\noc apply -f config/operatorhub -n global-load-balancer-operator\n```\n\nThis will create the appropriate OperatorGroup and Subscription and will trigger OLM to launch the operator in the specified namespace.\n\n### Deploying with Helm\n\nHere are the instructions to install the latest release with Helm.\n\n```shell\noc new-project global-load-balancer-operator\nhelm repo add global-load-balancer-operator https://redhat-cop.github.io/global-load-balancer-operator\nhelm repo update\nhelm install global-load-balancer-operator global-load-balancer-operator/global-load-balancer-operator\n```\n\nThis can later be updated with the following commands:\n\n```shell\nhelm repo update\nhelm upgrade global-load-balancer-operator global-load-balancer-operator/global-load-balancer-operator\n```\n\n## Metrics\n\nPrometheus compatible metrics are exposed by the Operator and can be integrated into OpenShift's default cluster monitoring. To enable OpenShift cluster monitoring, label the namespace the operator is deployed in with the label `openshift.io/cluster-monitoring=\"true\"`.\n\n```shell\noc label namespace \u003cnamespace\u003e openshift.io/cluster-monitoring=\"true\"\n```\n\n### Testing metrics\n\n```sh\nexport operatorNamespace=global-load-balancer-operator-local # or global-load-balancer-operator\noc label namespace ${operatorNamespace} openshift.io/cluster-monitoring=\"true\"\noc rsh -n openshift-monitoring -c prometheus prometheus-k8s-0 /bin/bash\nexport operatorNamespace=global-load-balancer-operator-local # or global-load-balancer-operator\ncurl -v -s -k -H \"Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\" https://global-load-balancer-operator-controller-manager-metrics.${operatorNamespace}.svc.cluster.local:8443/metrics\nexit\n```\n\n## Development\n\n### Running the operator locally\n\n```shell\nmake install\noc new-project global-load-balancer-operator-local\noc create serviceaccount global-load-balancer-controller-manager\nkustomize build ./config/local-development | oc apply -f - -n global-load-balancer-operator-local\nexport token=$(oc serviceaccounts get-token 'global-load-balancer-controller-manager' -n global-load-balancer-operator-local)\noc login --token ${token}\nmake run ENABLE_WEBHOOKS=false NAMESPACE=global-load-balancer-operator-local\n```\n\n### Test helm chart locally\n\nDefine an image and tag. For example...\n\n```shell\nexport imageRepository=\"quay.io/redhat-cop/global-load-balancer-operator\"\nexport imageTag=\"$(git -c 'versionsort.suffix=-' ls-remote --exit-code --refs --sort='version:refname' --tags https://github.com/redhat-cop/global-load-balancer-operator.git '*.*.*' | tail --lines=1 | cut --delimiter='/' --fields=3)\"\n```\n\nDeploy chart...\n\n```shell\nmake helmchart IMG=${imageRepository} VERSION=${imageTag}\nhelm upgrade -i global-load-balancer-operator-local charts/global-load-balancer-operator -n global-load-balancer-operator-local --create-namespace\n```\n\nDelete...\n\n```shell\nhelm delete global-load-balancer-operator-local -n global-load-balancer-operator-local\nkubectl delete -f charts/global-load-balancer-operator/crds/crds.yaml\n```\n\n### Building/Pushing the operator image\n\n```shell\nexport repo=raffaelespazzoli #replace with yours\ndocker login quay.io/$repo/global-load-balancer-operator\nmake docker-build IMG=quay.io/$repo/global-load-balancer-operator:latest\nmake docker-push IMG=quay.io/$repo/global-load-balancer-operator:latest\n```\n\n### Deploy to OLM via bundle\n\n```shell\nmake manifests\nmake bundle IMG=quay.io/$repo/global-load-balancer-operator:latest\noperator-sdk bundle validate ./bundle --select-optional name=operatorhub\nmake bundle-build BUNDLE_IMG=quay.io/$repo/global-load-balancer-operator-bundle:latest\ndocker login quay.io/$repo/global-load-balancer-operator-bundle\npodman push quay.io/$repo/global-load-balancer-operator-bundle:latest\noperator-sdk bundle validate quay.io/$repo/global-load-balancer-operator-bundle:latest --select-optional name=operatorhub\noc new-project global-load-balancer-operator\noc label namespace global-load-balancer-operator openshift.io/cluster-monitoring=\"true\"\noperator-sdk cleanup global-load-balancer-operator -n global-load-balancer-operator\noperator-sdk run bundle --install-mode AllNamespaces -n global-load-balancer-operator quay.io/$repo/global-load-balancer-operator-bundle:latest\n```\n\n### Releasing\n\n```shell\ngit tag -a \"\u003ctagname\u003e\" -m \"\u003ccommit message\u003e\"\ngit push upstream \u003ctagname\u003e\n```\n\nIf you need to remove a release:\n\n```shell\ngit tag -d \u003ctagname\u003e\ngit push upstream --delete \u003ctagname\u003e\n```\n\nIf you need to \"move\" a release to the current main\n\n```shell\ngit tag -f \u003ctagname\u003e\ngit push upstream -f \u003ctagname\u003e\n```\n\n### Cleaning up\n\n```shell\noperator-sdk cleanup global-load-balancer-operator -n global-load-balancer-operator\noc delete operatorgroup operator-sdk-og\noc delete catalogsource global-load-balancer-operator-catalog\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredhat-cop%2Fglobal-load-balancer-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fredhat-cop%2Fglobal-load-balancer-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredhat-cop%2Fglobal-load-balancer-operator/lists"}