{"id":20155278,"url":"https://github.com/redhat-cop/keepalived-operator","last_synced_at":"2026-03-06T18:04:12.920Z","repository":{"id":39630560,"uuid":"237202134","full_name":"redhat-cop/keepalived-operator","owner":"redhat-cop","description":"An operator to manage VIPs backed by keepalived","archived":false,"fork":false,"pushed_at":"2024-04-03T23:41:54.000Z","size":8222,"stargazers_count":122,"open_issues_count":37,"forks_count":37,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-04-08T12:26:23.155Z","etag":null,"topics":["container-cop","k8s-operator"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/redhat-cop.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-30T11:48:47.000Z","updated_at":"2025-03-16T14:00:06.000Z","dependencies_parsed_at":"2023-12-21T13:09:14.285Z","dependency_job_id":"88002f5e-6675-41fb-b094-f4e0887dce9b","html_url":"https://github.com/redhat-cop/keepalived-operator","commit_stats":null,"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/redhat-cop/keepalived-operator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-cop%2Fkeepalived-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-cop%2Fkeepalived-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-cop%2Fkeepalived-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-cop%2Fkeepalived-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/redhat-cop","download_url":"https://codeload.github.com/redhat-cop/keepalived-operator/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redhat-cop%2Fkeepalived-operator/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262753172,"owners_count":23358883,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["container-cop","k8s-operator"],"created_at":"2024-11-13T23:31:16.963Z","updated_at":"2026-03-06T18:04:07.863Z","avatar_url":"https://github.com/redhat-cop.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Keepalived operator\n\n![build status](https://github.com/redhat-cop/keepalived-operator/workflows/push/badge.svg)\n[![Go Report Card](https://goreportcard.com/badge/github.com/redhat-cop/keepalived-operator)](https://goreportcard.com/report/github.com/redhat-cop/keepalived-operator)\n![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/redhat-cop/keepalived-operator)\n\nThe objective of the keepalived operator is to allow for a way to create self-hosted load balancers in an automated way. From a user experience point of view the behavior is the same as of when creating [`LoadBalancer`](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer) services with a cloud provider able to manage them.\n\nThe keepalived operator can be used in all environments that allows nodes to advertise additional IPs on their NICs (and at least for now, in networks that allow multicast), however it's mainly aimed at supporting LoadBalancer services and ExternalIPs on bare metal installations (or other installation environments where a cloud provider is not available).\n\nOne possible use of the keepalived operator is also to support [OpenShift Ingresses](https://docs.openshift.com/container-platform/4.5/networking/configuring_ingress_cluster_traffic/overview-traffic.html) in environments where an external load balancer cannot be provisioned. See this [how-to](./Ingress-how-to.md) on how to configure keepalived-operator to support OpenShift ingresses\n\n## How it works\n\nThe keepalived operator will create one or more [VIPs](https://en.wikipedia.org/wiki/Virtual_IP_address) (an HA IP that floats between multiple nodes), based on the [`LoadBalancer`](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer) services and/or services requesting [`ExternalIPs`](https://kubernetes.io/docs/concepts/services-networking/service/#external-ips).\n\nFor `LoadBalancer` services the IPs found at `.Status.LoadBalancer.Ingress[].IP` will become VIPs.\n\nFor services requesting a `ExternalIPs`, the IPs found at `.Spec.ExternalIPs[]` will become VIPs.\n\nNote that a service can be of `LoadBalancer` type and also request `ExternalIPs`, it this case both sets of IPs will become VIPs.\n\nDue to a [keepalived](https://www.keepalived.org/manpage.html) limitation a single keepalived cluster can manage up to 256 VIP configurations. Multiple keepalived clusters can coexists in the same network as long as they use different multicast ports [TODO verify this statement].\n\nTo address this limitation the `KeepalivedGroup` [CRD](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) has been introduced. This CRD is supposed to be configured by an administrator and allows you to specify a node selector to pick on which nodes the keepalived pods should be deployed. Here is an example:\n\n```yaml\napiVersion: redhatcop.redhat.io/v1alpha1\nkind: KeepalivedGroup\nmetadata:\n  name: keepalivedgroup-router\nspec:\n  image: registry.redhat.io/openshift4/ose-keepalived-ipfailover\n  interface: ens3\n  nodeSelector:\n    node-role.kubernetes.io/loadbalancer: \"\"\n  blacklistRouterIDs:\n  - 1\n  - 2  \n```\n\nThis KeepalivedGroup will be deployed on all the nodes with role `loadbalancer`. Keepalived requires knowledge of the network device on which the VIPs will be exposed. If the interface name is the same on all nodes, it can be specified in the `interface` field. Alternatively, the `interfaceFromIP` field can be set to an IPv4 address to enable interface autodiscovery. In this scenario, the `interface` field will be ignored and each node in the KeepalivedGroup will expose the VIPs on the interface that would be used to reach the provided IP.\n\nServices must be annotated to opt-in to being observed by the keepalived operator and to specify which KeepalivedGroup they refer to. The annotation looks like this:\n\n`keepalived-operator.redhat-cop.io/keepalivedgroup: \u003ckeepalivedgroup namespace\u003e/\u003ckeepalivedgroup-name\u003e`\n\nThe image used for the keepalived containers can be specified with `.Spec.Image` it will default to `registry.redhat.io/openshift4/ose-keepalived-ipfailover` if undefined.\n\n## Requirements\n\n### Security Context Constraints\n\nEach KeepalivedGroup deploys a [daemonset](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) that requires the [privileged scc](https://docs.openshift.com/container-platform/4.5/authentication/managing-security-context-constraints.html), this permission must be given to the `default` service account in the namespace where the keepalived group is created by and administrator.\n\n```shell\noc adm policy add-scc-to-user privileged -z default-n \u003ckeepalivedgroup namespace\u003e\n```\n\n### Cluster Network Operator\n\nIn Openshift, use of an external IP address is governed by the following fields in the `Network.config.openshift.io` CR named `cluster`\n\n* `spec.externalIP.autoAssignCIDRs` defines an IP address block used by the load balancer when choosing an external IP address for the service. OpenShift supports only a single IP address block for automatic assignment.\n\n* `spec.externalIP.policy` defines the permissible IP address blocks when manually specifying an IP address. OpenShift does not apply policy rules to IP address blocks defined by `spec.externalIP.autoAssignCIDRs`\n\nThe following patch can be used to configure the Cluster Network Operator:\n\n```yaml\nspec:\n  externalIP:\n    policy:\n      allowedCIDRs:\n      - ${ALLOWED_CIDR}\n    autoAssignCIDRs:\n      - \"${AUTOASSIGNED_CIDR}\"\n```\n\nHere is an example of how to apply the patch:\n\n```shell\nexport ALLOWED_CIDR=\"192.168.131.128/26\"\nexport AUTOASSIGNED_CIDR=\"192.168.131.192/26\"\noc patch network cluster -p \"$(envsubst \u003c ./network-patch.yaml | yq r -j -)\" --type=merge\n```\n\nAdditionally, the fields can be edited manually via `oc edit Network.config.openshift.io cluster`\n\n## Blacklisting router IDs\n\nIf the Keepalived pods are deployed on nodes which are in the same network (same broadcast domain to be precise) with other keepalived the process, it's necessary to ensure that there is no collision between the used routers it.\nFor this purpose it is possible to provide a `blacklistRouterIDs` field with a list of black-listed IDs that will not be used.\n\n## Spreading VIPs across nodes to maximize load balancing\n\nIf a service contains multiple externalIPs or LoadBalancer IPs, it is possible to instruct keepalived-operator to maximize the spread of such VIPs across the nodes in the KeepalivedGroup by specifying the `keepalived-operator.redhat-cop.io/spreadvips: \"true\"` annotation on the service. This option ensures that different VIPs for the same service are always owned by different nodes (or, if the number of nodes in the group is less than the number of VIPs, that the VIPs are assigned maximizing the spread), to avoid creating a traffic bottleneck. However, in order to achieve this, keepalived-operator will create a separate VRRP instance per VIP of that service, which could exhaust the 256 available instances faster.\n\n## OpenShift RHV, vSphere, OSP and bare metal IPI instructions\n\nWhen IPI is used for RHV, vSphere, OSP or bare metal platforms, three keepalived VIPs are deployed. To make sure that keepalived-operator can work in these environment we need to discover and blacklist the corresponding VRRP router IDs.\n\nTo discover the VRRP router IDs being used, run the following command, you can run this command from you laptop:\n\n```shell\npodman run quay.io/openshift/origin-baremetal-runtimecfg:4.5 vr-ids \u003ccluster_name\u003e\n```\n\nIf you don't know your cluster name, run this command:\n\n```shell\npodman run quay.io/openshift/origin-baremetal-runtimecfg:4.5 vr-ids $(oc get cm cluster-config-v1 -n kube-system -o jsonpath='{.data.install-config}'| yq -r .metadata.name)\n```\n\nThen use these [instructions](#Blacklisting-router-IDs) to blacklist those VRRP router IDs.\n\n## Verbatim Configurations\n\nKeepalived has dozens of [configurations](https://www.keepalived.org/manpage.html). At the early stage of this project it's difficult to tell which one should be modeled in the API. Yet, users of this project may still need to use them. To account for that there is a way to pass verbatim options both at the keepalived group level (which maps to the keepalived config `global_defs` section) and at the service level (which maps to the keepalived config `vrrp_instance` section).\n\nKeepalivedGroup-level verbatim configurations can be passed as in the following example:\n\n```yaml\napiVersion: redhatcop.redhat.io/v1alpha1\nkind: KeepalivedGroup\nmetadata:\n  name: keepalivedgroup-router\nspec:\n  interface: ens3\n  nodeSelector:\n    node-role.kubernetes.io/loadbalancer: \"\"\n  verbatimConfig:  \n    vrrp_iptables: my-keepalived\n```\n\nthis will map to the following `global_defs`:\n\n```text\n    global_defs {\n        router_id keepalivedgroup-router\n        vrrp_iptables my-keepalived\n    }\n```\n\nService-level verbatim configurations can be passed as in the following example:\n\n```yaml\napiVersion: v1\nkind: Service\nmetadata:\n  annotations:\n    keepalived-operator.redhat-cop.io/keepalivedgroup: keepalived-operator/keepalivedgroup-router\n    keepalived-operator.redhat-cop.io/verbatimconfig: '{ \"track_src_ip\": \"\" }'\n```\n\nthis will map to the following `vrrp_instance` section\n\n```text\n    vrrp_instance openshift-ingress/router-default {\n        interface ens3\n        virtual_router_id 1  \n        virtual_ipaddress {\n          192.168.131.129\n        }\n        track_src_ip\n    }\n```\n\n## Advanced Users Only: Override Keepalived Configuration Template\n\n**NOTE**: This config customization feature can only be used via Helm.\n\nEach of the Keepalived daemon pods gets received it's configuration from a ConfigMap that gets generated by the Keepalived Operator from a configuration file template.\nIf you need to customize the configuration for your Keepalived daemon pods, you'll want to use the following steps.\n\nCreate a ConfigMap with the full contents of this configuration template file:\nhttps://github.com/redhat-cop/keepalived-operator/blob/master/config/templates/keepalived-template.yaml\n\n```\napiVersion: v1\nkind: ConfigMap\nmetadata:\nname: keepalived-template\nnamespace: {{ .KeepalivedGroup.ObjectMeta.Namespace }}\nlabels:\n  keepalivedGroup: {{ .KeepalivedGroup.ObjectMeta.Name }}    \ndata: \nkeepalived.conf: |\n    ...\n    # expected merge structure\n    # .KeepAlivedGroup\n    # .Services\n    - apiVersion: apps/v1\n      kind: DaemonSet\n      metadata:\n        name: {{ .KeepalivedGroup.ObjectMeta.Name }}\n        namespace: {{ .KeepalivedGroup.ObjectMeta.Namespace }}\n      spec:\n    ...\n```\n  \nThen in the Helm Chart set `keepalivedTemplateFromConfigMap: keepalived-template`\n\nThis will override the `/templates/keepalived-template.yaml` config file in the keepalived-operator pod which will allow you to update the configs without having to rebuild/push the operator docker image.\n\n\n## Metrics collection\n\nEach keepalived pod exposes a [Prometheus](https://prometheus.io/) metrics port at `9650`. Metrics are collected with [keepalived_exporter](github.com/gen2brain/keepalived_exporter), the available metrics are described in the project documentation.\n\nWhen a keepalived group is created a [`PodMonitor`](https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#podmonitor) rule to collect those metrics. All PodMonitor resources created that way have the label: `metrics: keepalived`. It is up to you to make sure your Prometheus instance watches for those `PodMonitor` rules. Here is an example of a fragment of a `Prometheus` CR configured to collect the keepalived pod metrics:\n\n```yaml\n  podMonitorSelector:\n    matchLabels:\n      metrics: keepalived\n```\n\nIn order to enable the collection of these metrics by the platform prometheus you have to appropriately label the namespace in which the `KeepalivedGroup` CR was created:\n\n```shell\noc label namespace \u003ckeepalived-group namespace\u003e openshift.io/cluster-monitoring=\"true\"\n```\n\n## Deploying the Operator\n\nThis is a cluster-level operator that you can deploy in any namespace, `keepalived-operator` is recommended.\n\nIt is recommended to deploy this operator via [`OperatorHub`](https://operatorhub.io/), but you can also deploy it using [`Helm`](https://helm.sh/).\n\n### Multiarch Support\n\n| Arch  | Support  |\n|:-:|:-:|\n| amd64  | ✅ |\n| arm64  | ✅  |\n| ppc64le  | ✅  |\n| s390x  | ❌  |\n\n### Deploying from OperatorHub\n\n\u003e **Note**: This operator supports being installed disconnected environments\n\nIf you want to utilize the Operator Lifecycle Manager (OLM) to install this operator, you can do so in two ways: from the UI or the CLI.\n\n#### Deploying from OperatorHub UI\n\n* If you would like to launch this operator from the UI, you'll need to navigate to the OperatorHub tab in the console.Before starting, make sure you've created the namespace that you want to install this operator to with the following:\n\n```shell\noc new-project keepalived-operator\n```\n\n* Once there, you can search for this operator by name: `keepalived`. This will then return an item for our operator and you can select it to get started. Once you've arrived here, you'll be presented with an option to install, which will begin the process.\n* After clicking the install button, you can then select the namespace that you would like to install this to as well as the installation strategy you would like to proceed with (`Automatic` or `Manual`).\n* Once you've made your selection, you can select `Subscribe` and the installation will begin. After a few moments you can go ahead and check your namespace and you should see the operator running.\n\n![Keepalived Operator](./media/keepalived-operator.png)\n\n#### Deploying from OperatorHub using CLI\n\nIf you'd like to launch this operator from the command line, you can use the manifests contained in this repository by running the following:\n\n```shell\noc new-project keepalived-operator\noc apply -f config/operatorhub -n keepalived-operator\n```\n\nThis will create the appropriate OperatorGroup and Subscription and will trigger OLM to launch the operator in the specified namespace.\n\n### Deploying with Helm\n\nHere are the instructions to install the latest release with Helm.\n\n```shell\noc new-project keepalived-operator\nhelm repo add keepalived-operator https://redhat-cop.github.io/keepalived-operator\nhelm repo update\nhelm install keepalived-operator keepalived-operator/keepalived-operator\n```\n\nThis can later be updated with the following commands:\n\n```shell\nhelm repo update\nhelm upgrade keepalived-operator keepalived-operator/keepalived-operator\n```\n\n## Metrics\n\nPrometheus compatible metrics are exposed by the Operator and can be integrated into OpenShift's default cluster monitoring. To enable OpenShift cluster monitoring, label the namespace the operator is deployed in with the label `openshift.io/cluster-monitoring=\"true\"`.\n\n```shell\noc label namespace \u003cnamespace\u003e openshift.io/cluster-monitoring=\"true\"\n```\n\n### Testing metrics\n\n```sh\nexport operatorNamespace=keepalived-operator-local # or keepalived-operator\noc label namespace ${operatorNamespace} openshift.io/cluster-monitoring=\"true\"\noc rsh -n openshift-monitoring -c prometheus prometheus-k8s-0 /bin/bash\nexport operatorNamespace=keepalived-operator-local # or keepalived-operator\ncurl -v -s -k -H \"Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\" https://keepalived-operator-controller-manager-metrics.${operatorNamespace}.svc.cluster.local:8443/metrics\nexit\n```\n\n## Development\n\n### Running the operator locally\n\n\u003e Note: this operator build process is tested with [podman](https://podman.io/), but some of the build files (Makefile specifically) use docker because they are generated automatically by operator-sdk. It is recommended [remap the docker command to the podman command](https://developers.redhat.com/blog/2020/11/19/transitioning-from-docker-to-podman#transition_to_the_podman_cli).\n\n```shell\nexport repo=raffaelespazzoli\ndocker login quay.io/$repo\noc new-project keepalived-operator\noc project keepalived-operator\ntilt up\n```\n\n### Test helm chart locally\n\nDefine an image and tag. For example...\n\n```shell\nexport imageRepository=\"quay.io/redhat-cop/keepalived-operator\"\nexport imageTag=\"$(git -c 'versionsort.suffix=-' ls-remote --exit-code --refs --sort='version:refname' --tags https://github.com/redhat-cop/keepalived-operator.git '*.*.*' | tail --lines=1 | cut --delimiter='/' --fields=3)\"\n```\n\nDeploy chart...\n\n```shell\nmake helmchart IMG=${imageRepository} VERSION=${imageTag}\nhelm upgrade -i keepalived-operator-local charts/keepalived-operator -n keepalived-operator-local --create-namespace\n```\n\nDelete...\n\n```shell\nhelm delete keepalived-operator-local -n keepalived-operator-local\nkubectl delete -f charts/keepalived-operator/crds/crds.yaml\n```\n\n### Building/Pushing the operator image\n\n```shell\nexport repo=raffaelespazzoli #replace with yours\ndocker login quay.io/$repo/keepalived-operator\nmake docker-build IMG=quay.io/$repo/keepalived-operator:latest\nmake docker-push IMG=quay.io/$repo/keepalived-operator:latest\n```\n\n### Deploy to OLM via bundle\n\n```shell\nmake manifests\nmake bundle IMG=quay.io/$repo/keepalived-operator:latest\noperator-sdk bundle validate ./bundle --select-optional name=operatorhub\nmake bundle-build BUNDLE_IMG=quay.io/$repo/keepalived-operator-bundle:latest\ndocker login quay.io/$repo/keepalived-operator-bundle\ndocker push quay.io/$repo/keepalived-operator-bundle:latest\noperator-sdk bundle validate quay.io/$repo/keepalived-operator-bundle:latest --select-optional name=operatorhub\noc new-project keepalived-operator\noc label namespace keepalived-operator openshift.io/cluster-monitoring=\"true\" --overwrite\noperator-sdk cleanup keepalived-operator -n keepalived-operator\noperator-sdk run bundle --install-mode AllNamespaces -n keepalived-operator quay.io/$repo/keepalived-operator-bundle:latest\n```\n\n## Integration Test\n\n```sh\nmake helmchart-test\n```\n\n### Testing\n\nAdd an external IP CIDR to your cluster to manage\n\n```shell\nexport CIDR=\"192.168.130.128/28\"\noc patch network cluster -p \"$(envsubst \u003c ./test/externalIP-patch.yaml | yq r -j -)\" --type=merge\n```\n\ncreate a project that uses a LoadBalancer Service\n\n```shell\noc new-project test-keepalived-operator\noc new-app django-psql-example -n test-keepalived-operator\noc delete route django-psql-example -n test-keepalived-operator\noc patch service django-psql-example -n test-keepalived-operator -p '{\"spec\":{\"type\":\"LoadBalancer\"}}' --type=strategic\nexport SERVICE_IP=$(oc get svc django-psql-example -n test-keepalived-operator -o jsonpath='{.status.loadBalancer.ingress[0].ip}')\n```\n\ncreate a keepalivedgroup\n\n```shell\noc adm policy add-scc-to-user privileged -z default -n test-keepalived-operator\noc apply -f ./test/keepalivedgroup.yaml -n test-keepalived-operator\n```\n\nannotate the service to be used by keepalived\n\n```shell\noc annotate svc django-psql-example -n test-keepalived-operator keepalived-operator.redhat-cop.io/keepalivedgroup=test-keepalived-operator/keepalivedgroup-test\n```\n\ncurl the app using the service IP\n\n```shell\ncurl http://$SERVICE_IP:8080\n```\n\ntest with a second keepalived group\n\n```shell\noc apply -f ./test/test-servicemultiple.yaml -n test-keepalived-operator\noc apply -f ./test/keepalivedgroup2.yaml -n test-keepalived-operator\noc apply -f ./test/test-service-g2.yaml -n test-keepalived-operator\n```\n\n### Releasing\n\n```shell\ngit tag -a \"\u003ctagname\u003e\" -m \"\u003ccommit message\u003e\"\ngit push upstream \u003ctagname\u003e\n```\n\nIf you need to remove a release:\n\n```shell\ngit tag -d \u003ctagname\u003e\ngit push upstream --delete \u003ctagname\u003e\n```\n\nIf you need to \"move\" a release to the current main\n\n```shell\ngit tag -f \u003ctagname\u003e\ngit push upstream -f \u003ctagname\u003e\n```\n\n### Cleaning up\n\n```shell\noperator-sdk cleanup keepalived-operator -n keepalived-operator\noc delete operatorgroup operator-sdk-og\noc delete catalogsource keepalived-operator-catalog\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredhat-cop%2Fkeepalived-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fredhat-cop%2Fkeepalived-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredhat-cop%2Fkeepalived-operator/lists"}