{"id":20510313,"url":"https://github.com/frobware/openshift-keda-examples","last_synced_at":"2025-03-05T22:26:09.066Z","repository":{"id":65183873,"uuid":"516332585","full_name":"frobware/openshift-keda-examples","owner":"frobware","description":"Scale OpenShift ingresscontroller CRD using KEDA","archived":false,"fork":false,"pushed_at":"2024-04-02T12:28:26.000Z","size":7408,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-24T07:38:28.565Z","etag":null,"topics":["autoscaler","custom-metrics-autoscaler","hpa","ingress","ingress-controller","keda","kubernetes","openshift","prometheus"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frobware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-21T10:51:55.000Z","updated_at":"2022-08-30T09:17:21.000Z","dependencies_parsed_at":"2024-06-20T17:28:54.933Z","dependency_job_id":null,"html_url":"https://github.com/frobware/openshift-keda-examples","commit_stats":{"total_commits":18,"total_committers":3,"mean_commits":6.0,"dds":"0.11111111111111116","last_synced_commit":"4ea8fb73f86e72625ed81acc5e12acd7f7f258bc"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frobware%2Fopenshift-keda-examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frobware%2Fopenshift-keda-examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frobware%2Fopenshift-keda-examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frobware%2Fopenshift-keda-examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frobware","download_url":"https://codeload.github.com/frobware/openshift-keda-examples/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242112680,"owners_count":20073650,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autoscaler","custom-metrics-autoscaler","hpa","ingress","ingress-controller","keda","kubernetes","openshift","prometheus"],"created_at":"2024-11-15T20:28:56.819Z","updated_at":"2025-03-05T22:26:09.043Z","avatar_url":"https://github.com/frobware.png","language":"Go","readme":"# Scale OpenShift ingresscontroller using Custom Metrics Autoscaler (KEDA)\n\nThis repo is an example of:\n\n- Installing OpenShift [Custom Metrics Autoscaler](https://cloud.redhat.com/blog/custom-metrics-autoscaler-on-openshift) operator on your cluster\n- Creating Custom Metrics Autoscaler resources (triggerauthentication, scaledobject, etc)\n- Scaling the default ingresscontroller based on the number of nodes\n\n## Useful links / prior work\n\n- https://github.com/zroubalik/keda-openshift-examples/tree/main/prometheus\n- https://access.redhat.com/articles/6718611\n- https://issues.redhat.com/browse/RHSTOR-1938\n\n## Prerequisites\n\n```sh\n$ git clone https://github.com/frobware/openshift-keda-examples \u0026\u0026 cd openshift-keda-examples\n```\n\n# Autoscaling the default ingress controller\n\n## Install Custom Metrics Autoscaler from OperatorHub\n\nIn the OperatorHub locate and install Custom Metrics Autoscaler.\n\n![Custom Metrics Adapter](screenshots/custom-metrics-adapter.png?raw=true \"Custom Metrics Autoscaler\")\n\nOnce the operator is installed also create a `KedaController`\ninstance. The documentation in the OperatorHub offers a 1-click option\nto do this once the operator is running.\n\n![Create KEDA controller instance](screenshots/create-keda-controller-instance.png?raw=true \"Create KEDA controller instance\")\n\nA functioning Custom Metrics Autoscaler setup will have 3 pods\nrunning:\n\n```sh\n% oc get pods -n openshift-keda\nNAME                                                  READY   STATUS    RESTARTS   AGE\ncustom-metrics-autoscaler-operator-5b865f6b96-p2jqw   1/1     Running   0          20h\nkeda-metrics-apiserver-7bb57b45b9-vxvmt               1/1     Running   0          20h\nkeda-operator-bd446d79c-skxjk                         1/1     Running   0          20h\n```\n\n## Do all operations in the openshift-ingress-operator namespace:\n\n```sh\n$ oc project openshift-ingress-operator\n```\n\n## Enable OpenShift monitoring for user-defined projects\n\nPlease refer to the\n[documentation](https://docs.openshift.com/container-platform/4.9/monitoring/enabling-monitoring-for-user-defined-projects.html),\nor just apply the following ConfigMap:\n\n```sh\n$ oc apply -f - \u003c\u003cEOF\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: cluster-monitoring-config\n  namespace: openshift-monitoring\ndata:\n  config.yaml: |\n    enableUserWorkload: true\nEOF\n```\n\n## Create a Service Account\n\nYou need a Service Account to authenticate with Thanos:\n\n```sh\n$ oc create serviceaccount thanos\n$ oc describe serviceaccount thanos\nName:                thanos\nNamespace:           openshift-ingress-operator\nLabels:              \u003cnone\u003e\nAnnotations:         \u003cnone\u003e\nImage pull secrets:  thanos-dockercfg-b4l9s\nMountable secrets:   thanos-dockercfg-b4l9s\nTokens:              thanos-token-c422q\nEvents:              \u003cnone\u003e\n```\n\n## Define a TriggerAuthentication with the Service Account's token\n\n```sh\n$ secret=$(oc get secret | grep thanos-token | head -n 1 | awk '{ print $1 }')\n$ echo $secret\nthanos-token-c422q\n\n$ oc process TOKEN=\"$secret\" -f - \u003c\u003cEOF | oc apply -f -\napiVersion: template.openshift.io/v1\nkind: Template\nparameters:\n- name: TOKEN\nobjects:\n- apiVersion: keda.sh/v1alpha1\n  kind: TriggerAuthentication\n  metadata:\n    name: keda-trigger-auth-prometheus\n  spec:\n    secretTargetRef:\n    - parameter: bearerToken\n      name: \\${TOKEN}\n      key: token\n    - parameter: ca\n      name: \\${TOKEN}\n      key: ca.crt\nEOF\n```\n\n## Create a role for reading metrics from Thanos\n\n```sh\n$ oc apply -f - \u003c\u003cEOF\napiVersion: rbac.authorization.k8s.io/v1\nkind: Role\nmetadata:\n  name: thanos-metrics-reader\nrules:\n- apiGroups:\n  - \"\"\n  resources:\n  - pods\n  - nodes\n  verbs:\n  - get\n- apiGroups:\n  - metrics.k8s.io\n  resources:\n  - pods\n  - nodes\n  verbs:\n  - get\n  - list\n  - watch\n- apiGroups:\n  - \"\"\n  resources:\n  - namespaces\n  verbs:\n  - get\nEOF\n```\n\n## Add the role for reading metrics from Thanos to the Service Account\n\n```sh\n$ oc adm policy add-role-to-user thanos-metrics-reader -z thanos --role-namespace=openshift-ingress-operator\n$ oc adm policy -n openshift-ingress-operator add-cluster-role-to-user cluster-monitoring-view -z thanos\n```\n\nThe previous `add-cluster-role-to-user` step is only required if you\nuse cross-namespace queries (which our examples will do); if you are\nrunning through the steps in this README then you will need to run the\n`add-cluster-role-to-user` step.\n\nOur scaledobject examples, which all run in the\n`openshift-ingress-operator` namespace, use a metric from the\n`kube-metrics` namespace (i.e., cross-namespace).\n\nIf you don't run both the `oc adm policy` steps above then the\nscaledobject will remain in a non-active state due to lack of\npermissions:\n\n```\n    $ oc get scaledobject\n    NAME             SCALETARGETKIND                              SCALETARGETNAME   MIN   MAX   TRIGGERS     AUTHENTICATION                 READY   ACTIVE   FALLBACK   AGE\n    ingress-scaler   operator.openshift.io/v1.IngressController   default           1     20    prometheus   keda-trigger-auth-prometheus   True    False    True       99s\n```\n\n## Create a scaledobject for scaling the default ingresscontroller\n\nFirst let's verify how many replicas we currently have for the\n`default` ingresscontroller; We are expecting 2:\n\n```sh\n$ oc get ingresscontroller/default -o yaml | grep replicas:\nreplicas: 2\n\n$ oc get pods -n openshift-ingress\nNAME                             READY   STATUS    RESTARTS   AGE\nrouter-default-7b5df44ff-l9pmm   2/2     Running   0          17h\nrouter-default-7b5df44ff-s5sl5   2/2     Running   0          3d21h\n```\n\nCreate a new scaledobject targeting the `default` ingresscontroller\ndeployment:\n\n```sh\n$ oc apply -f - \u003c\u003cEOF\napiVersion: keda.sh/v1alpha1\nkind: ScaledObject\nmetadata:\n  name: ingress-scaler\nspec:\n  scaleTargetRef:\n    apiVersion: operator.openshift.io/v1\n    kind: IngressController\n    name: default\n    envSourceContainerName: ingress-operator\n  minReplicaCount: 1\n  maxReplicaCount: 20\n  cooldownPeriod: 1\n  pollingInterval: 1\n  triggers:\n  - type: prometheus\n    metricType: AverageValue\n    metadata:\n      serverAddress: https://thanos-querier.openshift-monitoring.svc.cluster.local:9091\n      namespace: openshift-ingress-operator\n      metricName: 'kube-node-role'\n      threshold: '1'\n      query: 'sum(kube_node_role{role=\"worker\",service=\"kube-state-metrics\"})'\n      authModes: \"bearer\"\n    authenticationRef:\n      name: keda-trigger-auth-prometheus\nEOF\n```\n\nTo target a custom resource (i.e., the ingresscontroller) the changes\nover and above the different query amount to:\n\n```console\n   scaleTargetRef:\n     apiVersion: operator.openshift.io/v1\n     kind: IngressController\n     name: default\n     envSourceContainerName: ingress-operator\n```\n\nAnd we're purposely using a different query to prove that we can use\nmetrics that already exist in a deployed cluster:\n\n```sh\n$ oc apply -f ./ingresscontroller/scale-on-kube-node-role.yaml\nscaledobject.keda.sh/ingress-scaler created\n\n$ oc get scaledobject\nNAME             SCALETARGETKIND                              SCALETARGETNAME   MIN   MAX   TRIGGERS     AUTHENTICATION                 READY   ACTIVE   FALLBACK   AGE\ningress-scaler   operator.openshift.io/v1.IngressController   default           1     20    prometheus   keda-trigger-auth-prometheus   False   True     Unknown    4s\n\n$ oc get hpa\nNAME                      REFERENCE                   TARGETS             MINPODS   MAXPODS   REPLICAS   AGE\nkeda-hpa-ingress-scaler   IngressController/default   \u003cunknown\u003e/1 (avg)   1         20        0          7s\n```\n\nWaiting a little while we see the `default` ingresscontroler scaled\nout to 3 replicas which matches our kube-state-metrics query.\n\n```sh\n$ oc get ingresscontroller/default -o yaml | grep replicas:\nreplicas: 3\n\n$ oc get pods -n openshift-ingress\nNAME                             READY   STATUS    RESTARTS   AGE\nrouter-default-7b5df44ff-l9pmm   2/2     Running   0          17h\nrouter-default-7b5df44ff-s5sl5   2/2     Running   0          3d22h\nrouter-default-7b5df44ff-wwsth   2/2     Running   0          66s\n```\n\n![kube-state-metrics](screenshots/kube-state-metrics.png?raw=true \"kube-state-metrics\")\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrobware%2Fopenshift-keda-examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrobware%2Fopenshift-keda-examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrobware%2Fopenshift-keda-examples/lists"}