https://github.com/iamazeem/cwm-keda-external-scaler
CWM KEDA external scaler for scaling workers (https://github.com/CloudWebManage)
https://github.com/iamazeem/cwm-keda-external-scaler
k8s keda keda-external-scaler kubernetes kubernetes-deployment metrics scaledobject scaling
Last synced: 6 months ago
JSON representation
CWM KEDA external scaler for scaling workers (https://github.com/CloudWebManage)
- Host: GitHub
- URL: https://github.com/iamazeem/cwm-keda-external-scaler
- Owner: iamazeem
- License: mit
- Created: 2021-03-20T08:45:13.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2024-02-22T08:40:52.000Z (over 1 year ago)
- Last Synced: 2024-11-18T02:46:32.978Z (7 months ago)
- Topics: k8s, keda, keda-external-scaler, kubernetes, kubernetes-deployment, metrics, scaledobject, scaling
- Language: Go
- Homepage: https://github.com/iamAzeem/cwm-keda-external-scaler
- Size: 210 KB
- Stars: 4
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# cwm-keda-external-scaler
[](https://github.com/iamAzeem/cwm-keda-external-scaler/actions/workflows/ci.yml)
[](https://github.com/iamAzeem/cwm-keda-external-scaler/blob/main/LICENSE)
[CWM](https://github.com/CloudWebManage) [KEDA](https://keda.sh/)
[external scaler](https://keda.sh/docs/2.2/concepts/external-scalers/) scales
the target resrouce based on the metrics fetched from the Redis server. The
metrics are generated by the aggregated MinIO logs and pushed to the Redis
server by the
[cwm-worker-logger](https://github.com/CloudWebManage/cwm-worker-logger) using
[fluent-plugin-http-cwm](https://github.com/iamAzeem/fluent-plugin-http-cwm) as
its underlying logging engine. The Redis server stores a number of
[metrics](https://github.com/CloudWebManage/fluent-plugin-http-cwm#overview).
Based on its local configuration, the external scaler fetches only the required
one(s) to make its scaling decisions.
## Flow Diagram
```text
CONFIGURATION (global and local)
------------------------------------
Env Variables: { CWM_REDIS_HOST, ... }
{metrics} ScaledObject : { deploymentid, ... }
| |
| |
| |
v v
+-------------------+ +-------------------+
| | {metric} | |
| Redis Server |------------->| External Scaler |
| | | |
+-------------------+ +-------------------+
|
|
v
+-------------------+
| |
| Kubernetes |
| |
+-------------------+
|
| scale
v
+-------------------+
| |
| Target Resource |
| |
+-------------------+
```
## Configuration
The external scaler listens on port `50051`.
### Global Configuration: Environment Variables
| Environment Variable | Default Value | Description |
|:-----------------------------:|:-----------------------------:|:--------------------------------------|
| `CWM_REDIS_HOST` | `localhost` | ip/host of the Redis metrics server |
| `CWM_REDIS_PORT` | `6379` | port of the Redis metrics server |
| `CWM_REDIS_DB` | `0` | Redis database to use |
| `LAST_UPDATE_PREFIX` | `deploymentid:last_action` | prefix for last update key |
| `METRICS_PREFIX` | `deploymentid:minio-metrics` | prefix for metrics key |
### Local Configuration: Metadata in ScaledObject
Here is the generic YAML format of a `ScaledObject`:
```yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name:
namespace:
spec:
scaleTargetRef:
name:
triggers:
- type: external
metadata: # <<< local configuration >>> #
scalerAddress:
key1:
key2:
```
The local configuration will be under `metadata`:
```yaml
# ...
spec:
# ...
triggers:
- type: external
metadata:
# <<< local confiugration >>>
```
The following table lists the supported local configuration:
| Configuration (Key) | Default Value | Description |
|:-----------------------------:|:---------------:|:------------------------------------------------------|
| `deploymentid` | `minio` | value to append to the prefix |
| `isActiveTtlSeconds` | `600` | seconds since last update for workload to be active |
| `scaleMetricName` | `bytes_out` | metric for scaling (listed below) |
| `scalePeriodSeconds` | `600` | retention time for the metric value |
| `targetValue` | `10` | target value reported by the autoscaler |
Here are the supported options for `scaleMetricName`:
| Metric Name | Description |
|:-----------------------------:|:------------------------------------------------------------------------|
| `bytes_in` | number of incoming bytes |
| `bytes_out` | number of outgoing bytes (default) |
| `num_requests_in` | number of incoming requests (`WebUpload`, `PutObject`, `DeleteObject`) |
| `num_requests_out` | number of outgoing requests (`WebDownload`, `GetObject`) |
| `num_requests_misc` | number of requests other than input and output |
| `bytes_total` | `bytes_in` + `bytes_out` |
| `num_requests_in_out` | `num_requests_in` + `num_requests_out` |
| `num_requests_total` | `num_requests_in` + `num_requests_out` + `num_requests_misc` |
### Sample Configuration
Here's the
[configuration](https://keda.sh/docs/2.1/concepts/scaling-deployments/#scaledobject-spec)
format of a `ScaledObject`:
```yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: {scaled-object-name}
spec:
scaleTargetRef:
apiVersion: {api-version-of-target-resource} # Optional. Default: apps/v1
kind: {kind-of-target-resource} # Optional. Default: Deployment
name: {name-of-target-resource} # Mandatory. Must be in the same namespace as the ScaledObject
envSourceContainerName: {container-name} # Optional. Default: .spec.template.spec.containers[0]
pollingInterval: 30 # Optional. Default: 30 seconds
cooldownPeriod: 300 # Optional. Default: 300 seconds
minReplicaCount: 0 # Optional. Default: 0
maxReplicaCount: 100 # Optional. Default: 100
advanced: # Optional. Section to specify advanced options
restoreToOriginalReplicaCount: true/false # Optional. Default: false
horizontalPodAutoscalerConfig: # Optional. Section to specify HPA related options
behavior: # Optional. Use to modify HPA's scaling behavior
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 100
periodSeconds: 15
triggers:
# {list of triggers to activate scaling of the target resource}
```
Assuming that the global configuration via environment variables has properly
been set, the external scaler (`cwm-keda-external-scaler`) can be configured
as a `ScaledObject` under `triggers` like this:
```yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: {scaled-object-name}
spec:
scaleTargetRef:
name: {name-of-target-resource}
pollingInterval: 10
triggers:
- type: external
metadata:
scalerAddress: {host:port} # Mandatory.
deploymentid: {deployment-id} # Optional. Default: deploymentid
isActiveTtlSeconds: {seconds} # Optional. Default: 600
scaleMetricName: {supported-metric-name} # Optional. Default: bytes_out
scalePeriodSeconds: {seconds} # Optional. Default: 600
targetValue: {target-value} # Optional. Default: 10
```
## Build Docker Image
```shell
docker build -t cwm-keda-external-scaler:latest .
```
## Testing
### Prerequisite
| Software | Version |
|:-------------:|:-------------:|
| minikube | v1.12.3 |
| Kubernetes | v1.16.4 |
| KEDA | v2.1.0 |
Make sure that KEDA is already deployed and running (use v2.1.0 via YAML file):
```shell
# apply
kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.1.0/keda-2.1.0.yaml
# delete
kubectl delete -f https://github.com/kedacore/keda/releases/download/v2.1.0/keda-2.1.0.yaml
```
### Deploy
**Terminal-1**: Watch resources in all the namespaces
```shell
watch -x kubectl get all --all-namespaces
```
**Terminal-2**: Apply test [deployment](./test/deploy.yaml)
```shell
kubectl apply -f ./test/deploy.yaml
```
**Terminal-3**: Check logs of `pod/keda-operator-*` in `keda` namespace
```shell
kubectl logs -f -n keda pod/keda-operator-*
```
**Terminal-4**: Check logs of `pod/keda-operator-metrics-apiserver-*` in `keda` namespace
```shell
kubectl logs -f pod/keda-operator-metrics-apiserver-* -n keda
```
**Terminal-5**: Check logs of the custom external scaler `cwm-keda-external-scaler`
```shell
kubectl logs -f -n cwm-keda-external-scaler-ns pod/cwm-keda-external-scaler-* cwm-keda-external-scaler
```
**NOTE**: The trailing `*` in above `pod/-*` format denotes the actual
complete name of the pod.
### Run Tests
Start `minikube`:
```shell
minikube start --driver=docker --kubernetes-version=v1.16.14
```
Run:
```shell
./test/run_scaling_tests.sh
```
For details, please navigate to [test](./test) subdirectory.
## Contribute
- Fork the project.
- Check out the latest `main` branch.
- Create a feature or bugfix branch from `main`.
- Commit and push your changes.
- Make sure to add tests.
- Test locally.
- Submit the PR.
## License
[MIT](LICENSE)