{"id":13821326,"url":"https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler","last_synced_at":"2025-05-16T12:33:07.540Z","repository":{"id":41467112,"uuid":"443913031","full_name":"DevOps-Nirvana/Kubernetes-Volume-Autoscaler","owner":"DevOps-Nirvana","description":"Autoscaling volumes for Kubernetes (with the help of Prometheus)","archived":false,"fork":false,"pushed_at":"2024-05-30T18:52:01.000Z","size":432,"stargazers_count":273,"open_issues_count":5,"forks_count":32,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-11-19T21:36:01.918Z","etag":null,"topics":["automation","autoscaling","aws","cloud-native","devops","devops-nirvana","disk","drive","ebs-volumes","helm","k8s","kubernetes","persistentvolume","persistentvolumeclaim","volume"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DevOps-Nirvana.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-03T01:49:36.000Z","updated_at":"2024-10-25T08:24:40.000Z","dependencies_parsed_at":"2024-05-19T01:28:21.964Z","dependency_job_id":"89403e47-72af-4e4b-83a3-c1b2b456ef82","html_url":"https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevOps-Nirvana%2FKubernetes-Volume-Autoscaler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevOps-Nirvana%2FKubernetes-Volume-Autoscaler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevOps-Nirvana%2FKubernetes-Volume-Autoscaler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevOps-Nirvana%2FKubernetes-Volume-Autoscaler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DevOps-Nirvana","download_url":"https://codeload.github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254530641,"owners_count":22086651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","autoscaling","aws","cloud-native","devops","devops-nirvana","disk","drive","ebs-volumes","helm","k8s","kubernetes","persistentvolume","persistentvolumeclaim","volume"],"created_at":"2024-08-04T08:01:20.013Z","updated_at":"2025-05-16T12:33:05.495Z","avatar_url":"https://github.com/DevOps-Nirvana.png","language":"Python","funding_links":[],"categories":["kubernetes"],"sub_categories":[],"readme":"# Kubernetes Volume / Disk Autoscaler (via Prometheus)\n\n\u003ca href=\"https://hub.docker.com/r/devopsnirvana/kubernetes-volume-autoscaler\"\u003e\u003cimg src=\"https://img.shields.io/docker/pulls/devopsnirvana/kubernetes-volume-autoscaler?style=plastic\" alt=\"Docker Hub Pulls\"\u003e\u003c/a\u003e \u003ca href=\"https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/releases/tag/1.0.8\"\u003e\u003cimg src=\"https://img.shields.io/docker/v/devopsnirvana/kubernetes-volume-autoscaler/1.0.8?label=Latest%20Release\u0026style=plastic\" alt=\"Latest Release\"\u003e\u003c/a\u003e \u003ca href=\"https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/stargazers\"\u003e\u003cimg src=\"https://img.shields.io/github/stars/DevOps-Nirvana/Kubernetes-Volume-Autoscaler?style=social\" alt=\"Stargazers on Github\"\u003e\u003c/a\u003e\n\nThis repository contains a [Kubernetes controller](https://kubernetes.io/docs/concepts/architecture/controller/) that automatically increases the size of a Persistent Volume Claim (PVC) in Kubernetes when it is nearing full (either on space OR inode usage). Initially engineered based on AWS EKS, this should support any Kubernetes cluster or cloud provider which supports dynamically hot-resizing storage volumes in Kubernetes.\n\nKeeping your volumes at a minimal size can help reduce cost, but having to manually scale them up can be painful and a waste of time for a DevOps / Systems Administrator. This is often used on storage volumes against things in Kubernetes such as [Prometheus](https://prometheus.io), [MySQL](https://artifacthub.io/packages/helm/bitnami/mysql), [Redis](https://artifacthub.io/packages/helm/bitnami/redis), [RabbitMQ](https://bitnami.com/stack/rabbitmq/helm), or any other stateful service.\n\n\u003cimg src=\"./.github/screenshot.resize.png\" alt=\"Screenshot of usage\"\u003e\n\n## Requirements\n\n- [Kubernetes 1.17+ Cluster](https://kubernetes.io/releases/)\n- [kubectl binary](https://kubernetes.io/docs/tasks/tools/#kubectl) installed and setup with your cluster\n- [The Helm 3.0+ binary](https://github.com/helm/helm/releases)\n- [Prometheus](https://prometheus.io) installed on your cluster [Example 1](https://artifacthub.io/packages/helm/prometheus-community/prometheus) / [Example 2 (old)](https://github.com/helm/charts/tree/master/stable/prometheus)\n- Using a Storage Class with `allowVolumeExpansion == true`\n- Using a Volume provisioner which supports dynamic volume expansion\n  - EKS default driver on 1.17+ does\n  - [AWS EBS CSI driver](https://github.com/kubernetes-sigs/aws-ebs-csi-driver) also does\n\n\n## Prerequisites\n\nAs mentioned above, you must have a StorageClass which supports volume expansion, and the provisioner you're using must also support volume expansion. Ideally, \"hot\"-volume expansion so your services never have to restart. AWS EKS built-in provisioner `kubernetes.io/aws-ebs` supports this, and so does the `efs.csi.aws.com` CSI driver.  To check/enable this...\n\n```bash\n# First, check if your storage class supports volume expansion...\n$ kubectl get storageclasses\nNAME      PROVISIONER  RECLAIMPOLICY  VOLUMEBINDINGMODE  ALLOWVOLUMEEXPANSION  AGE\nstandard  aws-ebs      Delete         Immediate          false                 10d\n\n# If ALLOWVOLUMEEXPANSION is not set to true, patch it to enable this\nkubectl patch storageclass standard -p '{\"allowVolumeExpansion\": true}'\n```\n\n**NOTE:** The above StorageClass comes with EKS, however, it only supports gp2, which is largely a deprecated and much slower storage driver than [gp3](https://aws.amazon.com/about-aws/whats-new/2020/12/introducing-new-amazon-ebs-general-purpose-volumes-gp3/). I HIHGLY recommend before using EKS you install the [AWS EBS CSI driver](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html) to gain [gp3](https://aws.amazon.com/about-aws/whats-new/2020/12/introducing-new-amazon-ebs-general-purpose-volumes-gp3/) support and more future-proof support of Amazon's various storage volumes and their lifecycles.\n\nIf you do this, you can/should completely remove GP2 support, and after installing the above CSI driver, create a StorageClass with the new driver with best-practices in it by default including...\n\n- Retain-ing the volume if it was deleted (to prevent accidental data loss)\n- Having all disks encrypted-at-rest by default, for compliance/security\n- Using gp3 by default for faster disk bandwidth and IO\n\n```bash\n# For this, simply delete your old default StorageClass\nkubectl delete storageclass standard\n# Then apply/create a new default gp3 using the AWS EBS CSI driver you installed\nkubectl apply -f https://raw.githubusercontent.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/master/examples/gp3-default-encrypt-retain-allowExpansion-storageclass.yaml\n```\n\n## Installation with Helm\n\nNow that your cluster has a `StorageClass` which supports expansion, you can install the Volume Autoscaler\n\n```bash\n# First, setup this repo for your Helm\nhelm repo add devops-nirvana https://devops-nirvana.s3.amazonaws.com/helm-charts/\n\n# Example Pre-Install - DRY RUN + VERBOSE, Using auto-discovery, must be in the same namespace as Prometheus\n#   Do this if you want to see what this script \"would\" do.  To view it, view the logs of this service after you deploy it\nhelm upgrade --install volume-autoscaler devops-nirvana/volume-autoscaler \\\n  --namespace REPLACEME_WITH_PROMETHEUS_NAMESPACE \\\n  --set \"dry_run=true\" \\\n  --set \"verbose=true\"\n\n# Example Install 1 - Using auto-discovery, must be in the same namespace as Prometheus\nhelm upgrade --install volume-autoscaler devops-nirvana/volume-autoscaler \\\n  --namespace REPLACEME_WITH_PROMETHEUS_NAMESPACE\n\n# Example 2 - Manually setting where Prometheus is\nhelm upgrade --install volume-autoscaler devops-nirvana/volume-autoscaler \\\n  --namespace ANYWHERE_DOESNT_MATTER \\\n  --set \"prometheus_url=http://prometheus-server.namespace.svc.cluster.local\"\n\n# Example 3 - Recommended final usage, auto-discovery and use Slack notifications.  NOTE: Check helm-chart/values.yml for other variables you can override globally\nhelm upgrade --install volume-autoscaler devops-nirvana/volume-autoscaler \\\n  --namespace REPLACEME_WITH_PROMETHEUS_NAMESPACE \\\n  --set \"slack_webhook_url=https://hooks.slack.com/services/123123123/4564564564/789789789789789789\" \\\n  --set \"slack_channel=my-slack-channel-name\" \\\n  --set \"slack_prefix=This is my dev cluster\"\n```\n\nThere are also various other variables you're able to set as easily as above. Please see: [values.yaml](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/blob/master/helm-chart/values.yaml#L9) for all the simple values to adjust this service. In addition, review the rest of that file if you'd like to adjust other things (like resource limits, node selectors, taints/tolerances, adding labels, etc).\n\n### Advanced Helm usage...\n```bash\n# To update your local knowledge of remote repos, you may need to do this before upgrading...\nhelm repo update\n\n# To view what changes it will make, if you change things, this requires the helm diff plugin - https://github.com/databus23/helm-diff\nhelm diff upgrade volume-autoscaler --allow-unreleased devops-nirvana/volume-autoscaler \\\n  --namespace infrastructure \\\n  --set \"slack_webhook_url=https://hooks.slack.com/services/123123123/4564564564/789789789789789789\" \\\n  --set \"slack_channel=my-slack-channel-name\" \\\n  --set \"prometheus_url=http://prometheus-server.infrastructure.svc.cluster.local\"\n\n# To remove the service, simply run...\nhelm uninstall volume-autoscaler\n```\n\n\n## (Alternate) Installation with `kubectl`\n\n```bash\n# This simple installation will work as long as you put this in the same namespace as Prometheus\n# The default namespace of this yaml is hardcoded to is `infrastructure`.  If you'd like to change\n# the namespace you can run the first few commands below...\n\n# IF YOU USE `infrastructure` AS THE NAMESPACE FOR PROMETHEUS SIMPLY...\nkubectl --namespace infrastructure apply https://devops-nirvana.s3.amazonaws.com/volume-autoscaler/volume-autoscaler-1.0.8.yaml\n\n# OR, IF YOU NEED TO CHANGE THE NAMESPACE...\n# #1: Download the yaml...\nwget https://devops-nirvana.s3.amazonaws.com/volume-autoscaler/volume-autoscaler-1.0.8.yaml\n# #1: Or download with curl\ncurl https://devops-nirvana.s3.amazonaws.com/volume-autoscaler/volume-autoscaler-1.0.8.yaml -o volume-autoscaler-1.0.8.yaml\n# #2: Then replace the namespace in this, replacing\ncat volume-autoscaler-1.0.8.yaml | sed 's/\"infrastructure\"/\"PROMETHEUS_NAMESPACE_HERE\"/g' \u003e ./to_be_applied.yaml\n# #3: If you wish to have slack notifications, edit this to_be_applied.yaml and embed your webhook on the value: line for SLACK_WEBHOOK and set the SLACK_CHANNEL as well accordingly\n# #4: Finally, apply it...\nkubectl --namespace REPLACEME_WITH_PROMETHEUS_NAMESPACE apply ./to_be_applied.yaml\n```\n\n## Validation\n\nTo confirm the volume autoscaler is working properly this repo has an example which you can apply to your Kubernetes cluster which is a PVC and a pod which uses that PVC and fills the disk up constantly. To do this...\n\n```bash\n# Simply run this on your terminal\nkubectl apply -f https://raw.githubusercontent.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/master/examples/simple-pod-with-pvc.yaml\n# Then if you'd like to follow-along, \"follow\" the logs of your volume autoscaler pod to watch it detect full disk and scale up.\nkubectl get pods | grep -i volume-autoscaler\nkubectl logs --since=10m --follow volume-autoscaler-pod-goes-here-from-the-above-command-output\n```\n\n## Usage Information and Nuances\n\n#### 1. Volume MUST be in use (pod is running with volume mounted)\n\nFor this software to even work, it has to know how much disk is in-use. Most/any cloud providers do not \"know\" this information, which is why on the Cloud Providers' portal you can typically not see \"how much percent of the disk is in use\" unless you run a special service in your node to send this to your provider [like this one for AWS](https://github.com/DevOps-Nirvana/aws-missing-tools/tree/master/aws-push-cloudwatch-instance-metrics). So, unless you're actively using the disk, Prometheus doesn't know how full a disk is if you aren't running a pod which has mounted this volume, and if Prometheus doesn't know then we don't know either.\n\n#### 2. Must have waited long enough since the last resize\n\nAdditionally, since cloud providers don't let you just constantly resize disks, you will have to wait the amount of time required between resizes. On AWS this is 6 hours, and once this service has done this once, it will show the following warning in the logs.\n\n\u003cimg src=\"./.github/screenshot.recently-scaled.png\" alt=\"Screenshot of usage\"\u003e\n\n\n## Per-Volume Configuration via Annotations\n\nThis controller also supports tweaking your volume-autoscaler configuration per-PVC with annotations.  The annotations supported are...\n\n```yaml\napiVersion: v1\nkind: PersistentVolumeClaim\nmetadata:\n  name: sample-volume-claim\n  annotations:\n    # This is when we want to scale up after the disk is this percentage (out of 100) full\n    volume.autoscaler.kubernetes.io/scale-above-percent: \"80\"   # 80 is the default value\n    # This is how many intervals must go by above the scale-above-percent before triggering an autoscale action\n    volume.autoscaler.kubernetes.io/scale-after-intervals: \"5\"  # 5 is this default value\n    # This is how much to scale a disk up by, in percentage of the current size.\n    #   Eg: If this is set to \"10\" and the disk is 100GB, it will scale to 110GB\n    #   At larger disk sizes you may want to set this on your PVCs to like \"5\" or \"10\"\n    volume.autoscaler.kubernetes.io/scale-up-percent: \"20\"      # 20 (percent) is the default value\n    # This is the smallest increment to scale up by.  This helps when the disks are very small, and helps hit the minimum increment value per-provider (this is 1GB on AWS)\n    volume.autoscaler.kubernetes.io/scale-up-min-increment: \"1000000000\"  # 1GB by default (in bytes)\n    # This is the maximum increment to scale up by.  This helps when the disks are very LARGE, to prevent them from growing too much at a time.\n    #   This can be used instead of or in addition to scale-up-percent\n    volume.autoscaler.kubernetes.io/scale-up-max-increment: \"\"  # Unset by default(in \"bytes\")\n    # This is the largest disk size ever allowed for this tool to scale up to.  This is set to 16TB by default, because that's the limit of AWS EBS\n    volume.autoscaler.kubernetes.io/scale-up-max-size: \"16000000000000\"  # 16TB by default (in bytes)\n    # How long (in seconds) we must wait before scaling this volume again.  For AWS EBS, this is 6 hours which is 21600 seconds but for good measure we add an extra 10 minutes to this, so 22200\n    volume.autoscaler.kubernetes.io/scale-cooldown-time: \"22200\"\n    # If you want the autoscaler to completely ignore/skip this PVC, set this to \"true\"\n    volume.autoscaler.kubernetes.io/ignore: \"false\"\n    # Finally, Do not set this, and if you see this ignore this, this is how Volume Autoscaler keeps its \"state\"\n    volume.autoscaler.kubernetes.io/last-resized-at: \"123123123\"  # This will be an Unix epoch timestamp\nspec:\n  accessModes:\n  - ReadWriteOnce\n  resources:\n    requests:\n      storage: 1Gi\n  storageClassName: standard\n```\n\n\n## Victoriametrics compatibility\n\nIf you are using victoriametrics in the helm chart simply set this value to true as follows...\n\n```bash\nhelm upgrade --install volume-autoscaler devops-nirvana/volume-autoscaler \\\n  --namespace REPLACEME_WITH_PROMETHEUS_NAMESPACE \\\n  --set \"victoriametrics_mode=true\"\n```\n\n\n# Mimir or Cortex compatibility\n\nIf you are using Mimir or Cortex you will need to add an authentication header into our requests to Prometheus for them to function.  To do this, simply set a value in our helm chart as follows.  For more details see the [Mimir documentation on Authentication](https://grafana.com/docs/mimir/latest/references/http-api/#authentication) or the [Grafana Mimir docs here](https://grafana.com/docs/mimir/latest/operators-guide/secure/authentication-and-authorization/#grafana-mimir-authentication-and-authorization)\n\n```bash\nhelm upgrade --install volume-autoscaler devops-nirvana/volume-autoscaler \\\n  --namespace REPLACEME_WITH_PROMETHEUS_NAMESPACE \\\n  --set \"scope_orgid_auth_header=tenant-1|tenant-2|tenant-3\"\n```\n\n\n## Prometheus Metrics Supported\n\nThis controller also supports publishing prometheus metrics automatically.  It hosts a simple http server on port 8000 and publishes the following metrics\n\n| Metric Name                                | Type    | Description                                                        |\n|--------------------------------------------|---------|--------------------------------------------------------------------|\n| volume_autoscaler_resize_evaluated_total   | counter | Increased every time we evaluate resizing PVCs                     |\n| volume_autoscaler_resize_attempted_total   | counter | Increased every time we attempt to resize                          |\n| volume_autoscaler_resize_successful_total  | counter | Increased every time we successfully resize                        |\n| volume_autoscaler_resize_failure_total     | counter | Increased every time we fail to resize                             |\n| volume_autoscaler_num_valid_pvcs           | gauge   | The number of valid PVCs detected which we found to consider       |\n| volume_autoscaler_num_pvcs_above_threshold | gauge   | The number of PVCs detected above the desired percentage threshold |\n| volume_autoscaler_num_pvcs_below_threshold | gauge   | The number of PVCs detected below the desired percentage threshold |\n| volume_autoscaler_release_info             | info    | Version information in this volume autoscaler service (in label)   |\n| volume_autoscaler_settings_info            | info    | Settings currently used in this service (in labels)                |\n\n\n# Release History\n\n### [Current Release: 1.0.8 - October 17, 2023](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/releases/tag/1.0.8)\n```\n* Implemented scaling on reaching inode count limits (#18)\n```\n\n### [Release: 1.0.7 - September 17, 2023](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/releases/tag/1.0.7)\n```\n* Add GitHub workflow for automated docker build by @piec in (#12)\n* fix: various exception/failure path undefined by @24601 in (#14)\n* Fixing vulnerabilities September 2023 by @AndrewFarley in (#16)\n* Add Slack Prefix/Suffix support\n* Adding debounce to prevent two changes from being made to the same volume too quickly (10x interval time)\n* Updating helm chart for Kubernetes 1.21-1.26 support (tested/validated)\n* Adding Cortex/Mimir support\n* Updating to Python 3.11 Alpine 3.18 (related to security vulnerabilities)\n* Minor quality of life improvements (documentation, comments, improved startup message, etc)\n```\n\n### [Release: 1.0.6 - Oct 26, 2022](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/releases/tag/1.0.6)\n```\nMinor textual changes, grammar, etc (thanks @anthea-w)\nImplement victoriametrics mode (#8) (thanks @martin31821)\nUpgrade base docker image to python:3.9.16-alpine3.17 (#5) (thanks @sindvero / @abenoist)\nModified helm chart to add Linux node selector, as this only on linux\nAdding compatibility for up at least Kubernetes 1.25 (tested)\nFixing a fatal error that occurred\n```\n\n### [Release: 1.0.5 - Oct 26, 2022](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/releases/tag/1.0.5)\n```\nHandling low max disk size edge-case better (thanks @GuillaumeOuint)\nHuman-readable debug output much improved\n```\n\n### [Release: 1.0.4 - Oct 24, 2022](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/releases/tag/1.0.4)\n```\nGenerate informational Prometheus metrics (version number and settings)\nGenerate usage \u0026 health Prometheus metrics (number of pvcs, number of resizes, etc)\nUpdating upstream universal Helm chart\nScaled down default resize percentage to 20% (down from 50%) based on feedback\nUpdated to Python 3.9.15 (thanks @pblgomez)\n```\n\n### [Release: 1.0.3 - Jan 24, 2022](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/releases/tag/1.0.3)\n```\nHandle signal from Kubernetes to kill/restart properly/quickly\nAdd full env vars as documentation markdown table, inside notes for development section below\nAdding better exception logs via traceback, and more readable/reasonable log output especially when VERBOSE is enabled\nGenerate Kubernetes events so everyone viewing the event stream knows when actions occur. AKA, Be an responsible controller\n```\n\n### [Release: 1.0.2 - Jan 15, 2022](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/releases/tag/1.0.2)\n```\nAutomatically detecting version of Prometheus and using newer functions to de-bounce invalid PVCs automatically\nAdding max-increment annotation/variable support\nAdding exception handling in our main loop to handle jitter nicely and not fail catastrophically if someone has bad PVC annotations\nMaking all variables settable by a value in the Helm chart\nAdding verbose support, which when enabled prints out full data from the objects detected, and prints out even non-alerting disks\nPrinting the number of PVCs found in the log, useful when not in verbose mode\n```\n\n### [Release: 1.0.1 - Jan 4, 2022](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/releases/tag/1.0.1)\n```\nInitial Public Release\nBasic PVC Annotation support\nPublishing a Helm Chart\nA few variables configurable in the Helm Chart values\nSome basic documentation and installation help/guides\n```\n\n# TODO\n\nSee [issues](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/issues) and [Random Feature Ideas](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/issues/17)\n\n\n\n# Notes for Development\n\nThis tool can easily be run locally, as long as you have an functioning kubectl config, and as long as you can reach the http(s) endpoint for your cluster's prometheus.  If your prometheus is internal-only, you may need to have an VPN setup into your VPC/Cluster.  I recommend OpenVPN if you're installing one in your Kubernetes cluster.\n\n```bash\n# To test out/develop new features, first get your Prometheus IP address...\nkubectl get services --all-namespaces | grep -i prometheus-server\ninfrastructure  prometheus-server  ClusterIP  10.1.0.102  \u003cnone\u003e  80/TCP  109d\n# Then take its IP address and check if you can use it...\ncurl http://10.1.0.102\n\u003ca href=\"/graph\"\u003eFound\u003c/a\u003e\n# Alternatively, if your ops/devops person setup an ingress to make this accessible externally...\ncurl https://prometheus.mycompany.com\n# Alternatively, if you wish to port forward to your cluster...\nkubectl port-forward svc/prometheus-server 8001:80 \u0026\n# and check if it works...\ncurl http://localhost:8001\n# Once you have established a functioning URL to Prometheus, put it in the following command\n# and you'll be off and running in a safe way that won't affect anything because of DryRun\nVERBOSE=true DRY_RUN=true PROMETHEUS_URL=http://PUT_PROMETHEUS_URL_FROM_ABOVE_HERE ./main.py\n# Of course, remove DRY_RUN above if you want to actually have the software try to scale your disks by patching the PVC desired storage resources\n```\n\nThe following environment variables are settable during development to alter the default logic. These are also settable via the Helm Chart in values, and overridable [per-PVC in Annotations](#per-volume-configuration-via-annotations)\n\n| Variable Name          | Default        | Description |\n|------------------------|----------------|-------------|\n| INTERVAL_TIME          | 60             | How often (in seconds) to scan Prometheus for checking if we need to resize |\n| SCALE_ABOVE_PERCENT    | 80             | What percent out of 100 the volume must be consuming before considering to scale it |\n| SLACK_WEBHOOK          |                | A Slack webhook to automatically send alerts to upon resizing |\n| SLACK_CHANNEL          | devops         | The default Slack channel to send alerts to (if your webhook is allowed to send to different channels) |\n| SLACK_MESSAGE_PREFIX   |                | A prefix added to every Slack message send, to alert or inform you of what cluster it is on |\n| SLACK_MESSAGE_SUFFIX   |                | A suffix added to every Slack message send, to alert or inform you of what cluster it is on |\n| SCALE_AFTER_INTERVALS  | 5              | How many intervals of INTERVAL_TIME a volume must be above SCALE_ABOVE_PERCENT before we scale |\n| SCALE_UP_PERCENT       | 20             | How much percent of the current volume size to scale up by. (100 == (if disk is 10GB, scale to 20GB), eg: 20 == (if disk is 10GB, scale to 12GB) |\n| SCALE_UP_MIN_INCREMENT | 1000000000     | How many bytes is the minimum that we can resize up by, default is 1GB (in bytes, so 1000000000) |\n| SCALE_UP_MAX_INCREMENT | 16000000000000 | How many bytes is the maximum that we can resize up by, default is 16TB (in bytes, so 16000000000000) |\n| SCALE_UP_MAX_SIZE      | 16000000000000 | How many bytes is the maximum disk size that we can resize up, default is 16TB for EBS volumes in AWS (in bytes) |\n| SCALE_COOLDOWN_TIME    | 22200          | How long (in seconds) we must wait before scaling this volume again. For AWS EBS, this is 6 hours which is 21600 seconds but for good measure we add an extra 10 minutes to this, so 22200 |\n| PROMETHEUS_URL         | `auto-detect`  | Where Prometheus is, if not provided it can auto-detect it if it's in the same namespace as this Volume Autoscaler |\n| DRY_RUN                | false          | If we want to dry-run this, aka don't do any actions, only report (for dev/testing/poc purposes) |\n| PROMETHEUS_LABEL_MATCH |                | A PromQL label query to restrict volumes for this to see and scale, without braces. eg: 'namespace=\"dev\"' |\n| HTTP_TIMEOUT           | 15             | Allows to set the timeout for calls to Prometheus and Kubernetes. Adjust this if your Prometheus or Kubernetes is over a remote WAN link with high latency and/or is heavily loaded |\n| VERBOSE                | false          | If we want to verbose mode, prints out the raw data from each PVC and its status/state instead of the default \"\" |\n| VICTORIAMETRICS_COMPAT  | false          | Whether to skip the prometheus check and assume victoriametrics |\n| SCOPE_ORGID_AUTH_HEADER |                | The auth header to set when using Mimir or Cortex see [Mimir docs](https://grafana.com/docs/mimir/latest/references/http-api/#authentication) |\n\n\n# Contributors\n\nThanks for [your contributions](https://github.com/DevOps-Nirvana/Kubernetes-Volume-Autoscaler/graphs/contributors) both in the form of filing issues, PRs, and emailing me occasionally about this project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDevOps-Nirvana%2FKubernetes-Volume-Autoscaler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FDevOps-Nirvana%2FKubernetes-Volume-Autoscaler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDevOps-Nirvana%2FKubernetes-Volume-Autoscaler/lists"}