https://github.com/shuttl-tech/drone-autoscaler
Autoscale Drone CI agents
https://github.com/shuttl-tech/drone-autoscaler
autoscaling aws drone-ci
Last synced: about 2 months ago
JSON representation
Autoscale Drone CI agents
- Host: GitHub
- URL: https://github.com/shuttl-tech/drone-autoscaler
- Owner: Shuttl-Tech
- License: mpl-2.0
- Created: 2020-02-22T07:18:30.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2020-06-08T07:20:53.000Z (about 6 years ago)
- Last Synced: 2024-12-26T23:55:32.671Z (over 1 year ago)
- Topics: autoscaling, aws, drone-ci
- Language: Go
- Homepage:
- Size: 1.97 MB
- Stars: 3
- Watchers: 20
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# drone-autoscaler
This app scales a [Drone CI](https://drone.io/) agent cluster up or down based on build volume.
It was created because the [autoscaler provided by Drone](https://github.com/drone/autoscaler) has several limitations:
1. It doesn't support AWS's Autoscaling Groups (ASGs). Instead, it creates standalone ec2 instances. `drone-autoscaler` is designed to simply manipulate the desired capacity of the ASG that manages your Drone Agents.
2. It installs & configures an agent on a newly created machine. `drone-autoscaler` assumes that any new machine in the agent ASG is already provisioned.
3. It waits for upto 60 minutes for running builds to finish on an agent marked for termination. `drone-autoscaler` only terminates nodes that aren't running any builds.
## Usage
### Setup
This app assumes that your Drone agent cluster is managed by an AWS Autoscaling Group. New machinesi in this group should already be configured to run an agent upon startup and those being stopped/terminated should stop the agent gracefully.
The app needs access to IAM credentials with the following permissions:
```
ec2:DescribeTags
ec2:DescribeInstances
ec2:DescribeInstanceTypes
ec2:DescribeInstanceStatus
autoscaling:TerminateInstanceInAutoScalingGroup
autoscaling:SetInstanceProtection
autoscaling:SetDesiredCapacity
autoscaling:DetachInstances
autoscaling:DescribeScalingProcessTypes
autoscaling:DescribeScalingActivities
autoscaling:DescribeAutoScalingInstances
autoscaling:DescribeAutoScalingGroups"
```
### Configuration
The app's behaviour can be configured using various parameters
| Environment variable | Required |
| --- | ---- |
| `DRONE_AGENT_MAX_BUILDS` | Yes |
| `DRONE_AGENT_AUTOSCALING_GROUP` | Yes |
| `DRONE_SERVER_HOST` | Yes |
| `DRONE_SERVER_AUTH_TOKEN` | Yes |
| `SCALER_PROBE_INTERVAL` | No |
| `SCALER_LOG_FORMAT` | No |
| `SCALER_DEBUG` | No |
| `SCALER_DRY` | No |
| `DRONE_AGENT_MIN_RETIREMENT_AGE` | No |
| `DRONE_AGENT_MIN_COUNT` | No |
| `DRONE_SERVER_PROTO` | No |
| `DRONE_BUILD_PENDING_MAX_DURATION` | No |
| `DRONE_BUILD_RUNNING_MAX_DURATION` | No |
See [config.go](config/config.go) for parameter descriptions
Note that the autoscaler cannot scale beyond the maximum machine count set in your agent autoscaling group.
### Running
1. Download a pre-compiled binary from the releases page or build it from code using `make dist`.
2. Set the required configuration parameters via environment variables.
3. Run the acquired binary.
## Developing
The recommended way to run the app in development mode is to use the following configuration:
```bash
SCALER_LOG_FORMAT=text
SCALER_DEBUG=true
DRONE_BUILD_PENDING_MAX_DURATION="4h"
DRONE_BUILD_RUNNING_MAX_DURATION="1h"
# Outputs what the app plans to do, without making any changes
# to the actual infrastructure. This allows you to point the
# app to the actual agent ASG in development mode without
# worrying about accidental destruction of nodes.
SCALER_DRY=true
# If AWS profile is configured, use these params to load creds
# and all profile configuration.
AWS_PROFILE=my_profile
AWS_SDK_LOAD_CONFIG=true
```
Run `make fmt` to format the Go code. To run tests, use `make test`.
To create a new release, bump the app version in `main` and run `make dist`.