{"id":16701674,"url":"https://github.com/coryodaniel/ballast","last_synced_at":"2025-07-11T05:41:37.623Z","repository":{"id":66508384,"uuid":"183547268","full_name":"coryodaniel/ballast","owner":"coryodaniel","description":"Ballast manages kubernetes node pools to give you the cost of preemptible nodes with the confidence of on demand nodes.","archived":false,"fork":false,"pushed_at":"2019-10-08T21:39:54.000Z","size":157,"stargazers_count":16,"open_issues_count":10,"forks_count":0,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-24T05:34:45.191Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coryodaniel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-26T03:04:37.000Z","updated_at":"2024-05-17T20:38:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"ba530251-c130-44db-98c6-84d6f5b51b56","html_url":"https://github.com/coryodaniel/ballast","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coryodaniel%2Fballast","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coryodaniel%2Fballast/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coryodaniel%2Fballast/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coryodaniel%2Fballast/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coryodaniel","download_url":"https://codeload.github.com/coryodaniel/ballast/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248154994,"owners_count":21056543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T18:45:08.774Z","updated_at":"2025-04-10T04:12:10.863Z","avatar_url":"https://github.com/coryodaniel.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Ballast\n\nBallast manages kubernetes node pools to give you the cost of preemptible nodes with the confidence of on demand nodes.\n\n- [Ballast](#Ballast)\n  - [Getting Started](#Getting-Started)\n    - [Create a GCP Service Account](#Create-a-GCP-Service-Account)\n    - [Create a Kubernetes Secret w/ the GCP Service Account Keys](#Create-a-Kubernetes-Secret-w-the-GCP-Service-Account-Keys)\n    - [Deploy the operator](#Deploy-the-operator)\n      - [Environment Variables](#Environment-Variables)\n  - [Managing Ballast PoolPolicies](#Managing-Ballast-PoolPolicies)\n    - [Example `PoolPolicy`](#Example-PoolPolicy)\n    - [Optimizing costs with preemptible pools and node affinity](#Optimizing-costs-with-preemptible-pools-and-node-affinity)\n  - [Contributing](#Contributing)\n    - [Setting up a development/test cluster](#Setting-up-a-developmenttest-cluster)\n      - [Using docker-desktop](#Using-docker-desktop)\n      - [Using terraform and GKE](#Using-terraform-and-GKE)\n    - [Deploying operator CRDs to test against](#Deploying-operator-CRDs-to-test-against)\n    - [Testing](#Testing)\n    - [Developing](#Developing)\n  - [Links](#Links)\n\n## Getting Started\n\nThere are 3 steps to deploy the **ballast-operator**:\n\n1. Create a GCP Service Account\n2. Create a Kubernetes Secret w/ the GCP Service Account Keys\n3. Deploy the operator\n\n### Create a GCP Service Account\n\nThe ballast `Deployment` will need to run as a _GCP service account_ with access to your clusters' node pools.\n\nThe following script will create a GCP service account with permissions to view and manage cluster pool sizes.\n\n```shell\nexport GCP_PROJECT=my-project-id\nexport SERVICE_ACCOUNT=ballast-operator\n\ngcloud iam service-accounts create ${SERVICE_ACCOUNT}\n\ngcloud projects add-iam-policy-binding ${GCP_PROJECT} \\\n  --member serviceAccount:${SERVICE_ACCOUNT}@${GCP_PROJECT}.iam.gserviceaccount.com \\\n  --role roles/container.admin\n\ngcloud projects add-iam-policy-binding ${GCP_PROJECT} \\\n  --member serviceAccount:${SERVICE_ACCOUNT}@${GCP_PROJECT}.iam.gserviceaccount.com \\\n  --role roles/compute.viewer\n```\n\n*Note:* ballast only needs a few permissions. Security minded users may prefer to create a custom role with the following permissions instead:\n\n- container.clusters.get\n- container.clusters.update\n- compute.instanceGroups.get\n\n### Create a Kubernetes Secret w/ the GCP Service Account Keys\n\nThe following script will create a secret named `ballast-operator-sa-keys` that contains the GCP service account JSON keys.\n\n```shell\ngcloud iam service-accounts keys create /tmp/ballast-keys.json \\\n  --iam-account ${SERVICE_ACCOUNT}@${GCP_PROJECT}.iam.gserviceaccount.com\n\nkubectl create secret generic ballast-operator-sa-keys --from-file=gcp.json=/tmp/ballast-keys.json\n\nrm /tmp/ballast-keys.json\n```\n\n### Deploy the operator\n\nA kustomization [`base`](./manifests/base/kustomization.yaml) is included that deploys:\n\n- [`ClusterRole`](./manifests/base/cluster_role.yaml)\n- [`ClusterRoleBinding`](./manifests/base/cluster_role_binding.yaml)\n- [`CustomResourceDefinition`](./manifests/base/custom_resource_definition.yaml)\n- [`Deployment`](./manifests/base/deployment.yaml)\n- [`PodDisruptionBudget`](./manifests/base/pod_disruption_budget.yaml)\n- [`ServiceAccount`](./manifests/base/service_account.yaml)\n- [`Service`](./manifests/base/service.yaml)\n\nThe kustomization file expects `secret/ballast-operator-sa-keys` (created above) to exist in the same namespace the operator is deployed in.\n\n```shell\nkubectl apply -k ./manifests/base/\n```\n\nThe operator exposes prometheus metrics on port 9323 at `/metrics`.\n\n#### Environment Variables\n\n- `BALLAST_METRICS_PORT`=9323\n- `BALLAST_DEBUG`=true\n- `GOOGLE_APPLICATION_CREDENTIALS`=/abs/path/to/creds.json\n\n## Managing Ballast CRDs\n\n### Example `PoolPolicy`\n\nBallast requires that all node-pools be created in advanced. Ballast only scales *managed* pools' _minimum count_ (or _current size_ in the case autoscaling is disabled) to match the required minimums of the *source* pool.\n\n```yaml\napiVersion: ballast.bonny.run/v1\nkind: PoolPolicy\nmetadata:\n  name: ballast-example\nspec:\n  projectId: gcp-project-id-here\n  location: us-central1-a # zone that main/source pool of preemptible nodes exist in\n  clusterName: your-cluster-name\n  poolName: my-main-pool # name of the main/source pool\n  cooldownSeconds: 300\n  managedPools: # list of pools to scale relative to main pool\n  - poolName: pool-b\n    minimumInstances: 1\n    minimumPercent: 25\n    location: us-central1-a\n  - poolName: pool-c\n    minimumInstances: 5\n    minimumPercent: 50\n    location: us-central1-a\n```\n\nMultiple managed pools can be specified. A mix of autoscaling and fixed size pools can be used, as well as pools of different instance types/sizes.\n\n### Optimizing costs with preemptible pools and node affinity\n\nThe following steps will cause Kubernetes to *prefer* scheduling workloads on your preemptible nodes, but schedule workloads on your on-demand pools when it must.\n\n1. Add the label `node-group:a-good-name-for-your-node-group` to **_all_** of your node pools that will be referenced in your `PoolPolicy`.\n2. Add the following affinity to your `Pod`, `Deployment`, or other workload..\n\n```yaml\nspec:\n  affinity:\n    nodeAffinity:\n      requiredDuringSchedulingIgnoredDuringExecution:\n        nodeSelectorTerms:\n        - matchExpressions:\n          - key: node-group\n            operator: In\n            values:\n            - a-good-name-for-your-node-group\n      preferredDuringSchedulingIgnoredDuringExecution:\n      - weight: 1\n        preference:\n          matchExpressions:\n          - key: cloud.google.com/gke-preemptible\n            operator: In\n            values:\n            - \"true\"\n```\n\n### Example `EvictionPolicy`\n\nBallast also supports a CRD called an `EvictionPolicy`. Eviction policies allow you to specify rules for evicting pods from nodes. This can be useful for eviction pods off of unpreferred nodes effectively implementing ~`preferredDuringSchedulingPreferredDuringExecution`.\n\nThe schema is:\n\n- `mode` (*all*, *unpreferred*) evict off all nodes or only unpreferred nodes based on `preferredDuringSchedulingIgnoredDuringExecution`; Default: *all*\n- `maxLifetime` max lifetime of a pod matching `selector` ; Default: *600* seconds\n- `selector` matchLabel and matchExpressions for selecting pods to evict\n\n```yaml\napiVersion: ballast.bonny.run/v1\nkind: EvictionPolicy\nmetadata:\n  name: unpreferred-nodes-nginx\nspec:\n  mode: unpreferred\n  maxLifetime: 600\n  selector:\n    matchLabels:\n      app: nginx\n    matchExpressions:\n      - {key: tier, operator: In, values: [frontend]}\n      - {key: environment, operator: NotIn, values: [dev]}\n```\n\n## Contributing\n\nBallast is built with the [bonny operator framework](https://github.com/coryodaniel/bonny) and Elixir.\n\n[Terraform](https://terraform.io) is used to provision test clusters.\n\nA number of make commands exist to aid in development and testing:\n\n```shell\nmake help\n```\n\n### Setting up a development/test cluster\n\n#### Using docker-desktop\n\nTwo test suites are provided, both require a function kubernetes server. [Docker Desktop](https://www.docker.com/products/docker-desktop) ships with a version of kubernetes to get started locally quickly.\n\nAlternatively you can use [terraform](https://www.terraform.io/downloads.html) to provision a cluster on GKE with `make dev.cluster.apply`. You will be charged for resources when using this approach.\n\n#### Using terraform and GKE\n\nFirst you will need to configure terraform with your GCP project and credentials:\n\n```shell\ntouch ./terraform/terraform.tfvars\necho 'gcp_project = \"my-project-id\"' \u003e\u003e ./terraform/terraform.tfvars\necho 'gcp_credentials_path = \"path/to/my/gcp-credentials.json\"' \u003e\u003e ./terraform/terraform.tfvars\n```\n\nNow create the cluster, this can take a while:\n\n```shell\nmake dev.cluster.apply\n```\n\nWhen you are done destroy the cluster with:\n\n```shell\nmake dev.cluster.delete\n```\n\n### Deploying operator CRDs to test against\n\nAfter setting up your test cluster you'll need to deploy the operator CRDs so that the cluster has the features the test suite will exercise.\n\n```shell\nmake dev.start.in-cluster\n```\n\n### Testing\n\nTwo test suites exist:\n\n- `make test` - elixir unit test suite on underlying controller code\n- `make integration` - scales node pools on GKE\n\nTwo environment variables must be exported to run the full integration tests.\n\n```shell\nexport GOOGLE_APPLICATION_CREDENTIALS=/abs/path/to/creds.json\nexport GCP_PROJECT=your-project-id\n```\n\nAdditionally `make lint` will run the mix code formatter, credo, and dialyzer.\n\n### Developing\n\nYou'll need a function cluster to connect to. Ballast will use your `current-context` in `~/.kube/config`. This can be changed in `config/dev.exs`.\n\nGOOGLE_APPLICATION_CREDENTIALS must be set to start the application.\n\n```shell\nexport GOOGLE_APPLICATION_CREDENTIALS=/abs/path/to/creds.json\n```\n\nThen run the following to generate a development manifest, apply it to your cluster, and start `iex`:\n\n```shell\nmake dev.start.iex\n```\n\n## Links\n\n- GKE Docs\n  - [Instance Manager Groups REST API](https://cloud.google.com/compute/docs/reference/rest/v1/instanceGroupManagers)\n  - [Node Pools REST API](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters.nodePools)\n  - [Elixir Container Docs](https://hexdocs.pm/google_api_container/GoogleApi.Container.V1.Api.Projects.html)\n  - [Elixir Compute Docs](https://hexdocs.pm/google_api_compute)\n- GKE API Explorer\n  - [setAutoscaling](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters.nodePools/setAutoscaling)\n  - [setSize](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters.nodePools/setSize)\n  - [nodePools get](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.locations.clusters.nodePools/get)\n  - [instanceGroups get](https://cloud.google.com/compute/docs/reference/rest/v1/instanceGroups/get)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoryodaniel%2Fballast","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoryodaniel%2Fballast","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoryodaniel%2Fballast/lists"}