{"id":49432495,"url":"https://github.com/foundation-model-stack/fms-autoconf-k8s-controller","last_synced_at":"2026-04-29T14:04:32.681Z","repository":{"id":337630883,"uuid":"1115323279","full_name":"foundation-model-stack/fms-autoconf-k8s-controller","owner":"foundation-model-stack","description":null,"archived":false,"fork":false,"pushed_at":"2026-02-10T16:40:35.000Z","size":89,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-10T18:21:05.421Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/foundation-model-stack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-12T17:18:14.000Z","updated_at":"2026-02-10T16:40:00.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/foundation-model-stack/fms-autoconf-k8s-controller","commit_stats":null,"previous_names":["foundation-model-stack/fms-autoconf-k8s-controller"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/foundation-model-stack/fms-autoconf-k8s-controller","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foundation-model-stack%2Ffms-autoconf-k8s-controller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foundation-model-stack%2Ffms-autoconf-k8s-controller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foundation-model-stack%2Ffms-autoconf-k8s-controller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foundation-model-stack%2Ffms-autoconf-k8s-controller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/foundation-model-stack","download_url":"https://codeload.github.com/foundation-model-stack/fms-autoconf-k8s-controller/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foundation-model-stack%2Ffms-autoconf-k8s-controller/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32428622,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T13:34:34.882Z","status":"ssl_error","status_checked_at":"2026-04-29T13:34:29.830Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-29T14:04:31.080Z","updated_at":"2026-04-29T14:04:32.647Z","avatar_url":"https://github.com/foundation-model-stack.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Autoconf Resource Requirements for AI Jobs on Kubernetes\n\nThis Kubernetes controller integrates with the [ADO **autoconf** custom experiment](https://github.com/IBM/ado/tree/main/plugins/custom_experiments/autoconf) to automatically set resource requirements for AI tuning jobs that use [fms-hf-tuning](https://github.com/foundation-model-stack/fms-hf-tuning).\n\n---\n\n## What the controller does\n\nThe controller inspects the command line of your AI jobs (either `PyTorchJob` objects or `PyTorchJob` objects wrapped inside an `AppWrapper`) to extract details such as:\n- Model name\n- Tuning method\n- Effective batch size (i.e., `per_device_batch_size * NUM_GPUS`)\n- Maximum sequence length\n\nIt combines these with the target GPU model to request recommendations from the `autoconf` experiment and then **creates new derived objects** with the recommended resource requirements:\n- For `PyTorchJob` objects, and\n- For `AppWrapper` objects that contain a `PyTorchJob` object\n\nWe wish to use this controller to **enhance the execution of AI workloads on Kubernetes clusters** such that they use the right number of GPUs so as to avoid going out of GPU memory.\nThe design of the controller enables us to explore different algorithms for resource recommendation which we plan to explore in the future.\n\n### Kueue collaboration (design work in progress)\nWe are working with the Kueue maintainers on a Kubernetes Enhancement Proposal (KEP) to improve how external Kubernetes controllers interact with jobs managed by Kueue (including AI workloads). The design discussion is tracked here: \u003chttps://github.com/kubernetes-sigs/kueue/issues/6915\u003e.\n\nUntil that work lands, this controller demonstrates a way to interact with Kueue-managed jobs while operating within current Kueue capabilities.\n\n---\n\n## Examples\n\nThe controller in action!\n\nhttps://github.com/user-attachments/assets/ac0a5a0a-8966-40bd-9fce-8100086acef7\n\n\n### Test locally (no Docker image)\nYou can run the controller as a local process while it manages one or more namespaces on your cluster.\n\n**Assumptions for the example below**\n- You use a Kueue-managed namespace called `tuning` to run `AppWrapper` workloads.\n- These workloads:\n  1. use the Kueue `LocalQueue` named `default-queue`,\n  2. wrap a `PyTorchJob` that uses [fms-hf-tuning](https://github.com/foundation-model-stack/fms-hf-tuning),\n  3. request one or more NVIDIA GPUs of the same model (e.g., `NVIDIA-A100-SXM4-80GB`),\n  4. are subject to Kyverno policies requiring the `kueue.x-k8s.io/queue-name` label on `AppWrapper` objects.\n\n**Steps**\n1. Create and activate a Python virtual environment, then install the ADO autoconf client:\n   ```bash\n   python3 -m venv .venv \u0026\u0026 source .venv/bin/activate\n   pip install ado-autoconf==1.5.0 ipython\n   ```\n2. Log in to your cluster (via `kubectl` or `oc`).\n3. Build the controller locally:\n   ```bash\n   make\n   ```\n4. Start the controller with flags appropriate for the scenario:\n   ```bash\n   ./bin/manager \\\n     --done-label-key=kueue.x-k8s.io/queue-name \\\n     --done-label-value=default-queue \\\n     --namespaces \"tuning\" \\\n     --enable-appwrapper=true \\\n     --enable-pytorchjob=true \\\n     --unsuspend-derived-jobs=false \\\n     --default-gpu-model=NVIDIA-A100-SXM4-80GB \\\n     --path-wrapper-script=./cmd/wrapper_autoconf.py\n   ```\n5. Create an `AppWrapper` or `PyTorchJob` workload with the following labels:\n   ```yaml\n   # This setup both satisfies Kyverno (requires a queue-name) and\n   # allows Kueue to temporarily ignore the job until the controller updates it.\n   kueue.x-k8s.io/queue-name: fake\n   autoconf-plugin-name: resource-requirements-appwrapper\n   ```\n\nExample `AppWrapper` and `PyTorchJob` manifests are available under [`examples`](./examples).\n\n---\n\n### Deploy to the cluster (high-level)\nIf you prefer to run the controller in-cluster (e.g., as a `Deployment`), the high-level process is:\n\n1. **Build an image** for the controller.\n2. **Create RBAC**: ServiceAccount, Role/ClusterRole, and bindings that permit reading/updating/creating the resources you plan to manage (i.e. `AppWrapper` and/or `PyTorchJob`). The controller needs `read` and `create` permissions.\n3. **Deploy a `Deployment`** for the controller, setting the desired command-line flags (see **Configuration** below). Enable leader election if you run multiple replicas.\n4. **Optionally expose metrics/webhooks** via a `Service` if you enable those endpoints.\n5. **Label workloads** so the controller can discover them (see `--watch-label-key` / `--watch-label-value`), then create your `AppWrapper`/`PyTorchJob` objects.\n6. **Observe logs and job status** to confirm resources are being recommended and applied as expected. The controller will create new derived objects with owner references to the originals.\n\n\n---\n\n## Important Configuration flags\n\nBelow are the controller’s command-line options:\n\n### Core behavior\n- `--default-autoconf-model-version string` — Default autoconf model version to use (default `3.1.0`).\n- `--default-gpu-model string` — Default GPU model if not specified in the job.\n- `--patch-cpu-request` — Set job CPU request/limit to `max(1, 2 * NUM_GPUS)` in derived objects (default `true`).\n- `--unsuspend-derived-jobs` — Unsuspend derived jobs after creation.\n- `--path-wrapper-script string` — Path to the local Python wrapper for running models. **Mutually exclusive** with `--url-ado`. Exactly one of these must be set.\n- `--url-ado string` — URL of the ADO REST API serving the models. **Mutually exclusive** with `--path-wrapper-script`. Exactly one of these must be set.\n\n### Discovery \u0026 scope\n- `--namespaces string` — Comma-separated list of namespaces to watch.\n- `--watch-label-key string` — Limit monitoring to objects labeled `key=value` (default key `autoconf-plugin-name`).\n- `--watch-label-value string` — Label value used with `--watch-label-key` (default `resource-requirements-appwrapper`).\n- `--enable-appwrapper` — Watch `AppWrapper` objects and create derived objects with recommendations.\n- `--enable-pytorchjob` — Watch `PyTorchJob` objects and create derived objects with recommendations.\n\n### Completion labeling\n- `--done-label-key string` — Label key inserted on original object when processing is complete (default `autoconf-plugin-done`). Not set if key is `kueue.x-k8s.io/queue-name`.\n- `--done-label-value string` — Label value inserted on original object when processing is complete (default `yes`).\n- `--waiting-for-ado-request-id-label string` — Label used to mark jobs waiting for an ADO request ID (default `waiting-for-ado-request-id`).\n- `--recommendation-annotation-key string` — Annotation key to store recommendation results in JSON format (default `ado-autoconf.ibm.com/recommendation`).\n\n### Recommendation Annotations\n\nThe controller adds an annotation to each processed object containing the recommendation result in JSON format:\n\n**Successful Recommendation:**\n```json\n{\n  \"recommendation\": {\n    \"workers\": 2,\n    \"gpus\": 4\n  }\n}\n```\n\n**No Recommendation Available:**\n```json\n{\n  \"error\": \"No recommendation\"\n}\n```\n\nWhen the recommendation engine cannot generate recommendations (e.g., `can_recommend != 1` from the ado experiment), the controller:\n1. Adds the error annotation to the object\n2. Sets the done label to mark it as processed\n3. Allows the object to proceed through standard Kubernetes workflows without modifications\n4. Emits a warning event indicating no recommendation was available\n\nThis ensures objects are not stuck in a retry loop when recommendations cannot be generated.\n\n### Logging\n- `--zap-devel` — Development mode defaults (console encoder, debug log level, warn stack traces). Production mode defaults (JSON encoder, info log level, error stack traces). Default `true`.\n- `--zap-encoder [json|console]` — Zap log encoding.\n- `--zap-log-level value` — Log verbosity (`debug`, `info`, `error`, `panic`, or integer \u003e 0 for custom levels).\n- `--zap-stacktrace-level [info|error|panic]` — Level at and above which stack traces are captured.\n- `--zap-time-encoding [epoch|millis|nano|iso8601|rfc3339|rfc3339nano]` — Time encoding (default `epoch`).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoundation-model-stack%2Ffms-autoconf-k8s-controller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffoundation-model-stack%2Ffms-autoconf-k8s-controller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoundation-model-stack%2Ffms-autoconf-k8s-controller/lists"}