{"id":24638086,"url":"https://github.com/upbound/configuration-caas-ml","last_synced_at":"2026-05-17T18:37:01.207Z","repository":{"id":199740273,"uuid":"702038629","full_name":"upbound/configuration-caas-ml","owner":"upbound","description":null,"archived":false,"fork":false,"pushed_at":"2024-01-22T21:48:50.000Z","size":2834,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-05-08T07:40:58.652Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/upbound.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-08T10:18:21.000Z","updated_at":"2024-10-26T02:15:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"c29ac55a-7c13-4618-9386-e19e67f3dfe0","html_url":"https://github.com/upbound/configuration-caas-ml","commit_stats":null,"previous_names":["upbound/configuration-caas-ml"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/upbound/configuration-caas-ml","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-caas-ml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-caas-ml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-caas-ml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-caas-ml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/upbound","download_url":"https://codeload.github.com/upbound/configuration-caas-ml/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-caas-ml/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33149939,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-17T09:28:26.183Z","status":"ssl_error","status_checked_at":"2026-05-17T09:27:52.702Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-25T10:12:57.540Z","updated_at":"2026-05-17T18:37:01.202Z","avatar_url":"https://github.com/upbound.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003e **This repository is archived.** It is a historical example and is no longer maintained. No replacement is available.\n\n---\n\n\n# configuration-caas-ml\n\nConfiguration-caas-ml is a setup for deploying the Stable Diffusion Model using Cluster-as-a-Service in Upbound.\nSeveral tools are available to create and operate models, including those within Kubernetes.\nOne promising setup combines Upbound, Flux, JupyterHub, Ray, and Kubernetes.\n\n## Architecture\n\n*Upbound*\nis the easiest place to build, deploy, and manage cloud platforms using control planes. Many organizations find themselves embarking on similar journeys; Whether it's building “golden paths”, “launch zones”, or “cloud infrastructure self-service”, organizations are beginning to build their own internal developer platforms above the clouds–and they’re using control planes to power them. With Upbound's managed control planes your organization can operate control planes at scale.\n\n*Flux*\nis an open-source tool that automates the deployment and continuous synchronization of applications and configuration across Kubernetes clusters. It helps ensure that your Kubernetes-based applications are always up to date with the desired state defined in your version control system.\n\n*JupytherHub*\nprovides a shared platform for running notebooks that are popular in business, education, and research. It promotes interactive computing where users can execute code, visualize results, and work together. In the realm of GenAI, JupyterHub accelerates the experimentation process, especially in the feedback loop. It’s also where Data Engineers collaborate on models for Prompt Engineering.\n\n*Ray*\nis an open-source distributed computing framework that makes it easy to scale applications and to use state-of-the-art machine learning libraries. Ray is used to distribute the training of generative models across multiple nodes, which accelerates the training process and allows for the handling of larger datasets.\n\n*Kubernetes*\nis a powerful container orchestration platform that automates the deployment, scaling, and management of containerized applications. Kubernetes provides the infrastructure to run and scale GenAI models in containers, which ensures high availability, fault tolerance, and efficient resource utilization.\n\n## Solution Architecture\n\n![architecture](test/ml.png)\n\n## Run\nStep-by-Step guide on how tp deploy a generative model with Upbound on Kubernetes\n\n### Steps to deploy Stable Diffusion Model on Kubernetes\n#### Pre-requisites\n\n* Upbound Organization\n* Upbound Managed Control Plane (MCP)\n* kubectl\n* HuggingFaceToken with `write-scope`\n\nStep 1:\nUtilize this setup within an Upbound Managed Control Plane.\n\nStep 2: \nFirst, we need to set up a Kubernetes cluster.\n\n```bash\nkubectl apply -f ./up/examples/cluster.yaml\n```\n\nAfterward, we need to configure our Kubernetes services, including Kubernetes add-ons and the necessary tools for machine learning.\nNote: Please insert your own token into the Kubernetes secret.\n\n```bash\nkubectl apply -f ./up/examples/secret-hf-token.yaml\nkubectl apply -f ./up/examples/secret-notebook.yaml\nkubectl apply -f ./up/examples/ml.yaml\n```\n\nThis configuration provisions the following:\n* VPC\n* Subnets\n* Internet-Gateway\n* EKS Cluster\n* EKS managed node group for core components\n* EKS managed node group for gpu components\n* Amazon EBS CSI Driver\n* AWS Loadbalancer Controller\n* NVIDIA Device Plugin\n* JupyterHub\n* Ingress Nginx\n* Kuberay-Operator\n\nStep 3:\nFine-Tune Stable Diffusion Model\n\nYou are now prepared to begin experimenting with our model and create a notebook to customize it according to your requirements. Retrieve the Load Balancer DNS:\n\n```bash\nkubectl get svc proxy-public -n jupyterhub --output jsonpath='{.status.loadBalancer.ingress[0].hostname}'\n```\n\nOpen the provided DNS hostname (e.g., k8s-jupyterh-proxypub-xxx.elb.eu-central-1.amazonaws.com) in your web browser.\n\nLog in using the username \"admin1\" and the password found in the managed resource called `jupyterChart`.\n\nThis action initiates the provisioning of a pod named `jupyter-admin1` on the g5.12xlarge instance. You can observe this pod by running the command \"kubectl get pods -n jupyterhub.\n\n```bash\nkubectl get pods -n jupyterhub\nNAME                            READY   STATUS    RESTARTS   AGE\ncontinuous-image-puller-kbvwc   1/1     Running   0          1h\ncontinuous-image-puller-lftzx   1/1     Running   0          1h\ncontinuous-image-puller-mst4w   1/1     Running   0          1h\ncontinuous-image-puller-t9wfg   1/1     Running   0          1h\nhub-5f4d644989-49lz2            1/1     Running   0          1h\njupyter-admin1                  1/1     Running   0          1h\nproxy-5f89cffc9c-92qgz          1/1     Running   0          1h\n```\n\nAfter a successful launch, you will be automatically redirected to the notebook console in your web browser. Begin running the given Python notebook found in the `dogbooth` directory using the file browser interface in the notebook UI, as depicted in the following illustration.\n\n![overview](test/overview.png)\n\nThis takes about an 1.5 hours to complete.\n\nStep 4: \nServing the Large Language Model\n\nOnce applied, the RayCluster's head node and worker node will be scheduled on the GPU nodes, and the inference endpoint will be accessible through the load balancer's DNS hostname. Please note that due to the large image size of the GPU-based `rayproject/ray-ml:2.6.0-gpu` base image, this process may take up to eight minutes.\n\nTo monitor the pods and wait for them to be up, use the command `kubectl get pods -n dogbooth --watch`. Once they are ready, obtain the load balancer's DNS hostname and access the Ray Dashboard in your web browser.\n\n```bash\nkubectl get ingress dogbooth -n dogbooth --output jsonpath='{.status.loadBalancer.ingress[0].hostname}'\n```\n\nLastly, confirm the deployment of our Dogbooth model using a prompt like this:\n\n```bash\nhttp://k8s-ingressn-ingressn-xxx.elb.eu-central-1.amazonaws.com/dogbooth/serve/imagine?prompt=a photo of [v]dog on the sun\n```\n\n![dog-on-the-sun](test/imagine.png)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fupbound%2Fconfiguration-caas-ml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fupbound%2Fconfiguration-caas-ml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fupbound%2Fconfiguration-caas-ml/lists"}