{"id":19056850,"url":"https://github.com/provectus/sak-kubeflow","last_synced_at":"2025-04-18T00:31:42.954Z","repository":{"id":52162312,"uuid":"297894618","full_name":"provectus/sak-kubeflow","owner":"provectus","description":"🚀 Deploy Kubeflow on AWS EKS with Terraform 🤖","archived":false,"fork":false,"pushed_at":"2023-02-22T03:24:34.000Z","size":52528,"stargazers_count":63,"open_issues_count":13,"forks_count":23,"subscribers_count":11,"default_branch":"master","last_synced_at":"2024-07-30T18:48:12.456Z","etag":null,"topics":["ai","argocd","artificial-intelligence","automation","aws","cluster","data-science","deep-learning","devops","eks","gitops","iac","infrastructure","infrastructure-as-code","kubeflow","machine-learning","ml","open-source","terraform"],"latest_commit_sha":null,"homepage":"https://provectus.com","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/provectus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.MD","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-09-23T07:52:13.000Z","updated_at":"2024-07-25T14:00:10.000Z","dependencies_parsed_at":"2023-02-17T13:31:04.323Z","dependency_job_id":null,"html_url":"https://github.com/provectus/sak-kubeflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/provectus%2Fsak-kubeflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/provectus%2Fsak-kubeflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/provectus%2Fsak-kubeflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/provectus%2Fsak-kubeflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/provectus","download_url":"https://codeload.github.com/provectus/sak-kubeflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223768831,"owners_count":17199392,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","argocd","artificial-intelligence","automation","aws","cluster","data-science","deep-learning","devops","eks","gitops","iac","infrastructure","infrastructure-as-code","kubeflow","machine-learning","ml","open-source","terraform"],"created_at":"2024-11-08T23:52:05.752Z","updated_at":"2024-11-08T23:52:06.319Z","avatar_url":"https://github.com/provectus.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Maintenance](https://img.shields.io/maintenance/yes/2021?style=for-the-badge)]()\n[![Apache2](https://img.shields.io/badge/license-Apache2-green.svg?style=for-the-badge)](https://www.apache.org/licenses/LICENSE-2.0)\n[![Join us on Slack](https://img.shields.io/badge/%20-Join%20us%20on%20Slack-blue?style=for-the-badge\u0026logo=slack\u0026labelColor=5c5c5c)](https://sak-kubeflow.slack.com)\n\n\u003c!-- Swiss-Army-Kube_README --\u003e\n**[Quickstart](./QUICKSTART.md)** • **[Provectus](https://provectus.com/)**\n\n# Deploy Kubeflow on AWS EKS with Swiss Army Kube using Terraform\n\nExisting ways of deploying an Amazon EKS cluster with Kubeflow inside don’t offer a clear blueprint to set up, serve, update, and scale multiple clusters quickly. They require using several CLIs (`kubectl`, `kfctl`, ect.), limit what resources you can add to your cluster after deployment, and don’t allow for CI/CD automation or easy replication of cluster configurations you create.\n\n**Swiss Army Kube for Kubeflow (SAKK)** is a free open-source Terraform-based IaC tool that allows you to declaratively set up modular ML-ready AWS EKS clusters with Kubeflow,  automated with GitOps. SAKK provides a blueprint based on the best DevOps practices, which allows for one-click cluster replication, easy management, and augmenting your clusters with any resources, not limited to native tools. SAKK helps to quickly bring ML-ready clusters to production, and scale them by adding new modules as you go.  SAKK is built on top of Terraform (infrastructure as code), ArgoCD (deployment automation \u0026 management of all Kubernetes resources), and Cognito (AWS identity provider). As a result of deployment with SAKK, you get a scalable modular cluster like this:\n\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"./images/sakk-aws-eks-kubernetes-kubeflow-terraform-cluster.jpg\" width=\"600px\" alt=\"sakk-aws-eks-kubernetes-kubeflow-terraform-cluster\"/\u003e\u0026nbsp;\n\u003c/p\u003e\n    \n:point_right: **[QUICKSTART](./QUICKSTART.md)**\n\nWe believe that any organization or engineer using ML should be able to focus on their pipelines and applications without having to worry too much about the nitty-gritty of infrastructure deployment. Currently, SAKK is available for the [Amazon EKS](https://aws.amazon.com/eks/) (Elastic Kubernetes Service) cluster only. We plan to expand to other platforms soon.\n\nSwiss Army Kube for Kubeflow is based on the main [**Swiss Army Kube**](https://github.com/provectus/swiss-army-kube) repository. SAKK is a SAK modification for the Kubeflow EKS setup based on SAK's collection of modules.\n\n\u003cbr\u003e\n\n## Key Features\n\n### Deploy\n\n* Provision an AWS EKS cluster with Kubeflow inside in minutes\n* Use existing project structure to set up your ML cluster configuration\n* Configure your deployment with a single `.tf` file\n* Deploy with a couple of Terraform commands\n\n### Manage\n\n* Add any resources to your cluster before or after deployment\n* Deliver your projects and apps with GitOps CI/CD automation\n* Easily edit, reconfigure, rerun, add or destroy resources  \n* Build your own ML training pipelines in Kubeflow on AWS EKS\n* Manage your cluster with Terraform and Kubernetes CLI or ArgoCD CLI/UI\n\n### Scale\n\n* Replicate your cluster configuration with a couple of clicks   \n* Configure and deploy as many ML clusters as you need fast \n* Scale deployments by adding new resources as modules\n* Reduce your cloud infrastructure spend with spot instances \n* Maximize your workload cost-efficiency \n\n\u003cbr\u003e\n\n## How it Works\n\nThis repository is a template of a Kubeflow EKS cluster for your ML projects. Modify the `main.tf` file to set up a cluster and deploy it to AWS with Terraform commands. With this simple yet powerful workflow, you can provision as many ML-ready EKS clusters (with different settings of variables, networks, Kubernetes versions, etc.) as you want in no time.\n\n1. Prerequisites\n   + Prepare an AWS account with configured IAM user\n   + Fork and clone this repository\n   + Install Terraform\n   + Install AWS CLI\n2. Configure your EKS cluster before deployment using the repo as a template\n   + Configure `backend.hcl` \n   + Configure variables in `main.tf` \n3. Deploy your Kubeflow Kubernetes EKS cluster with Terraform commands\n4. Commit and push the repository\n5. Manage your Kubernetes cluster with ArgoCD (or configure `kubectl`) and deploy your ML apps to it.  \n\n\u003cbr\u003e\n\nTo see the cluster configuration and deployment process, you can check out this demo video: \n[Cluster configuration and deployment](https://asciinema.org/a/vdi54kQ0gjuWqcDU9UsNuwo1Z)\nTo see the deployment of an example located in the `sak-kubeflow/examples/simple` directory, you can check out this demo video: \n[Example deployment](https://github.com/provectus/sak-kubeflow/blob/init/examples/simple/main.tf)\n\n\u003cbr\u003e\n\n## Who is SAKK For\n\nSAKK is great for enterprises that work on ML/AI projects and want to deploy and manage Kubeflow clusters on AWS EKS in a declarative, modular, repeatable, GitOps way.\n\n\u003cbr\u003e\n\n## Get Started\n\nPlease visit our [Quickstart](./QUICKSTART.md) to get ready with prerequisites, configure your cluster, and deploy it with Terraform commands:\n\n``` \nterraform init\nterraform apply\naws --region \u003cregion\u003e eks update-kubeconfig --name \u003ccluster-name\u003e\n```  \nAfter the deployment, you'll have a Kubernetes cluster with Kubeflow and ArgoCD inside, deployed on AWS EKS and automated with GitOps (as shown below). You can manage your Kubernetes cluster with `kubectl` CLI or ArgoCD CLI/UI, and your AWS cluster with AWS or Terraform CLI. \n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"./images/declarative-modular-AWS-EKS-Kubeflow-GitOps-cluster.jpg\" width=\"600px\" alt=\"sakk-declarative-modular-AWS-EKS-Kubeflow-GitOps-cluster\"/\u003e\u0026nbsp;\n\u003c/p\u003e\n\n\u003cbr\u003e\n\n## How to Contribute\n\nTo get involved, please check out our [CONTRIBUTING.md](./CONTRIBUTING.md).\n\n\u003cbr\u003e\n\n## License\n\n[Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0.txt).\n\n\u003cbr\u003e\n\n## We Love Your Feedback!\n\nWe are always happy to hear your thoughts and questions about SAKK. Please join our Slack to discuss:\n\n\u003cbr\u003e\n\n[![Join us on Slack](https://img.shields.io/badge/%20-Join%20us%20on%20Slack-blue?style=for-the-badge\u0026logo=slack\u0026labelColor=5c5c5c)](https://sak-kubeflow.slack.com) \n\n\u003ca href=\"#top\"\u003eBack to top\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprovectus%2Fsak-kubeflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprovectus%2Fsak-kubeflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprovectus%2Fsak-kubeflow/lists"}