{"id":20494194,"url":"https://github.com/skypilot-org/new-cloud-skeleton","last_synced_at":"2025-04-13T17:21:34.654Z","repository":{"id":185715867,"uuid":"673988779","full_name":"skypilot-org/new-cloud-skeleton","owner":"skypilot-org","description":"Skeleton repo for how to add a new cloud to SkyPilot.","archived":false,"fork":false,"pushed_at":"2023-10-08T23:05:47.000Z","size":50,"stargazers_count":8,"open_issues_count":1,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-27T08:11:20.405Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/skypilot-org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-02T22:20:21.000Z","updated_at":"2025-01-29T22:55:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"144f5194-b692-465d-a42b-eab89d3276df","html_url":"https://github.com/skypilot-org/new-cloud-skeleton","commit_stats":null,"previous_names":["skypilot-org/new-cloud-skeleton"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skypilot-org%2Fnew-cloud-skeleton","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skypilot-org%2Fnew-cloud-skeleton/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skypilot-org%2Fnew-cloud-skeleton/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skypilot-org%2Fnew-cloud-skeleton/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/skypilot-org","download_url":"https://codeload.github.com/skypilot-org/new-cloud-skeleton/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248750851,"owners_count":21155795,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T17:38:41.576Z","updated_at":"2025-04-13T17:21:34.627Z","avatar_url":"https://github.com/skypilot-org.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# new-cloud-skeleton\n\n*ℹ️ NOTE: This guide will change as SkyPilot develops. Please check back often to make sure you have the most up-to-date version. The read me in this doc has been derived from the [Google Doc](https://docs.google.com/document/d/1iuPyQ47HloKuHfOYjcRNz7HAlUxFHs2WjMqedYmcPVQ/edit#heading=h.nby65cfuzxoq)*\n\nSkeleton repo for what's needed to add a new cloud.\n\nDetailed instructions upcoming! Contact the dev team on [SkyPilot Slack](https://slack.skypilot.co/) to get access.\n\n## Introduction to SkyPilot\n\n[SkyPilot](https://github.com/skypilot-org/skypilot) is an intercloud broker -- a framework for running workloads on any cloud. Here are some useful links to learn more:\n\n1. [Introductory Blogpost](https://medium.com/@zongheng_yang/skypilot-ml-and-data-science-on-any-cloud-with-massive-cost-savings-244189cc7c0f) [Start here if you are new]\n2. [Documentation](https://skypilot.readthedocs.io/en/latest/)\n3. [The Sky Above the Clouds](https://arxiv.org/abs/2205.07147)\n4. [GitHub](https://github.com/skypilot-org/skypilot)\n\n## How does SkyPilot work?\n\nHere's a simplified overview of SkyPilot's architecture.\n\nTODO: diagram in google doc\n\nIn this diagram, the user has two clouds enabled (AWS and GCP). This is what happens when a user launches a job with sky launch:\n\n1. The optimizer reads AWS Catalog and GCP Catalog and runs an algorithm to decide which cloud to run the job on. (Let's suppose the optimizer chooses AWS.) This information is then sent to the provisioner+executor.\n   - A catalog is a list of instance types and their prices.\n2. The provisioner+executor executes ray commands to launch a cluster on AWS.\n   - AWS Node Provider is the interface between ray and AWS, translating ray function calls to AWS API calls.\n3. Once the cluster is launched, the provisioner+executor ssh’s into the cluster to execute some AWS Setup commands. This is used to download some important packages on the cluster.\n4. The provisioner+executor submits the job to the cluster and the cluster runs the job.\n\nWhen all is done, the user can run sky down and provisioner+executor will tear down the cluster by executing more ray commands.\n\n## Getting Started\n\nNow let's say you have a new cloud, called FluffyCloud, that you want SkyPilot to support. What do you need to do?\n\nYou need to:\n\n1. Write a NodeProvider for FluffyCloud. This is the most important part.\n2. Add the FluffyCloud catalog to SkyPilot and write functions that read this catalog.\n3. Write FluffyCloud setup code.\n4. Add FluffyCloud credential check to verify locally stored credentials. This is needed for a user to enable FluffyCloud.\n\nFor reference, here is an actual merged PR for adding a new cloud to help you estimate what is required:\n\n- [Lambda Cloud](https://github.com/skypilot-org/skypilot/pull/1557)\n\nBy completing the following steps, you will be able to run SkyPilot on FluffyCloud.\n\n- [Step 0](/docs/integration_steps/step_0-api-library.md)\n- [Step 1](/docs/integration_steps/step_1-node-provider.md)\n- [Step 2](/docs/integration_steps/step_2-catalog.md)\n- [Step 3](/docs/integration_steps/step_3-setup-code.md)\n- [Step 4](/docs/integration_steps/step_4-setup-code.md.md)\n- [Step 5](/docs/integration_steps/step_5-e2e-failover.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskypilot-org%2Fnew-cloud-skeleton","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskypilot-org%2Fnew-cloud-skeleton","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskypilot-org%2Fnew-cloud-skeleton/lists"}