{"id":16544743,"url":"https://github.com/jlewitt1/runhouse-sagemaker","last_synced_at":"2026-04-11T13:35:12.685Z","repository":{"id":209539030,"uuid":"724264716","full_name":"jlewitt1/runhouse-sagemaker","owner":"jlewitt1","description":"Run code effortlessly on SageMaker with Runhouse","archived":false,"fork":false,"pushed_at":"2023-11-30T17:40:53.000Z","size":10,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-04-11T13:35:10.269Z","etag":null,"topics":["api","artificial-intelligence","aws","deployment","hyperparameter-tuning","inference","infrastructure","machine-learning","python","sagemaker","training"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jlewitt1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-11-27T18:20:17.000Z","updated_at":"2024-02-04T00:06:38.000Z","dependencies_parsed_at":"2023-11-27T23:26:24.316Z","dependency_job_id":"a2e69aab-dca0-45c3-a119-5e3aa1ffeffc","html_url":"https://github.com/jlewitt1/runhouse-sagemaker","commit_stats":null,"previous_names":["jlewitt1/runhouse-sagemaker"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jlewitt1/runhouse-sagemaker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlewitt1%2Frunhouse-sagemaker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlewitt1%2Frunhouse-sagemaker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlewitt1%2Frunhouse-sagemaker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlewitt1%2Frunhouse-sagemaker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jlewitt1","download_url":"https://codeload.github.com/jlewitt1/runhouse-sagemaker/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlewitt1%2Frunhouse-sagemaker/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31682953,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-11T13:07:20.380Z","status":"ssl_error","status_checked_at":"2026-04-11T13:06:47.903Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","artificial-intelligence","aws","deployment","hyperparameter-tuning","inference","infrastructure","machine-learning","python","sagemaker","training"],"created_at":"2024-10-11T19:04:29.386Z","updated_at":"2026-04-11T13:35:12.655Z","avatar_url":"https://github.com/jlewitt1.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🏃‍♀️Runhouse🏠 \u0026 SageMaker\n\n## ❣️ Why do we love SageMaker?\n\n* **Serverless compute**: SageMaker provides a more scalable experience than EC2, which means you don’t need to \nbe responsible for auto-stopping, scheduling, or worry about accessing compute in a K8s cluster and managing queueing \njobs or running them in parallel. With SageMaker you can easily launch multiple instances at the same time.  \n\n* **Launching with containers**: SageMaker allows you to launch a cluster with a docker container. This gives you a \nmore K8s like experience of launching compute with a lightweight image rather than an \n[AMI](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html), which is more difficult to publish and \nexpensive to maintain.\n\n* **More reliable scaling**: Instead of maintaining a fleet of GPUs with some orchestrator like ECS, \nSageMaker allows you to trigger many jobs in parallel. If you have training or batch jobs which are triggered by \ncustomers, SageMaker is a much more reliable way to maintain those jobs at scale. It's also very easy to mix and match \nSageMaker with other compute options within your existing fleet\n\n* **GPU availability**: We've observed that GPUs tend to be more available on SageMaker compared to EC2.\n\n## 🤝 Using Runhouse with SageMaker\n\n[Runhouse](https://www.run.house) makes it super easy to get started with SageMaker or continue to use SageMaker within your existing stack. \nWhether you are hacking on a single CPU / GPU instance or deploying a fleet of clusters to support training, \ninference, or hyperparameter tuning, SageMaker is an ideal platform to handle all of these needs and reliably scale \nas your ML stack evolves. Using the Runhouse uniform API allows you to transition from single box to multi-box to \na pool of compute without the need for any code migrations or refactors.\n\nThe Runhouse [SageMakerCluster](https://www.run.house/docs/main/en/api/python/cluster#sagemakercluster-class) \nmakes the process of onboarding to SageMaker more smooth, saving you the need to create estimators, or conform the \ncode to the SageMaker APIs. This translation step can take anywhere from days to months, and leads to rampant code \nduplication, forking, versioning and lineage issues.\n\n## 🚀 Getting Started\n\nSageMaker clusters require AWS CLI V2 and configuring the SageMaker IAM role with the AWS Systems Manager.\n\nIn order to launch a cluster, you must grant SageMaker the necessary permissions with an IAM role, \nwhich can be provided either by name, full ARN or with a profile name. You can also specify a profile explicitly or with the \n`AWS_PROFILE` environment variable.\n\nThe examples in this repo use an AWS profile name, which Runhouse extracts from the local `~/.aws/config` file. \nIf your config file contains the below profile: \n\n```ini\n[profile sagemaker]\nrole_arn = arn:aws:iam::123456789:role/service-role/AmazonSageMaker-ExecutionRole-20230717T192142\nregion = us-east-1\nsource_profile = default\n```\n\nYou can then pass in `profile=sagemaker` when initializing the cluster.\n\nFor a more detailed walkthrough, see the\n[SageMaker Hardware Setup](https://www.run.house/docs/stable/en/api/python/cluster#sagemaker-hardware-setup) section of the Runhouse docs.\n\n## 🛣️ Core Usage Paths\n\nThis repo contains examples highlighting some common SageMaker use cases: \n\n### Inference\n\nRunhouse facilitates easier access to the SageMaker compute from different environments. \nYou can interact with the compute from notebooks, IDEs, research, pipeline DAG, or any python interpreter. \nRunhouse allows you to SSH directly onto the cluster, update or suspend cluster autostop, and stream logs \ndirectly from the cluster. \n\nWe've highlighted to inference examples:\n- **[Stable Diffusion](inference/stable_diffusion.py)**: Create an inference service which receives a prompt input text and outputs a PIL image\n- **[Llama2](inference/llama2inference.py)**: Stand up an inference service using the Hugging Face chat model (Note: this requires a token)\n\nRunning the SD inference example:\n\n```bash\npython inference/stable_diffusion.py\n```\n\nRunning the Llama2 inference example:\n\n```bash\npython inference/llama2inference.py\n```\n\n### Training\n\nWe'll use a simple PyTorch model to illustrate the different ways we can run training on SageMaker compute via Runhouse.\nIn each of these examples, Runhouse is responsible for spinning up the requested SageMaker compute, and executing the \ntraining code on the cluster.\n\n(1) [**Simple train**](training/simple_train): Use Runhouse to create the SageMaker cluster and handle running the \ntraining code. In this example, we wrap the training code in a Runhouse function which we send to our cluster for \nexecution. The changes to the source code are minimal - we simply instantiate our SageMaker cluster, wrap the training \ncode in a function, and then call it in the same way we would call a local function.\n\nRun this example:\n\n```bash\npython training/simple_train/train.py\n```\n\n```bash\npython training/simple_train/train.py --epochs 5 --learning-rate 0.001 --batch-size 64\n```\n\n(2) [**Interactive train**](training/interactive_train): Convert the training code into a \nRunhouse [Module](https://www.run.house/docs/api/python/module) class, with separate methods for training, eval, and \ninference. While this requires slightly more modifications to the original source code, it gives us a stateful and \ninteractive experience with the model on the cluster, as if we are in a notebook environment. We can much more easily \nrun training epochs or try out the most recent checkpoint of the model that's been saved, without the need for \npackaging up the model and deploying it to a separate endpoint.\n\nRun this example:\n\n```bash\npython training/interactive_train/train.py\n```\n\n```bash\npython training/interactive_train/train.py --output-path ~/.cache/models/output\n```\n\n🦸 Both of these examples unlock a key superpower - the ability to easily run class methods on a remote cluster, \n**without** needing to translate or migrate the code onto another system.\n\n(3) [**Train with Estimator**](training/train_with_estimator): Use the SageMaker SDK to create an\n[estimator](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html). This is useful if you are already \nusing the SageMaker APIs. In this example, we define a SageMaker estimator which loads in the training code from \na separate file where the training code lives.\n\n*Note*: Logs for the training job can be viewed on the cluster in path: `/opt/ml/code/sm_cluster.out` or in \nAWS Cloudwatch in the default log group folder (e.g. `/aws/sagemaker/TrainingJobs`)\n\n### Hyperparameter Tuning (⚠️ Under active development)\n\nFor this [example](hyperparameter_tuning/hp_tuning.py), we use [Ray Tune](https://docs.ray.io/en/latest/tune/index.html) to try different hyperparameter \ncombinations on SageMaker compute. \n\n## 👨‍🏫 Resources\n[**Docs**](https://www.run.house/docs/api/python/cluster#sagemakercluster-class):\nHigh-level overviews of the architecture, detailed API references, and basic API examples for the SageMaker \nintegration.\n\n**Blog**: Coming soon... \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlewitt1%2Frunhouse-sagemaker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjlewitt1%2Frunhouse-sagemaker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlewitt1%2Frunhouse-sagemaker/lists"}