{"id":15017809,"url":"https://github.com/aws-samples/sagemaker-ssh-helper","last_synced_at":"2025-05-15T11:08:27.331Z","repository":{"id":62035071,"uuid":"551597084","full_name":"aws-samples/sagemaker-ssh-helper","owner":"aws-samples","description":"A helper library to connect into Amazon SageMaker with AWS Systems Manager and SSH (Secure Shell)","archived":false,"fork":false,"pushed_at":"2025-02-28T14:40:26.000Z","size":8614,"stargazers_count":241,"open_issues_count":5,"forks_count":34,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-22T02:37:31.338Z","etag":null,"topics":["amazon-sagemaker","aws","aws-systems-manager","machine-learning","pycharm","sagemaker","sagemaker-studio","ssh","vscode"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit-0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aws-samples.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-10-14T17:56:45.000Z","updated_at":"2025-04-14T08:25:08.000Z","dependencies_parsed_at":"2024-09-21T10:00:42.981Z","dependency_job_id":"148de8c4-8489-486a-aa65-6501d7cf6866","html_url":"https://github.com/aws-samples/sagemaker-ssh-helper","commit_stats":{"total_commits":308,"total_committers":8,"mean_commits":38.5,"dds":"0.045454545454545414","last_synced_commit":"6d6b9769f726e92a04515dbd07b45bc7436c58a6"},"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws-samples%2Fsagemaker-ssh-helper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws-samples%2Fsagemaker-ssh-helper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws-samples%2Fsagemaker-ssh-helper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws-samples%2Fsagemaker-ssh-helper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aws-samples","download_url":"https://codeload.github.com/aws-samples/sagemaker-ssh-helper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254328385,"owners_count":22052632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon-sagemaker","aws","aws-systems-manager","machine-learning","pycharm","sagemaker","sagemaker-studio","ssh","vscode"],"created_at":"2024-09-24T19:51:00.804Z","updated_at":"2025-05-15T11:08:27.296Z","avatar_url":"https://github.com/aws-samples.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SageMaker SSH Helper\n\n[![Latest Version](https://img.shields.io/pypi/v/sagemaker-ssh-helper.svg)](https://pypi.python.org/pypi/sagemaker-ssh-helper)\n[![Supported Python Versions](https://img.shields.io/pypi/pyversions/sagemaker-ssh-helper.svg)](https://pypi.python.org/pypi/sagemaker-ssh-helper)\n[![License](https://img.shields.io/github/license/aws-samples/sagemaker-ssh-helper.svg)](https://github.com/aws-samples/sagemaker-ssh-helper/blob/main/LICENSE)\n\nSageMaker SSH Helper is the \"army-knife\" library that helps you to securely connect to Amazon SageMaker training jobs, processing jobs, batch inference jobs\nand realtime inference endpoints as well as SageMaker Studio set of IDEs and SageMaker Notebook Instances for fast interactive experimentation, \nremote debugging, and advanced troubleshooting.\n\nThree most common tasks that motivated to create the library, sometimes referred as \"SSH into SageMaker\", are:\n1. A terminal session into a container running in SageMaker to diagnose a stuck training job, use CLI commands \nlike nvidia-smi and neuron-ls, or iteratively fix and re-execute your training script within seconds. \n2. Remote debugging of a code running in SageMaker from your local favorite IDE like \nPyCharm Professional Edition or Visual Studio Code.\n3. Port forwarding to access auxiliary tools running inside SageMaker, e.g., Dask dashboard, Streamlit apps, TensorBoard or Spark Web UI.\n\n![Screenshot](images/high-level-architecture.png)\n\nOther tasks include but not limited to connecting to a remote Jupyter Notebook in SageMaker Studio from your IDE, or starting a VNC session to SageMaker Studio to run GUI apps.  \n\n\n## How it works\nSageMaker SSH helper uses AWS Systems Manager (SSM) Session Manager, to register a SageMaker container in SSM, followed by creating a session between your client machine and the Session Manager service. From there, you can \"SSH into SageMaker\" by creating an SSH (Secure Shell) connection on top of the SSM sessions, that allows opening a Linux shell and configuring bidirectional SSH port forwarding to run applications like remote development, debugging, desktop GUI, and others.\n\n![Screenshot](images/guidance-diagram.png)\n\nIf you want to understand deeper how both the SageMaker service and the SageMaker SSH Helper library work, check the [Flow Diagrams](Flow_Diagrams.md) of the common use cases and carefully read all sections of the documentation.\n\nMake sure you also looked at our [Frequently Asked Questions](FAQ.md), especially at the [troubleshooting section](FAQ.md#troubleshooting), as well as at the existing both open and resolved [issues](https://github.com/aws-samples/sagemaker-ssh-helper/issues?q=is%3Aissue).\n\n## Getting started\n\nTo get started, your AWS system administrator must configure IAM and SSM in your AWS account as shown \nin [Setting up your AWS account with IAM and SSM configuration](IAM_SSM_Setup.md).\n\n\u003e **Note: This repository is a sample AWS content.** You should not use the sample content in your production accounts, in a production \n\u003e environment or on production or other critical data. If you plan to use the content in production, please, carefully review it with your security team. \n\u003e You are responsible for testing, securing, and optimizing the sample content \n\u003e as appropriate for production grade use based on your specific business requirements, including any quality control \npractices and standards.\n\nAlso make sure that the documentation you are reading corresponds the version you are installing. The best way to do that is to access the repository from the [Releases](https://github.com/aws-samples/sagemaker-ssh-helper/releases) page. In this case the URL in the address bar should look like [sagemaker-ssh-helper/tree/vX.Y.Z](https://github.com/aws-samples/sagemaker-ssh-helper/tree/vX.Y.Z).\n\n## Costs\n\nYou are responsible for the cost of the AWS services used while running this guidance. If you haven't used Systems Manager Session Manager in your AWS account before, it will be the only additional cost associated with using SageMaker SSH Helper. The costs of [SageMaker resources](https://aws.amazon.com/sagemaker/pricing/) do not change and you pay only for the resources that you use.  \n\nAs explained in the [IAM and SSM configuration](IAM_SSM_Setup.md), you need to enable Advanced instances tier in Systems Manager because this is the only way to enable Session Manager and access remote instances interactively with SSM and SSH. As of June 2024, the advanced instances tier comes at [a small fee](https://aws.amazon.com/systems-manager/pricing/#Session_Manager), which is $0.00695 per advanced on-premises instance per hour. \n\nSee [the sample cost breakdown table](Sample_Costs.md) for SageMaker and Systems Manager for AWS account that runs the routine [CI/CD tests](tests/) for this guidance in the EU (Ireland) Region.\n\nWe recommend creating a [Budget](https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-managing-costs.html) through [AWS Cost Explorer](https://aws.amazon.com/aws-cost-management/aws-cost-explorer/) to help manage costs. Prices are subject to change. For full details, refer to the pricing webpage for each AWS service used in this guidance.\n\n\n## Use Cases\nSageMaker SSH Helper supports a variety of use cases:\n- [Connecting to SageMaker training jobs with SSM](#training) - shell access only, open a shell to a single- or multi-node training job to examine its file systems, monitor resources, produce thread-dumps for stuck jobs, and interactively run your train script\n  - [Connecting to SageMaker inference endpoints with SSM](#inference)\n  - [Connecting to SageMaker batch transform jobs](#batch-transform)\n  - [Connecting to SageMaker processing jobs](#processing)  \n- [Forwarding TCP ports over SSH tunnel](#port-forwarding) - intermediate layer for debugging and remote code execution, enables through SSH and port forwarding the access to remote apps like Dask, Streamlit or Jupyter Notebook\n  - [sm-ssh](#sm-ssh) - SageMaker SSH Helper CLI, introduced in v2.2\n  - [~/.ssh/config](#sshconfig) - to use `ssh` CLI command instead of `sm-ssh`\n- [Remote code execution with PyCharm / VSCode over SSH](#remote-interpreter) - debugging and remote code execution, lets your IDE run or debug your code line-by-line inside SageMaker container with SSH interpreter, e.g. inside a SageMaker training job\n  - [Remote debugging with PyCharm Debug Server over SSH](#pycharm-debug-server) - debugging and remote code execution, uses debug server, lets SageMaker run your code that connects to PyCharm with a callback, to start line-by-line debugging with [PyDev.Debugger](https://pypi.org/project/pydevd-pycharm/), a.k.a. pydevd\n- [Local IDE integration with SageMaker Studio over SSH for PyCharm / VSCode](#studio) - debugging and remote code execution in SageMaker Studio, iterate fast on a single node at early stages of development without submitting SageMaker jobs\n  - [Support of SageMaker Notebook Instances](FAQ.md#are-sagemaker-notebook-instances-supported) - see FAQ\n- [Web VNC](#web-vnc) - run virtually any IDE or GUI tool in a browser though [AWS Jupyter Proxy](https://github.com/aws/aws-jupyter-proxy) extension and VNC remote desktop protocol\n\n**Pro Tip**: While multiple use cases allow debugging and remote code execution, the typical development journey looks as follows: (1) you start developing and running code in the IDE on your [local machine](https://aws.amazon.com/blogs/machine-learning/run-your-tensorflow-job-on-amazon-sagemaker-with-a-pycharm-ide/), then (2) you connect the IDE to [SageMaker Studio](https://docs.aws.amazon.com/sagemaker/latest/dg/studio.html) with SageMaker SSH Helper to test and troubleshoot it on the remote instance, then (3) you integrate your code with SageMaker and run it as a [training](https://docs.aws.amazon.com/sagemaker/latest/dg/train-model.html), [processing](https://docs.aws.amazon.com/sagemaker/latest/dg/processing-job.html) or [inference](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html) job, using SageMaker SSH Helper if needed, and finally (4) assemble jobs into MLOps pipelines with [SageMaker Projects](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-projects.html), to be deployed into multi-account structure on top of (5) the [secure enterprise ML platform](https://docs.aws.amazon.com/whitepapers/latest/build-secure-enterprise-ml-platform/build-secure-enterprise-ml-platform.html).\n\nIf you want to add a new use case or a feature, see [CONTRIBUTING](CONTRIBUTING.md).\n\n\n## \u003ca name=\"training\"\u003e\u003c/a\u003eConnecting to SageMaker training jobs with SSM\n\n[![Download Demo (.mov)](https://user-images.githubusercontent.com/87804596/205893540-d7a08259-94b3-48f2-b96e-93798b98a06c.png)](https://aws-blogs-artifacts-public.s3.amazonaws.com/artifacts/ML-4988/SSH_Helper-Shell-To-Training-Jobs.mov)\n[Download Demo (.mov)](https://aws-blogs-artifacts-public.s3.amazonaws.com/artifacts/ML-4988/SSH_Helper-Shell-To-Training-Jobs.mov)\n\n\u003e **Note**: This demo is recorded with a previous version of SSH Helper and may be not up-to-date with the recent features. Check the below documentation for the most up-to-date steps.\n\n### Step 1: Install the library\nBefore starting the whole procedure, check that both `pip` and `python` commands point to Python version 3.8 or higher with `python --version` command. Check all supported Python versions [in the PyPI repo](https://pypi.org/project/sagemaker-ssh-helper/) under \"Programming Language\" list.\n\n\u003e **Important:** Make sure you read the \"Getting started\" section and didn't skip the steps from [Setting up your AWS account with IAM and SSM configuration](IAM_SSM_Setup.md).\n\nInstall the latest stable version of library from the [PyPI repository](https://pypi.org/project/sagemaker-ssh-helper/):\n\n```shell\npip install sagemaker-ssh-helper\n```\n**Caution:** It's always recommended to install the library into a Python venv, not into the system env. If you want to use later the SSH plugins of your IDE that will use the system env and system Python, you should add the venv into the system PATH, as described in the section [Remote code execution with PyCharm / VSCode over SSH](#remote-interpreter).\n\nIf you're working on Windows, see [FAQ](FAQ.md#is-windows-supported).\n\n### Step 2: Modify your start training job code\n1. Add import for `SSHEstimatorWrapper`\n2. Add a `dependencies` parameter to the `Estimator` object constructor. Alternatively, instead of `dependencies` parameter, put `sagemaker_ssh_helper` into `source_dir/training/requirements.txt`.\n3. Add an `SSHEstimatorWrapper.create(estimator,...)` call before calling `fit()`.\n4. Add a call to `ssh_wrapper.print_ssh_info()` or `ssh_wrapper.get_instance_ids()` to get the SSM instance(s) id. You'll use this information\nto connect to the instance later on.   \n\nIn a nutshell:\n\n```python\nimport logging\nfrom sagemaker.pytorch import PyTorch\nfrom sagemaker_ssh_helper.wrapper import SSHEstimatorWrapper  # \u003c--NEW--\n\nrole = ...\n\nestimator = PyTorch(\n    entry_point='train.py',\n    source_dir='source_dir/training/',\n    dependencies=[SSHEstimatorWrapper.dependency_dir()],  # \u003c--NEW--\n    role=role,\n    framework_version='1.9.1',\n    py_version='py38',\n    instance_count=1,\n    instance_type='ml.m5.xlarge'\n)\n\nssh_wrapper = SSHEstimatorWrapper.create(estimator, connection_wait_time_seconds=600)  # \u003c--NEW--\n\nestimator.fit(wait=False)\n\nssh_wrapper.print_ssh_info()  # \u003c--NEW-- \n```\n\nThe `connection_wait_time_seconds` is the amount of time the SSH Helper will wait inside SageMaker before it continues the job execution. It's useful for training jobs, when you want to connect before training starts.\nIf you don't want the job to wait and start training as soon as the job starts, set it to `0`.\n\nAs an example, here's the full working code from a unit test: [test_end_to_end.py#L31-L56](https://github.com/aws-samples/sagemaker-ssh-helper/blob/v2.1.0/tests/test_end_to_end.py#L31-L56) . The method `start_ssm_connection_and_continue(port_number)` will connect to the instance through API, terminate the waiting loop and start training (useful for automation).\n\nIf you configured distributed training (i.e., `instance_count` is more than one), SSH Helper\nwill start by default only on the first two nodes (i.e., on `algo-1` and `algo-2`).\nIf you want to connect with SSH to other nodes, you can log in to either of these nodes, e.g., `algo-2`,\nand then SSH from this node to any other node of the training cluster, e.g., `algo-4`, without running SSH Helper \non these nodes, e.g., inside the pre-build SageMaker framework containers like PyTorch training container just run `ssh algo-4` from the shell.\n\nAlternatively, for distributed training, pass the additional parameter `ssh_instance_count` with the desired instance count \nto `SSHEstimatorWrapper.create()`, e.g., `SSHEstimatorWrapper.create(..., ssh_instance_count=3)`\n\n*Note:* if you a/ don't use script mode, b/ use basic `Estimator` class and c/ all code is already stored in your Docker container, check the code sample in [the corresponding section of the FAQ](FAQ.md#what-if-i-want-to-train-and-deploy-a-model-as-a-simple-estimator-in-my-own-container-without-passing-entry_point-and-source_dir).\n\nDon't run the modified code yet, see the next step.\n\n### Step 3: Modify your training script\n\n```python\nimport sagemaker_ssh_helper\nsagemaker_ssh_helper.setup_and_start_ssh()\n```\n\nThe `setup_and_start_ssh()` will start an SSM Agent that will connect the training instance to AWS Systems Manager.\n\nSee the [train.py](https://github.com/aws-samples/sagemaker-ssh-helper/blob/v2.1.0/tests/source_dir/training/train.py) from the corresponding unit test, as a full working code sample.\n\n### Step 4: Connect over SSM\nOnce you launched the job, you'll need to wait, a few minutes, for the SageMaker container to start and the SSM Agent\nto start successfully. Then you'll need to have the ID of the managed instance. The instance id is prefixed by `mi-` \nand will appear in the job's CloudWatch log like this:\n\n```text\nSuccessfully registered the instance with AWS SSM using Managed instance-id: mi-1234567890abcdef0\n``` \n\nTo fetch the instance IDs in an automated way without looking into the logs, you can call the Python method `ssh_wrapper.get_instance_ids()` or `ssh_wrapper.print_ssh_info()`, as mentioned in the step 1:\n\n```python\nestimator = ...\nssh_wrapper = ...\nestimator.fit(wait=False)\ninstance_ids = ssh_wrapper.get_instance_ids(timeout_in_sec=900)\n```\n\nThe method `get_instance_ids()` accepts the optional parameter `timeout_in_sec` (default is 900, i.e., 15 minutes). \nIf timeout is not 0, it will retry attempts to get instance IDs every 10 seconds.\n\nWith the instance ID at hand, you will be able to connect to the training container using the command line or the AWS web console. \n\nMethod A. Connecting using command line:  \n\n1. On the local machine, make sure that you installed the latest [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and the [AWS Session Manager CLI plugin](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html). Run the following command to perform the installation:\n\n```shell\nsm-local-configure\n```\n\n2. Run this command (replace the target value with the instance id for your SageMaker job). Example:\n```shell\naws ssm start-session --target mi-1234567890abcdef0\n```\n\n**Note:** Recently SageMaker has introduced [the native way to connect to training jobs with SSM](https://docs.aws.amazon.com/sagemaker/latest/dg/train-remote-debugging.html). \n\nMethod B. Connecting using the AWS Web Console:  \n\n  1. In AWS Web Console, navigate to Systems Manager \u003e Fleet Manager.     \n  2. Select the node, then Node actions \u003e Start terminal session.\n\nOnce connected to the container, you might want to switch to the root user with `sudo su -` command.\n\nMethod C. Connecting with SSH and port forwarding:\n\nThis method uses `sm-ssh connect` command and described in more details in the section [sm-ssh](#sm-ssh).\n\nMethod D. Connecting from SageMaker Studio\n\nSee the corresponding step [Connecting from SageMaker Studio](IAM_SSM_Setup.md#4-optional-connecting-from-sagemaker-studio) in IAM / SSM configuration section. Follow the same steps as in the Method A for the local machine, but run them inside SageMaker Studio. If you run the commands from SageMaker Studio image terminal, make sure that your Python environment is activated, e.g., with `conda activate base`.\n\n\n#### \u003ca name=\"cli-commands\"\u003e\u003c/a\u003eTip: Useful CLI commands\n\nHere are some useful commands to run in a terminal session:  \n- `ps xfaww` - Show running tree of processes\n- `ps xfawwe` - Show running tree of processes with environment variables \n- `ls -l /opt/ml/input/data` - Show input channels\n- `ls -l /opt/ml/code` - Show your training code\n- `pip freeze | less` - Show all Python packages installed\n- `dpkg -l | less` - Show all system packages installed\n\n#### \u003ca name=\"gdb\"\u003e\u003c/a\u003eTip: Generating a thread dump for stuck training jobs\nIn case your training job is stuck, it can be useful to observe what where its threads are waiting/busy.\nThis can be done without connecting to a python debugger beforehand.\n\n1. Having connected to the container as root, find the process id (pid) of the training process (assuming it's named `train.py`):\n`pgrep --newest -f train.py`  \n2. Install GNU debugger:  \n`apt-get -y install gdb python3.9-dbg`  \n3. Start the GNU debugger with python support:  \n`gdb python`  \n`source /usr/share/gdb/auto-load/usr/bin/python3.9-dbg-gdb.py`  \n4. Connect to the process (replace 361 with your pid):  \n`attach 361`  \n5. Show C low-level thread dump:  \n`info threads`  \n6. Show Python high-level thread dump:  \n`py-bt`  \n7. It might also be useful to observe what system calls the process is making:\n`apt-get install strace`\n8. Trace the process (replace 361 with your pid):  \n`sudo strace -p 361`\n\n#### \u003ca name=\"cli-commands\"\u003e\u003c/a\u003eTip: Pipeline automation\nIf you're looking for the full automation of the pipeline with SSM and SSH, and not only with `get_instance_ids()` method, take a look at the [automation question in the FAQ](FAQ.md#how-do-i-automate-my-pipeline-with-sagemaker-ssh-helper-end-to-end).\n\n## \u003ca name=\"inference\"\u003e\u003c/a\u003eConnecting to SageMaker inference endpoints with SSM\n\n**Note:** Recently SageMaker has introduced [the native way to connect to endpoints with SSM](https://docs.aws.amazon.com/sagemaker/latest/dg/ssm-access.html), but it requires allow-listing of an AWS account (as of writing).\n\nAdding SageMaker SSH Helper to inference endpoint is similar to training with the following differences.\n\n1. Wrap your model into `SSHModelWrapper` before calling `deploy()` and add SSH Helper to `dependencies`:\n\n```python\nfrom sagemaker import Predictor\nfrom sagemaker_ssh_helper.wrapper import SSHModelWrapper  # \u003c--NEW--\n\nestimator = ...\n...\nendpoint_name = ... \n\nmodel = estimator.create_model(\n    entry_point='inference_ssh.py',\n    source_dir='source_dir/inference/',\n    dependencies=[SSHModelWrapper.dependency_dir()]  # \u003c--NEW \n    # (alternatively, add sagemaker_ssh_helper into requirements.txt \n    # inside source dir) --\n)\n\nssh_wrapper = SSHModelWrapper.create(model, connection_wait_time_seconds=0)  # \u003c--NEW--\n\npredictor: Predictor = model.deploy(\n    initial_instance_count=1,\n    instance_type='ml.m5.xlarge',\n    endpoint_name=endpoint_name,\n    wait=True\n)\n\npredicted_value = predictor.predict(data=...)\n```\n\n*Note:* For the inference endpoint, which is always up and running, there's not too much value \nin setting `connection_wait_time_seconds`, so it's usually set to `0`.\n\nSimilar to training jobs, you can fetch the instance ids for connecting to the endpoint with SSM with \n`ssh_wrapper.get_instance_ids()` or `ssh_wrapper.print_ssh_info()`.\n\n2. Add the following lines at the top of your `inference_ssh.py` script:\n\n```python\nimport os\nimport sys\nsys.path.append(os.path.join(os.path.dirname(__file__), \"lib\"))\n\nimport sagemaker_ssh_helper\nsagemaker_ssh_helper.setup_and_start_ssh()\n```\n\n*Note:* adding `lib` dir to Python path is required, because SageMaker inference is putting dependencies \ninto the `code/lib` directory, while SageMaker training put libs directly to `code`. \n\nOn the following screenshot you see the sample code for the notebook [Deploy SD2.1 to Inferentia2 + SageMaker + HF Optimum Neuron + SageMaker SSH Helper](https://github.com/aws-samples/ml-specialized-hardware/blob/main/tutorials/04_ImageGenerationWithStableDiffusion/SDOnInf2AndHFOptimumNeuron_SMSSH.ipynb) that has the inference code running in SageMaker endpoint on AWS Inferentia accelerator chip.\nThe user has connected to the endpoint with the `sm-ssh connect` command and executed the `neuron-ls` command to list the currently utilized Inferentia cores:\n\n![](images/sm_ssh_inf2.png)\n\nYou can also notice on the screenshot that the user configured the remote Python interpreter that connects to SageMaker Studio, so the user also executes the notebook itself remotely inside SageMaker. This setup is further described in the section for [Local IDE Integration with SageMaker Studio](#studio).\n\n### Multi-model endpoints\n\nFor multi-model endpoints, the setup procedure is slightly different from regular endpoints:\n\n```python\nfrom sagemaker.multidatamodel import MultiDataModel\nfrom sagemaker_ssh_helper.wrapper import SSHModelWrapper, SSHMultiModelWrapper  # \u003c--NEW--\n\nmodel_data_prefix = \"s3://DOC-EXAMPLE-BUCKET/mms/\"\nmodel_name = ...\nendpoint_name = ...\nestimator = ...\n...\n\nmodel = estimator.create_model(entry_point='inference_ssh.py',\n                               source_dir='source_dir/inference/',\n                               dependencies=[SSHModelWrapper.dependency_dir()])  # \u003c--NEW--\n\nmdm = MultiDataModel(\n    name=model.name,\n    model_data_prefix=model_data_prefix,\n    model=model\n)\n\nssh_wrapper = SSHMultiModelWrapper.create(mdm, connection_wait_time_seconds=0)  # \u003c--NEW--\n\npredictor = mdm.deploy(initial_instance_count=1,\n                       instance_type='ml.m5.xlarge',\n                       endpoint_name=endpoint_name)\n\n\nmdm.add_model(model_data_source=model.repacked_model_data, model_data_path=model_name)\n\npredicted_value = predictor.predict(data=..., target_model=model_name)\n```\n\n**Important:** Make sure that you're passing to `add_model()` the model ready for deployment with dependencies located at `model.repacked_model_data`,\nnot the `estimator.model_data` that points to the trained model artifact. To obtain model suitable for inference, you might want to deploy first your model to a temporary single-node endpoint, so that SageMaker Python SDK takes care of repacking the model, or call the `prepare_container_def()` method, like in [the MMS test code](https://github.com/aws-samples/sagemaker-ssh-helper/blob/v2.1.0/tests/test_end_to_end.py#L252-L323).\n\nAlso note that SageMaker SSH Helper will be lazy loaded together with your model upon the first prediction request.\nSo you should try to connect to the multi-model endpoint only after calling `predict()`.\n\nThe `inference.py` script is the same as for regular endpoints.\n\n*Note:* If you are using PyTorch containers, make sure you select the latest versions, \ne.g. 1.12, 1.11, 1.10 (1.10.2), 1.9 (1.9.1).\nThis code might not work if you use PyTorch 1.8, 1.7 or 1.6.\n\n*Note:* If you're packing your models manually and don't pass the `model` object to the `MultiDataModel` constructor, i.e., pass only the `image_uri`, see corresponding sample code in the [FAQ.md](FAQ.md#what-if-i-want-to-deploy-a-multi-data-model-without-passing-a-reference-to-a-model-object-only-with-image_uri).\n\n## \u003ca name=\"batch-transform\"\u003e\u003c/a\u003eConnecting to SageMaker batch transform jobs\n\nFor batch transform jobs, you need to use both `SSHModelWrapper` and `SSHTransformerWrapper`, \nas in the following example:\n\n```python\nfrom sagemaker_ssh_helper.wrapper import SSHModelWrapper, SSHTransformerWrapper  # \u003c--NEW--\n\nsagemaker_session = ...\nbucket = ...\nestimator = ...\n...\n\nmodel = estimator.create_model(entry_point='inference_ssh.py',\n                               source_dir='source_dir/inference/',\n                               dependencies=[SSHModelWrapper.dependency_dir()])  # \u003c--NEW--\n\ntransformer_input = sagemaker_session.upload_data(path='data/batch_transform/input',\n                                                  bucket=bucket,\n                                                  key_prefix='batch-transform/input')\n\ntransformer_output = f\"s3://{bucket}/batch-transform/output\"\n\nssh_model_wrapper = SSHModelWrapper.create(model, connection_wait_time_seconds=600)  # \u003c--NEW--\n\ntransformer = model.transformer(instance_count=1,\n                                instance_type=\"ml.m5.xlarge\",\n                                accept='text/csv',\n                                strategy='SingleRecord',\n                                assemble_with='Line',\n                                output_path=transformer_output)\n\nssh_transformer_wrapper = SSHTransformerWrapper.create(transformer, ssh_model_wrapper)  # \u003c--NEW--\n\ntransformer.transform(data=transformer_input,\n                      content_type='text/csv',\n                      split_type='Line',\n                      join_source=\"Input\",\n                      wait=False)\n```\nThe `inference.py` script is the same as for regular endpoints.\n\n## \u003ca name=\"processing\"\u003e\u003c/a\u003eConnecting to SageMaker processing jobs\n\nSageMaker SSH Helper supports both Script Processors and Framework processors and setup procedure is similar \nto training jobs and inference endpoints.\n\n#### A. Framework processors\n\nThe code to set up a framework processor (e.g. PyTorch) is the following:\n\n```python\nfrom sagemaker.pytorch import PyTorchProcessor\nfrom sagemaker_ssh_helper.wrapper import SSHProcessorWrapper  # \u003c--NEW--\n\nrole = ...\n\ntorch_processor = PyTorchProcessor(\n    base_job_name='ssh-pytorch-processing',\n    framework_version='1.9.1',\n    py_version='py38',\n    role=role,\n    instance_count=1,\n    instance_type=\"ml.m5.xlarge\"\n)\n\nssh_wrapper = SSHProcessorWrapper.create(torch_processor, connection_wait_time_seconds=600)  # \u003c--NEW--\n\ntorch_processor.run(\n    source_dir=\"source_dir/processing/\",\n    dependencies=[SSHProcessorWrapper.dependency_dir()],  # \u003c--NEW--\n    code=\"process_framework.py\"\n)\n```\n\nAlso add the following lines at the top of `process_framework.py`:\n\n```python\nimport sagemaker_ssh_helper\nsagemaker_ssh_helper.setup_and_start_ssh()\n```\n\n#### B. Script Processors\n\nThe code to set up a script processor (e.g. PySpark) is the following:\n\n```python\nfrom sagemaker.spark import PySparkProcessor\nfrom sagemaker_ssh_helper.wrapper import SSHProcessorWrapper  # \u003c--NEW--\n\nrole = ...\n\nspark_processor = PySparkProcessor(\n    base_job_name='ssh-spark-processing',\n    framework_version=\"3.0\",\n    role=role,\n    instance_count=1,\n    instance_type=\"ml.m5.xlarge\"\n)\n\nssh_wrapper = SSHProcessorWrapper.create(spark_processor, connection_wait_time_seconds=600)  # \u003c--NEW--\n\nspark_processor.run(\n    submit_app=\"source_dir/processing/process.py\",\n    inputs=[ssh_wrapper.augmented_input()]  # \u003c--NEW--\n)\n```\n\nAlso add the following lines at the top of `process.py`:\n\n```python\nimport sys\nsys.path.append(\"/opt/ml/processing/input/\")\n\nimport sagemaker_ssh_helper\nsagemaker_ssh_helper.setup_and_start_ssh()\n```\n\n## \u003ca name=\"port-forwarding\"\u003e\u003c/a\u003eForwarding TCP ports over SSH tunnel\n\nPrevious sections focused on connecting to non-interactive SageMaker containers with SSM.\n\nNext sections rely on the Session Manager capability to create an SSH tunnel over SSM connection. SageMaker SSH Helper in turn runs SSH session over SSH tunnel and forwards the ports, including the SSH server port 22 itself.\n\nThe helper script behind this logic is `sm-local-start-ssh`:\n\n```shell\nsm-local-start-ssh \"$INSTANCE_ID\" \\\n  -R localhost:12345:localhost:12345 \\\n  -L localhost:8787:localhost:8787 \\\n  -L localhost:11022:localhost:22\n```\n\nYou can pass `-L` parameters for forwarding remote container port to local machine (e.g., `8787` for [Dask dashboard](https://docs.dask.org/en/stable/dashboard.html) or `8501` for [Streamlit apps](https://docs.streamlit.io/library/get-started)) or `-R` for forwarding local port to remote container. Read more about these options in the [SSH manual](https://man.openbsd.org/ssh).\n\nThis low-level script takes the managed instance ID as a parameter. Next sections use the high-level command `sm-ssh` that take the SageMaker resource name as a parameter and resolves it into the instance ID automatically.\n\n### sm-ssh\n\nThe syntax for the SSH Helper CLI command `sm-ssh` is the following:\n\n```bash\nsm-ssh [-h] [-v] {list,start-proxy,connect} [fqdn] [extra-connect-args]*\n```\n\nwhere `fqdn` is the resource name with `.sagemaker` suffix, respectively:\n\n* for model training, `.training.sagemaker`\n* for inference endpoints, i.e., real-time inference, `.inference.sagemaker`\n* for transform jobs, i.e., batch inference, `.transform.sagemaker`\n* for processing jobs, i.e. transforms without a trained model, `.processing.sagemaker`\n* for SageMaker Studio Classic, `.studio.sagemaker` - see [Local IDE integration with SageMaker Studio over SSH](#studio) for more details on FQDN format\n* for SageMaker Notebook instances, `.notebook.sagemaker`\n\nFor `list` command, the resource name and the dot `.` in the suffix can be omitted, i.e.:\n\n```bash\nsm-ssh list studio.sagemaker\n```\n\n– will list all running jupyter servers and kernel gateways and their SSH status, and\n\n```\nsm-ssh list \n```\nor\n```\nsm-ssh list sagemaker \n```\n\n– will list all resources of all types.\n\nThe instances with SSH Helper will be marked `Online` or `ConnectionLost` while the instances not registered with SSM be marked with `ssh:NotFound`.\n\nThe `connect` command starts interactive SSH session into container, e.g.:\n\n```bash\nsm-ssh connect ssh-training-example-2023-07-25-03-18-04-490.training.sagemaker\n```\n\nIt's possible to pass additional arguments and forward ports together with the `connect` command, e.g., to forward [SSH Agent](https://linux.die.net/man/1/ssh-agent)  and Streamlit web app port:\n\n```bash\nssh-add\nsm-ssh connect ssh-training-example-2023-07-25-03-18-04-490.training.sagemaker -A -L 8501:localhost:8501\n```\n\n#### ~/.ssh/config\n\nAlternatively, instead of using `sm-ssh connect` command, you can use the native `ssh` command, but it will require you to update your [ssh config](https://linux.die.net/man/5/ssh_config), typically `~/.ssh/config`, with `sm-ssh start-proxy` command as follows:\n\n```bash\nHost *.studio.sagemaker\n  User sagemaker-user\n\nHost *.*.sagemaker\n  IdentityFile ~/.ssh/%h\n  PasswordAuthentication no\n  ConnectTimeout 120\n  ServerAliveInterval 15\n  ServerAliveCountMax 8\n  ProxyCommand sm-ssh start-proxy %h\n  User root\n```\n\nYou can copy the same fragment from the [ssh_config_template.txt](ssh_config_template.txt) file.\n\nThe `sm-ssh start-proxy` command will set up the non-interactive SSH session that will serve as a proxy tunnel for SSH command. \n\nAs a benefit, you will be able to add additional SSH options like forwarding SSH agent connection with `-A` option, to securely pass your local SSH keys to remote machine, or forward ports with `-R` and `-L` options, akin to passing these options to `sm-local-start-ssh` command. \n\nAn example with [SSH Agent](https://linux.die.net/man/1/ssh-agent) and forwarding the web server port `8080`:\n```bash\nssh-add\nssh -A -L localhost:8080:localhost:8080 \\\n  ssh-training-example-2023-07-25-03-18-04-490.training.sagemaker\n```\n\nAs a drawback, you won't get a comprehensive logging since the output of `sm-ssh` will be suspended by ssh. In case you have connection issues with native `ssh`, try the `sm-ssh` command instead and check the output. \n\nFollow the steps in the next section for the IDE configuration, to prepare the `sm-ssh` for the use on the local machine. \n\n## \u003ca name=\"remote-interpreter\"\u003e\u003c/a\u003eRemote code execution with PyCharm / VSCode over SSH\n\n1. On the local machine, make sure that you installed the latest [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and the [AWS Session Manager CLI plugin](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html). To do so, perform the automated installation with the [sm-local-configure](sagemaker_ssh_helper/sm-local-configure) script:\n\n```shell\n# pip install sagemaker-ssh-helper\nsm-local-configure\n```\n\n**Caution**: You will use SSH plugins from the IDE running inside your system env with system Python, therefore you should add SSH Helper into your system PATH, e.g., on macOS:\n```bash\nsudo bash -c \"echo '/Users/janedoe/PycharmProjects/sagemaker-ssh-helper-dev-venv/bin' \u003e /etc/paths.d/42-sm-ssh\"\n```\n\nYou might need restart the Terminal and the IDE for changes to take an effect.\n\nAlternatively, use the trick with port forwarding - start the `sm-ssh` or `ssh` with `-L` option inside venv, and then use `localhost` as the host to connect to from IDE. This trick is used to make SSH Helper work on Windows, and it's described in [FAQ - Is Windows Supported?](FAQ.md#is-windows-supported).\n\n2. Submit your code to SageMaker with SSH Helper as described in previous sections, e.g. as a [training job](#step-1-install-the-library).\n\nMake sure you allow enough time for manually setting up the connection (do not set `connection_wait_time_seconds` to `0`, recommended minimum value is `600`, i.e. 10 minutes).  Don't worry to set it to higher values, e.g. to 30 min, because you will be able to terminate the waiting loop once you connected.\n\nInstead of using SSM to connect to the container from command line, proceed to the next step for configuring the IDE.\n\n3. Configure the remote interpreter in your IDE\n\nMake sure you've configured your ssh config as mentioned in the [~/.ssh/config](#sshconfig) section and your IDE can access `sm-ssh` command from the system env.\n\nIf you connect to your host for the first time, check that `ssh` command is working from CLI:\n\n```bash\nssh sh-training-manual-2023-10-02-14-38-56-744.training.sagemaker\n```\n\nA. Follow the [instructions in the PyCharm docs](https://www.jetbrains.com/help/pycharm/remote-debugging-with-product.html#remote-interpreter), to configure the remote interpreter in PyCharm.\n\nIn the field for host name, put the same value as for `fqdn` in the [`sm-ssh` command](#sm-ssh), e.g., `ssh-training-manual-2023-10-02-14-38-56-744.training.sagemaker`, and use `root` as the username. \nFor SageMaker Studio you might want to use `sagemaker-user` instead of `root`.\n\n![](images/pycharm_training.png)\n\nIf PyCharm asks for the SSH key, point to the `~/.ssh/\u003cfqdn\u003e` private key file that was automatically generated for you by SSH Helper:\n\n![](images/pycharm_training_ssh.png)\n\n*Note:* If PyCharm says connection refused, it can be due to timeout. Check that you can connect to this host from your system terminal with `ssh` and `sm-ssh` and try configuring the remote interpreter again.\n\n*Tip:* Instead of creating a new venv, point the Python interpreter to the existing location. \nYou can find this location by running a cell with `import sys; sys.executable` command in a SageMaker Studio notebook. You will get something like `/opt/conda/bin/python`.\n\n*Tip:* Now you also can [upload and download files from remote](https://www.jetbrains.com/help/pycharm/uploading-and-downloading-files.html) and [synchronize files with remote](https://www.jetbrains.com/help/pycharm/comparing-deployed-files-and-folders-with-their-local-versions.html).\n\nB. Follow the [instructions for VSCode](https://code.visualstudio.com/docs/remote/ssh), to configure local Visual Studio Code app\n\nPut the `root@fqdn` as the hostname to connect to, e.g., `root@ssh-training-example-2023-07-25-03-18-04-490.training.sagemaker` . \nFor SageMaker Studio you might want to use `sagemaker-user` instead of `root`, \ne.g. `sagemaker-user@default.ivankh.d-vdnygexample.studio.sagemaker`.\n\n![](images/vscode_training.png)\n\n\u003e **NOTE:**  The **Remote SSH** extension described in the above instructions is only for the [Visual Studio Code native app](https://code.visualstudio.com/). Code Editor in SageMaker Studio and other web apps based on [Code - OSS](https://github.com/microsoft/vscode#visual-studio-code---open-source-code---oss) such as [Code Server](https://github.com/coder/code-server) that use extensions from [Open VSX Registry](https://open-vsx.org/) might look and work differently from the native app that has Microsoft-specific customizations. SageMaker SSH Helper **DOES NOT** support browser-based implementations of VS Code and haven't been tested with any of Open VSX extensions. If you prefer to use the browser for development, take a look at the [Web VNC](#web-vnc) option. \n\nThere are few extension options that you might want to change for VS Code to work properly with SageMaker containers:\n\n* You might need to increase \"Remote.SSH: Connect Timeout\" option to `120` in VS Code. See [the StackOverflow post](https://stackoverflow.com/questions/59978826/why-ssh-connection-timed-out-in-vscode) for details. \n\n* If you see the error `tar: code: Cannot change ownership to uid 1000, gid 1000: Operation not permitted` when connecting, then try to set \"Remote.SSH: Use Exec server\" to `false`, as mentioned in [#58 - vscode connect fails](https://github.com/aws-samples/sagemaker-ssh-helper/issues/58).\n\n* You might also need to set \"Remote.SSH: Use Local Server\" to `false` and \"Remote.SSH: Lockfiles In Tmp\" to `true`, if you still have connection problems.\n\n4. Connect to the instance and stop the waiting loop\n\nWhen you set `connection_wait_time_seconds` to non-zero value, SSH Helper will run a waiting loop inside your training script, until waiting time is passed, or you manually terminate the loop.\n\nTo manually terminate the loop, run the `sm-wait stop` command from the container (under `root`):\n```bash\nssh root@ssh-training-example-2023-07-25-03-18-04-490.training.sagemaker \\\n  sm-wait stop\n```\n\nNote, that if you stop the waiting loop, SageMaker will run your training script only once, and you will be able to execute additional code from local machine from PyCharm only while your script is running. Once the script finishes, you will need to submit another training job and repeat the procedure again.\n\nHere's a useful trick: submit a dummy script `train_placeholder.py` with the infinite loop, and while this loop will be running, you can rerun your real training script again and again with the remote interpreter inside the same job.\n\nThe workflow in this case is roughly the following:\n\n  a. You submit a first job with your training script `train.py`, and it fails for some reason that you want to troubleshoot.\n  \n  b. You submit a second job with the placeholder script `train_placeholder.py`. You run your training script inside this job and change it few times until you find the cause of the problem and fix it. Setting `max_run` parameter of the estimator is highly recommended for the placeholder job, to avoid unnecessary charges.\n\n  c. You submit a third job with your fixed training script `train.py` to make sure it works now.\n\nThe dummy script may look like this:\n\n```python\nimport time\nfrom datetime import timedelta\n\nfrom sagemaker_ssh_helper import setup_and_start_ssh, is_last_session_timeout\n\nsetup_and_start_ssh()\n\nwhile not is_last_session_timeout(timedelta(minutes=30)):\n    time.sleep(10)\n```\n\nThe method `is_last_session_timeout()` will help to prevent unused resources and the job will end if there's no SSM or SSH sessions for the specified period of time. It will count active SSM sessions, and time out when there are no sessions left. \n\n**Caution:** Keep in mind that SSM sessions will [terminate automatically due to user inactivity](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-preferences-timeout.html), but SSH sessions will keep running until either a user terminates them manually or network timeout occurs, i.e., the user closes the laptop lid, disconnects from Wi-Fi, etc. If the user leaves the local machine unattended and connected to Internet, SSM sessions started by `aws ssm start-session` command will time out, but SSH-over-SSM sessions started with `sm-ssh connect` will stay open. Consider sending e-mail notifications for users of the long-running jobs, so the users don't forget to shut down unused resources. See [the related question in FAQ](FAQ.md#i-want-to-send-users-the-sms-or-email-notification-when-the-placeholder-training-job-has-issues-with-low-gpu-utilization-how-to-do-that) for more details and [train_placeholder.py](https://github.com/aws-samples/sagemaker-ssh-helper/blob/v2.1.0/tests/source_dir/training_placeholder/train_placeholder.py) that implements the similar logic.\n\n*Pro Tip:* Make sure that you're aware of [SageMaker Managed Warm Pools](https://docs.aws.amazon.com/sagemaker/latest/dg/train-warm-pools.html) \nfeature, which is also helpful in the scenario when you need to rerun your code remotely multiple times.\n\n*Pro Tip:* You can debug your code line by line in this scenario. See [the tutorial in PyCharm documentation](https://www.jetbrains.com/help/pycharm/debugging-your-first-python-application.html#debug).\n\n5. Run and debug your code\n\nNow when you have your training script or a placeholder script running, you can run additional code on the remote host, debug it line by line and set breakpoints.\n\nIf you want to change the control flow and let your training script call back the IDE for debugging, follow the next section on configuring the Debug Server.\n\n*Pro Tip*: The curious reader should also read the AWS blog post [Run your TensorFlow job on Amazon SageMaker with a PyCharm IDE](https://aws.amazon.com/blogs/machine-learning/run-your-tensorflow-job-on-amazon-sagemaker-with-a-pycharm-ide/). In contrast to the scenario with SageMaker SSH Helper, the blog instructions show how to use [SageMaker local mode](https://github.com/aws-samples/amazon-sagemaker-local-mode). As with Managed Warm Pools, SageMaker local mode helps to test your code faster, but it consumes local resources and doesn't provide the line by line debugging capability (as of writing).\n\n\n## \u003ca name=\"pycharm-debug-server\"\u003e\u003c/a\u003eRemote debugging with PyCharm Debug Server over SSH\n\nThere's another way to debug your code that is specific to the PyCharm Professional feature: [Remote debugging with the Python remote debug server configuration](https://www.jetbrains.com/help/pycharm/remote-debugging-with-product.html#remote-debug-config). The procedure assumes that you're running a training job, but the same steps apply to inference or data processing, too.\n\n1. In PyCharm, go to the Run/Debug Configurations (Run -\u003e Edit Configurations...), add a new Python Debug Server.\nChoose the fixed port, e. g. `12345`.\n\n2. Take the correct version of `pydevd-pycharm` package from the configuration window \nand install it either through `requirements.txt` or by calling `pip` from your source code.\n\n3. Add commands to connect to the Debug Server to your code **after** the `setup_and_start_ssh()`, e.g., into [a training script](https://github.com/aws-samples/sagemaker-ssh-helper/blob/main/tests/source_dir/training_debug/train_debug.py) that you submit as an entry point for a training job:\n```python\nimport sagemaker_ssh_helper\nsagemaker_ssh_helper.setup_and_start_ssh()\n\n...\n\nimport pydevd_pycharm\npydevd_pycharm.settrace('localhost', port=12345, stdoutToServer=True, stderrToServer=True, suspend=True)\n```\n*Tip*: Check the `settrace()` argument's description in [the library source code](https://github.com/JetBrains/intellij-community/blob/dee787ef05d1187a71b7667652f6b25f3f573a1b/python/helpers/pydev/pydevd.py#L1663).\n\n4. Set breakpoints in your code with PyCharm, as needed\n\n5. Start the Debug Server in PyCharm\n\n6. Submit your code to SageMaker with SSH Helper as described in previous sections.\n\n7. On your local machine, once the SSH helper connects to SSM and starts waiting inside the training job, connect to the host with SSH and start the port forwarding for the Debug Server:\n\n```shell\nssh -R localhost:12345:localhost:12345 \\\n  root@ssh-training-example-2023-07-25-03-18-04-490.training.sagemaker\n```\nIt will reverse-forward the remote debugger port `12345` to your local machine's Debug Server port.\n\n8. Stop the waiting loop\n\nAs already mentioned, make sure you've configured `connection_wait_time_seconds` to give yourself time to start the port forwarding before execution of the training script continues, and before it tries to connect to the debug server at port `12345`.\n\nInside the SSH session, run:\n```text\nsm-wait stop\n```\n\n9. After you stop the waiting loop, your code will continue running and will connect to your PyCharm Debug Server.\n\nIf everything is set up correctly, PyCharm will stop at your breakpoint, highlight the line and wait for your input. Debug Server window will say “connected”. You can now press, for example, F8 to \"Step Over\" the code line or F7 to \"Step Into\" the code line.\n\n\n## \u003ca name=\"studio\"\u003e\u003c/a\u003eLocal IDE integration with SageMaker Studio over SSH for PyCharm / VSCode\n\n![SSH_Helper-Remote-IDE.png](https://user-images.githubusercontent.com/87804596/205895890-e5e87f8b-1ca6-4ce6-bac1-5cb6e6f61dde.png)\n\nFor your local IDE integration with SageMaker Studio, follow the same steps as for configuring the IDE for [Remote code execution](#remote-interpreter), but instead of submitting the training / processing / inference code to SageMaker with Python SDK, execute the Jupyter notebook, as described in the next steps.\n\n\u003e **Important:** Make sure you read the \"Getting started\" section and didn't skip the steps from [Setting up your AWS account with IAM and SSM configuration](IAM_SSM_Setup.md).\n\n\n1. Copy [SageMaker_SSH_IDE.ipynb](SageMaker_SSH_IDE.ipynb) into SageMaker Studio and run it. \n\nNote that the `main` branch of this repo can contain changes that are not compatible with the version of `sagemaker-ssh-helper` that you installed from pip.\n\nTo be completely sure that you're using the version of the notebook that corresponds to the installed library, take a copy of the notebook from your filesystem after you install SSH Helper package, e.g.:\n\n```bash\npip install sagemaker-ssh-helper\nSM_SSH_PREFIX=`python -c 'import sys; print(sys.prefix);'`\necho $SM_SSH_PREFIX\n# check the output, e.g. /opt/conda\ncp $SM_SSH_PREFIX/sm_ssh/SageMaker_SSH_IDE.ipynb ~/user-default-efs/\n```\n\nYou can also check the version with `pip freeze | grep sagemaker-ssh-helper` and take the notebook from [the corresponding release tag](https://github.com/aws-samples/sagemaker-ssh-helper/tags).\n\nAlternatively, [attach](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-lcc-create.html) to a domain the KernelGateway lifecycle config script [kernel-lc-config.sh](kernel-lc-config.sh) \n(you may need to ask your administrator to do this).\nOnce configured, from the Launcher choose the environment, pick up the lifecycle script and choose \n'Open image terminal' (so, you don't even need to create a notebook).\n\nYou might want to change the `LOCAL_USER_ID` variable upon the first run, to prevent users from impersonating each other. For more details see the FAQ on [How SageMaker SSH Helper protects users from impersonating each other?](FAQ.md#how-sagemaker-ssh-helper-protects-users-from-impersonating-each-other).\n\n2. Configure remote interpreter in PyCharm / VS Code to connect to SageMaker Studio\n\nUse `app_name.app_space_name.domain_id.studio.sagemaker` or `app_name.studio.sagemaker` as the `fqdn` to connect.\nUse `sagemaker-user` as user. \nSee more details on how to configure the interpreter for the IDE in the [Remote code execution](#remote-interpreter) section, \nstep \"3. Configure the remote interpreter in your IDE\".\n\nTo see available apps to connect to, you may run the `list` command:\n\n```\nsm-ssh list studio.sagemaker\n```\n*Tip:* When you configure Python interpreter in PyCharm with SageMaker Studio, it's recommended to configure [the deployment path mapping](https://www.jetbrains.com/help/pycharm/creating-local-server-configuration.html#mapping) for you project to point into `/home/sagemaker-user/user-default-efs/project_name` instead of default `/tmp/pycharm_project_123`. \nThis is how you will be able to see your project in SageMaker Studio and PyCharm will automatically sync your local dir to the remote dir.\n\n*Note:* If you're using Windows, see [the FAQ](FAQ.md#is-windows-supported).\n\n\n3. Using the remote Jupyter Notebook\n\nIn recent versions of PyCharm, Jupyter Notebook is tunnelled automatically through remote interpreter connection. You might need to add `--allow-root` argument to the command line, when your remote interpreter runs under root:\n\n![](images/remote_jupyter.png)\n\nIf you wish to connect to the existing notebook server started by SSH Helper or don't use PyCharm, proceed with the next configuration steps.\n\nTo make the remote Jupyter Server port `8889` forwarded to the local machine, use SSH:\n\n```shell\nssh -L localhost:8889:localhost:8889 \\\n  root@ssh-ds2.jane-doe.d-egm0dexample.studio.sagemaker\n```\n\nNow you can also connect to a remote Jupyter Server started by SSH Helper inside SageMaker Studio as \nhttp://127.0.0.1:8889/?token=\u003c\u003cyour_token\u003e\u003e.\n\nYou will find the full URL with remote token in the [SageMaker_SSH_IDE.ipynb](SageMaker_SSH_IDE.ipynb) notebook in the output after running the cell  with `sm-ssh-ide start` command. If you use lifecycle configuration, run `tail /tmp/jupyter-notebook.log` from the image terminal to find the Jupyter Server URL.\n\n * [Instructions for remote Jupyter notebooks in PyCharm](https://www.jetbrains.com/help/pycharm/configuring-jupyter-notebook.html#configure-server)\n * [Instructions for remote Jupyter notebooks in VSCode](https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_connect-to-a-remote-jupyter-server) (don't forget to switch kernel to remote after configuring the remote server).\n\n4. Connecting to VNC\n\nTo make the remote VNC port `5901` forwarded to the local machine, use SSH:\n\n```shell\nssh -L localhost:5901:localhost:5901 \\\n  -R localhost:443:jetbrains-license-server.example.com:443 \\\n  root@ssh-ds2.jane-doe.d-egm0dexample.studio.sagemaker\n```\n\n*Note (PyCharm)*: The optional `-R` option will connect the remote port `443` to your local PyCharm license server address. Replace `jetbrains-license-server.example.com` with your server name and edit your `/etc/hosts` inside VNC to make this host point to `127.0.0.1` (should be done automatically if you didn't skip the `sm-ssh-ide set-jb-license-server` in the notebook).\n\nNow you can start the VNC session to [vnc://localhost:5901](vnc://localhost:5901) (e.g. on macOS with Screen Sharing app) and run IDE or any other GUI app on the remote desktop instead of your local machine. \n\nFor example, you can run inside VNC the `jupyter qtconsole` command to start the [Jupyter QT app](https://qtconsole.readthedocs.io/en/stable/index.html) as the alternative to Jupyter web UI:\n\n![Jupyter QT in VNC](images/vnc_jupyter.png)\n\n6. If you want to switch to another [kernel](https://docs.aws.amazon.com/sagemaker/latest/dg/notebooks-run-and-manage-change-image.html) \nor [instance](https://docs.aws.amazon.com/sagemaker/latest/dg/notebooks-run-and-manage-switch-instance-type.html), feel free to do so from SageMaker Studio UI and re-run\n[SageMaker_SSH_IDE.ipynb](SageMaker_SSH_IDE.ipynb).\n\nKeep in mind that in this case the previous kernel will stop and SSM Agent will stop, too.\nTo allow multiple kernel and instances to be up and running with SageMaker SSH Helper and SSM Agent,\nduplicate the notebook and give it a different name, e.g. `SageMaker_SSH_IDE-PyTorch.ipynb`.\nIn this case you'll be able to keep two environments in parallel. \n\nIf you're using lifecycle configuration script, just start another image terminal with different environment settings from Launcher.\n\n7. Don't forget to [shut down](https://docs.aws.amazon.com/sagemaker/latest/dg/notebooks-run-and-manage-shut-down.html) SageMaker Studio resources, if you don't need them anymore, e.g., launched notebooks, terminals, apps and instances.\n\n## \u003ca name=\"web-vnc\"\u003e\u003c/a\u003eWeb VNC\n\n\u003e **Note**: The Web VNC section is created for the earlier version of SSH Helper and was not tested with the recent features of SageMaker Studio released in late 2024 and in 2025.\n\nAt times, you cannot install all the software on your local machine, also because this is the software processes data, and you cannot copy massive amount of the data to your local machine. \n\nYou might have thought about [AWS Jupyter Proxy](https://github.com/aws/aws-jupyter-proxy), but some web apps like Dask may not fully work through the proxy, so VNC is the recommended alternative.\n\nBy combining the [noVNC](https://novnc.com/) tool with AWS Jupyter Proxy extension you can run virtually any IDE like PyCharm, VSCode, PyDev, or any tool like Blender (to work with 3D data), OpenShot (to work with audio-video data), etc., as well as other webapps from a SageMaker Studio web UI, without installing all of them to your local machine.\n\nIt's also helpful in situations when you cannot run SSH client on your local machine to forward ports for web tools, like Dask dashboard. In this case, you run a tool in the remote browser running through the web VNC (browser-in-a-browser), like on the below screenshot. You might notice that PyCharm and VSCode are also running in the background:\n![WebWNC Screenshot](images/webVNC.png)\n\nTo achieve this result, your Administrator should configure your SageMaker IAM role with both `SSHSageMakerServerPolicy` and `SSHSageMakerClientPolicy`. Configuration of IAM credentials for the local machine is not required in this case. See the Step 4 in [IAM_SSM_Setup.md](IAM_SSM_Setup.md#4-optional-connecting-from-sagemaker-studio-) for more details.\n\nThen follow these steps:\n\n1. On the SageMaker Studio System terminal run the commands from [server-lc-config.sh](server-lc-config.sh).\n\nAlternatively, ask the Administrator to [attach the lifecycle config](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-lcc-create.html) to the SageMaker Studio domain or to your profile as the default `JupyterServer` config, e.g., with the name `sagemaker-ssh-helper-webvnc`.\n\n2. Follow the step 1 for [the IDE configuration procedure](#studio), i.e., run the IDE notebook or lifecycle config inside the kernel gateway of your choice. \n\nInstead of your local user ID put the SageMaker Studio user ID (you can get it by running `aws sts get-caller-identity` from a SageMaker Studio terminal).\n\n3. On the System (!) terminal (not image terminal), run:\n\n```shell\nsm-ssh connect app_name.app_space_name.domain_id.studio.sagemaker\n```\n\nAlternatively, use SSH command to forward the VNC port and add more ports to the command, e.g., `-L localhost:8787:localhost:8787` to forward the Dask dashboard that is running inside the kernel gateway:\n\n```shell\nssh -L localhost:5901:localhost:5901 \\\n  -L localhost:8787:localhost:8787 \\\n  app_name.app_space_name.domain_id.studio.sagemaker\n```\n\n4. Navigate to `https://d-egm0dexample.studio.eu-west-1.sagemaker.aws/jupyter/default/proxy/6080/vnc.html?host=d-egm0dexample.studio.eu-west-1.sagemaker.aws\u0026port=443\u0026path=jupyter/default/proxy/6080/websockify`\n\nReplace both occurrences of `d-egm0dexample` with your SageMaker Studio domain ID, and `eu-west-1` with your AWS Region.\n\nYou will see the noVNC welcome screen.\n\n5. Press \"Connect\" and enter your password (default is `123456`).\n\nCongratulations! You now have successfully logged into the remote desktop environment running inside a SageMaker Studio kernel gateway.\n\nSome data handling application to try inside the VNC desktop, which you cannot run as a web app otherwise, are:\n* [3D Slicer](https://www.slicer.org/) - image computing platform for medical, biomedical, and other 3D images and meshes \n* [OpenShot](https://www.openshot.org/) - to work with video data\n* [LibreOffice](https://www.libreoffice.org/) - work with documents and spreadsheets\n\n*Tip:* If you have issues with copy-pasting through system clipboard, use the temp file, e.g. `clip.txt`, and open it in VNC session and SageMaker Studio file browser at the same time.\n\n*Pro Tip:* To set the resolution that matches your browser window size, make a page screenshot (in Firefox - right-click on an empty area -\u003e Take Screenshot -\u003e Save visible), then inspect the resolution of the image, e.g. 1920x970. Then add and switch resolution inside the VNC session:\n```shell\n$ cvt 1920 970 60\n# 1920x970 59.93 Hz (CVT) hsync: 60.35 kHz; pclk: 154.50 MHz\nModeline \"1920x970_60.00\"  154.50  1920 2040 2240 2560  970 973 983 1007 -hsync +vsync\n$ xrandr --newmode \"1920x970_60.00\"  154.50  1920 2040 2240 2560  970 973 983 1007 -hsync +vsync\n$ xrandr --addmode VNC-0 1920x970_60.00\n$ xrandr -s 1920x970_60.00\n```\n\n## Troubleshooting\n\nIf something doesn't work as expected, make sure you looked at our [FAQ](FAQ.md), especially at the [troubleshooting section](FAQ.md#troubleshooting), as well as at the existing both open and resolved [issues](https://github.com/aws-samples/sagemaker-ssh-helper/issues?q=is%3Aissue).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faws-samples%2Fsagemaker-ssh-helper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faws-samples%2Fsagemaker-ssh-helper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faws-samples%2Fsagemaker-ssh-helper/lists"}