https://github.com/goneri/ec2-cuda-podman-deploy

Bootstrap an EC2 instance with a GPU, enable CUDA and configure it as a remote Podman server
https://github.com/goneri/ec2-cuda-podman-deploy

Last synced: 25 days ago
JSON representation

Bootstrap an EC2 instance with a GPU, enable CUDA and configure it as a remote Podman server

• Host: GitHub
• URL: https://github.com/goneri/ec2-cuda-podman-deploy
• Owner: goneri
• Created: 2023-01-06T22:48:37.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-09-25T18:16:47.000Z (about 2 months ago)
• Last Synced: 2024-10-03T17:22:55.742Z (about 2 months ago)
• Language: Shell
• Size: 43 KB
• Stars: 1
• Watchers: 3
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Ansible playbook to deploy CUDA in an AWS instance and Ollama

- prepare the AWS instance (including a VPC and a subnet)

- install CUDA and all the stuff

- install Ollama

## Usage

To set-up the instance:

```

cp vars.yaml $USER.yaml

vim $USER.yaml

ansible-playbook -vvv deploy.yaml -e @vars.yaml

# If Ollama is already running locally, you need to turn it off

sudo systemctl stop ollama

# To stop ollama on Mac, use the ollama entry on the menu bar to Quit Ollama

# And finally, you can redirect locally the port 11434 from your remove instance

ssh -L 11434:127.0.0.1:11434 [email protected]

# In another terminal you can now use ollama locally

ollama run mistral

```

## Run jupyter notebook over created ec2-instance

```

# To update the currently installed ec2-instance packages

sudo dnf update

# Install python3.x, should be >=3.8.1 to support python langchain module

sudo dnf install python3.x

# Install python virtual env

python3.11 -m venv .env

source .env/bin/activate

# Install python pip

sudo dnf install python3-pip

pip install --upgrade pip

# Install required library

pip3 install torch

pip3 install jupyter-lab

pip3 install jupyter

jupyter notebook --generate-config

jupyter-lab --ip 0.0.0.0 --no-browser --allow-root

# Jupyter notebook should start running, with the token

# Start jupyter notebook in browser

- In the browser url-> "ec2_instance_public_ip:8888"

- Enter the token obtained from the previous step, to login to jupyter notebook instance

- Jupyter notebook should be running as expected

```