https://github.com/biocomputingup/elastic-slurm-in-openstack
Configure an elastic Slurm cluster on OpenStack cloud
https://github.com/biocomputingup/elastic-slurm-in-openstack
Last synced: about 1 year ago
JSON representation
Configure an elastic Slurm cluster on OpenStack cloud
- Host: GitHub
- URL: https://github.com/biocomputingup/elastic-slurm-in-openstack
- Owner: BioComputingUP
- Created: 2024-10-03T08:59:52.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-19T14:57:13.000Z (about 1 year ago)
- Last Synced: 2025-03-19T15:39:51.052Z (about 1 year ago)
- Language: Shell
- Size: 61.5 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Slurm cluster in OpenStack cloud
These Ansible playbooks create and manage a dynamically allocated (elastic) Slurm cluster in an OpenStack cloud.
The cluster is based on CentOS 8 (Rocky 8) and [OpenHPC 2.x](https://openhpc.community/downloads/). Slurm configurations are based on the work contained
in [Jetstream_Cluster](https://github.com/XSEDE/CRI_Jetstream_Cluster).
This repo is based on the project [slurm-cluster-in-openstack](https://github.com/CornellCAC/slurm-cluster-in-openstack)
adapted for use with [CloudVeneto](https://cloudveneto.ict.unipd.it/) OpenStack cloud.
## Prerequisites
### Install Ansible
Run the `install_ansible.sh` command:
```bash
./install_ansible.sh
```
### Configure CloudVeneto gateway (Gate) for SSH access
For this you should have a CloudVeneto account and access to the Gate machine (`cv_user` and `cv_pass`):
```bash
# generate a new key pair locally (preferably with passphrase). Skip and adapt if you already have a key pair:
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_vm
# copy the public key to Gate machine (it will ask for cv_pass):
cat ~/.ssh/id_ed25519_vm.pub | \
ssh cv_user@gate.cloudveneto.it 'cat >id_ed25519_vm.pub && \
mkdir -p .ssh && \
chmod 700 .ssh && \
mv id_ed25519_vm.pub .ssh/id_ed25519_vm.pub && \
cat .ssh/id_ed25519_vm.pub >>.ssh/authorized_keys'
# copy the private key to Gate machine (it will ask for cv_pass):
cat ~/.ssh/id_ed25519_vm | \
ssh cv_user@gate.cloudveneto.it \
'cat >.ssh/id_ed25519_vm && chmod 600 .ssh/id_ed25519_vm'
# connect to Gate machine (it will ask for SSH key passphrase, if used):
ssh -i ~/.ssh/id_ed25519_vm cv_user@gate.cloudveneto.it
```
If you have also the credentials and IP of a VM running in the cloud (`vm_user`, `vm_pass`, `vm_ip`), you can import the key pair to it:
```bash
# copy the public key from the Gate machine to VM (it will ask for vm_pass)
cat ~/.ssh/id_ed25519_vm.pub | \
ssh vm_user@vm_ip 'cat >.ssh/id_ed25519_vm.pub && \
cat .ssh/id_ed25519_vm.pub >>.ssh/authorized_keys'
# test connection to VM from Gate machine (it will ask for SSH passphrase, if used)
ssh -i ~/.ssh/id_ed25519_vm vm_user@vm_ip
exit
```
Accessing a VM from your local machine requires proxying the SSH connection through the CloudVeneto Gate. You can achieve this by using the following SSH command:
```bash
# (optionally) add key to ssh-agent (it may ask for SSH key passphrase)
ssh-add ~/.ssh/id_ed25519_vm
# connect to VM via proxy
ssh -i ~/.ssh/id_ed25519_vm \
-o StrictHostKeyChecking=accept-new \
-o ProxyCommand="ssh -i ~/.ssh/id_ed25519_vm \
-W %h:%p cv_user@gate.cloudveneto.it" \
vm_user@vm_ip
```
You can simplify the SSH connection to VM by configuring your SSH config file:
```bash
# update ssh config with proxy and headnode
cat <