https://github.com/markus-goetz/kseta-2023
https://github.com/markus-goetz/kseta-2023
Last synced: over 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/markus-goetz/kseta-2023
- Owner: Markus-Goetz
- License: mit
- Created: 2023-10-03T09:42:30.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2023-10-05T14:06:34.000Z (almost 3 years ago)
- Last Synced: 2025-02-08T10:31:10.279Z (over 1 year ago)
- Language: Jupyter Notebook
- Size: 3.9 MB
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# KSETA Course 2023 - Scalable AI
This repository contains the materials for the course on "Scalable AI" for the KSETA course 2023.
## Hands-on Session: `Pytorch DistributedDataParallel`
In this hands-on tutorial, you will learn how to train a data-parallel neural network using `PyTorch`'s `DistributedDataParallel` package.
You can find the corresponding exercises and solutions in `notebook/ddp_exercise.ipynb` and `notebook/ddp_solution.ipynb`, respectively.
You can work on and develop your code in the `exercise_ddp.ipynb` notebooks. However, truly parallel runs are inconvenient in Jupyter notebooks. This is why you need to create Python scripts from the code snippets as well as a job script to test and actually run your code in parallel as a batch job on HAICORE. You can find step-by-step instructions for how to do this in the exercises notebook. The actual `Python` and SLURM job scripts are in `py/` and `sh/`, respectively.