Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/davidrpugh/introduction-to-deep-learning
Course materials for a multi-day course on deep learning.
https://github.com/davidrpugh/introduction-to-deep-learning
Last synced: 9 days ago
JSON representation
Course materials for a multi-day course on deep learning.
- Host: GitHub
- URL: https://github.com/davidrpugh/introduction-to-deep-learning
- Owner: davidrpugh
- License: bsd-3-clause
- Created: 2022-10-19T11:59:21.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2023-11-22T16:05:20.000Z (about 1 year ago)
- Last Synced: 2023-11-22T17:27:56.291Z (about 1 year ago)
- Language: Jupyter Notebook
- Size: 6.65 MB
- Stars: 30
- Watchers: 3
- Forks: 12
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://mybinder.org/v2/gh/KAUST-Academy/introduction-to-machine-learning/HEAD)
# Introduction to Deep Learning
There is strong demand for machine learning (DL) skills and expertise to solve challenging business problems both globally and locally in KSA. This course will help learners build capacity in core DL tools and methods and enable them to develop their own deep learning applications. This course covers the basic theory behind DL algorithms but the majority of the focus is on hands-on examples using [PyTorch](https://pytorch.org/).
## Learning Objectives
The primary learning objective of this course is to provide students with practical, hands-on experience with state-of-the-art machine learning and deep learning tools that are widely used in industry.
This course covers portions of chapters 10-19 of [Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/) and chapters 11-19 of [Machine Learning with PyTorch and Scikit-Learn](https://www.packtpub.com/product/machine-learning-with-pytorch-and-scikit-learn/9781801819312). The following topics will be discussed.
* Introduction to Artificial Neural Networks (ANNs)
* Training Deep Neural Networks (DNNs)
* Custom Models and Training with PyTorch and Lightning
* Stratgeies for Loading and Preprocessing Data
* Training and Deploying PyTorch Models at Scale
## Lessons
The lessons are organizes into modules and sub-modules with the idea that they can taught somewhat independently to accommodate specific audiences.
### Module 1: [Introduction to Deep Learning](https://kaust-my.sharepoint.com/:p:/g/personal/pughdr_kaust_edu_sa/EfRHWqkIFjpBk9j8aL4I3fABUngt5d3uccvxjDuuurtYfA?e=ffNqUj)
* The morning session will focus on the theory behind neural networks for solving both classification and regression problems by covering relevant portions of [Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/) and [Machine Learning with PyTorch and Scikit-Learn](https://www.packtpub.com/product/machine-learning-with-pytorch-and-scikit-learn/9781801819312).
* The afternoon session will focus on applying the techniques learned in the morning session using [PyTorch](https://pytorch.org/), followed by a short assessment on the Kaggle data science competition platform.
| **Tutorial** | **Open in Google Colab** | **Open in Kaggle** |
|--------------|:------------------------:|:------------------:|
| First Steps with PyTorch | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/01a-first-steps-with-pytorch.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/01a-first-steps-with-pytorch.ipynb) |
| Building Data Pipelines with PyTorch | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/01b-building-data-pipelines-in-pytorch.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/01b-building-data-pipelines-in-pytorch.ipynb) |
| Building Neural Networks with PyTorch | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/01c-building-neural-networks-in-pytorch.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/notebooks/01c-building-neural-networks-in-pytorch.ipynb) |
| Introduction to PyTorch Lightning | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/01e-introduction-to-pytorch-lightning.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/01e-introduction-to-pytorch-lightning.ipynb) |
### Module 2: [Training DNNs](https://kaust-my.sharepoint.com/:p:/g/personal/pughdr_kaust_edu_sa/EeA0M9ydAEVGmfAWZhzVDEwBWtnOVos3YroubDAkfswWoQ?e=ryFLJy)
* Consolidation of previous days content via Q/A and live coding demonstrations.
* The morning session will focus on the theory behind neural networks for solving both classification and regression problems by covering chapters 12-13 of [Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/) and chapters 12-13 of [Machine Learning with PyTorch and Scikit-Learn](https://www.packtpub.com/product/machine-learning-with-pytorch-and-scikit-learn/9781801819312).
* The afternoon session will focus on applying the techniques learned in the morning session using [PyTorch](https://pytorch.org/), followed by a short assessment on the Kaggle data science competition platform.
| **Tutorial** | **Open in Google Colab** | **Open in Kaggle** |
|--------------|:------------------------:|:------------------:|
| Deep Dive into PyTorch | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/01d-deep-dive-into-pytorch.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/01d-deep-dive-into-pytorch.ipynb) |
| Vanishing and Exploding Gradients | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02c-vanishing-exploding-gradients.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02c-vanishing-exploding-gradients.ipynb) |
| Transfer Learning and Unsupervised Pre-training | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02d-transfer-learning-and-unsupervised-pretraining.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02d-transfer-learning-and-unsupervised-pretraining.ipynb) |
| Faster Optimizers | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02e-faster-optimizers.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02e-faster-optimizers.ipynb) |
| Learning Rate Schedulers | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02f-learning-rate-schedulers.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02f-learning-rate-schedulers.ipynb) |
| Regularization | [](https://colab.research.google.com/github/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02g-regularization.ipynb) | [](https://kaggle.com/kernels/welcome?src=https://github.com/KAUST-Academy/introduction-to-deep-learning/blob/master/notebooks/02g-regularization.ipynb) |
### Module 3: [Deploying and Scaling PyTorch Models](https://kaust-my.sharepoint.com/:p:/g/personal/pughdr_kaust_edu_sa/EZItqTR1l3VOs0UzJJ7NAVABaDT2FN5kclARg2Rv8cnoxA?e=vyaLcu)
* Consolidation of previous days content via Q/A and live coding demonstrations.
* The morning session will focus on various topics related to training and deploying PyTorch models as scale by covering chapter 19 of [Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/).
* The afternoon session will allow time for a final assessment as well as additional time for learners to complete any of the previous assessments.
## Assessment
Student performance on the course will be assessed through participation in a Kaggle classroom competition.
# Repository Organization
Repository organization is based on ideas from [_Good Enough Practices for Scientific Computing_](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005510).
1. Put each project in its own directory, which is named after the project.
2. Put external scripts or compiled programs in the `bin` directory.
3. Put raw data and metadata in a `data` directory.
4. Put text documents associated with the project in the `doc` directory.
5. Put all Docker related files in the `docker` directory.
6. Install the Conda environment into an `env` directory.
7. Put all notebooks in the `notebooks` directory.
8. Put files generated during cleanup and analysis in a `results` directory.
9. Put project source code in the `src` directory.
10. Name all files to reflect their content or function.
## Building the Conda environment
After adding any necessary dependencies that should be downloaded via `conda` to the
`environment.yml` file and any dependencies that should be downloaded via `pip` to the
`requirements.txt` file you create the Conda environment in a sub-directory `./env`of your project
directory by running the following commands.
```bash
export ENV_PREFIX=$PWD/env
mamba env create --prefix $ENV_PREFIX --file environment.yml --force
```
Once the new environment has been created you can activate the environment with the following
command.
```bash
conda activate $ENV_PREFIX
```
Note that the `ENV_PREFIX` directory is *not* under version control as it can always be re-created as
necessary.
For your convenience these commands have been combined in a shell script `./bin/create-conda-env.sh`.
Running the shell script will create the Conda environment, activate the Conda environment, and build
JupyterLab with any additional extensions. The script should be run from the project root directory
as follows.
```bash
./bin/create-conda-env.sh
```
### Ibex
The most efficient way to build Conda environments on Ibex is to launch the environment creation script
as a job on the debug partition via Slurm. For your convenience a Slurm job script
`./bin/create-conda-env.sbatch` is included. The script should be run from the project root directory
as follows.
```bash
sbatch ./bin/create-conda-env.sbatch
```
### Listing the full contents of the Conda environment
The list of explicit dependencies for the project are listed in the `environment.yml` file. To see
the full lost of packages installed into the environment run the following command.
```bash
conda list --prefix $ENV_PREFIX
```
### Updating the Conda environment
If you add (remove) dependencies to (from) the `environment.yml` file or the `requirements.txt` file
after the environment has already been created, then you can re-create the environment with the
following command.
```bash
$ mamba env create --prefix $ENV_PREFIX --file environment.yml --force
```
## Using Docker
In order to build Docker images for your project and run containers with GPU acceleration you will
need to install
[Docker](https://docs.docker.com/install/linux/docker-ce/ubuntu/),
[Docker Compose](https://docs.docker.com/compose/install/) and the
[NVIDIA Docker runtime](https://github.com/NVIDIA/nvidia-docker).
Detailed instructions for using Docker to build and image and launch containers can be found in
the `docker/README.md`.