Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/jeffheaton/app_deep_learning

T81-558: PyTorch - Applications of Deep Neural Networks @Washington University in St. Louis
https://github.com/jeffheaton/app_deep_learning

deep-learning python pytorch

Last synced: 15 days ago
JSON representation

T81-558: PyTorch - Applications of Deep Neural Networks @Washington University in St. Louis

Host: GitHub
URL: https://github.com/jeffheaton/app_deep_learning
Owner: jeffheaton
License: apache-2.0
Created: 2023-07-01T16:13:19.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2024-10-22T04:45:05.000Z (23 days ago)
Last Synced: 2024-10-22T19:20:26.466Z (22 days ago)
Topics: deep-learning, python, pytorch
Language: Jupyter Notebook
Homepage: https://sites.wustl.edu/jeffheaton/t81-558/
Size: 37.6 MB
Stars: 373
Watchers: 13
Forks: 145
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE
- Citation: CITATION.cff

Awesome Lists containing this project

README

# T81 558:Applications of Deep Neural Networks

[Washington University in St. Louis](http://www.wustl.edu)

Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/)

- Section 1. Spring 2024, Tuesday, 2:30 PM, Location: Cupples I / 215

# Course Description

Deep learning is a group of exciting new technologies for neural networks. Through a combination of advanced training techniques and neural network architectural components, it is now possible to create neural networks that can handle tabular data, images, text, and audio as both input and output. Deep learning allows a neural network to learn hierarchies of information in a way that is like the function of the human brain. This course will introduce the student to classic neural network structures, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Neural Networks (GRU), General Adversarial Networks (GAN) and reinforcement learning. Application of these architectures to computer vision, time series, security, natural language processing (NLP), and data generation will be covered. High Performance Computing (HPC) aspects will demonstrate how deep learning can be leveraged both on graphical processing units (GPUs), as well as grids. Focus is primarily upon the application of deep learning to problems, with some introduction to mathematical foundations. Students will use the Python programming language to implement deep learning using PyTorch. It is not necessary to know Python prior to this course; however, familiarity of at least one programming language is assumed. This course will be delivered in a hybrid format that includes both classroom and online instruction.

# Objectives

1. Explain how neural networks (deep and otherwise) compare to other machine learning models.
2. Determine when a deep neural network would be a good choice for a particular problem.
3. Demonstrate your understanding of the material through a final project uploaded to GitHub.

# Syllabus

This syllabus presents the expected class schedule, due dates, and reading assignments. [Download current syllabus](https://s3.amazonaws.com/data.heatonresearch.com/wustl/syllabus/jheaton-t81-558-fall-2024-syllabus.pdf)

| Module | Content |
| --------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Module 1](t81_558_class_01_1_overview.ipynb)
**Meet on 08/27/2024** | **Module 1: Python Preliminaries**

1.1: Course Overview

1.2: Introduction to Python

1.3: Python Lists, Dictionaries, Sets & JSON

1.4: File Handling

1.5: Functions, Lambdas, and Map/ReducePython Preliminaries

**We will meet on campus this week! (first meeting)**

|
| [Module 2](t81_558_class_02_1_python_pandas.ipynb)
Week of 09/03/2024 | **Module 2: Python for Machine Learning**

2.1: Introduction to Pandas for Deep Learning

2.2: Encoding Categorical Values in Pandas

2.3: Grouping, Sorting, and Shuffling

2.4: Using Apply and Map in Pandas

2.5: Feature Engineering in Padas

[Module 1 Program](./assignments/assignment_yourname_class1.ipynb) due: 09/04/2024

Icebreaker due: 09/04/2024

|
| [Module 3](t81_558_class_03_1_neural_net.ipynb)
Week of 09/10/2024 | **Module 3: PyTorch for Neural Networks**

3.1: Deep Learning and Neural Network Introduction

3.2: Introduction to PyTorch

3.3: Encoding a Feature Vector for PyTorch Deep Learning

3.4: Early Stopping and Network Persistence

3.5: Sequences vs Classes in PyTorch

[Module 2: Program](./assignments/assignment_yourname_class2.ipynb) due: 09/11/2024

|
| [Module 4](t81_558_class_04_1_kfold.ipynb)
Week of 09/17/2024 | **Module 4: Training for Tabular Data**

4.1: Using K-Fold Cross-validation with PyTorch

4.2: Training Schedules for PyTorch

4.3: Dropout Regularization

4.4: Batch Normalization

4.5: RAPIDS for Tabular Data

[Module 3 Program](./assignments/assignment_yourname_class3.ipynb) due: 09/18/2024

|
| [Module 5](t81_558_class_05_1_python_images.ipynb)
Week of 09/24/2024 | **Module 5: CNN and Computer Vision**

5.1 Image Processing in Python

5.2 Using Convolutional Neural Networks

5.3 Using Pretrained Neural Networks

5.4 Looking at Generators and Image Augmentation

5.5 Recognizing Multiple Images with YOLO

[Module 4 Program](./assignments/assignment_yourname_class4.ipynb) due: 09/25/2024

|
| [Module 6](t81_558_class_06_1_transformers.ipynb)
**Meet on 10/01/2024** | **Module 6: ChatGPT and Large Language Models**

6.1: Introduction to Transformers

6.2: Accessing the ChatGPT API

6.3: LLM Memory

6.4: Introduction to Embeddings

6.5: Prompt Engineering

[Module 5 Program](./assignments/assignment_yourname_class5.ipynb) due: 10/02/2024

**We will meet on campus this week! (second meeting)**

|
| [Module 7](t81_558_class_07_1_img_generative.ipynb)
Week of 10/15/2024 | **Module 7: Image Generative Models**

7.1: Introduction to Generative AI

7.2: Generating Faces with StyleGAN3

7.3: GANS to Enhance Old Photographs Deoldify

7.4: Text to Images with StableDiffusion

7.5: Finetuning with Dreambooth

[Module 6 Program](./assignments/assignment_yourname_class6.ipynb) due: 10/16/2024

|
| [Module 8](t81_558_class_08_1_kaggle_intro.ipynb)
**Meet on 10/22/2024** | **Module 8: Kaggle**

8.1 Introduction to Kaggle

8.2 Building Ensembles with Scikit-Learn and PyTorch

8.3 How Should you Architect Your PyTorch Neural Network: Hyperparameters

8.4 Bayesian Hyperparameter Optimization for PyTorch

8.5 Current Semester's Kaggle

[Module 7 Program](./assignments/assignment_yourname_class7.ipynb) due: 10/23/2024

**We will meet on campus this week! (third meeting)**

|
| [Module 9](t81_558_class_09_1_faces.ipynb)
Week of 10/29/2024 | **Module 9: Facial Recognition**

9.1 Detecting Faces in an Image

9.2 Detecting Facial Features

9.3 Image Augmentation

9.4 Application: Emotion Detection

9.5 Application: Blink Efficiency

[Module 8 Assignment](./assignments/assignment_yourname_class8.ipynb) due: 10/30/2024

|
| [Module 10](t81_558_class_10_1_timeseries.ipynb)
Week of 11/05/2024 | **Module 10: Time Series in PyTorch**

Time Series Data Encoding for Deep Learning, PyTorch

Seasonality and Trend

LSTM-Based Time Series with PyTorch

CNN-Based Time Series with PyTorch

Predicting with Meta Prophet

[Module 9 Program](./assignments/assignment_yourname_class9.ipynb) due: 11/06/2024

|
| [Module 11](t81_558_class_11_1_hf.ipynb)
Week of 11/12/2024 | **Module 11: Natural Language Processing**

11.1 Introduction to Natural Language Processing

11.2 Hugging Face Introduction

11.3 Hugging Face Tokenizers

11.4 Hugging Face Data Sets

11.5 Training a Model in Hugging Face

[Module 10 Program](./assignments/assignment_yourname_class10.ipynb) due: 11/13/2024

|
| [Module 12](t81_558_class_12_1_ai_gym.ipynb)
Week of 11/19/2024 | **Module 12: Reinforcement Learning**

Introduction to Gymnasium

Introduction to Q-Learning

Stable Baselines Q-Learning

Atari Games with Stable Baselines Neural Networks

Future of Reinforcement Learning

|
| [Module 13](t81_558_class_13_1_auto_encode.ipynb)
Week of 11/26/2024 | **Module 13: Deployment and Monitoring**

13.1: Using Denoising AutoEncoders

13.2: Anomaly Detection

13.3: Model Drift and Retraining

13.4: Tensor Processing Units (TPUs)

13.5: Future Directions in Artificial Intelligence

Kaggle Assignment due: 11/26/2024 (approx 4-6PM, due to Kaggle GMT timezone)

|
| Week 14
**Meet on 12/3/2024** | **Week 14: Kaggle Presentations**

Top Kaggle teams will present

**We will meet on campus this week! (fourth meeting)**

Final project due: 12/4/2024

# Datasets

- [Datasets can be downloaded here](https://data.heatonresearch.com/data/t81-558/index.html)