{"id":13737214,"url":"https://github.com/jeffheaton/t81_558_deep_learning","last_synced_at":"2025-05-14T00:11:00.774Z","repository":{"id":39635261,"uuid":"64573324","full_name":"jeffheaton/t81_558_deep_learning","owner":"jeffheaton","description":"T81-558: Keras - Applications of Deep Neural Networks @Washington University in St. Louis","archived":false,"fork":false,"pushed_at":"2025-01-22T03:00:47.000Z","size":249175,"stargazers_count":5731,"open_issues_count":2,"forks_count":3032,"subscribers_count":319,"default_branch":"master","last_synced_at":"2025-04-12T04:13:56.155Z","etag":null,"topics":["convolutional-neural-networks","deeplearning","gan","keras","machine-learning","neural-network","tensorflow"],"latest_commit_sha":null,"homepage":"https://sites.wustl.edu/jeffheaton/t81-558/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jeffheaton.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"jeffheaton","patreon":"jeffheaton"}},"created_at":"2016-07-31T02:32:59.000Z","updated_at":"2025-04-10T08:56:38.000Z","dependencies_parsed_at":"2023-02-19T16:45:42.558Z","dependency_job_id":"7b3dc05d-2a43-418f-8577-df1669956fcb","html_url":"https://github.com/jeffheaton/t81_558_deep_learning","commit_stats":{"total_commits":959,"total_committers":44,"mean_commits":"21.795454545454547","dds":"0.11678832116788318","last_synced_commit":"cac591e478a6f19f3d4193d8514bfce89564ae6e"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffheaton%2Ft81_558_deep_learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffheaton%2Ft81_558_deep_learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffheaton%2Ft81_558_deep_learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffheaton%2Ft81_558_deep_learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jeffheaton","download_url":"https://codeload.github.com/jeffheaton/t81_558_deep_learning/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254044380,"owners_count":22005149,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","deeplearning","gan","keras","machine-learning","neural-network","tensorflow"],"created_at":"2024-08-03T03:01:37.669Z","updated_at":"2025-05-14T00:10:55.752Z","avatar_url":"https://github.com/jeffheaton.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/jeffheaton","https://patreon.com/jeffheaton"],"categories":["Jupyter Notebook","Security"],"sub_categories":[],"readme":"# Important Note\n\nCurrent students of this course at Washington University should refer to the [PyTorch version of this course](https://github.com/jeffheaton/app_deep_learning), which is what is currently offered at the university. This repository contains the previous Keras/TensorFlow version.\n\n# T81 558:Applications of Deep Neural Networks - TensorFlow\n[Washington University in St. Louis](http://www.wustl.edu)\n\nInstructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/)\n\n**The content of this course changes as technology evolves**, to keep up to date with changes [follow me on GitHub](https://github.com/jeffheaton).\n\n* Section 1. Spring 2023, Monday, 2:30 PM, Location: Eads / 216 \n* Section 2. Spring 2023, Online\n\n# Course Description\n\nDeep learning is a group of exciting new technologies for neural networks. Through a combination of advanced training techniques and neural network architectural components, it is now possible to create neural networks that can handle tabular data, images, text, and audio as both input and output. Deep learning allows a neural network to learn hierarchies of information in a way that is like the function of the human brain. This course will introduce the student to classic neural network structures, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Neural Networks (GRU), General Adversarial Networks (GAN) and reinforcement learning. Application of these architectures to computer vision, time series, security, natural language processing (NLP), and data generation will be covered. High Performance Computing (HPC) aspects will demonstrate how deep learning can be leveraged both on graphical processing units (GPUs), as well as grids. Focus is primarily upon the application of deep learning to problems, with some introduction to mathematical foundations. Students will use the Python programming language to implement deep learning using Google TensorFlow and Keras. It is not necessary to know Python prior to this course; however, familiarity of at least one programming language is assumed. This course will be delivered in a hybrid format that includes both classroom and online instruction.\n\n# Textbook\n\nThe complete text for this course is here on GitHub. This same material is also available in [book format](https://www.heatonresearch.com/book/applications-deep-neural-networks-keras.html). The course textbook is “Applications of Deep Neural networks with Keras“, ISBN 9798416344269.\n\nIf you would like to cite the material from this course/book, please use the following BibTex citation:\n\n```\n@misc{heaton2020applications,\n    title={Applications of Deep Neural Networks},\n    author={Jeff Heaton},\n    year={2020},\n    eprint={2009.05673},\n    archivePrefix={arXiv},\n    primaryClass={cs.LG}\n}\n```\n\n# Objectives\n\n1. Explain how neural networks (deep and otherwise) compare to other machine learning models.\n2. Determine when a deep neural network would be a good choice for a particular problem.\n3. Demonstrate your understanding of the material through a final project uploaded to GitHub.\n\n# Syllabus\nThis syllabus presents the expected class schedule, due dates, and reading assignments.  [Download current syllabus.](https://data.heatonresearch.com/wustl/jheaton-t81-558-spring-2023-syllabus.pdf)\n\nModule|Content\n---|---\n[Module 1](t81_558_class_01_1_overview.ipynb)\u003cbr\u003e**Meet on 01/23/2023** | **Module 1: Python Preliminaries**\u003cul\u003e\u003cli\u003ePart 1.1: Course Overview\u003cli\u003ePart 1.2: Introduction to Python\u003cli\u003ePart 1.3: Python Lists, Dictionaries, Sets \u0026 JSON\u003cli\u003ePart 1.4: File Handling\u003cli\u003ePart 1.5: Functions, Lambdas, and Map/ReducePython Preliminaries\u003cli\u003e**We will meet on campus this week! (first meeting)**\u003c/ul\u003e\n[Module 2](t81_558_class_02_1_python_pandas.ipynb)\u003cbr\u003eWeek of 01/30/2023 | **Module 2: Python for Machine Learning**\u003cul\u003e\u003cli\u003e\tPart 2.1: Introduction to Pandas for Deep Learning\u003cli\u003ePart 2.2: Encoding Categorical Values in Pandas\u003cli\u003ePart 2.3: Grouping, Sorting, and Shuffling\u003cli\u003ePart 2.4: Using Apply and Map in Pandas\u003cli\u003ePart 2.5: Feature Engineering in Pandas\u003cli\u003e[Module 1 Program](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class1.ipynb) due: 01/31/2023\u003cli\u003e Icebreaker due: 01/31/2023\u003c/ul\u003e\n[Module 3](t81_558_class_03_1_neural_net.ipynb)\u003cbr\u003eWeek of 02/06/2023 | **Module 3: TensorFlow and Keras for Neural Networks**\u003cul\u003e\u003cli\u003ePart 3.1: Deep Learning and Neural Network Introduction\u003cli\u003ePart 3.2: Introduction to Tensorflow \u0026 Keras\u003cli\u003ePart 3.3: Saving and Loading a Keras Neural Network\u003cli\u003ePart 3.4: Early Stopping in Keras to Prevent Overfitting\u003cli\u003ePart 3.5: Extracting Keras Weights and Manual Neural Network Calculation\u003cli\u003e[Module 2: Program](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class2.ipynb) due: 02/07/2023\u003c/ul\u003e\n[Module 4](t81_558_class_04_1_feature_encode.ipynb)\u003cbr\u003eWeek of 02/13/2023 |**Module 4: Training for Tabular Data**\u003cul\u003e\u003cli\u003ePart 4.1: Encoding a Feature Vector for Keras Deep Learning\u003cli\u003ePart 4.2: Keras Multiclass Classification for Deep Neural Networks with ROC and AUC\u003cli\u003ePart 4.3: Keras Regression for Deep Neural Networks with RMSE\u003cli\u003ePart 4.4: Backpropagation, Nesterov Momentum, and ADAM Training\u003cli\u003ePart 4.5: Neural Network RMSE and Log Loss Error Calculation from Scratch\u003cli\u003e[Module 3 Program](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class3.ipynb) due: 02/14/2023\u003c/ul\u003e\n[Module 5](t81_558_class_05_1_reg_ridge_lasso.ipynb)\u003cbr\u003e**Meet on 02/20/2023** | **Module 5: Regularization and Dropout**\u003cul\u003e\u003cli\u003ePart 5.1: Introduction to Regularization: Ridge and Lasso\u003cli\u003ePart 5.2: Using K-Fold Cross Validation with Keras\u003cli\u003ePart 5.3: Using L1 and L2 Regularization with Keras to Decrease Overfitting\u003cli\u003ePart 5.4: Drop Out for Keras to Decrease Overfitting\u003cli\u003ePart 5.5: Bootstrapping and Benchmarking Hyperparameters\u003cli\u003e[Module 4 Program](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class4.ipynb) due: 02/21/2023\u003cli\u003e**We will meet on campus this week! (second meeting)**\u003c/ul\u003e\n[Module 6](t81_558_class_06_1_python_images.ipynb)\u003cbr\u003eWeek of 02/27/2023 | **Module 6: CNN for Vision**\u003cul\u003e\tPart 6.1: Image Processing in Python\u003cli\u003ePart 6.2: Using Convolutional Networks with Keras\u003cli\u003ePart 6.3: Using Pretrained Neural Networks\u003cli\u003ePart 6.4: Looking at Keras Generators and Image Augmentation\u003cli\u003ePart 6.5: Recognizing Multiple Images with YOLOv5\u003cli\u003e[Module 5 Program](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class5.ipynb) due: 02/28/2023\u003c/ul\u003e\n[Module 7](t81_558_class_07_1_gan_intro.ipynb)\u003cbr\u003eWeek of 03/06/2023 | **Module 7: Generative Adversarial Networks (GANs)**\u003cul\u003e\u003cli\u003ePart 7.1: Introduction to GANS for Image and Data Generation\u003cli\u003ePart 7.2: Train StyleGAN3 with your Own Images\u003cli\u003ePart 7.3: Exploring the StyleGAN Latent Vector\u003cli\u003ePart 7.4: GANS to Enhance Old Photographs Deoldify\u003cli\u003ePart 7.5: GANs for Tabular Synthetic Data Generation\u003cli\u003e[Module 6 Assignment](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class6.ipynb) due: 03/07/2023\u003c/ul\u003e\n[Module 8](t81_558_class_08_1_kaggle_intro.ipynb)\u003cbr\u003eWeek of 03/20/2023 | **Module 8: Kaggle**\u003cul\u003e\u003cli\u003ePart 8.1: Introduction to Kaggle\u003cli\u003ePart 8.2: Building Ensembles with Scikit-Learn and Keras\u003cli\u003ePart 8.3: How Should you Architect Your Keras Neural Network: Hyperparameters\u003cli\u003ePart 8.4: Bayesian Hyperparameter Optimization for Keras\u003cli\u003ePart 8.5: Current Semester's Kaggle\u003cli\u003e[Module 7 Assignment](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class7.ipynb) due: 03/21/2023\u003c/ul\u003e\n[Module 9](t81_558_class_09_1_keras_transfer.ipynb)\u003cbr\u003e**Meet on 03/27/2023** | **Module 9: Transfer Learning**\u003cul\u003e\u003cli\u003ePart 9.1: Introduction to Keras Transfer Learning\u003cli\u003ePart 9.2: Keras Transfer Learning for Computer Vision\u003cli\u003ePart 9.3: Transfer Learning for NLP with Keras\u003cli\u003ePart 9.4: Transfer Learning for Facial Feature Recognition\u003cli\u003ePart 9.5: Transfer Learning for Style Transfer\u003cli\u003e**We will meet on campus this week! (third meeting)**\u003cli\u003e[Module 8 Assignment](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class8.ipynb) due: 03/28/2023\u003c/ul\u003e\n[Module 10](t81_558_class_10_1_timeseries.ipynb)\u003cbr\u003eWeek of 04/03/2023 | **Module 10: Time Series in Keras**\u003cul\u003e\u003cli\u003ePart 10.1: Time Series Data Encoding for Deep Learning, Keras\u003cli\u003ePart 10.2: Programming LSTM with Keras and\u003cli\u003ePart 10.3: Text Generation with Keras\u003cli\u003ePart 10.4: Introduction to Transformers\u003cli\u003ePart 10.5: Transformers for Timeseries\u003cli\u003e[Module 9 Assignment](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class9.ipynb) due: 04/04/2023\u003c/ul\u003e\n[Module 11](t81_558_class_11_01_huggingface.ipynb)\u003cbr\u003eWeek of 04/10/2023 | **Module 11: Natural Language Processing**\u003cul\u003e\u003cli\u003ePart 11.1: Hugging Face Introduction\u003cli\u003ePart 11.2: Hugging Face Tokenizers\u003cli\u003ePart 11.3: Hugging Face Data Sets\u003cli\u003ePart 11.4: Training a Model in Hugging Face\u003cli\u003ePart 11.5: What are Embedding Layers in Keras\u003cli\u003e[Module 10 Assignment](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class10.ipynb) due: 04/11/2023\u003c/ul\u003e\n[Module 12](t81_558_class_12_01_ai_gym.ipynb)\u003cbr\u003eWeek of 04/17/2023 | **Module 12: Reinforcement Learning**\u003cul\u003e\u003cli\u003eKaggle Assignment due: 04/17/2023 (approx 4-6PM, due to Kaggle GMT timezone)\u003cli\u003ePart 12.1: Introduction to the OpenAI Gym\u003cli\u003ePart 12.2: Introduction to Q-Learning for Keras\u003cli\u003ePart 12.3: Keras Q-Learning in the OpenAI Gym\u003cli\u003ePart 12.4: Atari Games with Keras Neural Networks\u003cli\u003ePart 12.5: Application of Reinforcement Learning\u003c/ul\u003e\n[Module 13](t81_558_class_13_01_flask.ipynb)\u003cbr\u003e**Meet on 04/24/2023** | **Module 13: Deployment and Monitoring**\u003cul\u003e\u003cli\u003ePart 13.1: Flask and Deep Learning Web Services \u003cli\u003ePart 13.2: Interrupting and Continuing Training\u003cli\u003ePart 13.3: Using a Keras Deep Neural Network with a Web Application\u003cli\u003ePart 13.4: When to Retrain Your Neural Network\u003cli\u003ePart 13.5: Tensor Processing Units (TPUs)\u003cli\u003eFinal Project due 05/08/2023\u003cli\u003e**We will meet on campus this week! (fourth meeting)**\u003c/ul\u003e\n\n# Datasets\n\n* [Datasets can be downloaded here](https://data.heatonresearch.com/data/t81-558/index.html)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeffheaton%2Ft81_558_deep_learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjeffheaton%2Ft81_558_deep_learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeffheaton%2Ft81_558_deep_learning/lists"}