{"id":20261322,"url":"https://github.com/lightning-ai/dl-fundamentals","last_synced_at":"2025-04-04T19:13:13.206Z","repository":{"id":64292393,"uuid":"564043804","full_name":"Lightning-AI/dl-fundamentals","owner":"Lightning-AI","description":"Deep Learning Fundamentals -- Code material and exercises","archived":false,"fork":false,"pushed_at":"2024-02-28T20:30:26.000Z","size":22841,"stargazers_count":368,"open_issues_count":3,"forks_count":193,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-03-28T18:13:21.926Z","etag":null,"topics":["ai","deep-learning","machine-learning","python","pytorch"],"latest_commit_sha":null,"homepage":"https://lightning.ai/pages/courses/deep-learning-fundamentals/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Lightning-AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-09T21:48:20.000Z","updated_at":"2025-03-25T10:31:19.000Z","dependencies_parsed_at":"2024-11-14T11:39:22.361Z","dependency_job_id":null,"html_url":"https://github.com/Lightning-AI/dl-fundamentals","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2Fdl-fundamentals","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2Fdl-fundamentals/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2Fdl-fundamentals/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lightning-AI%2Fdl-fundamentals/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Lightning-AI","download_url":"https://codeload.github.com/Lightning-AI/dl-fundamentals/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247234923,"owners_count":20905854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deep-learning","machine-learning","python","pytorch"],"created_at":"2024-11-14T11:25:15.621Z","updated_at":"2025-04-04T19:13:13.173Z","avatar_url":"https://github.com/Lightning-AI.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Learning Fundamentals: Code Materials and Exercises\n\n\n\n*This repository contains code materials \u0026amp; exercises for Deep Learning Fundamentals course by [Sebastian Raschka](https://sebastianraschka.com) and [Lightning AI](https://lightning.ai).*\n\n\n\n- Link to the course website: https://lightning.ai/pages/courses/deep-learning-fundamentals/\n- Link to the discussion forum: https://github.com/Lightning-AI/dl-fundamentals/discussions\n- Reach out to Lightning \u0026 Sebastian on social media: [@LightningAI](https://twitter.com/LightningAI) [@rasbt](https://twitter.com/rasbt)\n\n\n\n---\n\nFor other announcements, updates, and additional materials, you can follow [Lightning AI](https://twitter.com/LightningAI) and [Sebastian](https://twitter.com/rasbt) on Twitter!\n\n---\n\n\n\n## Links to the materials\n\n\n\n### Unit 1. Welcome to Machine Learning and Deep Learning [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/unit-1/) ] \n\n- 1.1 What Is Machine Learning?\n- 1.2 How Can We Use Machine Learning?\n- 1.3 A Typical Machine Learning Workflow (The Supervised Learning Workflow)\n- 1.4 The First Machine Learning Classifier\n- 1.5 Setting Up Our Computing Environment\n- [1.6 Implementing a Perceptron in Python](https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit01-ml-intro/1.6-perceptron-in-python)\n- 1.7 Evaluating Machine Learning Models\n- Unit 1 exercises\n  - [Exercise 1: Add early-stopping to make the Perceptron more efficient](https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit01-ml-intro/exercises/1_early-stop)\n  - [Exercise 2: Initialize the model parameters with small random numbers instead of 0's](https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit01-ml-intro/exercises/2_random-weights)\n  - [Exercise 3: Use a learning rate for updating the weights and bias unit](https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit01-ml-intro/exercises/3_learning-rate)\n\n### Unit 2. First Steps with PyTorch: Using Tensors [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/2-0-unit-2-overview/) ] \n\n- 2.1 Introducing PyTorch\n- [2.2 What Are Tensors?](https://github.com/Lightning-AI/dl-fundamentals/blob/main/unit02-pytorch-tensors/2.2-tensors/torch-tensors.ipynb)\n- [2.3 How Do We Use Tensors in PyTorch?](https://github.com/Lightning-AI/dl-fundamentals/blob/main/unit02-pytorch-tensors/2.3-using-tensors/top10-tensor-commands.ipynb)\n- [2.4 Improving Code Efficiency with Linear Algebra](https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit02-pytorch-tensors/2.4-linalg)\n- [2.5 Debugging Code](https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit02-pytorch-tensors/2.5-debugging)\n- [2.6 Revisiting the Perceptron Algorithm](https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit02-pytorch-tensors/2.6-revisiting-perceptron)\n- 2.7 Seeing Predictive Models as Computation Graphs\n- Unit 2 exercises\n  - [Exercise 1: Introducing more PyTorch functions to make your code more efficient](https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit02-pytorch-tensors/exercises/1_torch-where)\n  - [Exercise 2: Make the perceptron more efficient using matrix multiplication](https://github.com/Lightning-AI/dl-fundamentals/tree/main/unit02-pytorch-tensors/exercises/2_perceptron-matmul)\n\n\n\n### Unit 3. Model Training in PyTorch [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/3-0-overview-model-training-in-pytorch/) ] \n\n- 3.1 Using Logistic Regression for Classification\n- 3.2 The Logistic Regression Computation Graph\n- 3.3 Model Training with Stochastic Gradient Descent\n- 3.4 Automatic Differentiation in PyTorch\n- 3.5 The PyTorch API\n- [3.6 Training a Logistic Regression Model in PyTorch](https://github.com/Lightning-AI/dl-fundamentals/tree/main/3.6-logreg-in-pytorch)\n- 3.7 Feature Normalization\n- Unit 3 exercises\n  - [Exercise 1: Banknote Authentication](https://github.com/Lightning-AI/dl-fundamentals/tree/main/exercises/1_banknotes)\n  - [Exercise 2: Standardization](https://github.com/Lightning-AI/dl-fundamentals/tree/main/exercises/2_standardization) \n\n### Unit 4. Training Multilayer Neural Networks [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/training-multilayer-neural-networks-overview/) ] \n\n- 4.1 Dealing with More than Two Classes: Softmax Regression\n- 4.2 Multilayer Neural Networks and Why We Need Them\n- [4.3 Training a Multilayer Perceptron in PyTorch](unit04-multilayer-nets/4.3-mlp-pytorch)\n  - [XOR data](unit04-multilayer-nets/4.3-mlp-pytorch/4.3-mlp-pytorch-part1-2-xor)\n  - [MNIST data](unit04-multilayer-nets/4.3-mlp-pytorch/4.3-mlp-pytorch-part3-5-mnist)\n- [4.4 Defining Efficient Data Loaders](unit04-multilayer-nets/4.4-dataloaders)\n- [4.5 Multilayer Neural Networks for Regression](unit04-multilayer-nets/4.5-mlp-regression)\n- 4.6 Speeding Up Model Training Using GPUs\n- [Unit 4 exercises](./unit04-multilayer-nets/exercises)\n  - [Excercise 1: Changing the Number of Layers](./unit04-multilayer-nets/exercises/1_changing-layers)\n  - [Exercise 2: Implementing a Custom Dataset Class for Fashion MNIST](./unit04-multilayer-nets/exercises/2_fashion-mnist)\n\n### Unit 5. Organizing your PyTorch Code with Lightning [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/overview-organizing-your-code-with-pytorch-lightning/) ] \n\n- 5.1 Organizing Your Code with PyTorch Lightning\n- [5.2 Training a Multilayer Perceptron in PyTorch Lightning](./unit05-lightning/5.2-mlp-lightning)\n- [5.3 Computing Metrics Efficiently with TorchMetrics](./unit05-lightning/5.3-torchmetrics)\n- [5.4 Making Code Reproducible](./unit05-lightning/5.4-reproducibility)\n- [5.5 Organizing Your Data Loaders with Data Modules](./unit05-lightning/5.5-datamodules)\n- [5.6 The Benefits of Logging Your Model Training](./unit05-lightning/5.6-logging)\n- [5.7 Evaluating and Using Models on New Data](./unit05-lightning/5.7-evaluating)\n- 5.8 Add functionality with callbacks\n- [Unit 5 exercises](./unit05-lightning/exercises)\n\n### Unit 6. Essential Deep Learning Tips \u0026 Tricks [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/unit-6-overview-essential-deep-learning-tips-tricks/) ] \n\n- [6.1 Model Checkpointing and Early Stopping](./unit06-dl-tips/6.1-checkpointing)\n- [6.2 Learning Rates and Learning Rate Schedulers](./unit06-dl-tips/6.2-learning-rates)\n- 6.3 Using More Advanced Optimization Algorithms\n- 6.4 Choosing Activation Functions\n- [6.5 Automating The Hyperparameter Tuning Process](./unit06-dl-tips/6.5-hparam-opt)\n- 6.6 Improving Convergence with Batch Normalization\n- 6.7 Reducing Overfitting With Dropout\n- [6.8 Debugging Deep Neural Networks](./unit06-dl-tips/6.8-debugging)\n- [Unit 6 exercises](./unit06-dl-tips/exercises)\n\n\n### Unit 7. Getting Started with Computer Vision [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/unit-7-overview-getting-started-with-computer-vision/) ] \n\n- 7.1 Working With Images\n- 7.2 How Convolutional Neural Networks Work\n- 7.3 Convolutional Neural Network Architectures\n- [7.4 Training Convolutional Neural Networks](./unit07-computer-vision/unit07-computer-vision/7.4-cnn-training)\n- [7.5 Improving Predictions with Data Augmentation](./unit07-computer-vision/unit07-computer-vision/)\n- [7.6 Leveraging Pre-trained Models with Transfer Learning](./unit07-computer-vision/unit07-computer-vision/)\n- [7.7 Using Unlabeled Data with Self-Supervised](./unit07-computer-vision/unit07-computer-vision/)\n- [Unit 7 exercises](./unit07-computer-vision/exercises)\n\n\n\n### Unit 8. Introduction to Natural Language Processing and Large Language Models  [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/unit-8.0-natural-language-processing-and-large-language-models/) ] \n\n- 8.1 Working with Text Data\n- [8.2 Training Text Classifier Baseline](unit08-large-language-models/8.2-bag-of-words)\n- 8.3. Introduction to Recurrent Neural Networks\n- 8.4 From RNNas to the Transformer Architecture\n- 8.5 Understanding Self-Attention\n- 8.6 Large Language Models\n- [8.7 Using Large Language Model for Classification](unit08-large-language-models/8.7-distilbert-finetuning)\n- [Unit 8 exercises](unit08-large-language-models/exercises)\n\n### Unit 9. Techniques for Speeding Up Model Training  [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/9.0-overview-techniques-for-speeding-up-model-training/) ] \n\n- [9.1 Accelerated Model Training via Mixed-Precision Training](unit09-performance/9.1-mixed-precision)\n- 9.2 Multi-GPU Training Strategies\n- [9.3 Deep Dive Into Data Parallelism](unit09-performance/9.3-multi-gpu)\n- [9.4 Compiling PyTorch Models](unit09-performance/9.4-compile)\n- [9.5 Increasing Batch Sizes to Increase Throughput](unit09-performance/9.5-batchsize-finder)\n- [Unit 9 exercises](unit09-performance/exercises)\n\n### Unit 10. The Finale: Our Next Steps After AI Model Training  [ [Link to videos](https://lightning.ai/pages/courses/deep-learning-fundamentals/10.0-overview-the-finale-our-next-steps-after-ai-model-training/) ] \n\n- [10.1 Trustworthy and Reliable Machine Learning](unit10-after-training/10.1-confidence-intervals)\n- [10.2 Fabric - Scaling PyTorch Model Training without Boilerplate Code](unit10-after-training/10.2-fabric)\n- 10.3 Designing Machine Learning Systems\n- 10.4 Conclusion\n- [Unit 10 exercises](unit10-after-training/exercises)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightning-ai%2Fdl-fundamentals","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flightning-ai%2Fdl-fundamentals","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightning-ai%2Fdl-fundamentals/lists"}