{"id":22799704,"url":"https://github.com/git-vish/pytorch_mnist","last_synced_at":"2025-03-30T19:17:01.800Z","repository":{"id":255913833,"uuid":"264673996","full_name":"git-vish/PyTorch_MNIST","owner":"git-vish","description":"Simple Feed Forward networks for image classification on MNIST dataset, using PyTorch Library. ","archived":false,"fork":false,"pushed_at":"2020-05-17T16:21:04.000Z","size":118,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-05T21:53:25.693Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/git-vish.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-17T13:33:25.000Z","updated_at":"2020-05-17T16:21:06.000Z","dependencies_parsed_at":"2024-09-07T20:42:17.106Z","dependency_job_id":"5726a191-a2a8-4a37-af2e-9f5a319133aa","html_url":"https://github.com/git-vish/PyTorch_MNIST","commit_stats":null,"previous_names":["git-vish/pytorch_mnist"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/git-vish%2FPyTorch_MNIST","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/git-vish%2FPyTorch_MNIST/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/git-vish%2FPyTorch_MNIST/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/git-vish%2FPyTorch_MNIST/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/git-vish","download_url":"https://codeload.github.com/git-vish/PyTorch_MNIST/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246365647,"owners_count":20765549,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-12T07:09:42.321Z","updated_at":"2025-03-30T19:17:01.520Z","avatar_url":"https://github.com/git-vish.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PyTorch MNIST\n   AI algorithms will be incorporated into more and more everyday applications. For example, you might want to include an image classifier in a smartphone app. To do this, you'd use a deep learning model trained on hundreds of thousands of images as part of the overall application architecture. A large part of software development in the future will be using these types of models as common parts of applications.\n\n\n   In this project, I have trained 2 image classifiers to recognize Hand written digits and Clothing Types. I have used **_[PyTorch](https://pytorch.org/)_** library and created very basic 3-4 layer small models. In practice, one would use Convolutional Neural Networks (CNN) for these type of problems. We'll be using popular MNIST and Fashion MNIST dataset of 24*24 Gray scale images.\n1. [Hand Written Digit Classification(MNIST)](https://github.com/cloud-VG/PyTorch_MNIST/blob/master/MNIST_PyTorch.ipynb)\n2. [Clothing Classification(Fashion MNIST)](https://github.com/cloud-VG/PyTorch_MNIST/blob/master/Fashion_MNIST_PyTorch.ipynb)\n### Sample Output (MNIST)\n![Output](img/mnist_pred.png)\n\n# Overfitting\n\n   If we look at the training and validation losses as we train the **_Fashion MNIST_** network, we can see a phenomenon known as overfitting.\n\n\n![Overfitting](img/loss.png)\n\n   \n   The network learns the training set better and better, resulting in lower training losses. However, it starts having problems generalizing to data outside the training set leading to the validation loss increasing. The ultimate goal of any deep learning model is to make predictions on new data, so we should strive to get the lowest validation loss possible. One option is to use the version of the model with the lowest validation loss, here the one around 8-10 training epochs. This strategy is called early-stopping. In practice, you'd save the model frequently as you're training then later choose the model with the lowest validation loss.\n\nThe most common method to reduce overfitting (outside of early-stopping) is dropout, where we randomly drop input units. This forces the network to share information between weights, increasing it's ability to generalize to new data. Adding dropout in PyTorch is straightforward using the **_[nn.Dropout](https://pytorch.org/docs/stable/nn.html?highlight=dropout#torch.nn.Dropout)_** module.\n\n```python\nclass Model(nn.Module):\n    def __init__(self):\n        ...\n        # Dropout module with 0.2 drop probability\n        self.dropout = nn.Dropout(p=0.2)\n        \n    def forward(self, x):\n        ...\n        x = self.dropout(F.relu(self.layer(x)))\n```\n\nDuring training we want to use dropout to prevent overfitting, but during inference we want to use the entire network. So, we need to turn off dropout during validation, testing, and whenever we're using the network to make predictions. To do this, you use **_model.eval()_**. This sets the model to evaluation mode where the dropout probability is 0. You can turn dropout back on by setting the model to train mode with **_model.train()_**. In general, the pattern for the validation loop will look like this, where you turn off gradients, set the model to evaluation mode, calculate the validation loss and metric, then set the model back to train mode.\n\n```python\n# turn off gradients\nwith torch.no_grad():\n\n    # set model to evaluation mode\n    model.eval()\n\n    # validation pass here\n    for images, labels in testloader:\n        ...\n\n# set model back to train mode\nmodel.train()\n```\n\nAfter Implementing Dropout technique, the result will look like,\n\n\n![Dropout loss](img/dropout-loss.png)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgit-vish%2Fpytorch_mnist","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgit-vish%2Fpytorch_mnist","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgit-vish%2Fpytorch_mnist/lists"}