https://github.com/kinwaicheuk/reducing-overfitting-with-gaussian-noise

Preventing overfitting by putting Gaussian noise before the input activation
https://github.com/kinwaicheuk/reducing-overfitting-with-gaussian-noise

Last synced: about 1 month ago
JSON representation

Preventing overfitting by putting Gaussian noise before the input activation

• Host: GitHub
• URL: https://github.com/kinwaicheuk/reducing-overfitting-with-gaussian-noise
• Owner: KinWaiCheuk
• Created: 2018-09-25T09:22:09.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2018-09-25T09:33:07.000Z (over 6 years ago)
• Last Synced: 2025-02-05T02:49:04.547Z (3 months ago)
• Language: Jupyter Notebook
• Size: 151 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# solving-overfitting-with-Gaussian-Noise

Preventing overfitting by putting Gaussian noise before the input activation

# Dependencies

Tensorflow 1.5

keras 2.2

sklearn 0.19.1

matplotlib 2.2.2

numpy 1.14.5

# Instruction

All the required codes are contained inside the jupyter-notebook. The code is designed that you can also run it quickly and obtain the results in a PC without a GPU

# Summary

### A Fancy model that overfits

When using a complicated model on a small dataset, the model will overfit easily. As you can see the validation loss (orange curve) for is increasing after training for a long time.

![alt text](https://raw.githubusercontent.com/KinWaiCheuk/solving-overfitting-with-Gaussian-Noise/master/overfits.png)

### The data distribution of MNIST dataset after applying Triplet Neural Newtork

By adding some Gaussian Noise to the input data before the first activation, the overfitting problem can be mitigated

![alt test](https://raw.githubusercontent.com/KinWaiCheuk/solving-overfitting-with-Gaussian-Noise/master/withGaussianNoise.png)