https://github.com/Alkamist/odin_simple_neural_network

https://github.com/Alkamist/odin_simple_neural_network

Last synced: 18 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/Alkamist/odin_simple_neural_network
• Owner: Alkamist
• Created: 2024-07-15T20:11:23.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-07-27T21:26:04.000Z (5 months ago)
• Last Synced: 2024-07-27T22:37:09.091Z (5 months ago)
• Language: Odin
• Size: 15.1 MB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-odin - Simple Neural Network
• awesome-odin - Simple Neural Network

README

        
This is a little experiment with the goal of understanding more about machine learning.

It is based on 3Blue1Brown's series on basic deep learning:

https://www.youtube.com/watch?v=aircAruvnKk

And the simple network code from:

https://github.com/mnielsen/neural-networks-and-deep-learning/blob/master/src/network.py

It is also based on the article found here:

https://gpuopen.com/learn/deep_learning_crash_course/

Here I implemented a simple multi-layer perceptron in 100% Odin that runs on the CPU.

There are 3 main functions that you can run:

- train

- load_and_validate

- predict_digit

`train` will train the model off of the MNIST dataset, which you will need to unzip first. It will dump the model's best performing weights and biases into a json file as it is training. You should probably run it with `-o:speed`.

`load_and_validate` will load the model from the json file and then run it on the MNIST validation dataset to check for accuracy.

`predict_digit` will load the model from the json file and try to predict what digit is present in the file `digit.png`.

The model uses:

- Leaky Relu

- Cross Entropy Loss

- Dropout

- He and Xavier Initialization

- L2 Regularization

- ADAM Optimization

And a quick note: All of the digits in the MNIST dataset are centered, I haven't yet made a way to augment the dataset to be more robust to off-center and oddly scaled digits, so if you draw a digit in `digit.png` make sure it is in the center.