https://github.com/fr3fou/gone
neural network library in go from scratch
https://github.com/fr3fou/gone
go golang gradient-descent hacktoberfest neural-network
Last synced: 10 months ago
JSON representation
neural network library in go from scratch
- Host: GitHub
- URL: https://github.com/fr3fou/gone
- Owner: fr3fou
- License: mit
- Created: 2020-03-06T13:07:31.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2023-07-04T08:49:23.000Z (almost 3 years ago)
- Last Synced: 2024-06-19T05:59:04.407Z (about 2 years ago)
- Topics: go, golang, gradient-descent, hacktoberfest, neural-network
- Language: Go
- Homepage:
- Size: 185 KB
- Stars: 34
- Watchers: 5
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# gone
[![Github Actions Widget]][github actions] [![GoReport Widget]][goreport] [![GoDoc Widget]][godoc]
A simple neural network library in Go from scratch. 0 dependencies\*
_there are 0 neural network related dependencies, the only dependency is for persisting the weights to a file ([golang/protobuf](github.com/golang/protobuf))_
[goreport widget]: https://goreportcard.com/badge/github.com/fr3fou/gone
[goreport]: https://goreportcard.com/report/github.com/fr3fou/gone
[github actions widget]: https://github.com/fr3fou/gone/workflows/Test/badge.svg
[github actions]: https://github.com/fr3fou/gone/actions
[godoc]: http://pkg.go.dev/github.com/fr3fou/gone
[godoc widget]: https://godoc.org/github.com/fr3fou/gone?status.svg
## Example
### Getting started
```go
func main() {
g := gone.New(
0.1,
gone.MSE(),
gone.Layer{
Nodes: 2,
},
gone.Layer{
Nodes: 4,
Activator: gone.Sigmoid(),
},
gone.Layer{
Nodes: 1,
},
)
g.Train(gone.SGD(), gone.DataSet{
{
Inputs: []float64{1, 0},
Targets: []float64{1},
},
{
Inputs: []float64{0, 1},
Targets: []float64{1},
},
{
Inputs: []float64{1, 1},
Targets: []float64{0},
},
{
Inputs: []float64{0, 0},
Targets: []float64{0},
},
}, 5000)
g.Predict([]float64{1, 1})
}
```
### Saving model to disk
```go
g.Save("test.gone")
```
### Loading model back into memory
```go
g, err := gone.Load("test.gone")
```
## TODO
### `gone/`
- [x] Types of task:
- [x] Classification - `softmax` (soon to be implemented) as the last layer's activation function
- [x] Regression - `sigmoid` as the last layer's activation function
- [x] Bias
- [x] Matrix, rather than a single number
- [x] Feedforward (Predict)
- [ ] Train
- [x] Support shuffling the data
- [x] Epochs
- [x] Backpropagation
- [x] Batching
- [ ] Different loss functions
- [x] Mean Squared Error
- [ ] Cross Entropy Error
- [x] Saving data - Done thanks to protobuf
- [x] Loading data
- [ ] Adam optimizer
- [ ] Nestrov + Momentum for GD
- [x] Fix MSE computation in debug mode (not used in actual backpropagation)
- [ ] Somehow persist configurations for Activation, Loss and Optimizer functions in the protobuf messages (???, if we want to do it like it tensorflow, we'd have to do `interface{}` and do type assertions)
- [ ] Convolutional Layers
- [ ] Flatten layer
- [x] Copy
- [x] Crossover
- [x] Mutate
- [x] Gaussian Mutator
### `matrix/`
NOTE: all of this was migrated to [github.com/fr3fou/matrigo](https://github.com/fr3fou/matrigo)
- [x] Randomize
- [x] Transpose
- [x] Scale
- [x] AddMatrix
- [x] Add
- [x] SubtractMatrix
- [x] Subtract
- [x] Multiply
- [x] Multiply
- [x] Flatten
- [x] Unflatten
- [x] NewFromArray - makes a single row
- [x] Map
- [x] Fold
- [x] Methods to support chaining
```go
n.Weights[i].
Multiply(output). // weighted sum of the previous layer
Add(n.Layers[i+1].Bias). // bias
Map(func(val float64, x, y int) float64 { // activation
return n.Layers[i+1].Activator.F(val)
})
```
### Research
- [x] Derivatives ~
- [x] Partial Derivatives ~
- [ ] Linear vs non-linear problems (activation function)
- [x] Gradient Descent
- [x] (Batch) Gradient Descent (GD)
- [x] Stochastic Gradient Descent (SGD)
- [x] Mini-Batch Gradient Descent (MBGD?)
- [ ] Softmax (needed for multi class classification!)
- [x] Mean Squared Error
- [ ] Cross Entropy Error (needed for multi class classification!)
- [ ] How to determine how many layers and nodes to use
- [x] One Hot Encoding
- [ ] Convolutional Layers
- [ ] Reinforcment learning
- [x] Genetic Algorithms~
- [x] Neuroevolution~
- [ ] Simulated Annealing
- [ ] Q-Learning
- [ ] Linear vs Logistic Regression
- [ ] 3D inputs (regarding Video and CNNs)
### Questions
These are some (stupid) questions I have that confuse me:
- Is Neuroevolution considered Reinforcement learning?
- How is training done with HUGE datasets when they can't fit on your storage device?
- Imagine your dataset is a copule of TB big, what do you do?
- Is Q-Learning only done with a single agent (unlike genetic algorithms / neuroevolution)?
- Is Q-Learning the only method for Reinforcement Learning?
- What's the difference between a Convolutional Neuron and a normal weight matrix?
- Is Deep Learning really just a Neural Network with a lot of layers? (more than 2)
- Why do you need multiple CNN layers? Is it to go to a smaller and smaller version of the image? (when working with images that is) (because of MaxPooling?) Why can't you go directly to the smallest size (512x512 -> 16x16 vs 512x512 -> 256x256 -> 128x128 -> ...)?
- So if images are stored in a 2D array (but with the RGB channels, making it a 3D array with 3 layers), do we use `Conv2D` or `Conv3D`?
- If 3D inputs are used for videos, how is that represented? Is a single input basically an array of 2D arrays (array of images - frames)? So basically a single observation is a single video and your entire dataset is a lot of videos, right?
### Examples
- [x] XOR Problem
- [x] [Digit Classifier](https://github.com/fr3fou/digit-classifier)
- [x] [Flappy Bird AI](https://github.com/fr3fou/flappy-ai)
### Shoutouts
- [David Josephs](https://github.com/josephsdavid) - was of HUGE help with algebra and other ML-related questions; also helped me spot some nasty bugs!
## References
_Note: some of the references weren't used during the development, but are in this section as they were a helpful guidance throughout my AI journey_
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-