{"id":24847765,"url":"https://github.com/fr3fou/gone","last_synced_at":"2025-09-02T15:38:02.957Z","repository":{"id":57522054,"uuid":"245422374","full_name":"fr3fou/gone","owner":"fr3fou","description":"neural network library in go from scratch","archived":false,"fork":false,"pushed_at":"2023-07-04T08:49:23.000Z","size":189,"stargazers_count":34,"open_issues_count":1,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-06-19T05:59:04.407Z","etag":null,"topics":["go","golang","gradient-descent","hacktoberfest","neural-network"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fr3fou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-06T13:07:31.000Z","updated_at":"2024-06-06T03:54:32.000Z","dependencies_parsed_at":"2024-06-19T05:41:03.165Z","dependency_job_id":null,"html_url":"https://github.com/fr3fou/gone","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fr3fou%2Fgone","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fr3fou%2Fgone/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fr3fou%2Fgone/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fr3fou%2Fgone/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fr3fou","download_url":"https://codeload.github.com/fr3fou/gone/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236508956,"owners_count":19160476,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","gradient-descent","hacktoberfest","neural-network"],"created_at":"2025-01-31T11:32:24.978Z","updated_at":"2025-01-31T11:32:25.723Z","avatar_url":"https://github.com/fr3fou.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gone\n\n[![Github Actions Widget]][github actions] [![GoReport Widget]][goreport] [![GoDoc Widget]][godoc]\n\nA simple neural network library in Go from scratch. 0 dependencies\\*\n\n_there are 0 neural network related dependencies, the only dependency is for persisting the weights to a file ([golang/protobuf](github.com/golang/protobuf))_\n\n[goreport widget]: https://goreportcard.com/badge/github.com/fr3fou/gone\n[goreport]: https://goreportcard.com/report/github.com/fr3fou/gone\n[github actions widget]: https://github.com/fr3fou/gone/workflows/Test/badge.svg\n[github actions]: https://github.com/fr3fou/gone/actions\n[godoc]: http://pkg.go.dev/github.com/fr3fou/gone\n[godoc widget]: https://godoc.org/github.com/fr3fou/gone?status.svg\n\n## Example\n\n### Getting started\n\n```go\nfunc main() {\n g := gone.New(\n    0.1,\n    gone.MSE(),\n    gone.Layer{\n      Nodes: 2,\n    },\n    gone.Layer{\n      Nodes:     4,\n      Activator: gone.Sigmoid(),\n    },\n    gone.Layer{\n      Nodes: 1,\n    },\n )\n\n g.Train(gone.SGD(), gone.DataSet{\n    {\n      Inputs:  []float64{1, 0},\n      Targets: []float64{1},\n    },\n    {\n      Inputs:  []float64{0, 1},\n      Targets: []float64{1},\n    },\n    {\n      Inputs:  []float64{1, 1},\n      Targets: []float64{0},\n    },\n    {\n      Inputs:  []float64{0, 0},\n      Targets: []float64{0},\n    },\n }, 5000)\n\n g.Predict([]float64{1, 1})\n}\n```\n\n### Saving model to disk\n\n```go\n g.Save(\"test.gone\")\n\n```\n\n### Loading model back into memory\n\n```go\n g, err := gone.Load(\"test.gone\")\n```\n\n## TODO\n\n### `gone/`\n\n- [x] Types of task:\n  - [x] Classification - `softmax` (soon to be implemented) as the last layer's activation function\n  - [x] Regression - `sigmoid` as the last layer's activation function\n- [x] Bias\n  - [x] Matrix, rather than a single number\n- [x] Feedforward (Predict)\n- [ ] Train\n  - [x] Support shuffling the data\n  - [x] Epochs\n  - [x] Backpropagation\n  - [x] Batching\n  - [ ] Different loss functions\n    - [x] Mean Squared Error\n    - [ ] Cross Entropy Error\n- [x] Saving data - Done thanks to protobuf\n- [x] Loading data\n- [ ] Adam optimizer\n- [ ] Nestrov + Momentum for GD\n- [x] Fix MSE computation in debug mode (not used in actual backpropagation)\n- [ ] Somehow persist configurations for Activation, Loss and Optimizer functions in the protobuf messages (???, if we want to do it like it tensorflow, we'd have to do `interface{}` and do type assertions)\n- [ ] Convolutional Layers\n  - [ ] Flatten layer\n- [x] Copy\n- [x] Crossover\n- [x] Mutate\n  - [x] Gaussian Mutator\n\n### `matrix/`\n\nNOTE: all of this was migrated to [github.com/fr3fou/matrigo](https://github.com/fr3fou/matrigo)\n\n- [x] Randomize\n- [x] Transpose\n- [x] Scale\n- [x] AddMatrix\n- [x] Add\n- [x] SubtractMatrix\n- [x] Subtract\n- [x] Multiply\n- [x] Multiply\n- [x] Flatten\n- [x] Unflatten\n- [x] NewFromArray - makes a single row\n- [x] Map\n- [x] Fold\n- [x] Methods to support chaining\n\n```go\nn.Weights[i].\n    Multiply(output).                         // weighted sum of the previous layer\n    Add(n.Layers[i+1].Bias).                  // bias\n    Map(func(val float64, x, y int) float64 { // activation\n      return n.Layers[i+1].Activator.F(val)\n    })\n```\n\n### Research\n\n- [x] Derivatives ~\n- [x] Partial Derivatives ~\n- [ ] Linear vs non-linear problems (activation function)\n- [x] Gradient Descent\n  - [x] (Batch) Gradient Descent (GD)\n  - [x] Stochastic Gradient Descent (SGD)\n  - [x] Mini-Batch Gradient Descent (MBGD?)\n- [ ] Softmax (needed for multi class classification!)\n- [x] Mean Squared Error\n- [ ] Cross Entropy Error (needed for multi class classification!)\n- [ ] How to determine how many layers and nodes to use\n- [x] One Hot Encoding\n- [ ] Convolutional Layers\n- [ ] Reinforcment learning\n- [x] Genetic Algorithms~\n- [x] Neuroevolution~\n- [ ] Simulated Annealing\n- [ ] Q-Learning\n- [ ] Linear vs Logistic Regression\n- [ ] 3D inputs (regarding Video and CNNs)\n\n### Questions\n\nThese are some (stupid) questions I have that confuse me:\n\n- Is Neuroevolution considered Reinforcement learning?\n- How is training done with HUGE datasets when they can't fit on your storage device?\n  - Imagine your dataset is a copule of TB big, what do you do?\n- Is Q-Learning only done with a single agent (unlike genetic algorithms / neuroevolution)?\n  - Is Q-Learning the only method for Reinforcement Learning?\n- What's the difference between a Convolutional Neuron and a normal weight matrix?\n- Is Deep Learning really just a Neural Network with a lot of layers? (more than 2)\n- Why do you need multiple CNN layers? Is it to go to a smaller and smaller version of the image? (when working with images that is) (because of MaxPooling?) Why can't you go directly to the smallest size (512x512 -\u003e 16x16 vs 512x512 -\u003e 256x256 -\u003e 128x128 -\u003e ...)?\n- So if images are stored in a 2D array (but with the RGB channels, making it a 3D array with 3 layers), do we use `Conv2D` or `Conv3D`?\n- If 3D inputs are used for videos, how is that represented? Is a single input basically an array of 2D arrays (array of images - frames)? So basically a single observation is a single video and your entire dataset is a lot of videos, right?\n\n### Examples\n\n- [x] XOR Problem\n- [x] [Digit Classifier](https://github.com/fr3fou/digit-classifier)\n- [x] [Flappy Bird AI](https://github.com/fr3fou/flappy-ai)\n\n### Shoutouts\n\n- [David Josephs](https://github.com/josephsdavid) - was of HUGE help with algebra and other ML-related questions; also helped me spot some nasty bugs!\n\n## References\n\n_Note: some of the references weren't used during the development, but are in this section as they were a helpful guidance throughout my AI journey_ \n\n- \u003chttps://www.analyticsvidhya.com/blog/2020/01/fundamentals-deep-learning-activation-functions-when-to-use-them/\u003e\n- \u003chttps://www.youtube.com/watch?v=XJ7HLz9VYz0\u0026list=PLRqwX-V7Uu6Y7MdSCaIfsxc561QI0U0Tb\u003e\n- \u003chttps://www.youtube.com/watch?v=aircAruvnKk\u0026list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi\u003e\n- \u003chttp://matrixmultiplication.xyz/\u003e\n- \u003chttps://www.khanacademy.org/math/precalculus/x9e81a4f98389efdf:matrices/x9e81a4f98389efdf:properties-of-matrix-addition-and-scalar-multiplication/a/properties-of-matrix-addition\u003e\n- \u003chttps://www.wikiwand.com/en/Matrix_(mathematics)\u003e\n- \u003chttps://www.wikiwand.com/en/Activation_function\u003e\n- \u003chttps://www.wikiwand.com/en/Delta_rule\u003e\n- \u003chttps://www.jeremyjordan.me/intro-to-neural-networks/\u003e\n- \u003chttps://www.arxiv-vanity.com/papers/2003.02139/\u003e\n- \u003chttps://machinelearningmastery.com/gentle-introduction-mini-batch-gradient-descent-configure-batch-size/\u003e\n- \u003chttp://neuralnetworksanddeeplearning.com/chap2.html\u003e\n- \u003chttps://arxiv.org/pdf/1802.01528.pdf\u003e\n- \u003chttps://github.com/stevenmiller888/mind/blob/master/index.js\u003e\n- \u003chttps://github.com/stevenmiller888/go-mind\u003e\n- \u003chttps://medium.com/yottabytes/everything-you-need-to-know-about-gradient-descent-applied-to-neural-networks-d70f85e0cc14\u003e\n- \u003chttps://towardsdatascience.com/deep-learning-which-loss-and-activation-functions-should-i-use-ac02f1c56aa8\u003e\n- \u003chttps://github.com/milosgajdos/go-neural\u003e\n- \u003chttps://ml-cheatsheet.readthedocs.io/en/latest/forwardpropagation.html\u003e\n- \u003chttps://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/\u003e\n- \u003chttps://medium.com/coinmonks/representing-neural-network-with-vectors-and-matrices-c6b0e64db9fb\u003e\n- \u003chttps://towardsdatascience.com/classifying-cat-pics-with-a-logistic-regression-model-e35dfb9159bb\u003e\n- \u003chttps://towardsdatascience.com/math-neural-network-from-scratch-in-python-d6da9f29ce65\u003e\n- \u003chttps://towardsdatascience.com/gradient-descent-from-scratch-e8b75fa986cc\u003e\n- \u003chttps://rstudio-pubs-static.s3.amazonaws.com/337306_79a7966fad184532ab3ad66b322fe96e.html\u003e\n- \u003chttps://gombru.github.io/2018/05/23/cross_entropy_loss\u003e\n- \u003chttps://medium.com/@tomvykruta/memory-aid-for-softmax-and-cross-entropy-loss-5704c66d795d\u003e\n- \u003chttps://gombru.github.io/2018/05/23/cross_entropy_loss/\u003e\n- \u003chttps://cs.stackexchange.com/questions/90228/crossover-operator-in-genetic-algorithms-in-neural-networks\u003e\n- \u003chttps://stackoverflow.com/questions/54625643/where-is-the-gaussian-distribution-function-in-the-pseudocode-below\u003e\n- \u003chttps://www.wikiwand.com/en/Normal_distribution\u003e\n- \u003chttps://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/partial-derivative-and-gradient-articles/a/introduction-to-partial-derivatives\u003e\n- \u003chttps://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/partial-derivative-and-gradient-articles/a/the-gradient\u003e\n- \u003chttps://towardsdatascience.com/e2e-the-every-purpose-ml-method-5d4f20dafee4\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffr3fou%2Fgone","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffr3fou%2Fgone","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffr3fou%2Fgone/lists"}