https://github.com/ameobea/neural-network-from-scratch

A neural network library written from scratch in Rust along with a web-based application for building + training neural networks + visualizing their outputs
https://github.com/ameobea/neural-network-from-scratch

backpropagation data-visualization gradient-descent neural-network neural-networks neural-networks-from-scratch rust simd wasm wasm-bindgen webassembly

Last synced: 10 days ago
JSON representation

A neural network library written from scratch in Rust along with a web-based application for building + training neural networks + visualizing their outputs

• Host: GitHub
• URL: https://github.com/ameobea/neural-network-from-scratch
• Owner: Ameobea
• Created: 2021-07-29T22:33:07.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2022-04-09T00:13:59.000Z (over 2 years ago)
• Last Synced: 2024-08-03T09:11:11.412Z (3 months ago)
• Topics: backpropagation, data-visualization, gradient-descent, neural-network, neural-networks, neural-networks-from-scratch, rust, simd, wasm, wasm-bindgen, webassembly
• Language: Rust
• Homepage: https://nn.ameo.dev
• Size: 1.69 MB
• Stars: 122
• Watchers: 6
• Forks: 11
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Neural Network from Scratch in the Browser

![A screenshot of the neural network web application itself which shows the full UI, network response visualization, and cost plot](https://ameo.link/u/97k.png)

**Try it yourself**: 

## About

The goal of this project was to understand neural networks better by building them from the ground up.  I wanted to be able to see dynamically how choosing different network architectures + training parameters affects network performance and how well networks can various functions.

It allows users to define a neural network infrastructure by adding hidden layers, picking neuron counts + activation function types, and setting training parameters like learning rate.  The network then learns one of a variety of selectable target functions from which examples are randomly sampled to train it.

The "response" of the network over the entire range of possible inputs is then plotted as a 3D surface along with the target function to show how well the network has learned.

### Technical Details

The neural network implementation itself is built in Rust and compiled to WebAssembly using Wasm SIMD to accelerate training.

The training takes place on a dedicated thread by using a web worker and the excellent [Comlink](https://github.com/GoogleChromeLabs/comlink) library to communicate between the main/render thread and the training thread.

All the charts + visualizations are created using the excellent [echarts](https://echarts.apache.org/en/index.html) library.

The UI is created using [react-control-panel](https://github.com/ameobea/react-control-panel) which is a React port of the excellent [`control-panel`](https://github.com/freeman-lab/control-panel) library for easy GUI creation.

## Building + Developing

You'll need Rust nightly with WebAssembly support.  You can install Rust via easily via rustup: 

Then, add the latest nightly toolchain + switch to it:

`rustup default nightly`

Add WebAssembly support:

`rustup target add wasm32-unknown-unknown`

This project uses the [`just` command runner](https://github.com/casey/just) to simplify many tasks.  Install it with:

`cargo install just`.

You'll also need [`wasm-bindgen`](https://github.com/rustwasm/wasm-bindgen):

`cargo install wasm-bindgen --version=0.2.74`

You'll need to install the `wasm-opt` tool from [`binaryen`](https://github.com/WebAssembly/binaryen).  You can download the executable from the Releases section on Github or build it yourself with CMake.

Then, you'll need tools for the web stack.  I use yarn for node package management, and you can either update the `Justfile` to change `yarn` to `npm` or install yarn 1.0 from here: 

That should be all you need!  To start the webpack dev server for hot-reloading and development, execute `just run` in the project root.

To create a release build, execute `just build` in the project root.  That will produce a fully functional static website in the `dist` directory.

## References

This is a list of some of the resources that I made use of while learning about neural networks and building this project:

* 

* 

* [MIT 6.S094: Recurrent Neural Networks for Steering Through Time](https://www.youtube.com/watch?v=nFTQ7kHQWtc&t=475s) <- Really helped me break through the wall of understanding some of the core concepts behind neural networks