Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/frozenassassine/neuralnetwork-fromscratch

Neural Network from scratch in C# with CUDA support
https://github.com/frozenassassine/neuralnetwork-fromscratch

ai classification csharp cuda gpu gpu-acceleration neural-network neural-networks nvidia

Last synced: 3 months ago
JSON representation

Neural Network from scratch in C# with CUDA support

Awesome Lists containing this project

README 

        



Neural Network from scratch with CUDA Support




    

    

    




## 🤔 What is this project?

This project is a neural network implementation from scratch in C# with CUDA support written in C++. It currently supports Optical Digit Recognition (ODR) trained with 60,000 images and can also perform XOR as a simple initial test.

More complex image classification is in progress. I trained it with 2000 rgb images of 150 * 150 pixels and got some ok results.



## ❗Info

At the current point I would not recommend this in any production environment, for me it's just a fun project to learn more about CUDA and Neural Networks.

Also I tried to implement Convolution and Pooling layer from scratch, but failed in the back propagation. Currently they are not working in any way😢



## 🛠️ Features

- **Optical Digit Recognition (ODR)**: Trained with the MNIST dataset of 60,000 images.

- **XOR Test**: A simple test to demonstrate the neural network's basic functionality.

- **CUDA Support**: Accelerates neural network training using GPU resources.

### 📎See also [DeepReinforcementLearning](https://github.com/FrozenAssassine/DeepReinforcementLearning) from scratch using this project



## 📊 Benchmarks

| Training Details | GPU (CUDA, RTX 3050) | CPU (i9-10900) | (CPU) Ryzen 5 3500U |

|------------------|----------------------|----------------|----------------|

| 100 images, 150x150x3 (67500 inputs, 1024 hidden, 512 hidden, 256 hidden, 6 outputs) | 2.231 sec | 9.514 sec | 34.472 sec

| 100 images, 150x150x3 (67500 inputs, 2048 hidden, 1024 hidden, 6 outputs) | (old)6.832 sec | 9.426 sec | 31.467 sec



## 🚀 Performance History

### Sequential to true Parallel 📈 ...



The initial Optical Digit Recognition (ODR) implementation, using 28x28 black-and-white images as input with a neural network consisting of 128 and 64 hidden neurons and 10 output neurons, took 2.8 seconds to train on 1000 images.  

To improve performance, I added **Parallel.For** support, which accelerated the training process. Enabling Release mode further optimized the training time, reducing it to around 780ms for 1000 images.   

However, this was not sufficient. I began integrating CUDA support, which proved challenging but significantly reduced the training time. With CUDA, I brought the training time down to 400ms for 1000 images. In the latest build, I achieved a training time of approximately 200ms per 1000 images.   

Overall, this resulted in a 10-fold increase in performance.



## 🏗️ Get Started

1. Clone the repository.

2. Ensure you have the necessary dependencies for C# and CUDA development.

   (https://developer.nvidia.com/cuda-downloads)

4. Open the solution file (`.sln`) in Visual Studio.

5. Build and run the project.



## Example code

```cs

//XOR prediction

var nnmodel = NetworkBuilder.Create()

    .Stack(new InputLayer(2))

    .Stack(new DenseLayer(4, ActivationType.Sigmoid))

    .Stack(new OutputLayer(1, ActivationType.Sigmoid))

    .Build();



nnmodel.Summary();



float[][] inputs = new float[][] { new float[] { 0, 0 }, new float[] { 0, 1 }, new float[] { 1, 0 }, new float[] { 1, 1 } };

float[][] desired = new float[][] { new float[] { 0 }, new float[] { 1 }, new float[] { 1 }, new float[] { 0 } };

nnmodel.Train(inputs, desired, 15900, 0.01f, 1000, 100);



var prediction = nnmodel.Predict(new float[] { 0, 0 });

Console.WriteLine("Prediction: " + MathHelper.GetMaximumIndex(prediction));

```