{"id":19945621,"url":"https://github.com/frozenassassine/neuralnetwork-fromscratch","last_synced_at":"2026-02-20T15:31:57.835Z","repository":{"id":244699910,"uuid":"803979130","full_name":"FrozenAssassine/NeuralNetwork-FromScratch","owner":"FrozenAssassine","description":"Neural Network from scratch in C# with CUDA support","archived":false,"fork":false,"pushed_at":"2024-11-08T08:57:26.000Z","size":210,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-11-08T09:24:23.723Z","etag":null,"topics":["ai","classification","csharp","cuda","gpu","gpu-acceleration","neural-network","neural-networks","nvidia"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FrozenAssassine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-21T18:14:51.000Z","updated_at":"2024-11-08T08:57:29.000Z","dependencies_parsed_at":"2024-09-14T12:56:40.983Z","dependency_job_id":"4beed47d-7967-44ea-b6cd-d8ffecd9328a","html_url":"https://github.com/FrozenAssassine/NeuralNetwork-FromScratch","commit_stats":null,"previous_names":["frozenassassine/neuralnetwork-fromscratch"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FrozenAssassine%2FNeuralNetwork-FromScratch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FrozenAssassine%2FNeuralNetwork-FromScratch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FrozenAssassine%2FNeuralNetwork-FromScratch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FrozenAssassine%2FNeuralNetwork-FromScratch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FrozenAssassine","download_url":"https://codeload.github.com/FrozenAssassine/NeuralNetwork-FromScratch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224368207,"owners_count":17299662,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","classification","csharp","cuda","gpu","gpu-acceleration","neural-network","neural-networks","nvidia"],"created_at":"2024-11-13T00:26:13.631Z","updated_at":"2026-02-20T15:31:57.828Z","avatar_url":"https://github.com/FrozenAssassine.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!--\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"path_to_your_logo\" height=\"150px\" width=\"auto\" alt=\"Neural Network Logo\"\u003e\n\u003c/p\u003e\n--\u003e\n\n\u003ch1 align=\"center\"\u003eNeural Network from scratch with CUDA Support\u003c/h1\u003e\n\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/stars/FrozenAssassine/NeuralNetwork-FromScratch?style=flat\"/\u003e\n    \u003cimg src=\"https://img.shields.io/github/issues-pr/FrozenAssassine/NeuralNetwork-FromScratch?style=flat\"/\u003e\n    \u003cimg src=\"https://img.shields.io/github/repo-size/FrozenAssassine/NeuralNetwork-FromScratch?style=flat\"/\u003e\n\u003c/div\u003e\n\n## 🤔 What is this project?\nThis project is a neural network implementation from scratch in C# with CUDA support written in C++. It currently supports Optical Digit Recognition (ODR) trained with 60,000 images and can also perform XOR as a simple initial test.\nI also tried more complex image recognition using a cnn, but I was not able to implement it from scratch.\n\n## ❗Info\nAt the current point I would not recommend this in any production environment, for me it's just a fun project to learn more about CUDA and Neural Networks.\nAlso I tried to implement Convolution and Pooling layer from scratch, but failed in the back propagation. Currently they are not working in any way😢\n\n\n## 🛠️ Features\n- **Optical Digit Recognition (ODR)**: Trained with the MNIST dataset of 60,000 images.\n- **XOR Test**: A simple test to demonstrate the neural network's basic functionality.\n- **CUDA Support**: Accelerates neural network training using GPU resources.\n- **CUDA or CPU**: Simply switch between CUDA or CPU processing.\n\n## 📎See also \n- [Deep reinforcement learning](https://github.com/FrozenAssassine/DeepReinforcementLearning) from scratch using this project\n- [ESP32 \u0026 Arduino](https://github.com/FrozenAssassine/NeuralNetwork-Arduino) running XOR-Demo with a simplified version of this project\n\n- [Interactive Demo](https://frozenassassine.de/nn/xor?ref=github) on my website\n\n## 📊 Benchmarks\n| Training Details | GPU (CUDA, RTX 3050) | CPU (i9-10900) \n|------------------|----------------------|----------------|\n| 54000 images, 28x28x1 (784 inputs, 512 dense, 256 dense, 10 outputs) | 13.813 sec | 44.001 sec\n\n## 🚀 Performance History\n### Sequential to true Parallel 📈 ...\n\nThe initial Optical Digit Recognition (ODR) implementation, using 28x28 black-and-white images as input with a neural network consisting of 128 and 64 hidden neurons and 10 output neurons, took 2.8 seconds to train on 1000 images.  \nTo improve performance, I added **Parallel.For** support, which accelerated the training process. Enabling Release mode further optimized the training time, reducing it to around 780ms for 1000 images.   \nHowever, this was not sufficient. I began integrating CUDA support, which proved challenging but significantly reduced the training time. With CUDA, I brought the training time down to 400ms for 1000 images. In the latest build, I achieved a training time of approximately 200ms per 1000 images.   \nOverall, this resulted in a 10 times increase in performance.\n\n\n## 🏗️ Get Started\n1. Clone the repository.\n2. Ensure you have the necessary dependencies for C# and CUDA development.\n   (https://developer.nvidia.com/cuda-downloads)\n4. Open the solution file (`.sln`) in Visual Studio.\n5. Build and run the project.\n\n## Example code\n```cs\n//XOR prediction\nvar nnmodel = NetworkBuilder.Create()\n    .Stack(new InputLayer(2))\n    .Stack(new DenseLayer(4, ActivationType.Sigmoid))\n    .Stack(new OutputLayer(1, ActivationType.Sigmoid))\n    .Build(true); //set to false to train on CPU\n\nnnmodel.Summary();\n\nfloat[][] inputs = new float[][] { new float[] { 0, 0 }, new float[] { 0, 1 }, new float[] { 1, 0 }, new float[] { 1, 1 } };\nfloat[][] desired = new float[][] { new float[] { 0 }, new float[] { 1 }, new float[] { 1 }, new float[] { 0 } };\nnnmodel.Train(inputs, desired, 15900, 0.01f, 1000, 100);\n\nvar prediction = nnmodel.Predict(new float[] { 0, 0 });\nConsole.WriteLine(\"Prediction: \" + MathHelper.GetMaximumIndex(prediction));\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrozenassassine%2Fneuralnetwork-fromscratch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrozenassassine%2Fneuralnetwork-fromscratch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrozenassassine%2Fneuralnetwork-fromscratch/lists"}