{"id":22547476,"url":"https://github.com/matsjfunke/feedforward-neural-network-from-scratch","last_synced_at":"2025-04-10T01:12:51.197Z","repository":{"id":245187603,"uuid":"810520827","full_name":"matsjfunke/feedforward-neural-network-from-scratch","owner":"matsjfunke","description":"Fully Connected Neural Network (FCNN) from scratch in python \u0026 Notes to aid understanding the workings of neural networks","archived":false,"fork":false,"pushed_at":"2025-01-10T03:28:40.000Z","size":31489,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-10T01:12:25.618Z","etag":null,"topics":["backpropagation","forward-propagation","gradient-descent","mnist","neural-network","numpy","pandas","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matsjfunke.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-04T21:22:40.000Z","updated_at":"2025-01-10T03:28:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"c12e652b-2f7d-4524-a5c7-1ecd895e4cc9","html_url":"https://github.com/matsjfunke/feedforward-neural-network-from-scratch","commit_stats":null,"previous_names":["matsjfunke/feedforward-neural-networks","matsjfunke/feedforward-neural-network-from-scratch"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matsjfunke%2Ffeedforward-neural-network-from-scratch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matsjfunke%2Ffeedforward-neural-network-from-scratch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matsjfunke%2Ffeedforward-neural-network-from-scratch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matsjfunke%2Ffeedforward-neural-network-from-scratch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matsjfunke","download_url":"https://codeload.github.com/matsjfunke/feedforward-neural-network-from-scratch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248137891,"owners_count":21053775,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backpropagation","forward-propagation","gradient-descent","mnist","neural-network","numpy","pandas","python"],"created_at":"2024-12-07T15:11:59.954Z","updated_at":"2025-04-10T01:12:51.188Z","avatar_url":"https://github.com/matsjfunke.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Feedforward Neural Network on MNIST dataset from scratch\n\n## check out my notes on:\n**forward propagation** (computing the output of a neural network):\n- [Single Neurons \u0026 activation functions](./notes/single-neuron.pdf)\n- [Neural Networks (multi-layer perceptron) \u0026 walkthrough handwirting interpretation](./notes/neural-networks.pdf)\n\n**back propagation** (updating the network's weights by correcting its mistakes):   \n- [learning (hill-climb \u0026 gradient descent](./notes/learning-optimization.pdf)\n- [back propagation algorithm](./notes/backpropagation.pdf)\n\n# implementing  Neural Network:\nThis project implements a simple neural network to classify handwritten numbers from the mnist dataset. \nYou can define the number of layers, neurons per layer, activation functions, and other training parameters via command-line arguments.\n\nThe following libraries are used:\n- zipfile: For extracting dataset files.\n- numpy: For numerical computations.\n- pandas: For data manipulation.\n- matplotlib.pyplot: For plotting images.\n- sklearn.metrics: For evaluating the model's performance.\n\n## Workflow\n**Dataset Loading**\n- Extract the dataset from a ZIP file and load it into a Pandas DataFrame.\n- Convert the DataFrame to a NumPy array and prepare the training and test datasets.\n\n**Dataset Preparation**\n- Shuffle the dataset and split it into training and test sets.\n- Normalize the features to ensure pixel values are between 0 and 1.\n\n**Initialize Parameters**\n- Define a function to initialize random weights and biases for each layer in the network.\n\n**Activation Functions**\n- Implement the ReLU and Softmax activation functions.\n\n**Forward Propagation**\n- Define a function to perform forward propagation through the network using the activation functions.\n\n**Cost Function Calculation**\n- Implement a cost function to calculate the cross-entropy loss between predictions and true labels.\n\n**Backward Propagation**\n- Define a function to perform backward propagation to compute gradients of the cost function with respect to the parameters.\n\n**Parameter Update**\n- Implement gradient descent to update the parameters using the computed gradients.\n\n**Train Network**\n- Define a function to train the neural network using the training data, specified architecture, learning rate, and tolerance for convergence.\n\n**Predict**\n- Implement a function to predict labels for given input features using the trained network.\n\n**Predict Single Example**\n- Define a function to predict the label for a single input sample.\n- Implement a function to visualize an input image and compare the true label with the predicted label.\n\n## Usage\n\n```sh\n# install dependencies\npip install -r requirements.txt\n# get to know the dataset\npython visualize_mnist.py\n# train network with default parameters\npython neural_network.py\n# train neural_network with specific parameters (don't specify to use the default option)\npython neural_network.py --layers \u003clayer_sizes\u003e --activations \u003cactivation_functions\u003e --learning_rate \u003clearning_rate\u003e --max_iterations \u003citerations\u003e --tolerance \u003ctolerance\u003e --train_file \u003ctrain_file\u003e --test_file \u003ctest_file\u003e\n```\n\n### Arguments\n\n- `--layers`: Neurons per layer (default: `128 64 10`).\n- `--activations`: Activation functions per layer (default: `relu relu softmax`).\n- `--learning_rate`: Learning rate for training (default: `0.01`).\n- `--max_iterations`: Max training iterations (default: `200`).\n- `--tolerance`: Cost change tolerance for stopping (default: `0.001`).\n- `--train_file`: Path to the ZIP file with the training CSV (default: `./input/train.csv.zip`).\n- `--test_file`: Path to the ZIP file with the test CSV (default `./input/test.csv.zip`).\n\n### Example\n\n```bash\npython neural_network.py --layers 128 64 10 --activations relu relu softmax --learning_rate 0.01 --max_iterations 200 --tolerance 0.001 --train_file ./data/train.csv.zip --test_file ./data/test.csv.zip\n```\nthe output should look like:\n![output](./train_test_model_output.png)\n\nhave fun tinkering :)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatsjfunke%2Ffeedforward-neural-network-from-scratch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatsjfunke%2Ffeedforward-neural-network-from-scratch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatsjfunke%2Ffeedforward-neural-network-from-scratch/lists"}