{"id":25819008,"url":"https://github.com/pngo1997/neural-network-implementation-evaluation","last_synced_at":"2026-05-10T12:16:54.804Z","repository":{"id":229206611,"uuid":"776118880","full_name":"pngo1997/Neural-Network-Implementation-Evaluation","owner":"pngo1997","description":"Implements the NNDL book's \"network.py\" code (Chapter 1) to improve its functionality and performance.","archived":false,"fork":false,"pushed_at":"2025-01-31T20:53:13.000Z","size":672,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-31T21:26:41.028Z","etag":null,"topics":["back-propagation","cross-entropy","epoch","hyperparameter-optimization","hyperparameter-tuning","log-likelihood","mini-batch-gradient-descent","neural-network","python","sgd"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pngo1997.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-22T18:07:34.000Z","updated_at":"2025-01-31T20:53:17.000Z","dependencies_parsed_at":"2025-01-31T21:36:48.279Z","dependency_job_id":null,"html_url":"https://github.com/pngo1997/Neural-Network-Implementation-Evaluation","commit_stats":null,"previous_names":["pngo1997/neural-networks-practices","pngo1997/neural-network-implementation-evaluation"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pngo1997%2FNeural-Network-Implementation-Evaluation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pngo1997%2FNeural-Network-Implementation-Evaluation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pngo1997%2FNeural-Network-Implementation-Evaluation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pngo1997%2FNeural-Network-Implementation-Evaluation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pngo1997","download_url":"https://codeload.github.com/pngo1997/Neural-Network-Implementation-Evaluation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241122313,"owners_count":19913455,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["back-propagation","cross-entropy","epoch","hyperparameter-optimization","hyperparameter-tuning","log-likelihood","mini-batch-gradient-descent","neural-network","python","sgd"],"created_at":"2025-02-28T08:14:22.346Z","updated_at":"2026-05-10T12:16:49.576Z","avatar_url":"https://github.com/pngo1997.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🧠 Neural Network Implementation \u0026 Evaluation  \n\n## 📜 Overview  \nThis project involves modifying the **NNDL book's \"network.py\"** code (Chapter 1) to improve its functionality and performance. The goal is to deepen the understanding of neural networks by implementing **evaluation metrics, efficiency improvements, and visualization techniques**.  \n\n## 🎯 Objectives  \n- Modify the **network definition code** to include additional evaluation metrics.  \n- Optimize **SGD, backpropagation, and mini-batch updates** for efficiency.  \n- Implement **early stopping** based on accuracy and MSE loss trends.  \n- Develop a **custom test application** to validate the modified network.  \n- Visualize **training performance metrics** (Accuracy, MSE, CE, and Log-Likelihood).  \n\n📌 **Dataset**: The **Iris dataset** (iris-3.csv) is used for training and testing.  \n\n## ⚙️ Installation \u0026 Dependencies  \n### **Requirements**  \n- **Python 3.x**  \n- **Jupyter Notebook**  \n- **Required Libraries**:  \n  - `numpy` – Matrix computations  \n  - `matplotlib` – Data visualization  \n  - `pandas` – Dataset handling  \n  - `scipy` – Scientific computations  \n\n## 🔍 Project Structure\n### 1️⃣ Initial Setup \u0026 Testing\n📌 **Objective**: Ensure environment compatibility and run baseline tests.\n\n### 2️⃣ Modifications to the Network Code\n📌 **Objective**: Extend the existing evaluate(), SGD(), backprop(), and update_mini_batch() functions.\n\n✅ **Key Modifications**:\n- **evaluate() function**: Compute and return\n  - ✅ Correctly classified instances (Count)\n  - ✅ Accuracy (Accuracy)\n  - ✅ Mean Squared Error (MSE)\n  - ✅ Cross-Entropy (CE)\n  - ✅ Log-Likelihood (LL)\n- **SGD() function**:\n  - Include **test data evaluation** in a similar format.\n  - Implement **early stopping** if accuracy reaches **1.0** or **MSE stops decreasing** (checks last three epochs for stagnation).\n  - Return **training and test performance metrics** as nested lists [train_results, test_results].\n- **backprop() function**:\n  - Preallocate activation matrices for **all layers** to improve efficiency.\n- **update_mini_batch() function**:\n  - Optimize weight updates by accumulating gradients **without redundant memory allocations.**\n\n### 3️⃣ Testing \u0026 Validation\n📌 **Objective**: Verify the correctness of the modified network.\n\n✅ **Steps**:\n- Run the second test application.\n- Ensure it works with a **deeper network** (iris4-20-7-3.dat).\n\n### 4️⃣ Custom Test Application \u0026 Visualization\n📌 **Objective**: Implement and visualize a new test case.\n- Given graphs here are using eta = 0.315, so eta 0.3 should yields consistent results on the performances of both train and test sets. From the graphs, Accuracy increases over time, at epoch 100, trainset yields 98.10% and test set yields 93.33%. Mean-Squared Error, Cross Entropy and Log Likelihood are decreasing over time and both train and test sets.\n- Around epoch 60th forward, errors on testing data are slightly higher than training data but not significant. The two lines are almost overlap indicates there is no underfitting/overfitting of the data using eta approximately 0.3. The neural network is performing well on unseen data, correctly classifying the three Iris species.\n![Neural Network](https://github.com/pngo1997/Images/blob/main/Neural%20Network.png)  \n\n## 📊 Results \u0026 Findings\n✅ Successfully extended the evaluation metrics (Accuracy, MSE, CE, LL).\n\n✅ Optimized backpropagation and mini-batch processing for better efficiency.\n\n✅ Implemented early stopping based on accuracy and loss trends.\n\n✅ Verified correctness using multiple test cases.\n\n✅ Created visualizations for training progress.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpngo1997%2Fneural-network-implementation-evaluation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpngo1997%2Fneural-network-implementation-evaluation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpngo1997%2Fneural-network-implementation-evaluation/lists"}