{"id":15158298,"url":"https://github.com/pejpero/neural_network_regression_and_classification","last_synced_at":"2026-02-09T12:03:29.437Z","repository":{"id":253581622,"uuid":"843433175","full_name":"PejperO/Neural_Network_Regression_and_Classification","owner":"PejperO","description":"This repository contains neural network regression models built from scratch and using Keras for comparison. It visualizes training and testing performance, analyzing MSE, R², and decision boundaries. The project demonstrates learning techniques and optimization for regression tasks.","archived":false,"fork":false,"pushed_at":"2024-08-23T12:13:26.000Z","size":1892,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-13T18:21:35.759Z","etag":null,"topics":["data-visualization","keras","machine-learning","neural-network","regression"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PejperO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-16T13:59:53.000Z","updated_at":"2024-08-23T12:13:29.000Z","dependencies_parsed_at":"2024-08-23T13:46:56.728Z","dependency_job_id":null,"html_url":"https://github.com/PejperO/Neural_Network_Regression_and_Classification","commit_stats":null,"previous_names":["pejpero/neural_network_regression_and_classification"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PejperO%2FNeural_Network_Regression_and_Classification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PejperO%2FNeural_Network_Regression_and_Classification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PejperO%2FNeural_Network_Regression_and_Classification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PejperO%2FNeural_Network_Regression_and_Classification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PejperO","download_url":"https://codeload.github.com/PejperO/Neural_Network_Regression_and_Classification/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247684480,"owners_count":20979072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-visualization","keras","machine-learning","neural-network","regression"],"created_at":"2024-09-26T20:42:30.285Z","updated_at":"2026-02-09T12:03:24.395Z","avatar_url":"https://github.com/PejperO.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Neural Network Regression and Classification - A Comparative Study\n\nThis repository contains a collection of Python scripts that implement neural network models for both regression and classification tasks using custom-built neural networks and Keras-based networks. The goal of this project is to explore the use of neural networks for different types of data and tasks, examining the performance through key metrics and visualizations.\n\n## Overview\n### Files Included:\n1. **4.a.fa.py:** A custom-built neural network implemented from scratch for regression using Sigmoid activation functions.\n2. **4.a.py:** A similar custom-built network but using ReLU as the activation function.\n3. **4.b.I.py:** A regression neural network built using the Keras Functional API, with early stopping and two hidden layers.\n4. **4.b.II.py:** A classification neural network built using Keras to solve the classic moons dataset problem.\n\n## Goals of the Project:\n- Compare different activation functions (Sigmoid vs. ReLU) in a custom neural network for regression.\n- Utilize Keras API to implement a more optimized neural network with early stopping for regression tasks.\n- Train a neural network for a binary classification task using Keras and visualize decision boundaries.\n- Understand the training behavior through visualizations like MSE, R², and accuracy during training.\n\n## Running the Scripts\n- **4.a.fa.py** and **4.a.py:** These scripts train a custom-built neural network for a regression task. The input is expected from a txt file located in a folder called Dane (the dataset).\n- **4.b.I.py:** A regression model built using Keras, including early stopping to prevent overfitting. The model uses two hidden layers.\n- **4.b.II.py:** A binary classification task (Moons dataset) using Keras, with visualization of decision boundaries.\n\n## Key Results and Insights\n### Activation Functions: Sigmoid vs ReLU\n- 4.a.fa.py (Sigmoid Activation): The network tends to saturate for large values and struggles with gradient issues. The MSE and R² plots show slower convergence compared to ReLU.\n- 4.a.py (ReLU Activation): The network shows faster convergence in both MSE and R² metrics. The ReLU activation function avoids the vanishing gradient problem seen with Sigmoid and provides more accurate predictions, as demonstrated in the test vs. prediction scatter plots.\n\n### Keras-Based Models (Regression and Classification)\n- 4.b.I.py (Keras Regression): With the addition of early stopping, the model reaches its optimal performance early, preventing overfitting. The MSE and R² values show significant improvement over custom-built models. The plots reflect smoother convergence with validation data.\n- 4.b.II.py (Keras Classification): The neural network successfully learns the decision boundary of the moon-shaped data. The accuracy plots show strong performance, with decision boundary visualizations reflecting how well the model separates the classes.\n\n## Visualizations\nEach script generates visualizations that track the model’s performance over time, including:\n\n- Mean Squared Error (MSE) vs. Epochs: Demonstrates how the error decreases during training.\n- R² Score vs. Epochs: Shows the improvement in fit quality.\n- Data Scatter Plots with Predictions: Compares the true data against predictions, showing how well the model generalizes.\n- Classification Decision Boundaries: In the classification model, the decision boundary shows how well the model classifies the dataset.\n\n![DataSet_3](https://github.com/user-attachments/assets/61d19f86-9576-4229-b5d8-bee6268e86b8)\n![DataSet_9](https://github.com/user-attachments/assets/f2ebbbc8-57ab-4325-a351-0259f9ce9182)\n![DataSet_4](https://github.com/user-attachments/assets/0982012b-a3aa-4302-a483-7f69082b6635)\n![DataSet_7](https://github.com/user-attachments/assets/f0726f8d-0179-4842-833b-4970aca5b79a)\n\n## Conclusions\nFrom these visualizations, we can conclude the following:\n\n1. Activation Functions: ReLU generally performs better than Sigmoid for regression tasks in terms of speed and accuracy.\n2. Early Stopping: The use of early stopping in the Keras model prevents overfitting and ensures the model does not train for unnecessary epochs.\n3. Binary Classification: Keras efficiently handles classification tasks with a relatively simple network, providing clear decision boundaries and high accuracy.\n\n## What I Learned\n- **Activation Functions Matter:** The choice of activation function significantly affects the training dynamics of neural networks. Sigmoid suffers from vanishing gradients, while ReLU is much more efficient for deeper models.\n- **Custom vs. Framework-Based Models:** Writing neural networks from scratch is a great learning experience, but using frameworks like Keras simplifies development and optimization, allowing for more complex architectures with less code.\n- **Early Stopping and Optimization:** Incorporating callbacks such as early stopping helps in achieving better model performance and prevents overfitting, especially when dealing with noisy or smaller datasets.\n- **Visualization is Key:** Visualizing the training process is crucial to understanding how well the model is learning and identifying potential issues such as overfitting or underfitting.\n\n## License\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpejpero%2Fneural_network_regression_and_classification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpejpero%2Fneural_network_regression_and_classification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpejpero%2Fneural_network_regression_and_classification/lists"}