{"id":24717988,"url":"https://github.com/steq28/polynomial-regression","last_synced_at":"2025-03-22T10:29:33.799Z","repository":{"id":257992325,"uuid":"868548120","full_name":"steq28/polynomial-regression","owner":"steq28","description":"Polynomial Regression with Gradient Descent using PyTorch. This repo includes functions for plotting polynomials, generating noisy datasets, and training a linear regression model. Features visualizations for loss tracking and coefficient comparison.","archived":false,"fork":false,"pushed_at":"2024-10-15T21:10:22.000Z","size":1128,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-27T10:12:57.959Z","etag":null,"topics":["gradient-descent","polynomial-regression"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/steq28.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-06T16:59:51.000Z","updated_at":"2025-01-15T18:00:44.000Z","dependencies_parsed_at":"2024-10-18T02:50:20.560Z","dependency_job_id":null,"html_url":"https://github.com/steq28/polynomial-regression","commit_stats":null,"previous_names":["steq28/polynomial-regression"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steq28%2Fpolynomial-regression","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steq28%2Fpolynomial-regression/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steq28%2Fpolynomial-regression/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steq28%2Fpolynomial-regression/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/steq28","download_url":"https://codeload.github.com/steq28/polynomial-regression/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244943083,"owners_count":20536153,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gradient-descent","polynomial-regression"],"created_at":"2025-01-27T10:13:00.565Z","updated_at":"2025-03-22T10:29:33.776Z","avatar_url":"https://github.com/steq28.png","language":"Python","readme":"# Polynomial Regression with Gradient Descent\n\nThis repository contains Python code for performing polynomial regression using gradient descent, implemented with PyTorch, as required for **Assignment 1**. The code provides functions for polynomial plotting, dataset generation, model training, and visualization.\n\n## Assignment Overview\n\nThe task involves performing polynomial regression on a dataset generated from a given polynomial. The regression is done using gradient descent with PyTorch's `torch.nn.Linear` model to estimate the coefficients of the polynomial. The steps include:\n\n-   Plotting the polynomial.\n-   Generating noisy datasets for training and evaluation.\n-   Training a linear regression model using gradient descent.\n-   Visualizing the model's performance and results.\n\n## Features\n\n-   **Polynomial Plotting**: A function to visualize any polynomial by defining its coefficients and plotting range.\n-   **Dataset Generation**: Create noisy datasets for training and validation using polynomial values with added Gaussian noise.\n-   **Polynomial Regression Model**: Train a neural network to learn the polynomial's coefficients using gradient descent.\n-   **Loss Visualization**: Track the loss function over time to observe model convergence.\n-   **Parameter Tracking**: Visualize how the model's parameters (coefficients) converge towards the true polynomial values.\n\n## Requirements\n\nMake sure you have the following Python packages installed:\n\n-   `matplotlib`\n-   `numpy`\n-   `torch`\n\nYou can install them using:\n\n```bash\npip install matplotlib numpy torch\n```\n\n## Neural Network Model for Polynomial Regression\n\nIn this project, we perform **polynomial regression** using a simple linear regression model in PyTorch. While the polynomial is of degree 4, the regression model estimates the coefficients by mapping the inputs to the polynomial's features. Here is how the model is structured:\n\n### Model Definition\n\nWe use PyTorch's `torch.nn.Linear` module, which implements a basic fully connected layer (linear transformation). In this case, it is used to map the polynomial features (`z`, `z²`, `z³`, `z⁴`) to the target output (`y`).\n\n-   **Input features**: The model receives a vector `x = [1, z, z², z³, z⁴]`, where `z` is the independent variable.\n-   **Output**: The model outputs a scalar `y_hat`, which is the predicted value based on the learned coefficients.\n\n### Model Architecture\n\nThe architecture is a single-layer linear model, defined as:\n\n```python\nclass PolynomialRegressionModel(nn.Module):\n    def __init__(self):\n        super(PolynomialRegressionModel, self).__init__()\n        self.linear = nn.Linear(in_features=5, out_features=1, bias=True)  # Linear layer\n\n    def forward(self, x):\n        return self.linear(x)\n```\n\n-   **Input**: A tensor of shape `(N, 5)`, where `N` is the batch size and `5` represents the 5 input features `[1, z, z², z³, z⁴]`.\n-   **Output**: A tensor of shape `(N, 1)`, representing the predicted values.\n\n### Training Process\n\nWe use **gradient descent** to minimize the **mean squared error (MSE)** between the predicted values (`y_hat`) and the true values (`y`). The training process involves iterating over the dataset and updating the model's parameters (coefficients) to reduce the loss function.\n\n#### Steps:\n\n1. **Initialize the Model**:\n    ```python\n    model = PolynomialRegressionModel()\n    ```\n2. **Define the Loss Function**: We use MSELoss for regression.\n    ```python\n    criterion = nn.MSELoss()\n    ```\n3. **Define the Optimizer**: We use the `Adam` optimizer to perform gradient-based optimization.\n    ```python\n    optimizer = torch.optim.Adam(model.parameters(), lr=0.01)\n    ```\n4. **Training Loop**: In each iteration, we:\n    - Perform a forward pass to compute predictions.\n    - Calculate the loss (error).\n    - Backpropagate the error and update the model's parameters.\n    - Track the loss over time to ensure the model is converging.\n\n### Polynomial Features\n\nTo perform polynomial regression, we transform the original data points `z` into polynomial features `[1, z, z², z³, z⁴]`. This allows us to fit a higher-degree polynomial using linear regression.\n\n### Learning Rate and Convergence\n\nThe **learning rate** controls how large the parameter updates are at each step of gradient descent. If the learning rate is too high, the model might overshoot the minimum of the loss function, causing instability. If it is too small, the model will take a long time to converge. During the training process, different learning rates are experimented with to find the optimal value.\n\n## Functions\n\n### 1. `plot_polynomial(coeffs: np.array, z_range: Tuple[float, float], color='b')`\n\nPlots a polynomial based on the given coefficients and range.\n\n**Parameters:**\n\n-   `coeffs`: A numpy array of coefficients `[w0, w1, w2, w3, w4]`.\n-   `z_range`: The interval `[z_min, z_max]` for the `z` variable.\n-   `color`: The color of the plot (default: blue).\n\n### 2. `create_dataset(coeffs: np.array, z_range: Tuple[float, float], sample_size: int, sigma: float, seed: int = 42)`\n\nGenerates a noisy dataset based on the polynomial defined by the coefficients.\n\n**Parameters:**\n\n-   `coeffs`: Polynomial coefficients.\n-   `z_range`: Range of `z` values.\n-   `sample_size`: Number of data points to generate.\n-   `sigma`: Standard deviation of noise added to the data.\n-   `seed`: Seed for random number generation (default: 42).\n\nReturns two `torch.tensor` objects representing the dataset.\n\n### 3. `visualize_data(X, y, coeffs, z_range, title=\"\")`\n\nPlots the true polynomial and the generated data (training or validation). Includes scatter plots for visualizing the dataset.\n\n**Parameters:**\n\n-   `X`, `y`: Dataset generated by `create_dataset`.\n-   `coeffs`: Polynomial coefficients.\n-   `z_range`: Range of `z` values.\n-   `title`: Title of the plot.\n\n### 4. **Polynomial Regression Using Gradient Descent**\n\nThis step trains a model to fit the generated dataset using gradient descent in PyTorch. The training loop involves:\n\n-   Forward pass to compute the predicted polynomial values.\n-   Calculation of the loss (MSE).\n-   Backpropagation to compute gradients and update model parameters.\n\n## Usage\n\n1. **Plot a Polynomial:**\n\n```python\nfrom main import plot_polynomial\nplot_polynomial(np.array([1, -1, 5, -0.1, 1/30]), (-4, 4))\n```\n\n2. **Generate a Dataset:**\n\n```python\nfrom main import create_dataset\ntrain_data, train_labels = create_dataset(np.array([1, -1, 5, -0.1, 1/30]), (-2, 2), 500, sigma=0.5, seed=0)\n```\n\n3. **Visualize Data:**\n\n```python\nfrom main import visualize_data\nvisualize_data(train_data, train_labels, np.array([1, -1, 5, -0.1, 1/30]), (-2, 2), title=\"Training Data\")\n```\n\n4. **Train the Model**:\n\n```python\n# Initialize the model\nmodel = PolynomialRegressionModel()\n\n# Define loss and optimizer\ncriterion = nn.MSELoss()\noptimizer = torch.optim.Adam(model.parameters(), lr=0.01)\n\n# Training loop\nfor epoch in range(num_epochs):\n    # Forward pass, loss calculation, backward pass, optimizer step\n    pass  # Example loop\n```\n\n## License\n\nThis project is licensed under the MIT License.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsteq28%2Fpolynomial-regression","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsteq28%2Fpolynomial-regression","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsteq28%2Fpolynomial-regression/lists"}