{"id":23914836,"url":"https://github.com/alessiochen/multi-class-logistic-regression-with-optimization-methods","last_synced_at":"2025-02-23T18:44:49.551Z","repository":{"id":233338357,"uuid":"786494493","full_name":"AlessioChen/Multi-Class-Logistic-Regression-with-Optimization-Methods","owner":"AlessioChen","description":"Implementation of Trust Region and Gradient Descent methods for Multinomial Regression","archived":false,"fork":false,"pushed_at":"2024-05-15T18:46:41.000Z","size":3472,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-05T10:31:43.376Z","etag":null,"topics":["armijo","armijo-condition","cauchy-point","line-search","mlr","multiclass-logistic-regression","optimization","optimization-algorithms","trust-region","trust-region-optimization"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlessioChen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-14T16:28:53.000Z","updated_at":"2024-05-16T06:33:11.000Z","dependencies_parsed_at":"2025-01-05T10:38:38.581Z","dependency_job_id":null,"html_url":"https://github.com/AlessioChen/Multi-Class-Logistic-Regression-with-Optimization-Methods","commit_stats":null,"previous_names":["alessiochen/project-work-in-optmization-techniques-for-machine-learning","alessiochen/multi-class-logistic-regression-with-optimization-methods"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlessioChen%2FMulti-Class-Logistic-Regression-with-Optimization-Methods","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlessioChen%2FMulti-Class-Logistic-Regression-with-Optimization-Methods/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlessioChen%2FMulti-Class-Logistic-Regression-with-Optimization-Methods/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlessioChen%2FMulti-Class-Logistic-Regression-with-Optimization-Methods/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlessioChen","download_url":"https://codeload.github.com/AlessioChen/Multi-Class-Logistic-Regression-with-Optimization-Methods/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240364298,"owners_count":19789756,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["armijo","armijo-condition","cauchy-point","line-search","mlr","multiclass-logistic-regression","optimization","optimization-algorithms","trust-region","trust-region-optimization"],"created_at":"2025-01-05T10:28:33.248Z","updated_at":"2025-02-23T18:44:49.510Z","avatar_url":"https://github.com/AlessioChen.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"#  Multi-Class Logstic Regression with Optmization Methods\n\nThis repository contains Python code for implementing multi-class logistic regression using gradient descent with Armijo line search and trust region optimization methods with cauchy step. \n\n\n\n## Introduction \nMulti-class logistic regression is a popular technique for classification problems where the target variable can take multiple classes. This repository provides an implementation of multi-class logistic regression along with two optimization methods:\n\n- **Gradient Descent with Armijo Line Search**: This method optimizes the model parameters by iteratively updating them based on the gradient of the loss function, with step size determined using the Armijo line search algorithm.\n- **Trust Region Algorithm:**  This method iteratively minimizes the loss function while considering a trust region around the current solution. It utilizes Taylor's second-order model and the Cauchy step to update the model parameters within the trust region.\n\n## Implementation \nThe implementation is provided in Python using NumPy for numerical computations. The main components of the implementation include:\n\n- MultiClassLogisticRegression class: Implements multi-class logistic regression with methods for training the model using gradient descent and trust region optimization.\n- Functions for computing loss, gradient, Hessian matrix, Armijo line search, and Cauchy step.\n\n## Experimantation \nThis repository includes experimentation on a benchmark of multi-class classification datasets. It measures the training loss decrease and runtime for both optimization methods. Additionally, it reports the accuracy of the final model solutions.\n\n## Datasets\nSmall set of real-world dataset for multi-class classification taska are used. These dataset are directy imported from sklearn datasets or they are download from [LIBSVM](https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/) and imported in the datasets directory.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falessiochen%2Fmulti-class-logistic-regression-with-optimization-methods","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falessiochen%2Fmulti-class-logistic-regression-with-optimization-methods","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falessiochen%2Fmulti-class-logistic-regression-with-optimization-methods/lists"}