{"id":20182509,"url":"https://github.com/hunar4321/rls-neural-net","last_synced_at":"2025-10-29T11:02:09.778Z","repository":{"id":159811232,"uuid":"287931521","full_name":"hunar4321/RLS-neural-net","owner":"hunar4321","description":"Recursive Leasting Squares (RLS) with Neural Network for fast learning","archived":false,"fork":false,"pushed_at":"2023-11-16T14:07:28.000Z","size":522,"stargazers_count":53,"open_issues_count":3,"forks_count":9,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-24T06:22:26.436Z","etag":null,"topics":["artificial-intelligence","kalman-filter","machine-learning","multiple-regression","neural-network","oneshot-learning","oneshotlearning","recursive-least-squares","regression-analysis"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hunar4321.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-08-16T11:36:20.000Z","updated_at":"2025-01-20T12:01:01.000Z","dependencies_parsed_at":"2023-07-03T12:13:00.481Z","dependency_job_id":"353ad611-92d8-4601-acf4-d1e0e63b80ef","html_url":"https://github.com/hunar4321/RLS-neural-net","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hunar4321%2FRLS-neural-net","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hunar4321%2FRLS-neural-net/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hunar4321%2FRLS-neural-net/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hunar4321%2FRLS-neural-net/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hunar4321","download_url":"https://codeload.github.com/hunar4321/RLS-neural-net/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248161273,"owners_count":21057555,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","kalman-filter","machine-learning","multiple-regression","neural-network","oneshot-learning","oneshotlearning","recursive-least-squares","regression-analysis"],"created_at":"2024-11-14T02:38:54.837Z","updated_at":"2025-10-29T11:02:09.712Z","avatar_url":"https://github.com/hunar4321.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n![GitHub repo size](https://img.shields.io/github/repo-size/hunar4321/RLS_Learning)\n![GitHub](https://img.shields.io/github/license/hunar4321/RLS_Learning)\n\n# 1. Recursive Least Squares by predicting errors\nThis is a simple intuitive method to solve linear equations using recursive least squares\n\nCheckout the step by step video tutorial here: https://youtu.be/4vGaN1dTVhw\n\n------------\n\n#### Illustration - RLS Error Prediction:\n\n![](images/rls_figure.jpg)\n\n#### Comparision between how errors are shared among the inputs in Gradient based methods vs. RLS based methods\n![](images/comparision.jpg)\n\nAlgorithmically, this method is faster than matrix inversion due to the fewer operations required. However, in practice, it's hard to fairly compare this method with the already established linear solvers because many optimalization tricks have been done at the level of the hardware for matrix operations. We added a simple C++ implementation using Eigen library to compare the performance of this method to the matrix inversion method.\n\nInspired from the following post by whuber: https://stats.stackexchange.com/q/166718\n\n# 2. Fast Learning in Neural Networks (Real time optimization)\n-----------------------------------\nThere is an example usage at the end of *RLS_Neural_Network.py* which showcases how this network can learn XOR data in a single iteration. Run the code and see the output.\n\n**Advantages of using RLS for learning instead of gradient descent**\n1. Fast learning and sample efficiency (can learn in one-shot).\n2. Online Learning (suitable for real time learning).\n3. No worries about local minima.\n\n**Disadvantages:**\n1. Computationally inefficient if the size of the input is big (Quadratic Complexity).\n2. Sensitive to overflow and underflow and this can lead to unstability in some cases\n3. The current implementation works with a single hidden unit neural network. It is not clear if adding more layers will be useful since learning only happens in the last layer \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhunar4321%2Frls-neural-net","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhunar4321%2Frls-neural-net","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhunar4321%2Frls-neural-net/lists"}