{"id":13415618,"url":"https://github.com/unslothai/hyperlearn","last_synced_at":"2025-10-04T04:58:14.722Z","repository":{"id":39412752,"uuid":"146321178","full_name":"unslothai/hyperlearn","owner":"unslothai","description":"2-2000x faster ML algos, 50% less memory usage, works on all hardware - new and old.","archived":false,"fork":false,"pushed_at":"2024-11-19T02:09:54.000Z","size":48965,"stargazers_count":2294,"open_issues_count":2,"forks_count":148,"subscribers_count":94,"default_branch":"master","last_synced_at":"2025-10-02T15:16:06.067Z","etag":null,"topics":["data-analysis","data-science","deep-learning","econometrics","gpu","machine-learning","neural-network","optimization","python","pytorch","regression-models","research","scikit-learn","statistics","statsmodels","tensor"],"latest_commit_sha":null,"homepage":"https://unsloth.ai","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/unslothai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-27T16:00:47.000Z","updated_at":"2025-10-02T12:35:48.000Z","dependencies_parsed_at":"2024-11-19T03:28:28.434Z","dependency_job_id":null,"html_url":"https://github.com/unslothai/hyperlearn","commit_stats":{"total_commits":241,"total_committers":5,"mean_commits":48.2,"dds":0.02904564315352698,"last_synced_commit":"18d3aebd1086c95660772bf360bf7668c0ae94eb"},"previous_names":["unslothai/hyperlearn","danielhanchen/hyperlearn"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/unslothai/hyperlearn","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unslothai%2Fhyperlearn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unslothai%2Fhyperlearn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unslothai%2Fhyperlearn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unslothai%2Fhyperlearn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/unslothai","download_url":"https://codeload.github.com/unslothai/hyperlearn/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unslothai%2Fhyperlearn/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278266894,"owners_count":25958733,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis","data-science","deep-learning","econometrics","gpu","machine-learning","neural-network","optimization","python","pytorch","regression-models","research","scikit-learn","statistics","statsmodels","tensor"],"created_at":"2024-07-30T21:00:50.821Z","updated_at":"2025-10-04T04:58:14.700Z","avatar_url":"https://github.com/unslothai.png","language":"Jupyter Notebook","readme":"\u003cp align=\"center\"\u003e\n\u003cimg src=\"Images/HyperLearn_Logo.png\" alt=\"drawing\" width=\"300\"/\u003e\n\u003c/p\u003e\n\n\u003ch4 align=\"center\"\u003e\n2-2000x faster algos, 50% less memory usage, works on all hardware - new and old.\n\u003c/h4\u003e\n\u003ch2 align=\"center\"\u003e\nIf you want to collab on fast algorithms - msg me!!\nJoin our Discord server on making AI faster, or if you just wanna chat about AI!! https://discord.gg/unsloth\n\u003c/h2\u003e\n\n\n\u003ca href=\"http://unsloth.ai/\" style=\"text-align: center\"\u003e\u003cp align=\"center\"\u003eUnsloth Website\u003c/p\u003e\u003c/a\u003e\n\u003ca href=\"https://hyperlearn.readthedocs.io/en/latest/index.html\" style=\"text-align: center\"\u003e\u003cp align=\"center\"\u003eDocumentation\u003c/p\u003e\u003c/a\u003e\n\u003ca href=\"https://drive.google.com/file/d/18fxyBiPE0G4e5yixAj5S--YL_pgTh3Vo/view?usp=sharing\" style=\"text-align: center\"\u003e\u003cp align=\"center\"\u003e50 Page Modern Big Data Algorithms PDF\u003c/p\u003e\u003c/a\u003e\n\n----\n\u003ch3\u003e Hyperlearn's algorithms, methods and repo has been featured or mentioned in 5 research papers! \u003c/h3\u003e\n\n```diff\n+ Microsoft, UW, UC Berkeley, Greece, NVIDIA\n```\n\n* **Microsoft**: Yu et al. Making Classical Machine Learning Pipelines Differentiable http://learningsys.org/nips18/assets/papers/45CameraReadySubmissionfinetune.pdf\n* **University of Washington**: Ariel Rokem, Kendrick Kay. Fractional ridge regression: a fast, interpretable reparameterization of ridge regression https://arxiv.org/abs/2005.03220\n* **National Center for Scientific Research 'Demokritos', Greece**: Christos Platias, Georgios Petasis. A Comparison of Machine Learning Methods for Data Imputation https://dl.acm.org/doi/10.1145/3411408.3411465\n* **UC Berkeley** David Chan. GPU Accelerated T-Distributed Stochastic Neighbor Embedding https://digitalassets.lib.berkeley.edu/techreports/ucb/incoming/EECS-2020-89.pdf _(Incorporated Hyperlearn methods into NVIDIA RAPIDS TSNE)_\n* **NVIDIA**: Raschka et al. RAPIDS: Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence https://arxiv.org/abs/2002.04803 _(Incorporated Hyperlearn methods into NVIDIA RAPIDS TSNE)_\n\n----\n\u003ch3\u003e Hyperlearn's methods and algorithms have been incorporated into more than 6 organizations and repositories! \u003c/h3\u003e\n\n```diff\n+ NASA + Facebook's Pytorch, Scipy, Cupy, NVIDIA, UNSW\n```\n\n* **Facebook's Pytorch**: SVD very very slow and GELS gives nans, -inf #11174 https://github.com/pytorch/pytorch/issues/11174\n* **Scipy**: EIGH very very slow --\u003e suggesting an easy fix #9212 https://github.com/scipy/scipy/issues/9212\n* **Cupy**: Make SVD overwrite temporary array x https://github.com/cupy/cupy/pull/2277\n* **NVIDIA**: Accelerating TSNE with GPUs: From hours to seconds https://medium.com/rapids-ai/tsne-with-gpus-hours-to-seconds-9d9c17c941db\n* **UNSW** Abdussalam et al. Large-scale Sku-level Product Detection In Social Media Images And Sales Performance https://www.abstractsonline.com/pp8/#!/9305/presentation/465\n\n----\n\n\u003ch3\u003e During Hyperlearn's development, bugs and issues were notified to GCC! \u003c/h3\u003e\n\n* GCC 10 ignoring function attribute optimize for all x86 since r11-1019 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96535\n* Vector Extensions aligned(1) not generating unaligned loads/stores https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98317\n* GCC \u003e= 6 cannot inline _mm_cmp_ps on SSE targets https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98387\n* GCC 10.2 AVX512 Mask regression from GCC 9 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98348\n\n----\n\n\u003ca href=\"\" style=\"text-align: center\"\u003e\u003cimg src=\"Images/Packages_Used_long.png\" alt=\"Packages Used\" align=\"center\"/\u003e\u003c/a\u003e\n\nHyperLearn is written completely in PyTorch, NoGil Numba, Numpy, Pandas, Scipy \u0026 LAPACK, C++, C, Python, Cython and Assembly, and mirrors (mostly) Scikit Learn.\nHyperLearn also has statistical inference measures embedded, and can be called just like Scikit Learn's syntax.\n\nSome key current achievements of HyperLearn:\n\n* 70% less time to fit Least Squares / Linear Regression than sklearn + 50% less memory usage\n* 50% less time to fit Non Negative Matrix Factorization than sklearn due to new parallelized algo\n* 40% faster full Euclidean / Cosine distance algorithms\n* 50% less time LSMR iterative least squares\n* New Reconstruction SVD - use SVD to impute missing data! Has .fit AND .transform. Approx 30% better than mean imputation\n* 50% faster Sparse Matrix operations - parallelized\n* RandomizedSVD is now 20 - 30% faster\n\n\u003ca href=\"\" style=\"text-align: center\"\u003e\u003cimg src=\"Images/SVD.png\" alt=\"Modern Big Data Algorithms\" align=\"center\"/\u003e\u003c/a\u003e\n---\n\n\n### Comparison of Speed / Memory\n\n| Algorithm         |  n    |  p  | Time(s) |            | RAM(mb) |            | Notes                   |\n| ----------------- | ----- | --- | ------- | ---------- | ------- | ---------- | ----------------------- |\n|                   |       |     | Sklearn | Hyperlearn | Sklearn | Hyperlearn |                         |\n| QDA (Quad Dis A)  |1000000| 100 |   54.2  |   *22.25*  |  2,700  |  *1,200*   | Now parallelized        |\n| LinearRegression  |1000000| 100 |   5.81  |   *0.381*  |   700   |    *10*    | Guaranteed stable \u0026 fast|\n\nTime(s) is Fit + Predict. RAM(mb) = max( RAM(Fit), RAM(Predict) )\n \nI've also added some preliminary results for N = 5000, P = 6000\n\n\u003cimg src=\"Images/Preliminary Results N=5000 P=6000.png\" alt=\"drawing\" width=\"500\"/\u003e\n\n---\n#### Help is really needed! Message me!\n---\n# Key Methodologies and Aims\n#### 1. [Embarrassingly Parallel For Loops](#1)\n#### 2. [50%+ Faster, 50%+ Leaner](#2)\n#### 3. [Why is Statsmodels sometimes unbearably slow?](#3)\n#### 4. [Deep Learning Drop In Modules with PyTorch](#4)\n#### 5. [20%+ Less Code, Cleaner Clearer Code](#5)\n#### 6. [Accessing Old and Exciting New Algorithms](#6)\n---\n\u003ca id='1'\u003e\u003c/a\u003e\n### 1. Embarrassingly Parallel For Loops\n  * Including Memory Sharing, Memory Management\n  * CUDA Parallelism through PyTorch \u0026 Numba\n  \n\u003ca id='2'\u003e\u003c/a\u003e\n### 2. 50%+ Faster, 50%+ Leaner  \n  * Matrix Multiplication Ordering: https://en.wikipedia.org/wiki/Matrix_chain_multiplication\n  * Element Wise Matrix Multiplication reducing complexity to O(n^2) from O(n^3): https://en.wikipedia.org/wiki/Hadamard_product_(matrices)\n  * Reducing Matrix Operations to Einstein Notation: https://en.wikipedia.org/wiki/Einstein_notation\n  * Evaluating one-time Matrix Operations in succession to reduce RAM overhead.\n  * If p\u003e\u003en, maybe decomposing X.T is better than X.\n  * Applying QR Decomposition then SVD might be faster in some cases.\n  * Utilise the structure of the matrix to compute faster inverse (eg triangular matrices, Hermitian matrices).\n  * Computing SVD(X) then getting pinv(X) is sometimes faster than pure pinv(X)\n  \n\u003ca id='3'\u003e\u003c/a\u003e\n### 3. Why is Statsmodels sometimes unbearably slow?\n  * Confidence, Prediction Intervals, Hypothesis Tests \u0026 Goodness of Fit tests for linear models are optimized.\n  * Using Einstein Notation \u0026 Hadamard Products where possible.\n  * Computing only what is neccessary to compute (Diagonal of matrix and not entire matrix).\n  * Fixing the flaws of Statsmodels on notation, speed, memory issues and storage of variables.\n\n\u003ca id='4'\u003e\u003c/a\u003e\n### 4. Deep Learning Drop In Modules with PyTorch\n  * Using PyTorch to create Scikit-Learn like drop in replacements.\n\n\u003ca id='5'\u003e\u003c/a\u003e\n### 5. 20%+ Less Code, Cleaner Clearer Code\n  * Using Decorators \u0026 Functions where possible.\n  * Intuitive Middle Level Function names like (isTensor, isIterable).\n  * Handles Parallelism easily through hyperlearn.multiprocessing\n\n\u003ca id='6'\u003e\u003c/a\u003e\n### 6. Accessing Old and Exciting New Algorithms\n  * Matrix Completion algorithms - Non Negative Least Squares, NNMF\n  * Batch Similarity Latent Dirichelt Allocation (BS-LDA)\n  * Correlation Regression\n  * Feasible Generalized Least Squares FGLS\n  * Outlier Tolerant Regression\n  * Multidimensional Spline Regression\n  * Generalized MICE (any model drop in replacement)\n  * Using Uber's Pyro for Bayesian Deep Learning\n\n---\n\n---\n# Extra License Terms\n1. The Apache 2.0 license is adopted.\n","funding_links":[],"categories":["Jupyter Notebook","其他_机器学习与深度学习"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funslothai%2Fhyperlearn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funslothai%2Fhyperlearn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funslothai%2Fhyperlearn/lists"}