{"id":16949237,"url":"https://github.com/f4str/ml-algorithms","last_synced_at":"2026-05-16T09:34:14.744Z","repository":{"id":48537074,"uuid":"233739023","full_name":"f4str/ml-algorithms","owner":"f4str","description":"Implementations of various machine learning algorithms ","archived":false,"fork":false,"pushed_at":"2021-08-26T04:52:34.000Z","size":28806,"stargazers_count":0,"open_issues_count":5,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-10T13:30:28.442Z","etag":null,"topics":["algorithms","data-science","machine-learning","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/f4str.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-14T02:28:46.000Z","updated_at":"2021-08-26T04:52:36.000Z","dependencies_parsed_at":"2022-09-23T12:51:36.243Z","dependency_job_id":null,"html_url":"https://github.com/f4str/ml-algorithms","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/f4str/ml-algorithms","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/f4str%2Fml-algorithms","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/f4str%2Fml-algorithms/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/f4str%2Fml-algorithms/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/f4str%2Fml-algorithms/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/f4str","download_url":"https://codeload.github.com/f4str/ml-algorithms/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/f4str%2Fml-algorithms/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33096950,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T04:41:52.686Z","status":"ssl_error","status_checked_at":"2026-05-16T04:41:52.009Z","response_time":115,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithms","data-science","machine-learning","python"],"created_at":"2024-10-13T21:53:31.292Z","updated_at":"2026-05-16T09:34:14.729Z","avatar_url":"https://github.com/f4str.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Machine Learning Algorithms\n\nImplementations of commonly-used machine learning algorithms from scratch using only [numpy](https://numpy.org/). Each algorithm is standalone with no other dependencies of other algorithms.\n\nAll models are intended for a transparent look into their implementation. They are not intended to be efficient or used to practical applications, but simply offer aid to anyone studying machine learning.\n\n## Installation\n\nClone the repository.\n\n```bash\ngit clone https://github.com/f4str/ml-algorithms\n```\n\nChange directories into the cloned repository.\n\n```bash\ncd ml-algorithms\n```\n\nInstall Python and create a virtual environment.\n\n```bash\npython3 -m venv venv\nsource venv/bin/activate\n```\n\nInstall the dev dependencies using pip.\n\n```bash\npip install -e .[dev]\n```\n\n## User Guide\n\nAll implementations are based on [scikit-learn](https://scikit-learn.org/) and [keras](https://keras.io/) by following a similar class style and structure.\n\n### Model Creation\n\nAll models are created by initializing a class with their hyperparameters. Each model will always have default hyperparameters so models can also be created without any parameters.\n\n```python\nclassifier = LogisticRegression(penalty='l1', C=0.001) # specify hyperparameters\nregressor = RidgeRegression() # use default hyperparameters\n```\n\nSince various other parameters are setup when creating a model. It is recommended to completely reinitialize the model rather than changing the hyperparameter.\n\n```python\ntree_clf = DecisionTreeClassifier(criterion='gini')\ntree_clf.criterion = 'entropy' # will not work, do not use\ntree_clf = DecisionTreeClassifier(criterion='entropy') # use this instead\n```\n\n### Model Training\n\nAll models are trained using the `fit(X, y)` method. This will always take parameters `X`, a matrix of training features, and `y`, the training labels. If the algorithm uses gradient descent, it may also take optional parameters for the `epochs` and `lr` to override the defaults. If the model uses gradient descent, the `fit(X, y)` method will return two lists for the training loss and evaluation metric (accuracy or R2 score) per training epoch. Otherwise, the model will return the final loss and evaluation metric from the trained model equivalent to calling `evaluate(X, y)`.\n\n```python\ntraining_loss, training_acc = classifier.fit(X, y) # returns Tuple[list, list]\nloss, r2 = regressor.fit(X, y) # returns Tuple[float, float]\n```\n\n### Model Prediction\n\nAll models have a `predict(X)` method which can be called after training. This will return the predicted values based on the weights learned from training.\n\n```python\ny_pred = classifier.predict(X) # returns class labels\ny_pred = regressor.predict(X) # returns real value predictions\n```\n\nIn addition, some classifiers have a `predict_proba(X)` and `predict_log_proba(X)` to get the class probabilities and log probabilities.\n\n```python\ny_pred_prob = classifier.predict_proba(X)\ny_pred_log_prob = classifier.predict_log_proba(X)\n```\n\n### Model Evaluation\n\nTo evaluate a model, it is recommended to run `predict(X)` and use your evaluation metrics of choice (accuracy, R2 score, F1 score, cross entropy, MSE, etc.). However, to get a quick and rough estimate of the model performance, all models have an `evaluate(X, y)` method which will return the default loss and evaluation metric. These metrics are model specific.\n\n```python\nce, acc = classifier.evaluate(X, y) # cross entropy and binary accuracy\nmse, r2 = regressor.evaluate(X, y) # mean square error and R2 score\n```\n\n## Algorithm Implementations\n\nVarious algorithms are implemented for both supervised and unsupervised learning tasks. All models are separated into their own category located in their respective subdirectory. Aside from the `utils` submodule with helper functions, all implementations are completely standalone so there are no other dependencies and the class can be used immediately out of the box.\n\n### Linear Models\n\n* Linear Regression\n* Ridge Regression\n* Lasso Regression\n* ElasticNet Regression\n* Logistic Regression\n  * L1 Penalty\n  * L2 Penalty\n  * ElasticNet Penalty\n\n### Decision Trees\n\n* Decision Tree Classifier\n  * Gini Split\n  * Entropy Split\n  * Misclassification Split\n* Decision Tree Regressor\n  * Mean Squared Error Split\n  * Mean Absolute Error Split\n  * Poisson Deviance Split\n\n### Nearest Neighbors\n\n* K-Nearest Neighbors Classifier (in-progress)\n* K-Nearest Neighbors Regressor (in-progress)\n\n### Support Vector Machines\n\n* Support Vector Classifier (in-progress)\n* Support Vector Regressor (in-progress)\n\n### Neural Networks\n\n* Multilayer Perceptron Regressor\n* Multilayer Perceptron Classifier\n\n### Decomposition\n\n* Principal Component Analysis (in-progress)\n\n## Development\n\nThe `tox` library is used to run all tests and code formatting. This is automatically installed with the dev requirements. The available options are as follows.\n\n* Run linting checks using `flake8`.\n\n    ```bash\n    tox -e lint\n    ```\n\n* Run type checks using `mypy`.\n\n    ```bash\n    tox -e type\n    ```\n\n* Run unit tests `pytest`.\n\n    ```bash\n    tox -e test\n    ```\n\n* Run all three of the tests above.\n\n    ```bash\n    tox\n    ```\n\n* Format the code using `black` and `isort` to comply with linting conventions.\n\n    ```bash\n    tox -e format\n    ```\n\nUpon pull request, merge, or push to the `master` branch, the three tests with `tox` will be run using GitHub Actions. The workflow will fail if any of the tests fail. See `.github/workflows/python-package.yml` for more information on how the CI works.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ff4str%2Fml-algorithms","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ff4str%2Fml-algorithms","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ff4str%2Fml-algorithms/lists"}