{"id":25286138,"url":"https://github.com/stdlib-js/ml-incr-binary-classification","last_synced_at":"2025-10-27T20:30:42.466Z","repository":{"id":41421722,"uuid":"377266629","full_name":"stdlib-js/ml-incr-binary-classification","owner":"stdlib-js","description":"Incrementally perform binary classification using stochastic gradient descent (SGD).","archived":false,"fork":false,"pushed_at":"2024-08-01T13:28:52.000Z","size":3793,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-11T01:22:17.287Z","etag":null,"topics":["algorithm","binary","class","classification","gradient-descent","incremental","javascript","logistic","machine-learning","math","mathematics","ml","node","node-js","nodejs","online","prediction","statistics","stats","stdlib"],"latest_commit_sha":null,"homepage":"https://github.com/stdlib-js/stdlib","language":"JavaScript","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stdlib-js.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["stdlib-js"],"open_collective":"stdlib","tidelift":"npm/@stdlib/stdlib"}},"created_at":"2021-06-15T19:00:04.000Z","updated_at":"2024-08-01T06:14:20.000Z","dependencies_parsed_at":"2024-01-16T15:41:39.295Z","dependency_job_id":"eed0e71e-7f99-48f2-8628-fd20a17b0429","html_url":"https://github.com/stdlib-js/ml-incr-binary-classification","commit_stats":{"total_commits":47,"total_committers":1,"mean_commits":47.0,"dds":0.0,"last_synced_commit":"952ceca77f3a4384777537bbaabecf6bbf404aaf"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stdlib-js%2Fml-incr-binary-classification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stdlib-js%2Fml-incr-binary-classification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stdlib-js%2Fml-incr-binary-classification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stdlib-js%2Fml-incr-binary-classification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stdlib-js","download_url":"https://codeload.github.com/stdlib-js/ml-incr-binary-classification/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238461528,"owners_count":19476343,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithm","binary","class","classification","gradient-descent","incremental","javascript","logistic","machine-learning","math","mathematics","ml","node","node-js","nodejs","online","prediction","statistics","stats","stdlib"],"created_at":"2025-02-12T21:25:17.480Z","updated_at":"2025-10-27T20:30:42.457Z","avatar_url":"https://github.com/stdlib-js.png","language":"JavaScript","readme":"\u003c!--\n\n@license Apache-2.0\n\nCopyright (c) 2018 The Stdlib Authors.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n   http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n--\u003e\n\n\n\u003cdetails\u003e\n  \u003csummary\u003e\n    About stdlib...\n  \u003c/summary\u003e\n  \u003cp\u003eWe believe in a future in which the web is a preferred environment for numerical computation. To help realize this future, we've built stdlib. stdlib is a standard library, with an emphasis on numerical and scientific computation, written in JavaScript (and C) for execution in browsers and in Node.js.\u003c/p\u003e\n  \u003cp\u003eThe library is fully decomposable, being architected in such a way that you can swap out and mix and match APIs and functionality to cater to your exact preferences and use cases.\u003c/p\u003e\n  \u003cp\u003eWhen you use stdlib, you can be absolutely certain that you are using the most thorough, rigorous, well-written, studied, documented, tested, measured, and high-quality code out there.\u003c/p\u003e\n  \u003cp\u003eTo join us in bringing numerical computing to the web, get started by checking us out on \u003ca href=\"https://github.com/stdlib-js/stdlib\"\u003eGitHub\u003c/a\u003e, and please consider \u003ca href=\"https://opencollective.com/stdlib\"\u003efinancially supporting stdlib\u003c/a\u003e. We greatly appreciate your continued support!\u003c/p\u003e\n\u003c/details\u003e\n\n# incrBinaryClassification\n\n[![NPM version][npm-image]][npm-url] [![Build Status][test-image]][test-url] [![Coverage Status][coverage-image]][coverage-url] \u003c!-- [![dependencies][dependencies-image]][dependencies-url] --\u003e\n\n\u003e Incrementally perform binary classification using [stochastic gradient descent][stochastic-gradient-descent] (SGD).\n\n\u003csection class=\"installation\"\u003e\n\n## Installation\n\n```bash\nnpm install @stdlib/ml-incr-binary-classification\n```\n\nAlternatively,\n\n-   To load the package in a website via a `script` tag without installation and bundlers, use the [ES Module][es-module] available on the [`esm`][esm-url] branch (see [README][esm-readme]).\n-   If you are using Deno, visit the [`deno`][deno-url] branch (see [README][deno-readme] for usage intructions).\n-   For use in Observable, or in browser/node environments, use the [Universal Module Definition (UMD)][umd] build available on the [`umd`][umd-url] branch (see [README][umd-readme]).\n\nThe [branches.md][branches-url] file summarizes the available branches and displays a diagram illustrating their relationships.\n\nTo view installation and usage instructions specific to each branch build, be sure to explicitly navigate to the respective README files on each branch, as linked to above.\n\n\u003c/section\u003e\n\n\u003csection class=\"usage\"\u003e\n\n## Usage\n\n```javascript\nvar incrBinaryClassification = require( '@stdlib/ml-incr-binary-classification' );\n```\n\n#### incrBinaryClassification( N\\[, options] )\n\nReturns an accumulator `function` which incrementally performs binary classification using [stochastic gradient descent][stochastic-gradient-descent].\n\n```javascript\n// Create an accumulator for performing binary classification on 3-dimensional data:\nvar accumulator = incrBinaryClassification( 3 );\n```\n\nThe function accepts the following `options`:\n\n-   **intercept**: `boolean` indicating whether to include an intercept. If `true`, an element equal to one is implicitly added to each provided feature vector (note, however, that the model does not perform regularization of the intercept term). If `false`, the model assumes that feature vectors are already centered. Default: `true`.\n\n-   **lambda**: regularization parameter. The regularization parameter determines the amount of shrinkage inflicted on the model coefficients. Higher values reduce the variance of the model coefficient estimates at the expense of introducing bias. Default: `1.0e-4`.\n\n-   **learningRate**: an array-like object containing the learning rate function and associated parameters. The learning rate function decides how fast or slow the model coefficients will be updated toward the optimal coefficients. Must be one of the following:\n\n    -   `['constant', ...]`: constant learning rate function. To set the learning rate, provide a second array element. By default, when the learn rate function is 'constant', the learning rate is set to `0.02`.\n    -   `['basic']`: basic learning rate function according to the formula `10/(10+t)` where `t` is the current iteration.\n    -   `['invscaling', ...]`: inverse scaling learning rate function according to the formula `eta0/pow(t, power_t)` where `eta0` is the initial learning rate and `power_t` is the exponent controlling how quickly the learning rate decreases. To set the initial learning rate, provide a second array element. By default, the initial learning rate is `0.02`. To set the exponent, provide a third array element. By default, the exponent is `0.5`.\n    -   `['pegasos']`: [Pegasos][@shalevshwartz:2011a] learning rate function according to the formula `1/(lambda*t)` where `t` is the current iteration and `lambda` is the regularization parameter.\n\n    Default: `['basic']`.\n\n-   **loss**: loss function. Must be one of the following:\n\n    -   `hinge`: hinge loss function. Corresponds to a soft-margin linear Support Vector Machine (SVM), which can handle non-linearly separable data.\n    -   `log`: logistic loss function. Corresponds to Logistic Regression.\n    -   `modifiedHuber`: Huber loss function [variant][@zhang:2004a] for classification.\n    -   `perceptron`: hinge loss function without a margin. Corresponds to the original perceptron by Rosenblatt (1957).\n    -   `squaredHinge`: squared hinge loss function SVM (L2-SVM).\n\n    Default: `'log'`.\n\nBy default, the model contains an intercept term. To omit the intercept, set the `intercept` option to `false`:\n\n```javascript\nvar array = require( '@stdlib/ndarray-array' );\n\n// Create a model with the intercept term:\nvar acc = incrBinaryClassification( 2, {\n    'intercept': true\n});\nvar coefs = acc( array( [ 1.4, 0.5 ] ), 1 );\n// returns \u003cndarray\u003e\n\nvar dim = coefs.length;\n// returns 3\n\n// Create a model without the intercept term:\nacc = incrBinaryClassification( 2, {\n    'intercept': false\n});\ncoefs = acc( array( [ 1.4, 0.5 ] ), -1 );\n// returns \u003cndarray\u003e\n\ndim = coefs.length;\n// returns 2\n```\n\n#### accumulator( x, y )\n\nIf provided a feature vector `x` and response value `y` (either `+1` or `-1`), the accumulator function updates a binary classification model; otherwise, the accumulator function returns the current binary classification model coefficients.\n\n```javascript\nvar array = require( '@stdlib/ndarray-array' );\n\n// Create an accumulator:\nvar acc = incrBinaryClassification( 2 );\n\n// Provide data to the accumulator...\nvar x = array( [ 1.0, 0.0 ] );\n\nvar coefs = acc( x, -1 );\n// returns \u003cndarray\u003e\n\nx.set( 0, 0.0 );\nx.set( 1, 1.0 );\n\ncoefs = acc( x, 1 );\n// returns \u003cndarray\u003e\n\nx.set( 0, 0.5 );\nx.set( 1, 1.0 );\n\ncoefs = acc( x, 1 );\n// returns \u003cndarray\u003e\n\ncoefs = acc();\n// returns \u003cndarray\u003e\n```\n\n#### accumulator.predict( X\\[, type] )\n\nComputes predicted response values for one or more observation vectors `X`.\n\n```javascript\nvar array = require( '@stdlib/ndarray-array' );\n\n// Create a model with the intercept term:\nvar acc = incrBinaryClassification( 2 );\n\n// ...\n\nvar label = acc.predict( array( [ 0.5, 2.0 ] ) );\n// returns \u003cndarray\u003e\n```\n\nProvided an [`ndarray`][@stdlib/ndarray/ctor] having shape `(..., N)`, where `N` is the number of features, the returned [`ndarray`][@stdlib/ndarray/ctor] has shape `(...)` (i.e., the number of dimensions is reduced by one) and data type `float64`. For example, if provided a one-dimensional [`ndarray`][@stdlib/ndarray/ctor], the method returns a zero-dimensional [`ndarray`][@stdlib/ndarray/ctor] whose only element is the predicted response value.\n\nBy default, the method returns the predict label (`type='label'`). In order to return a prediction probability of a `+1` response value given either the logistic (`log`) or modified Huber (`modifiedHuber`) loss functions, set the second argument to `'probability'`.\n\n```javascript\nvar array = require( '@stdlib/ndarray-array' );\n\n// Create a model with the intercept term:\nvar acc = incrBinaryClassification( 2, {\n    'loss': 'log'\n});\n\n// ...\n\nvar phat = acc.predict( array( [ 0.5, 2.0 ] ), 'probability' );\n// returns \u003cndarray\u003e\n```\n\nIn order to return the linear predictor (i.e., the signed distance to the hyperplane, which is computed as the dot product between the model coefficients and the provided feature vector `x`, plus the intercept), set the second argument to `'linear'`.\n\n```javascript\nvar array = require( '@stdlib/ndarray-array' );\n\n// Create a model with the intercept term:\nvar acc = incrBinaryClassification( 2, {\n    'loss': 'log'\n});\n\n// ...\n\nvar lp = acc.predict( array( [ 0.5, 2.0 ] ), 'linear' );\n// returns \u003cndarray\u003e\n```\n\nGiven a feature vector `x = [x_0, x_1, ...]` and model coefficients `c = [c_0, c_1, ...]`, the linear predictor is equal to `(x_0*c_0) + (x_1*c_1) + ... + c_intercept`.\n\n\u003c/section\u003e\n\n\u003c!-- /.usage --\u003e\n\n\u003csection class=\"notes\"\u003e\n\n## Notes\n\n-   The underlying binary classification model performs [L2 regularization][tikhonov-regularization] of model coefficients, shrinking them toward zero by penalizing their squared [euclidean norm][euclidean-norm].\n-   [Stochastic gradient descent][stochastic-gradient-descent] is sensitive to the scaling of the features. One is advised to either scale each feature to `[0,1]` or `[-1,1]` or to transform each feature into z-scores with zero mean and unit variance. One should keep in mind that the same scaling has to be applied to training data in order to obtain accurate predictions.\n-   In general, the more data provided to an accumulator, the more reliable the model predictions.\n\n\u003c/section\u003e\n\n\u003c!-- /.notes --\u003e\n\n\u003csection class=\"examples\"\u003e\n\n## Examples\n\n\u003c!-- eslint no-undef: \"error\" --\u003e\n\n```javascript\nvar normal = require( '@stdlib/random-base-normal' );\nvar binomial = require( '@stdlib/random-base-binomial' );\nvar array = require( '@stdlib/ndarray-array' );\nvar exp = require( '@stdlib/math-base-special-exp' );\nvar incrBinaryClassification = require( '@stdlib/ml-incr-binary-classification' );\n\n// Create a new accumulator:\nvar acc = incrBinaryClassification( 2, {\n    'intercept': true,\n    'lambda': 1.0e-3,\n    'loss': 'log'\n});\n\n// Incrementally update the classification model...\nvar phat;\nvar x;\nvar i;\nfor ( i = 0; i \u003c 10000; i++ ) {\n    x = array( [ normal( 0.0, 1.0 ), normal( 0.0, 1.0 ) ] );\n    phat = 1.0 / ( 1.0+exp( -( ( 3.0*x.get(0) ) - ( 2.0*x.get(1) ) + 1.0 ) ) );\n    acc( x, ( binomial( 1, phat ) ) ? 1.0 : -1.0 );\n}\n\n// Retrieve model coefficients:\nvar coefs = acc();\nconsole.log( 'Feature coefficients: %d, %d', coefs.get( 0 ), coefs.get( 1 ) );\nconsole.log( 'Intercept: %d', coefs.get( 2 ) );\n\n// Predict new observations...\nx = array( [ [ 0.9, 0.1 ], [ 0.1, 0.9 ], [ 0.9, 0.9 ] ] );\n\nvar out = acc.predict( x );\nconsole.log( 'x = [%d, %d]; label = %d', x.get( 0, 0 ), x.get( 0, 1 ), out.get( 0 ) );\nconsole.log( 'x = [%d, %d]; label = %d', x.get( 1, 0 ), x.get( 1, 1 ), out.get( 1 ) );\nconsole.log( 'x = [%d, %d]; label = %d', x.get( 2, 0 ), x.get( 2, 1 ), out.get( 2 ) );\n\nout = acc.predict( x, 'probability' );\nconsole.log( 'x = [%d, %d]; P(y=1|x) = %d', x.get( 0, 0 ), x.get( 0, 1 ), out.get( 0 ) );\nconsole.log( 'x = [%d, %d]; P(y=1|x) = %d', x.get( 1, 0 ), x.get( 1, 1 ), out.get( 1 ) );\nconsole.log( 'x = [%d, %d]; P(y=1|x) = %d', x.get( 2, 0 ), x.get( 2, 1 ), out.get( 2 ) );\n\nout = acc.predict( x, 'linear' );\nconsole.log( 'x = [%d, %d]; lp = %d', x.get( 0, 0 ), x.get( 0, 1 ), out.get( 0 ) );\nconsole.log( 'x = [%d, %d]; lp = %d', x.get( 1, 0 ), x.get( 1, 1 ), out.get( 1 ) );\nconsole.log( 'x = [%d, %d]; lp = %d', x.get( 2, 0 ), x.get( 2, 1 ), out.get( 2 ) );\n```\n\n\u003c/section\u003e\n\n\u003c!-- /.examples --\u003e\n\n\u003csection class=\"references\"\u003e\n\n## References\n\n-   Rosenblatt, Frank. 1957. \"The Perceptron–a perceiving and recognizing automaton.\" 85-460-1. Buffalo, NY, USA: Cornell Aeronautical Laboratory.\n-   Zhang, Tong. 2004. \"Solving Large Scale Linear Prediction Problems Using Stochastic Gradient Descent Algorithms.\" In _Proceedings of the Twenty-First International Conference on Machine Learning_, 116. New York, NY, USA: Association for Computing Machinery. doi:[10.1145/1015330.1015332][@zhang:2004a].\n-   Shalev-Shwartz, Shai, Yoram Singer, Nathan Srebro, and Andrew Cotter. 2011. \"Pegasos: primal estimated sub-gradient solver for SVM.\" _Mathematical Programming_ 127 (1): 3–30. doi:[10.1007/s10107-010-0420-4][@shalevshwartz:2011a].\n\n\u003c/section\u003e\n\n\u003c!-- /.references --\u003e\n\n\u003c!-- Section for related `stdlib` packages. Do not manually edit this section, as it is automatically populated. --\u003e\n\n\u003csection class=\"related\"\u003e\n\n* * *\n\n## See Also\n\n-   \u003cspan class=\"package-name\"\u003e[`@stdlib/ml-incr/sgd-regression`][@stdlib/ml/incr/sgd-regression]\u003c/span\u003e\u003cspan class=\"delimiter\"\u003e: \u003c/span\u003e\u003cspan class=\"description\"\u003eonline regression via stochastic gradient descent (SGD).\u003c/span\u003e\n\n\u003c/section\u003e\n\n\u003c!-- /.related --\u003e\n\n\u003c!-- Section for all links. Make sure to keep an empty line after the `section` element and another before the `/section` close. --\u003e\n\n\n\u003csection class=\"main-repo\" \u003e\n\n* * *\n\n## Notice\n\nThis package is part of [stdlib][stdlib], a standard library for JavaScript and Node.js, with an emphasis on numerical and scientific computing. The library provides a collection of robust, high performance libraries for mathematics, statistics, streams, utilities, and more.\n\nFor more information on the project, filing bug reports and feature requests, and guidance on how to develop [stdlib][stdlib], see the main project [repository][stdlib].\n\n#### Community\n\n[![Chat][chat-image]][chat-url]\n\n---\n\n## License\n\nSee [LICENSE][stdlib-license].\n\n\n## Copyright\n\nCopyright \u0026copy; 2016-2025. The Stdlib [Authors][stdlib-authors].\n\n\u003c/section\u003e\n\n\u003c!-- /.stdlib --\u003e\n\n\u003c!-- Section for all links. Make sure to keep an empty line after the `section` element and another before the `/section` close. --\u003e\n\n\u003csection class=\"links\"\u003e\n\n[npm-image]: http://img.shields.io/npm/v/@stdlib/ml-incr-binary-classification.svg\n[npm-url]: https://npmjs.org/package/@stdlib/ml-incr-binary-classification\n\n[test-image]: https://github.com/stdlib-js/ml-incr-binary-classification/actions/workflows/test.yml/badge.svg?branch=main\n[test-url]: https://github.com/stdlib-js/ml-incr-binary-classification/actions/workflows/test.yml?query=branch:main\n\n[coverage-image]: https://img.shields.io/codecov/c/github/stdlib-js/ml-incr-binary-classification/main.svg\n[coverage-url]: https://codecov.io/github/stdlib-js/ml-incr-binary-classification?branch=main\n\n\u003c!--\n\n[dependencies-image]: https://img.shields.io/david/stdlib-js/ml-incr-binary-classification.svg\n[dependencies-url]: https://david-dm.org/stdlib-js/ml-incr-binary-classification/main\n\n--\u003e\n\n[chat-image]: https://img.shields.io/gitter/room/stdlib-js/stdlib.svg\n[chat-url]: https://app.gitter.im/#/room/#stdlib-js_stdlib:gitter.im\n\n[stdlib]: https://github.com/stdlib-js/stdlib\n\n[stdlib-authors]: https://github.com/stdlib-js/stdlib/graphs/contributors\n\n[umd]: https://github.com/umdjs/umd\n[es-module]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules\n\n[deno-url]: https://github.com/stdlib-js/ml-incr-binary-classification/tree/deno\n[deno-readme]: https://github.com/stdlib-js/ml-incr-binary-classification/blob/deno/README.md\n[umd-url]: https://github.com/stdlib-js/ml-incr-binary-classification/tree/umd\n[umd-readme]: https://github.com/stdlib-js/ml-incr-binary-classification/blob/umd/README.md\n[esm-url]: https://github.com/stdlib-js/ml-incr-binary-classification/tree/esm\n[esm-readme]: https://github.com/stdlib-js/ml-incr-binary-classification/blob/esm/README.md\n[branches-url]: https://github.com/stdlib-js/ml-incr-binary-classification/blob/main/branches.md\n\n[stdlib-license]: https://raw.githubusercontent.com/stdlib-js/ml-incr-binary-classification/main/LICENSE\n\n[@stdlib/ndarray/ctor]: https://github.com/stdlib-js/ndarray-ctor\n\n[euclidean-norm]: https://en.wikipedia.org/wiki/Norm_%28mathematics%29#Euclidean_norm\n\n[tikhonov-regularization]: https://en.wikipedia.org/wiki/Tikhonov_regularization\n\n[stochastic-gradient-descent]: https://en.wikipedia.org/wiki/Stochastic_gradient_descent\n\n[@zhang:2004a]: https://doi.org/10.1145/1015330.1015332\n\n[@shalevshwartz:2011a]: https://doi.org/10.1007/s10107-010-0420-4\n\n\u003c!-- \u003crelated-links\u003e --\u003e\n\n[@stdlib/ml/incr/sgd-regression]: https://github.com/stdlib-js/ml-incr-sgd-regression\n\n\u003c!-- \u003c/related-links\u003e --\u003e\n\n\u003c/section\u003e\n\n\u003c!-- /.links --\u003e\n","funding_links":["https://github.com/sponsors/stdlib-js","https://opencollective.com/stdlib","https://tidelift.com/funding/github/npm/@stdlib/stdlib"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstdlib-js%2Fml-incr-binary-classification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstdlib-js%2Fml-incr-binary-classification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstdlib-js%2Fml-incr-binary-classification/lists"}