{"id":16599745,"url":"https://github.com/tupol/online-stats","last_synced_at":"2026-03-05T15:32:39.928Z","repository":{"id":130092484,"uuid":"118917003","full_name":"tupol/online-stats","owner":"tupol","description":"Online statistics implementations, including average, variance and standard deviation; exponentially weighted versions as well.","archived":false,"fork":false,"pushed_at":"2022-10-26T05:59:57.000Z","size":2995,"stargazers_count":10,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-30T12:27:28.881Z","etag":null,"topics":["covariance","exponential-moving-average","exponential-moving-variance","kurtosis","library","online-stats","scala","skewness","variance"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tupol.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-25T13:35:46.000Z","updated_at":"2024-04-26T12:18:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"7331c26f-2a16-4183-b122-e65bf91a3e9a","html_url":"https://github.com/tupol/online-stats","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/tupol/online-stats","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tupol%2Fonline-stats","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tupol%2Fonline-stats/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tupol%2Fonline-stats/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tupol%2Fonline-stats/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tupol","download_url":"https://codeload.github.com/tupol/online-stats/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tupol%2Fonline-stats/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30133248,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-05T14:41:47.141Z","status":"ssl_error","status_checked_at":"2026-03-05T14:41:21.567Z","response_time":93,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["covariance","exponential-moving-average","exponential-moving-variance","kurtosis","library","online-stats","scala","skewness","variance"],"created_at":"2024-10-12T00:12:28.501Z","updated_at":"2026-03-05T15:32:39.909Z","avatar_url":"https://github.com/tupol.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# online-stats #\n\n[![Maven Central](https://img.shields.io/maven-central/v/org.tupol/online-stats_2.11.svg)](https://mvnrepository.com/artifact/org.tupol/online-stats) \u0026nbsp;\n[![GitHub](https://img.shields.io/github/license/tupol/online-stats.svg)](https://github.com/tupol/online-stats/blob/master/LICENSE) \u0026nbsp; \n[![Travis (.org)](https://img.shields.io/travis/tupol/online-stats.svg)](https://travis-ci.com/tupol/online-stats) \u0026nbsp; \n[![Codecov](https://img.shields.io/codecov/c/github/tupol/online-stats.svg)](https://codecov.io/gh/tupol/online-stats) \u0026nbsp;\n[![Javadocs](https://www.javadoc.io/badge/org.tupol/online-stats_2.11.svg)](https://www.javadoc.io/doc/org.tupol/online-stats_2.11) \u0026nbsp;\n[![Gitter](https://badges.gitter.im/online-stats/community.svg)](https://gitter.im/online-stats/community?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge) \u0026nbsp; \n[![Twitter](https://img.shields.io/twitter/url/https/_tupol.svg?color=%2317A2F2)](https://twitter.com/_tupol) \u0026nbsp; \n\n## Scope ##\n\nNaive implementation of a few online statistical algorithms.\n\nThe idea behind this implementation is to be used as a tool for stateful streaming computations.\n\nAlgos covered so far:\n- 4 statistical moments and the derived features:\n  - average\n  - variance and standard deviation\n  - skewness\n  - kurtosis\n- covariance\n- exponentially weighted moving averages and variance\n\nAlgos to be researched:\n- exponentially weighted moving skewness\n- exponentially weighted moving kurtosis\n\nUsing a more formal and mature library like **[Apache Commons Math](http://commons.apache.org/proper/commons-math/)** \nis probably a better idea for production applications, but this is also tested against it. \n\n## Description ##\n\nThe main concepts introduced in this library are the `Stats`, `EWeightedStats` (exponentially\nweighted stats), `VectorStats` and `Covariance`. Each of them can be composed using either the\n`append` or the `|+|` functions. \n\nFor example, if we have a sequence of numbers, we can compute the statistics like this:\n\n```scala\n  val xs1 = Seq(1.0, 3.0)\n  val stats1: Stats = xs1.foldLeft(Stats.Nil)((s, x) =\u003e s |+| x)\n  val xs2 = Seq(5.0, 7.0)\n  val stats2: Stats = xs2.foldLeft(Stats.Nil)((s, x) =\u003e s |+| x)\n  val totalStats = stats1 |+| stats2\n  val newStats = totalStats |+| 4.0\n```\n\nThe `Stats` type with the `|+|` operation also form a *monoid*, since `|+|` has an *identity* \n(unit) element, `Stats.Nil`, and it is *associative*. \n \nAlso the `|+|` operation is also *commutative*, which makes appealing for distributed computing \nas well.\n \nSame goes for `VectorStats` and `Covariance`.\n\n`EWeightedStats` is an exception for now, as two `EWeightedStats` instances can not be composed.\nHowever, the `|+|` works between an `EWeightedStats` instance and a double. \n\n\n## Complexity ##\n\n| Feature                         | Space Complexity (*O*) | Time Complexity (*O*) |\n| ------------------------------- | :--------------------: | :-------------------: |\n| Count, Sum, Min, Max            | ***O***(1)  (1 * MU)   | ***O***(1)            |\n| Average                         | ***O***(1)  (2 * MU)   | ***O***(1)            |\n| Variance, Standard deviation    | ***O***(1)  (3 * MU)   | ***O***(1)            |\n| Skewness                        | ***O***(1)  (4 * MU)   | ***O***(1)            |\n| Kurtosis                        | ***O***(1)  (5 * MU)   | ***O***(1)            |\n| Exponentially weighted average  | ***O***(1)  (2 * MU)   | ***O***(1)            |\n| Exponentially weighted variance | ***O***(1)  (2 * MU)   | ***O***(1)            |\n\n*MU*: Memory Unit, e.g. Int: 4 bytes, Double 8: bytes\n\n## Demos and Examples ##\n\nThe [`streaming-anomalies-demos`](https://github.com/tupol/streaming-anomalies-demos) project was created to explore and demonstrate some basic use cases for the `online-stats` library.\n\n## References ##\n\n- [*\"Formulas for Robust, One-Pass Parallel Computation of Covariances and Arbitrary-Order Statistical Moments\"* by Philippe Pebay](https://digital.library.unt.edu/ark:/67531/metadc837537/m1/7/?utm_source=email\u0026utm_medium=client\u0026utm_content=ark_sidebar\u0026utm_campaign=ark_permanent)(http://prod.sandia.gov/techlib/access-control.cgi/2008/086212.pdf)\n- [*\"Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights\"* by Philippe Pebay, Timothy B. Terriberry, Hemanth Kolla, Janine Bennett4](https://zenodo.org/record/1232635/files/article.pdf)\n- [*\"The Exponentially Weighted Moving Variance\"* by J. F. Macgregor and T. J. Harris](https://www.tandfonline.com/doi/abs/10.1080/00224065.1993.11979433)\n- [*\"Incremental calculation of weighted mean and variance\"* by Tony Finch, February 2009](https://fanf2.user.srcf.net/hermes/doc/antiforgery/stats.pdf)\n- [Bessel Correction](https://en.wikipedia.org/wiki/Bessel%27s_correction)\n- [Skewness](https://en.wikipedia.org/wiki/Skewness)\n- [Kurtosis](https://en.wikipedia.org/wiki/Kurtosis)\n- [Pearson's Correlation Coefficient](https://en.wikipedia.org/wiki/Correlation_and_dependence#Pearson's_product-moment_coefficient)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftupol%2Fonline-stats","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftupol%2Fonline-stats","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftupol%2Fonline-stats/lists"}