{"id":23338237,"url":"https://github.com/sengelha/streaming-percentiles","last_synced_at":"2025-04-09T22:30:57.685Z","repository":{"id":16785745,"uuid":"19544165","full_name":"sengelha/streaming-percentiles","owner":"sengelha","description":"Cross-platform, multi-language implementation of multiple streaming percentile algorithms","archived":false,"fork":false,"pushed_at":"2023-02-18T02:31:19.000Z","size":1570,"stargazers_count":61,"open_issues_count":9,"forks_count":9,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-18T14:17:34.794Z","etag":null,"topics":["analytics","c-plus-plus","javascript","percentiles","streaming","streaming-percentiles"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sengelha.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-05-07T17:45:09.000Z","updated_at":"2025-01-08T02:04:42.000Z","dependencies_parsed_at":"2024-06-21T05:43:25.189Z","dependency_job_id":"b4ef7c3b-0f60-4027-bedf-96e26874b0e7","html_url":"https://github.com/sengelha/streaming-percentiles","commit_stats":{"total_commits":211,"total_committers":5,"mean_commits":42.2,"dds":"0.10426540284360186","last_synced_commit":"76341f4bb4c17020e1a69a98a98c19d4ad4a5fa4"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sengelha%2Fstreaming-percentiles","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sengelha%2Fstreaming-percentiles/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sengelha%2Fstreaming-percentiles/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sengelha%2Fstreaming-percentiles/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sengelha","download_url":"https://codeload.github.com/sengelha/streaming-percentiles/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248123320,"owners_count":21051444,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","c-plus-plus","javascript","percentiles","streaming","streaming-percentiles"],"created_at":"2024-12-21T03:12:51.537Z","updated_at":"2025-04-09T22:30:57.637Z","avatar_url":"https://github.com/sengelha.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# streaming-percentiles\n\n## Build Status\n\n![ci](https://github.com/sengelha/streaming-percentiles/actions/workflows/ci.yml/badge.svg)\n![nightly](https://github.com/sengelha/streaming-percentiles/actions/workflows/nightly.yml/badge.svg)\n\n## About the Library\n\nThis is a cross-platform library with implementations of various\npercentile algorithms on streams of data.  These algorithms allow\nyou to calculate approximate percentiles (e.g. 50th percentile,\n95th percentile) in a single pass over a data set.\nThey are particularly useful for calculating percentiles for\nimmense data sets, for extremely high-throughput systems, or\nfor near-real time use cases.\n\nThe library supports the following languages:\n\n- C++\n- JavaScript\n\nThe library implements the following streaming percentile algorithms:\n\n- [Greenwald-Khanna](doc/methodology/GK01.pdf)\n- [Cormode-Korn-Muthukrishnan-Srivastava](doc/methodology/CKMS05.pdf) for high-biased, low-biased, uniform, and targeted quantiles\n- [Q-Digest](doc/methodology/q-digest.pdf)\n\nFor more background on streaming\npercentiles, see [Calculating Percentiles on Streaming\nData](//www.stevenengelhardt.com/postseries/calculating-percentiles-on-streaming-data/).\n\n## Obtaining the Library\n\nThe current version of the library is 3.1.0, and it was released\nMarch 28, 2019.\n\n### C++\n\nYou can download the latest release of the library from the\n[streaming-percentiles latest release\npage](//github.com/sengelha/streaming-percentiles-cpp/releases/latest).\n\n### JavaScript\n\nIf you use NPM, `npm install streaming-percentiles`. Otherwise, download\nthe latest release of the library from the [streaming-percentiles latest\nrelease\npage](//github.com/sengelha/streaming-percentiles-cpp/releases/latest)\npage.\n\nFor convenience, you can also use the latest release JS directly\nfrom `sengelha.github.io`:\n\n- [Unminified version](//sengelha.github.io/streaming-percentiles/streamingPercentiles.v1.js)\n- [Minified version](//sengelha.github.io/streaming-percentiles/streamingPercentiles.v1.min.js)\n\n### Source Code\n\nYou can download the latest release's source code from the\n[streaming-percentiles latest release\npage](//github.com/sengelha/streaming-percentiles-cpp/releases/latest).\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for instructions on how to build the release from\nsource.\n\n### Historical Releases\n\nHistorical releases may be downloaded from the [streaming-percentiles release page](//github.com/sengelha/streaming-percentiles/releases).\n\n## Using the Library\n\n### C++\n\nHere's an example on how to use the Greenwald-Khanna streaming\npercentile algorithm from C++:\n\n```cpp\n#include \u003cstmpct/gk.hpp\u003e\n\ndouble epsilon = 0.1;\nstmpct::gk\u003cdouble\u003e g(epsilon);\nfor (int i = 0; i \u003c 1000; ++i)\n    g.insert(rand());\ndouble p50 = g.quantile(0.5); // Approx. median\ndouble p95 = g.quantile(0.95); // Approx. 95th percentile\n```\n\n### JavaScript\n\n#### Node.JS\n\nHere's an example of how to use the library from Node.JS:\n```javascript\nvar sp = require('streaming-percentiles');\n\nvar epsilon = 0.1;\nvar g = new sp.GK(epsilon);\nfor (var i = 0; i \u003c 1000; ++i)\n    g.insert(Math.random());\nvar p50 = g.quantile(0.5); // Approx. median\nvar p95 = g.quantile(0.95); // Approx. 95th percentile\n```\n\n#### Browser\n\nHere's an example of how to use the library from a browser.  Note that the\ndefault module name is `streamingPercentiles`:\n```html\n\u003cscript src=\"//sengelha.github.io/streaming-percentiles/streamingPercentiles.v1.min.js\"\u003e\u003c/script\u003e\n\u003cscript\u003e\nvar epsilon = 0.1;\nvar g = new streamingPercentiles.GK(epsilon);\nfor (var i = 0; i \u003c 1000; ++i)\n    g.insert(Math.random());\nvar p50 = g.quantile(0.5);\n\u003c/script\u003e\n```\n\n## Performance\n\nBelow is the performance of the C++ implementation of these algorithms\ntested on December 22, 2021 on a Intel(R) Core(TM) i7-8809G CPU @ 3.10GHz:\n\n| Algorithm | Epsilon |         N | Input Type | Throughput (thousands of insertions/s) |\n| --------- | ------: | --------: | ---------- | -------------------------------------: |\n| ckms_hbq  |     0.1 | 1,000,000 | sorted     |                                1,517.7 |\n| ckms_hbq  |    0.01 | 1,000,000 | sorted     |                                1,094.1 |\n| ckms_hbq  |   0.001 | 1,000,000 | sorted     |                                  289.8 |\n| ckms_uq   |     0.1 | 1,000,000 | sorted     |                                1,533.9 |\n| ckms_uq   |    0.01 | 1,000,000 | sorted     |                                1,654.2 |\n| ckms_uq   |   0.001 | 1,000,000 | sorted     |                                  399.2 |\n| gk        |     0.1 | 1,000,000 | sorted     |                                1,392.1 |\n| gk        |    0.01 | 1,000,000 | sorted     |                                1,444.9 |\n| gk        |   0.001 | 1,000,000 | sorted     |                                1,114.9 |\n\nSee `//cpp:benchmark` for the benchmark program.\n\nBenchmarks are auto-generated on every `master` build.  They can be viewed\nat [https://sengelha.github.io/streaming-percentiles/dev/bench/](https://sengelha.github.io/streaming-percentiles/dev/bench/).\n\n## API Reference\n\nSee [doc/api_reference/](doc/api_reference/index.md) for detailed API reference documentation.\n\n## License\n\nThis project is licensed under the MIT License.  See\n[LICENSE](LICENSE) for more information.\n\n## Contributing\n\nIf you are interested in contributing to the library, please see\n[CONTRIBUTING.md](CONTRIBUTING.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsengelha%2Fstreaming-percentiles","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsengelha%2Fstreaming-percentiles","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsengelha%2Fstreaming-percentiles/lists"}