https://github.com/residentmario/streaming-algos
Notebooks implementing a selection of streaming quantile algorithms.
https://github.com/residentmario/streaming-algos
Last synced: over 1 year ago
JSON representation
Notebooks implementing a selection of streaming quantile algorithms.
- Host: GitHub
- URL: https://github.com/residentmario/streaming-algos
- Owner: ResidentMario
- Created: 2018-06-04T17:59:42.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2018-06-04T18:03:24.000Z (about 8 years ago)
- Last Synced: 2025-01-28T10:25:13.507Z (over 1 year ago)
- Language: Jupyter Notebook
- Size: 20.5 KB
- Stars: 1
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# streaming-algos
These handful of notebooks implement three different quantile estimation algorithms. These algorithms are meant to be used to accurately estimate quantiles in large streaming data whilst using fewer computational and memory resources than simple histograms.
References:
* GK --- https://stevenengelhardt.com/2018/03/07/calculating-percentiles-on-streaming-data-part-2-notes-on-implementing-greenwald-khanna/
* Q-digest --- https://graphics.stanford.edu/courses/cs468-05-winter/Papers/Information_Aggregation/Suri_sensys04.pdf
* T-digest --- file:///Users/alex/Downloads/histo.pdf
TODOs:
- [ ] Implement forward decay (http://dimacs.rutgers.edu/~graham/pubs/papers/fwddecay.pdf)
- [ ] Browse through and implement more things from https://gist.github.com/debasishg/8172796