{"id":22780217,"url":"https://github.com/sitegui/fast-quantiles","last_synced_at":"2025-03-30T14:16:07.014Z","repository":{"id":150084966,"uuid":"205544284","full_name":"sitegui/fast-quantiles","owner":"sitegui","description":"Space-efficient quantile algorithms in Rust","archived":false,"fork":false,"pushed_at":"2020-05-10T15:43:24.000Z","size":159,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-05T15:53:59.695Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sitegui.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-31T12:54:05.000Z","updated_at":"2020-05-10T15:43:27.000Z","dependencies_parsed_at":"2023-06-05T01:45:33.759Z","dependency_job_id":null,"html_url":"https://github.com/sitegui/fast-quantiles","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sitegui%2Ffast-quantiles","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sitegui%2Ffast-quantiles/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sitegui%2Ffast-quantiles/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sitegui%2Ffast-quantiles/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sitegui","download_url":"https://codeload.github.com/sitegui/fast-quantiles/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246326783,"owners_count":20759439,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-11T20:12:32.613Z","updated_at":"2025-03-30T14:16:07.009Z","avatar_url":"https://github.com/sitegui.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Space efficient quantile\n\nThis repo will implement one (or more) space-efficient algorithm to compute quantiles (like median).\n\nThis is mostly an exercise of Rust :)\n\n*IN PROGRESS*\n\n## Benchmark\n\nThe results of benchmark implemented by the file [./run_benchmark.sh](run_benchmark.sh) and run in AMD® Ryzen 7 2700x were as follows:\n\nAlgorithm | Max error | Num threads | Num elements | Time | Memory\n---|---|---|---|---|---\nNaive | 0% | 1 | 100M | 18s | 1100MB\nGK | 1% | 1 | 100M | 23s | 3.0MB\nGK | 1% | 8 | 100M | 3.1s | 3.1MB\nModified GK | 1% | 1 | 100M | 9.1s | 2.7MB\nModified GK | 1% | 8 | 100M | 1.3s | 3.1MB\nModified GK | 1% | 1 | 1G | 88s | 2.7MB\nModified GK | 1% | 8 | 1G | 13s | 3.2MB\nModified GK | 0.1% | 8 | 1G | 17s | 3.8MB\n\nSome notes:\n\n1. The naive algorithm simply stores all values in memory, sorts them and then grabs the median term. It has maximum precision however it has linear memory complexity.\n2. The GK algorithm was implemented as close to the original [Space-Efficient Online Computation of QuantileSummaries](http://infolab.stanford.edu/~datar/courses/cs361a/papers/quantiles.pdf) article as I could, with a custom modification to allow for parallel execution, since the article does not give any guidance on how to merge different structures.\n3. The modified GK algorithm was inspired by the original but has some fundamental changes in the structure and implementation. This is the algorithm exposed at root level by this library.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsitegui%2Ffast-quantiles","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsitegui%2Ffast-quantiles","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsitegui%2Ffast-quantiles/lists"}