{"id":19054229,"url":"https://github.com/datadog/sketches-go","last_synced_at":"2025-05-14T09:06:50.473Z","repository":{"id":35518351,"uuid":"150104350","full_name":"DataDog/sketches-go","owner":"DataDog","description":"Go implementations of the distributed quantile sketch algorithm DDSketch","archived":false,"fork":false,"pushed_at":"2025-04-29T20:56:40.000Z","size":332,"stargazers_count":173,"open_issues_count":3,"forks_count":27,"subscribers_count":357,"default_branch":"master","last_synced_at":"2025-05-14T09:05:58.570Z","etag":null,"topics":["ddsketch","quantile"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataDog.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-09-24T13:09:55.000Z","updated_at":"2025-03-07T18:05:18.000Z","dependencies_parsed_at":"2024-06-24T16:13:07.041Z","dependency_job_id":"ef7a01f7-881d-4abc-99cb-5e24e090c974","html_url":"https://github.com/DataDog/sketches-go","commit_stats":{"total_commits":156,"total_committers":14,"mean_commits":"11.142857142857142","dds":0.5512820512820513,"last_synced_commit":"35c0e1cd3a53990c638ac834b9edf4c1af57a984"},"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataDog%2Fsketches-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataDog%2Fsketches-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataDog%2Fsketches-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataDog%2Fsketches-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataDog","download_url":"https://codeload.github.com/DataDog/sketches-go/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254110374,"owners_count":22016391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ddsketch","quantile"],"created_at":"2024-11-08T23:37:15.644Z","updated_at":"2025-05-14T09:06:50.423Z","avatar_url":"https://github.com/DataDog.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sketches-go \n\nThis repo contains Go implementations of the distributed quantile sketch algorithm\nDDSketch [1]. DDSketch has relative-error guarantees for any quantile q in [0, 1].\nThat is if the true value of the qth-quantile is `x` then DDSketch returns a value `y` \nsuch that `|x-y| / x \u003c e` where `e` is the relative error parameter. DDSketch is also \nfully mergeable, meaning that multiple sketches from distributed systems can be combined \nin a central node.\n\nOur default implementation, returned from `NewDefaultDDSketch(relativeAccuracy)`, is\nguaranteed [1] not to grow too large in size for any data that can be described by a\ndistribution whose tails are sub-exponential.\n\nWe also provide implementations, returned by `LogCollapsingLowestDenseDDSketch(relativeAccuracy, maxNumBins)`\nand `LogCollapsingHighestDenseDDSketch(relativeAccuracy, maxNumBins)`, where the q-quantile\nwill be accurate up to the specified relative error for q that is not too small (or large).\nConcretely, the q-quantile will be accurate up to the specified relative error as long as it\nbelongs to one of the `m` bins kept by the sketch. For instance, If the values are time in seconds, \n`maxNumBins = 2048` covers a time range from 80 microseconds to 1 year.\n\n### Usage\n\n```go\nimport \"github.com/DataDog/sketches-go/ddsketch\"\n\nrelativeAccuracy := 0.01\nsketch := ddsketch.NewDefaultDDSketch(relativeAccuracy)\n```\n\nAdd values to the sketch.\n\n```go\nimport \"math/rand\"\n\nfor i := 0; i \u003c 500; i++ {\n  v := rand.NormFloat64()\n  sketch.Add(v)\n}\n```\n\nFind the quantiles to within alpha relative error.\n\n```go\nqs := []float64{0.5, 0.75, 0.9, 1}\nquantiles, err := sketch.GetValuesAtQuantiles(qs)\n```\n\nMerge another `DDSketch` into `sketch`.\n\n```go\nanotherSketch := ddsketch.NewDefaultDDSketch(relativeAccuracy)\nfor i := 0; i \u003c 500; i++ {\n  v := rand.NormFloat64()\n  anotherSketch.Add(v)\n}\nsketch.MergeWith(anotherSketch)\n```\n\nThe quantiles are in `sketch` are still accurate to within `relativeAccuracy`.\n\n## References\n\n[1] Charles Masson and Jee E Rim and Homin K. Lee. DDSketch: A fast and fully-mergeable quantile sketch with \nrelative-error guarantees. PVLDB, 12(12): 2195-2205, 2019. (The code referenced in the paper, including our \nimplementation of the the Greenwald-Khanna (GK) algorithm, can be found at: \nhttps://github.com/DataDog/sketches-go/releases/tag/v0.0.1 )\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatadog%2Fsketches-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatadog%2Fsketches-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatadog%2Fsketches-go/lists"}