{"id":16726319,"url":"https://github.com/brunobonacci/reservoir","last_synced_at":"2025-07-13T00:38:10.317Z","repository":{"id":147148744,"uuid":"277117391","full_name":"BrunoBonacci/reservoir","owner":"BrunoBonacci","description":"A fast implementation of Reservoir Sampling with Immutable Persistent data structures.","archived":false,"fork":false,"pushed_at":"2020-08-07T15:43:09.000Z","size":24,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-12-16T15:16:01.877Z","etag":null,"topics":["clojure","fast","immutable","persistent","reservoir","reservoir-sampling"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BrunoBonacci.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-07-04T13:41:45.000Z","updated_at":"2020-09-23T05:41:00.000Z","dependencies_parsed_at":"2023-06-09T22:30:29.783Z","dependency_job_id":null,"html_url":"https://github.com/BrunoBonacci/reservoir","commit_stats":{"total_commits":13,"total_committers":1,"mean_commits":13.0,"dds":0.0,"last_synced_commit":"0a5bf5545eb6d45926f3ec5db4445d5f4ab946c7"},"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrunoBonacci%2Freservoir","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrunoBonacci%2Freservoir/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrunoBonacci%2Freservoir/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrunoBonacci%2Freservoir/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BrunoBonacci","download_url":"https://codeload.github.com/BrunoBonacci/reservoir/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243747698,"owners_count":20341546,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","fast","immutable","persistent","reservoir","reservoir-sampling"],"created_at":"2024-10-12T22:52:51.636Z","updated_at":"2025-03-15T15:24:45.350Z","avatar_url":"https://github.com/BrunoBonacci.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# reservoir\n[![Clojars Project](https://img.shields.io/clojars/v/com.brunobonacci/reservoir.svg)](https://clojars.org/com.brunobonacci/reservoir)  [![cljdoc badge](https://cljdoc.org/badge/com.brunobonacci/reservoir)](https://cljdoc.org/d/com.brunobonacci/reservoir/CURRENT) ![CircleCi](https://img.shields.io/circleci/project/BrunoBonacci/reservoir.svg) ![last-commit](https://img.shields.io/github/last-commit/BrunoBonacci/reservoir.svg)\n\nA fast implementation of Reservoir Sampling with Immutable Persistent data structures.\n\n`*** WORK IS STILL IN PROGRESS ***`\n\n## Usage\n\nIn order to use the library add the dependency to your `project.clj`\n\n``` clojure\n;; Leiningen project\n[com.brunobonacci/reservoir \"0.1.0-SNAPSHOT\"]\n\n;; deps.edn format\n{:deps { com.brunobonacci/reservoir {:mvn/version \"0.1.0-SNAPSHOT\"}}}\n```\n\nCurrent version: [![Clojars Project](https://img.shields.io/clojars/v/com.brunobonacci/reservoir.svg)](https://clojars.org/com.brunobonacci/reservoir)\n\n\nThen require the namespace:\n\n``` clojure\n(ns foo.bar\n  (:require [com.brunobonacci.reservoir :as rsv]))\n```\n\nCheck the [online documentation](https://cljdoc.org/d/com.brunobonacci/reservoir/CURRENT)\n\n``` clojure\n;; create a reservoir\n(def r (rsv/reservoir 3))\n\n;; add some items, returns a new reservoir\n(def r' (conj r :item1 :item2 :item3 :item4 :item4))\n\n(frequencies r')\n;; =\u003e {:item4 2, :item3 1}\n\n;; to add many items you can use `into`\n(def r2\n  (-\u003e\u003e (repeatedly #(rand-int 1000))\n    (take 100000)\n    (into (rsv/reservoir 1000))))\n\n;; the distribution of the samples should mimic the normal\n;; distribution (pseudo)\n(-\u003e\u003e (frequencies r2)\n  (sort-by second \u003e)\n  (take 10))\n;; =\u003e ([481 6]\n;;     [186 5]\n;;     [647 5]\n;;     [194 5]\n;;     [362 5]\n;;     [887 4]\n;;     [592 4]\n;;     [694 4]\n;;     [21 4]\n;;     [708 4])\n\n;; to see how many items passed by the reservoir\n(rsv/total-items r2)\n;; =\u003e 100000\n\n;; to get the samples\n(rsv/samples r2)\n;; =\u003e [205 831 423 830 238  ...]\n\n;; reservoir and just normal clojure sequences\n(seq r2)\n;; =\u003e (205 831 423 830 238  ...)\n\n\n(def r3 (into (rsv/reservoir 100) (range 30)))\n\n;; this is the total capacity of the reservoir\n(rsv/capacity r3)\n;; =\u003e 100\n\n;; here the current number of items in the reservoir\n;; it will be always between 0 and `capacity`\n(count r3)\n;; =\u003e 30\n\n;; this is the total number of items which passed\n;; into the reservoir. let's add more elements\n(rsv/total-items (into r3 (range 120)))\n;; =\u003e 150\n\n\n;; you can merge two or more reservoir as following\n;; the resulting reservoir will have the characteristics\n;; of the first reservoir with all the sample combined\n;; based on their probability to be in the final sample.\n(rsv/merge-reservoirs\n  (into (rsv/reservoir 5) (range 20))\n  (into (rsv/reservoir 3) (range 10))\n  (into (rsv/reservoir 4 :algorithm :algorithm-R) (range 20)))\n;; =\u003e #com.brunobonacci/reservoir [:algorithm-L 5 50 [0 5 8 17 10]]\n```\n\n### Performances\n\nThere are two implementation of the reservoir sampling.\nThe **Simple** algorithm (also known as [Algorithm-R](https://en.wikipedia.org/wiki/Reservoir_sampling#Simple_algorithm)\nwhich has a time complexity of `O(n)` and an optimal version called\n[Algorithm-L](https://en.wikipedia.org/wiki/Reservoir_sampling#An_optimal_algorithm)\nwhich is `O(k(1+log(n/k)))`. *The default implementation is Algorithm-L*\n\n``` clojure\n  ;; :simple :algorithm-R\n  (quick-bench\n    (into (reservoir 10000 :algorithm :simple) (range 1000000))\n\n  ;; Time per million of items\n  ;; --------------------------------\n  ;; Execution time mean : 48.510345 ms\n  ;; Execution time std-deviation : 1.994757 ms\n  ;; Execution time lower quantile : 46.083439 ms ( 2.5%)\n  ;; Execution time upper quantile : 51.081795 ms (97.5%)\n  ;; Overhead used : 2.309092 ns\n\n\n  ;; :algorithm-L  (default)\n  (quick-bench\n    (into (reservoir 10000 :algorithm :algorithm-L) (range 1000000)))\n\n  ;; Time per million of items\n  ;; --------------------------------\n  ;; Execution time mean : 30.131117 ms\n  ;; Execution time std-deviation : 1.487733 ms\n  ;; Execution time lower quantile : 28.247301 ms ( 2.5%)\n  ;; Execution time upper quantile : 31.998845 ms (97.5%)\n  ;; Overhead used : 8.211514 ns\n\n```\n\nThis reservoir implementation is up to two orders or magnitude faster\n(~50x) than the excellent\n[BigML/sampling](https://github.com/bigmlcom/sampling) implementation.\n\n``` clojure\n  (require '[bigml.sampling.reservoir :as reservoir])\n\n  (quick-bench\n    (into (reservoir/create  10000 :implementation :efraimdis) (range 1000000)))\n\n  ;; Execution time mean : 1439.723 ms\n  ;; Execution time std-deviation : 56.219255 ms\n  ;; Execution time lower quantile : 1.384248 sec ( 2.5%)\n  ;; Execution time upper quantile : 1.513384 sec (97.5%)\n  ;; Overhead used : 1.827186 ns\n\n  (quick-bench\n    (into (reservoir/create  10000 :implementation :insertion) (range 1000000)))\n\n  ;; Execution time mean : 462.243933 ms\n  ;; Execution time std-deviation : 8.206758 ms\n  ;; Execution time lower quantile : 447.970058 ms ( 2.5%)\n  ;; Execution time upper quantile : 469.125423 ms (97.5%)\n  ;; Overhead used : 1.827186 ns\n\n```\n\n## License\n\nCopyright © 2020 Bruno Bonacci - Distributed under the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrunobonacci%2Freservoir","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrunobonacci%2Freservoir","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrunobonacci%2Freservoir/lists"}