{"id":19320572,"url":"https://github.com/aluriak/linear_choosens","last_synced_at":"2026-05-13T01:34:45.031Z","repository":{"id":143140995,"uuid":"65145462","full_name":"Aluriak/linear_choosens","owner":"Aluriak","description":"Choose randomly n in k elements, in linear time and constant memory complexity","archived":false,"fork":false,"pushed_at":"2018-12-06T10:47:46.000Z","size":60338,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-24T05:23:36.516Z","etag":null,"topics":["benchmark","choice","python","random"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Aluriak.png","metadata":{"files":{"readme":"README.mkd","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-08-07T17:24:56.000Z","updated_at":"2018-12-06T10:47:48.000Z","dependencies_parsed_at":"2023-04-27T05:31:16.899Z","dependency_job_id":null,"html_url":"https://github.com/Aluriak/linear_choosens","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Aluriak/linear_choosens","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aluriak%2Flinear_choosens","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aluriak%2Flinear_choosens/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aluriak%2Flinear_choosens/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aluriak%2Flinear_choosens/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Aluriak","download_url":"https://codeload.github.com/Aluriak/linear_choosens/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aluriak%2Flinear_choosens/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32964052,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-12T23:30:32.555Z","status":"ssl_error","status_checked_at":"2026-05-12T23:30:18.191Z","response_time":102,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","choice","python","random"],"created_at":"2024-11-10T01:29:31.125Z","updated_at":"2026-05-13T01:34:45.004Z","avatar_url":"https://github.com/Aluriak.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# n choose k problem, and related implementations\nComparison of four Python implementations of the n choose k problem: *Choose randomly n elements in a list of k elements*.\nThe distribution should be uniform: equiprobability is a necessary condition.\n\nThis works takes its roots from [this blog post](https://getkerf.wordpress.com/2016/03/30/the-best-algorithm-no-one-knows-about/),that explain both interests and history of the problem and its solutions.\n\n## Notes on the linear implementation\nThe complexity in time is linear, and constant in memory (if you do not save the choosen items).\n\nThe linear implementation provides not only a performance boost compared to stdlib when the number of item is greater than 20 in the tests, but also a greater flexibility: you don't have to provides the full items in order to walk them, just their number.\n\nYou can therefore choose N elements into a generator of K elements, without having to store the full generator\nor even the N elements if you treat them immediatly.\n\nThe current implementation of linear_choosen implements these uses, and is a standalone ;\nyou can therefore copy-paste it whereever you need it.\n\n\n\n## Repository\n\n- **plot.py:** plotting with matplotlib\n- **test.py:** launch benchmarks\n- **linear_choosen.py:** methods implementation\n- **results/:** images outputs\n\n## Implemented methods\nThe four following methods are implemented. The third one is the main interest of this repository.\nFurther formal analysis of the method needs to be done.\n\n### stdlib\nThe `random.sample` method do exactly the job. It automatically choose between two internal implementations,\nfunction to n. [Link to the doc](https://hg.python.org/cpython/file/3.5/Lib/random.py#l280).\n\n[Source](https://github.com/Aluriak/linear_choosens/blob/master/linear_choosens.py#L11).\n\n\n### dumb\nThe very obvious *mix it, then take the n firsts*.\n[Very costly](https://github.com/Aluriak/linear_choosens#runtime-comparison).\n\n[Source](https://github.com/Aluriak/linear_choosens/blob/master/linear_choosens.py#L16).\n\n\n### linear\nThis algorithm is detailed in [Vitter paper](http://www.ittc.ku.edu/~jsv/Papers/Vit87.RandomSampling.pdf)\n*An Efficient Algorithm for Sequential Random Sampling*,\npublished in 1987, with the mathematical proof.\nYou can also find it in Knutt's *Seminumerical Algorithms*.\n(source: Jon Bentley's *Programming Pearls*)\n\nThis is an implementation proposal, that could probably be improved.\nIts less efficient than stdlib when n is small, but notably quicker when n comes near k.\n\nThe first element have a `n/k` likelihood to be in the output subset.\nIf this element is choosen, then find the next choosens is like n-1 choose k-1.\nThis treatment is recursively applied on the k items.\n\nThis method allows an O(1) complexity in memory, and a O(k) complexity in time (worst case),\nbecause elements are walked only once, and deciding whether an element is choosen or not\nis a comparison of a random number against a likelihood.\nThe subset is consequently constructed during the walk of all elements.\nAll the k elements don't need to be walked, when the search is 0 choose k-i.\n\n[Source](https://github.com/Aluriak/linear_choosens/blob/master/linear_choosens.py#L31).\n\n\n### (linear) recursive\nPurely recursive implementation of the linear algorithm.\nSlower, don't work on big dataset without modification of python stack size.\n\n[Source](https://github.com/Aluriak/linear_choosens/blob/master/linear_choosens.py#L65).\n\n\n## Runtime comparison\n![runtimes](results/runtime_100.png)\nThe stdlib implementation is better when n is small, but the linear implementation\nis quicker in other cases.\n\n\n## linear method distribution\nFollowing graphics don't show any obvious distribution bias between firsts and lasts elements.\n\n![runtimes](./results/benchmark_l_100_1000_100.png)\n\n![runtimes](./results/benchmark_l_900_1000_100.png)\n\nThese results are comparable to stdlib:\n\n![runtimes](./results/benchmark_s_100_1000_100.png)\n\n![runtimes](./results/benchmark_s_900_1000_100.png)\n\nAnd to the dumb method:\n\n![runtimes](./results/benchmark_d_100_1000_100.png)\n\n![runtimes](./results/benchmark_d_900_1000_100.png)\n\n\nFurther analysis of the data could highlight a bias induced by the linear method.\nIf so, it is probably a bias of implementation (bad source code) or a random bias (bad random generator).\nThe method itself is proven.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faluriak%2Flinear_choosens","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faluriak%2Flinear_choosens","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faluriak%2Flinear_choosens/lists"}