{"id":16572736,"url":"https://github.com/lemire/testingrng","last_synced_at":"2026-02-26T09:16:08.474Z","repository":{"id":37405846,"uuid":"100496064","full_name":"lemire/testingRNG","owner":"lemire","description":"Testing common random-number generators (RNG)","archived":false,"fork":false,"pushed_at":"2025-12-22T21:29:00.000Z","size":7226,"stargazers_count":194,"open_issues_count":6,"forks_count":25,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-12-24T10:38:47.187Z","etag":null,"topics":["benchmark","pcg","random-number-generators","rng","seed","statistical-tests","xorshift-generator"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lemire.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-08-16T14:05:23.000Z","updated_at":"2025-12-22T21:29:08.000Z","dependencies_parsed_at":"2022-07-13T16:44:30.556Z","dependency_job_id":"222fefd3-2d91-4fb7-9794-57fc7208574e","html_url":"https://github.com/lemire/testingRNG","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lemire/testingRNG","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2FtestingRNG","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2FtestingRNG/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2FtestingRNG/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2FtestingRNG/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lemire","download_url":"https://codeload.github.com/lemire/testingRNG/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lemire%2FtestingRNG/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29855200,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-26T08:51:08.701Z","status":"ssl_error","status_checked_at":"2026-02-26T08:50:19.607Z","response_time":89,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","pcg","random-number-generators","rng","seed","statistical-tests","xorshift-generator"],"created_at":"2024-10-11T21:28:28.499Z","updated_at":"2026-02-26T09:16:08.468Z","avatar_url":"https://github.com/lemire.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# testingRNG : testing popular random-number generators\n\nThere are several benchmarks that can be used to test (pseudo-)random number\ngenerators (RNG). Of particular interest are TestU01 and PractRand. We want to test these popular RNGs.\n\n\nThis project lead to the following publication:\n\n- [Xorshift1024*, Xorshift1024+, Xorshift128+ and Xoroshiro128+ fail statistical tests for linearity](https://www.sciencedirect.com/science/article/pii/S0377042718306265?dgcid=author), Journal of Computational and Applied Mathematics, Volume 350, 2019. (Available online 22 October 2018)\n\n## Scope Limitation\n\nThis project is meant to test some well-known non-cryptographic random number generators written in C/C++ using pre-existing frameworks (TestU01 and PractRand). If you would like to contribute new RNG functions, please open a pull request.\n\nThis project is not a tutorial on how to use TestU01 and PractRand. If you have questions about how to use TestU01 and PractRand, or have issues with TestU01 and PractRand, please refer to the relevant projects. We have not modified, in the least, the TestU01 and PractRand frameworks.\n\n\n\n## Prerequisites\n\nWe assume Linux or macOS. Under Windows 10, you can get the [Linux Subsystem](https://msdn.microsoft.com/en-us/commandline/wsl/install_guide).\n\n- Our scripts should work out of the box on  most Linux boxes, except maybe when they are configured to act as secure servers (e.g., without a C compiler).\n- If you have a mac, it is assumed that [you have installed a recent Xcode package](https://developer.apple.com/xcode/) to get\nthe C (and C++) compiler. Make sure you have installed the command-line utilities.\n\n(Note: We assume a recent x64 processor. TestU01, in particular, does not easily build on some ARM-based systems.)\n\n## Usage\n\nYou can run the tests by going to a bash shell (Terminal) and executing a few commands.\n\n#### PractRand:\n```\ncd practrand\n./runtests.sh\n```\n\nThe PractRand benchmark takes some time to complete because we analyze a large volume of random numbers.\n\n#### TestU01:\n```\ncd testu01\n./bigcrushall.sh\n```\n\nThe TestU01 benchmark \"big crush\" (``bigcrushall.sh``) might take days. It outputs its results in the current\ndirectory (``testu1``, but we copied already computed in the ``results`` subdirectory.\nA parallel version (``bigcrushallparallel.sh``) will test multiple generators at the same time, up to the number of detected CPU threads.\n\nThere are also extensive scripts that generate many (100) seeds and test generators, the script take\nthe form ``rand*.sh``, one per generator. They output their results in their ``longresults`` subdirectory.\n\n#### Speed:\nIt is interesting to assess running speed as well. This can be done quickly:\n\n```\ncd speed\nmake\nmake test\n```\n\nNote  that the speed tests assume a recent x64 processor (e.g., they would not work\n  on ARM processors).\n\n## The contenders\n\n- splitmix64 is a random number generator in widespread use and part of the standard Java API, we adapted a port to C produced by Vigna. It produces 64-bit numbers.\n- pcg32 and pcg64 are instances of the PCG family designed by O'Neill. They produce either 32-bit or 64-bit outputs.\n- xorshift32 is a classical xorshift random number generator. We do not expect it to do well.\n- xorshift128plus, xorshift1024star, xorshift1024plus and xoroshiro128plus are recently proposed random number generator by Vigna. There are many parameters possible, but we used those recommended by Vigna. For xorshift128plus, the V8 JavaScript runtime opted for other constants, so we add a new generator \"v8xorshift128plus\" which relies on constants that Vigna recommended against using, but that are apparently used by V8.\n- rand is whatever random number number generator your C standard library provides. It is a useful point of reference when assessing speed.\n- lehmer64 is a simple (but fast) Multiplicative Linear Congruential Generator\n- aesctr is a random number generator based on the AES cipher (contributed by Samuel Neves)\n- wyhash and wyrand are random number generators based on the MUM hashing function\n\n\n## Methodology\n\nFor historical reasons, it appears that TestU01 is designed to test 32-bit numbers\nwhereas many modern random number generators produce 64-bit numbers. Indeed, years ago,\nmost random number generators would produce, at best 31-bit random values.\n\nHow do we assess modern-day 64-bit random number generators? In such cases,\na sensible approach is to \"cast\" the result of the 64-bit test to a 32-bit integer.\n(In C/C++, we do ``uint32_t x32 = (uint32_t) x;``.)\nHowever, as pointed out by [Vigna (2016)](https://arxiv.org/pdf/1402.6246.pdf), we should\nmake sure that permuting the bit order does not lead to a test failure. So we test with\nboth the original, and reversed bit order. We also test with a reversed byte order.\n\nWe can also test the most significant bits (msb) instead of the least significant bits (lsb)\n(In C/C++, we do ``uint32_t x32 = (uint32_t) (x \u003e\u003e 32)``.) By convention, we refer to this\napproach with the ``-H`` flag in our command-line executables.\n\nThere are other possibilities, but if a random number generator were to require a very\nparticular approach to extract good 32-bit values from a 64-bit value, then it would be\na good sign that something is not quite right with the original 64-bit values.\n\nA given tests might fail with one seed, but this tells us little. So we check that\nthe test fails with at least four seeds.\n\nFor PractRand, we do not need to truncate the produced random bits.\n\n## TestU01 results\n\nSee testu01/results for detailed outputs.\nType ``./summarize.pl *.log |more``.\nThere are also supplementary results in the ``longresults`` subdirectory for\nspecific generators, to confirm beyond a doubt systematic failures.\n\n```\n$ ./summarize.pl testsplitmix64-*.log\nSummary for splitmix64 lsb 32-bits (4 crushes):\n- 3 unnoteworthy blips (#31, #40, #87)\n\nSummary for splitmix64 lsb 32-bits (bit reverse) (4 crushes):\n- 1 unnoteworthy blips (#16)\n\nSummary for splitmix64 lsb 32-bits (byte reverse) (4 crushes):\n- 1 unnoteworthy blips (#49)\n\nSummary for splitmix64 msb 32-bits (4 crushes):\n- 2 unnoteworthy blips (#102, #102)\n\nSummary for splitmix64 msb 32-bits (bit reverse) (4 crushes):\n- 3 unnoteworthy blips (#11, #65, #87)\n\nSummary for splitmix64 msb 32-bits (byte reverse) (4 crushes):\n- 4 unnoteworthy blips (#11, #38, #58, #74)\n```\n\n```\n$ ./summarize.pl testpcg32*.log\nSummary for pcg32 lsb 32-bits (4 crushes):\n- 1 unnoteworthy blips (#9)\n\nSummary for pcg32 lsb 32-bits (bit reverse) (4 crushes):\n- 1 unnoteworthy blips (#23)\n```\n\n```\n$ ./summarize.pl testpcg64*.log\nSummary for pcg64 lsb 32-bits (bit reverse) (4 crushes):\n- 2 unnoteworthy blips (#49, #74)\n\nSummary for pcg64 lsb 32-bits (byte reverse) (4 crushes):\n- 1 unnoteworthy blips (#38)\n\nSummary for pcg64 msb 32-bits (4 crushes):\n- 1 unnoteworthy blips (#6)\n\nSummary for pcg64 msb 32-bits (bit reverse) (4 crushes):\n- 2 unnoteworthy blips (#33, #76)\n\nSummary for pcg64 msb 32-bits (byte reverse) (4 crushes):\n- 1 unnoteworthy blips (#7)\n```\n\n```\n$ ./summarize.pl testxoroshiro*.log\nSummary for xoroshiro128plus lsb 32-bits (4 crushes):\n- 1 unnoteworthy blips (#76)\n\nSummary for xoroshiro128plus lsb 32-bits (bit reverse) (4 crushes):\n- #68: MatrixRank, L=1000, r=0: FAIL!! -- p-values too unlikely (eps, eps, eps, eps) -- ALL CRUSHES FAIL!!\n- #71: MatrixRank, L=5000: FAIL!! -- p-values too unlikely (eps, eps, eps, eps) -- ALL CRUSHES FAIL!!\n- #80: LinearComp, r = 0: FAIL!! -- p-values too unlikely (1 - eps1, 1 - eps1, 1 - eps1, 1 - eps1) -- ALL CRUSHES FAIL!!\n\nSummary for xoroshiro128plus lsb 32-bits (byte reverse) (4 crushes):\n- #71: MatrixRank, L=5000: FAIL!! -- p-values too unlikely (eps, eps, eps, eps) -- ALL CRUSHES FAIL!!\n\nSummary for xoroshiro128plus msb 32-bits (4 crushes):\n- 4 unnoteworthy blips (#11, #38, #97, #98)\n\nSummary for xoroshiro128plus msb 32-bits (bit reverse) (4 crushes):\n- 2 unnoteworthy blips (#48, #88)\n```\n\n```\n$ ./summarize.pl testxorshift128plus*.log\nreviewing xorshift128plus lsb 32-bits\nSummary for xorshift128plus lsb 32-bits (4 crushes):\n- 3 unnoteworthy blips (#7, #22, #59)\n\nreviewing xorshift128plus lsb 32-bits (bit reverse)\nSummary for xorshift128plus lsb 32-bits (bit reverse) (4 crushes):\n- #68: MatrixRank, L=1000, r=0: FAIL!! -- p-values too unlikely (eps, eps, eps, eps) -- ALL CRUSHES FAIL!!\n- #71: MatrixRank, L=5000: FAIL!! -- p-values too unlikely (eps, eps, eps, eps) -- ALL CRUSHES FAIL!!\n- #80: LinearComp, r = 0: FAIL!! -- p-values too unlikely (1 - eps1, 1 - eps1, 1 - eps1, 1 - eps1) -- ALL CRUSHES FAIL!!\n- 2 unnoteworthy blips (#24, #24)\n\nreviewing xorshift128plus lsb 32-bits (byte reverse)\nSummary for xorshift128plus lsb 32-bits (byte reverse) (4 crushes):\n- #71: MatrixRank, L=5000: FAIL!! -- p-values too unlikely (eps, eps, eps, eps) -- ALL CRUSHES FAIL!!\n```\n```\n$ ./summarize.pl testv8xorshift128plus-*.logreviewing v8xorshift128plus lsb 32-bits\nSummary for v8xorshift128plus lsb 32-bits (4 crushes):\n- 2 unnoteworthy blips (#47, #50)\n\nreviewing v8xorshift128plus lsb 32-bits (bit reverse)\nSummary for v8xorshift128plus lsb 32-bits (bit reverse) (4 crushes):\n- #68: MatrixRank, L=1000, r=0: FAIL!! -- p-values too unlikely (eps, eps, eps, eps) -- ALL CRUSHES FAIL!!\n- #71: MatrixRank, L=5000: FAIL!! -- p-values too unlikely (eps, eps, eps, eps) -- ALL CRUSHES FAIL!!\n- #80: LinearComp, r = 0: FAIL!! -- p-values too unlikely (1 - eps1, 1 - eps1, 1 - eps1, 1 - eps1) -- ALL CRUSHES FAIL!!\n- 3 unnoteworthy blips (#24, #43, #78)\n\nreviewing v8xorshift128plus lsb 32-bits (byte reverse)\nSummary for v8xorshift128plus lsb 32-bits (byte reverse) (4 crushes):\n- #71: MatrixRank, L=5000: FAIL!! -- p-values too unlikely (eps, eps, eps, eps) -- ALL CRUSHES FAIL!!\n- 2 unnoteworthy blips (#9, #102)\n\nreviewing v8xorshift128plus msb 32-bits (bit reverse)\nSummary for v8xorshift128plus msb 32-bits (bit reverse) (4 crushes):\n- 1 unnoteworthy blips (#11)\n\nreviewing v8xorshift128plus msb 32-bits (byte reverse)\nSummary for v8xorshift128plus msb 32-bits (byte reverse) (4 crushes):\n- 1 unnoteworthy blips (#48)\n```\n\n\n```\n$ ./summarize.pl testlehmer64*.log\nreviewing lehmer64 lsb 32-bits\nSummary for lehmer64 lsb 32-bits (4 crushes):\n- 2 unnoteworthy blips (#50, #74)\n\nreviewing lehmer64 lsb 32-bits (bit reverse)\nSummary for lehmer64 lsb 32-bits (bit reverse) (4 crushes):\n- 1 unnoteworthy blips (#77)\n\nreviewing lehmer64 lsb 32-bits (byte reverse)\nSummary for lehmer64 lsb 32-bits (byte reverse) (4 crushes):\n- 3 unnoteworthy blips (#59, #64, #76)\n\nreviewing lehmer64 msb 32-bits (bit reverse)\nSummary for lehmer64 msb 32-bits (bit reverse) (4 crushes):\n- 4 unnoteworthy blips (#10, #74, #80, #91)\n\nreviewing lehmer64 msb 32-bits (byte reverse)\nSummary for lehmer64 msb 32-bits (byte reverse) (4 crushes):\n- 1 unnoteworthy blips (#12)\n```\n```\n$ ./summarize.pl testaesctr*.log\nreviewing aesctr lsb 32-bits\nSummary for aesctr lsb 32-bits (4 crushes):\n- 3 unnoteworthy blips (#10, #22, #88)\n\nreviewing aesctr lsb 32-bits (bit reverse)\nSummary for aesctr lsb 32-bits (bit reverse) (4 crushes):\n- 1 unnoteworthy blips (#11)\n```\n\n```\n$ ./results/summarize.pl testxorshift1024star*.log\nreviewing xorshift1024star lsb 32-bits\nSummary for xorshift1024star lsb 32-bits (4 crushes):\n- #81: LinearComp, r = 29: FAIL!! -- p-values too unlikely (1 - eps1, 1 - eps1, 1 - eps1, 1 - eps1) -- ALL CRUSHES FAIL!!\n\nreviewing xorshift1024star lsb 32-bits (bit reverse)\nSummary for xorshift1024star lsb 32-bits (bit reverse) (4 crushes):\n- #80: LinearComp, r = 0: FAIL!! -- p-values too unlikely (1 - eps1, 1 - eps1, 1 - eps1, 1 - eps1) -- ALL CRUSHES FAIL!!\n- 2 unnoteworthy blips (#9, #75)\n\nreviewing xorshift1024star msb 32-bits\nSummary for xorshift1024star msb 32-bits (4 crushes):\n- 2 unnoteworthy blips (#9, #48)\n\nreviewing xorshift1024star msb 32-bits (bit reverse)\nSummary for xorshift1024star msb 32-bits (bit reverse) (4 crushes):\n- 5 unnoteworthy blips (#7, #23, #78, #99, #104)\n```\n```\nreviewing xorshift1024plus lsb 32-bits (bit reverse)\nSummary for xorshift1024plus lsb 32-bits (bit reverse) (4 crushes):\n- #80: LinearComp, r = 0: FAIL!! -- p-values too unlikely (1 - eps1, 1 - eps1, 1 - eps1, 1 - eps1) -- ALL CRUSHES FAIL!!\n- 1 unnoteworthy blips (#74)\n\nreviewing xorshift1024plus msb 32-bits\nSummary for xorshift1024plus msb 32-bits (4 crushes):\n- 2 unnoteworthy blips (#77, #85)\n\nreviewing xorshift1024plus msb 32-bits (bit reverse)\nSummary for xorshift1024plus msb 32-bits (bit reverse) (4 crushes):\n- 5 unnoteworthy blips (#18, #21, #25, #51, #55)\n\nreviewing xorshift1024plus msb 32-bits (byte reverse)\nSummary for xorshift1024plus msb 32-bits (byte reverse) (4 crushes):\n- 1 unnoteworthy blips (#95)\n```\n\nThe xorshift32 generator fails very badly.\n\n## PractRand results (512 GB)\n\nSee practrand/results for detailed outputs.\n\nRaw output:\n\n```\n$ ./summarize.sh \ntestmersennetwister.log:  [Low16/64]BRank(12):12K(1)        R= +3016  p~=  6.7e-909   FAIL !!!!!!!\ntestmitchellmoore.log:  [Low1/64]BRank(12):256(2)         R= +73.5  p~=  3.8e-23    FAIL !!\ntestv8xorshift128plus-H.log:  BCFN(2+1,13-0,T)                  R= +28.7  p =  6.9e-15    FAIL !\ntestv8xorshift128plus.log:  [Low4/64]BRank(12):768(1)         R= +1272  p~=  5.4e-384   FAIL !!!!!!!\ntestxoroshiro128plus.log:  [Low4/64]BRank(12):768(1)         R= +1272  p~=  5.4e-384   FAIL !!!!!!!\ntestxorshift1024plus.log:  [Low1/64]BRank(12):1536(1)        R=+10916  p~=  3e-3287    FAIL !!!!!!!!\ntestxorshift1024star.log:  [Low4/64]BRank(12):1536(1)        R= +2650  p~=  9.8e-799   FAIL !!!!!!!\ntestxorshift128plus-H.log:  BCFN(2+1,13-0,T)                  R= +27.9  p =  1.9e-14    FAIL\ntestxorshift128plus.log:  [Low4/64]BRank(12):768(1)         R= +1272  p~=  5.4e-384   FAIL !!!!!!!\ntestxorshift32.log:  BCFN(2+0,13-2,T)                  R=+179.4  p =  2.8e-91    FAIL !!!!!\ntestxorshift-k4.log:  BRank(12):256(4)                  R= +5300  p~=  1e-2819    FAIL !!!!!!!!\ntestxorshift-k5.log:  [Low4/64]BRank(12):768(1)         R=+583.3  p~=  1.2e-176   FAIL !!!!!!\n```\n\n\n\n## Speed results\n\nFor a report on what might be the fastest generator, see [The fastest conventional random number generator that can pass Big Crush?](https://lemire.me/blog/2019/03/19/the-fastest-conventional-random-number-generator-that-can-pass-big-crush/)\n\nOn a recent (Skylake) processor, on a Linux box, I got the following results:\n\n\n```\nxorshift_k4:  1.26 cycles per byte\nxorshift_k5:  1.26 cycles per byte\nmersennetwister:  2.34 cycles per byte\nmitchellmoore:  3.49 cycles per byte\nwidynski:  1.26 cycles per byte\nxorshift32:  2.51 cycles per byte\npcg32:  1.51 cycles per byte\nrand:  4.74 cycles per byte\naesdragontamer:  0.92 cycles per byte\naesctr:  1.18 cycles per byte\nlehmer64:  0.63 cycles per byte\nxorshift128plus:  0.89 cycles per byte\nxoroshiro128plus:  0.83 cycles per byte\nsplitmix64:  1.01 cycles per byte\npcg64:  0.97 cycles per byte\nxorshift1024star:  1.74 cycles per byte\nxorshift1024plus:  1.03 cycles per byte\nwyhash64:  0.76 cycles per byte\n```\n\nResults will depend on your specific hardware and might be quite different on ARM processors. Tweaking the benchmark could also change the results. In particular, our benchmark stresses throughput as opposed to latency.\n\n\n## Visual Summary\n\n|                  | TestU01 (big crush)| PractRand (64 GB)        | time (cycles/byte) |\n|------------------|--------------------|--------------------------|--------------------|\n| wyhash (MUM)     |  :+1:              |   :+1:                   | 1.0                |\n| lehmer64         |  :+1:              |   :+1:                   | 1.0                |\n| splitmix64       |  :+1:              |   :+1:                   | 1.0                |\n| aes              |  :+1:              |   :+1:                   | 1.0                |\n| xorshift128plus  |  fails!            |   fails!                 | 1.0                |\n| v8xorshift128plus|  fails!            |   fails!                 | 1.0                |\n| xorshift1024plus |  fails!            |   fails!                 | 1.0                |\n| xoroshiro128plus |  fails!            |   fails!                 | 1.1                |\n| pcg64            |  :+1:              |   :+1:                   | 1.4                |\n| xorshift1024star |  fails!            |   fails!                 | 1.5                |\n| pcg32            |  :+1:              |   :+1:                   | 2.1                |\n| xorshift32       |  fails!            |   fails!                 | 2.5                |\n\n## Interpreting the results\n\nTests are subject to both false positives and false negatives, and should be interpreted with care.\n\n- That your random number generator passes several statistical tests does not prove that it is of high quality. There might be other tests that it would fail. However, the more tests one passes, the greater the evidence is in favor of your random number generator, evidently.\n- A statistical test might fail even if the bits are indeed \"random\". Thus a failed test is not \"proof\" that the random number generation is faulty. However, failing tests can be further investigated and evidence of a fault can be ascertained.\n\nOne source of concern is that random number generators are initialized with a seed, and sometimes a \"sequence\" is selected. Testing successfully with a given seed does not preclude the possibility that there might be \"bad seeds\". However, if we pick different seeds an a failure keeps happening, we gain confidence that the failure is easily reproducible and thus problematic. Repeated test failures with different seeds gives us confidence that there is a fault.\n\nTo summarize, caution is required when interpreting the results. It is not black and white, good and bad... One might say that a given generator passes a given test, but it is possible that with different seeds, it could fail, and so forth.\n\nStill, for convenience, it is necessary to express results in a comprehensible manner. Thus we often say that a generator \"passes a given test\" or does not.\n\n\n\n## Testing frameworks\n\nWe use the following testing framework:\n\n- [Practically Random](https://sourceforge.net/projects/pracrand/)\n- [TestU01](http://simul.iro.umontreal.ca/testu01/)\n\nAs of 2017, these represent the state-of-the-art.\n\n## Academic references\n\n- D. H. Lehmer, Mathematical methods in large-scale computing units.  Proceedings of a Second Symposium on Large Scale Digital Calculating Machinery; Annals of the Computation Laboratory, Harvard Univ. 26 (1951), pp. 141-146.\n\n\n## Talks\n\n- [Stanford Seminar - PCG: A Family of Better Random Number Generators](https://www.youtube.com/watch?v=45Oet5qjlms) (by O'Neill)\n\n## Cite this work\n\nIf you use results or code from this repository in a publication, please consider citing the project as:\n\n```\n@misc{lemire2025testingRNG,\n  author = {Daniel Lemire},\n  title = {{testingRNG: testing popular random-number generators}},\n  year = {2025},\n  howpublished = {GitHub repository},\n  note = {Commit on 2025-12-22},\n  url = {https://github.com/dlemire/testingRNG}\n}\n```\n\nYou can copy the BibTeX entry above into your references file.\n\n## Links of interest (technical)\n\n- [PRNG shootout](http://xoroshiro.di.unimi.it) by Vigna\n- How to test PCG with  [PractRand](http://www.pcg-random.org/posts/how-to-test-with-practrand.html) and [TestU01](http://www.pcg-random.org/posts/how-to-test-with-testu01.html) (by O'Neill)\n\n## Blog posts\n\n- [Testing non-cryptographic random number generators: my results](https://lemire.me/blog/2017/08/22/testing-non-cryptographic-random-number-generators-my-results/) by Lemire\n- [Cracking random number generators (xoroshiro128+)](https://lemire.me/blog/2017/08/22/cracking-random-number-generators-xoroshiro128/) by Lemire\n- [On Melissa O’Neill’s PCG random number generator](http://lemire.me/blog/2017/08/15/on-melissa-oneills-pcg-random-number-generator/) by Lemire\n- [Testing RNGs with PractRand](https://www.johndcook.com/blog/2017/08/14/testing-rngs-with-practrand/), [Manipulating a random number generator](https://www.johndcook.com/blog/2017/08/16/manipulating-a-random-number-generator/) by Cook\n- [PCG Passes PractRand](http://www.pcg-random.org/posts/pcg-passes-practrand.html) by O'Neill\n\n## More reading (interesting quotes)\n\n\u003e When a p-value is extremely close to 0 or to 1 (for example, if it is less than 10^−10), one can obviously conclude that the generator fails the test. If the p-value is suspicious but failure is not clear enough, (p = 0.0005, for example), then the test can be replicated independently until either failure becomes obvious or suspicion disappears (i.e., one finds that the suspect p-value was obtained only by chance). This approach is possible because there is no limit (other than CPU time) on the amount of data that can be produced by a RNG to increase the sample size and the power of the test. (TestU01 manual)\n\n\u003e A recent example shows the importance of testing the reverse generator. Saito and Matsumoto [2014] propose a different way to eliminate linear artifacts: instead of multiplying the output of an underlying xorshift generator (with 128 bits of state and based on 32-bit shifts) by a constant, they add it (in Z/232Z) with the previous output. Since the sum in Z/232Z is not linear over Z/2Z, the result should be free of linear artifacts. However, while their generator passes BigCrush, its reverse fails systematically the LinearComp, MatrixRank, MaxOft and Permutation test of BigCrush, which highlights a significant weakness in its lower bits. [Vigna, An experimental exploration of Marsaglia’s xorshift generators, scrambled, ACM Transactions on Mathematical Software (TOMS), 2016 ](https://arxiv.org/pdf/1402.6246.pdf)\n\n\u003e xoroshiro128plus seems like an interesting variation on LFSRs, it maintains the excessive linearity problem, but the variation in structure (compared to classic LFSRs) costs little and may significantly improve quality, not sure yet. In PractRand it quickly fails binary rank tests, and also eventually fails a short-medium range linear tests in the expanded test suite (DC6-5x4Byte-1). The author suggests avoiding reliance on the lowest bit to work around the binary rank problem, but the 2nd lowest bit also fails binary rank tests (though it takes substantially larger matrices to detect that). A non-linear output function could fix that stuff, but I'd also prefer a larger statespace. It's not clear how well this PRNG supports random access - a function to skip forward exactly 2^64 is provided, but there's no obvious way to skip forward/backward other amounts - I'd guess it's possible, but requires significant precomputation on a per-power-of-2 basis and might be slow at runtime for distances that are not neat powers of two. I think xoroshiro has a single cycle of length 2^128-1.\n[Chris Doty-Humphrey, 2016-09-07](https://sourceforge.net/p/pracrand/discussion/366935/thread/2b43b548/#a33d)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flemire%2Ftestingrng","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flemire%2Ftestingrng","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flemire%2Ftestingrng/lists"}