{"id":19289802,"url":"https://github.com/ragibson/levy-stable-benchmarks","last_synced_at":"2025-04-12T03:09:47.243Z","repository":{"id":143220408,"uuid":"285120420","full_name":"ragibson/levy-stable-benchmarks","owner":"ragibson","description":"Accuracy and performance benchmark of stable (\"fat-tailed\") distribution libraries in Python.","archived":false,"fork":false,"pushed_at":"2024-01-29T22:58:37.000Z","size":38323,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-12T03:09:23.904Z","etag":null,"topics":["accuracy","benchmark","heavy-tailed-distributions","stable-distribution","statistics"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ragibson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-04T23:02:15.000Z","updated_at":"2024-08-07T19:13:49.000Z","dependencies_parsed_at":"2025-01-05T19:51:34.450Z","dependency_job_id":null,"html_url":"https://github.com/ragibson/levy-stable-benchmarks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ragibson%2Flevy-stable-benchmarks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ragibson%2Flevy-stable-benchmarks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ragibson%2Flevy-stable-benchmarks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ragibson%2Flevy-stable-benchmarks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ragibson","download_url":"https://codeload.github.com/ragibson/levy-stable-benchmarks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248510001,"owners_count":21116131,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accuracy","benchmark","heavy-tailed-distributions","stable-distribution","statistics"],"created_at":"2024-11-09T22:17:12.095Z","updated_at":"2025-04-12T03:09:47.216Z","avatar_url":"https://github.com/ragibson.png","language":"Python","readme":"# levy-stable-benchmarks\n\nStable distributions (sometimes called Lévy alpha-stable distributions) are\nimportant for modelling data across several disciplines including signal\nprocessing, physics, and finance. Despite this, many Python libraries\nprovide buggy and/or inaccurate implementations for computing its PDF/CDF.\n\nThis repository attempts to define a benchmark to test the accuracy of\nsuch implementations. We also provide some alternative calculation methods.\n\n![accuracy figures](figures/accuracy_figures.png)\n\nAmong other things, this repository has helped to\n * Identify a typo in Theorem 1, case (d) of Nolan's original paper, [Numerical calculation of stable densities and distribution functions](https://www.tandfonline.com/doi/abs/10.1080/15326349708807450). This is corrected in [Nolan's new textbook](https://www.springer.com/us/book/9783030529147).\n * Further progress on [a scipy PR to improve stable distribution computations](https://github.com/scipy/scipy/pull/9523).\n * Identify [some issues in pylevy](https://github.com/josemiotto/pylevy/issues/15), a Python package for applications involving stable distributions.\n\n# Table of Contents\n * [CDF accuracy percentages](#CDFAccuracy)\n * [PDF accuracy percentages](#PDFAccuracy)\n * [Average runtimes](#AverageRuntimes)\n * [FAQ, notes, and limitations](#FAQ)\n\n\u003ca name = \"CDFAccuracy\"\u003e\u003c/a\u003e\n## CDF accuracy percentages\n\n### CDF table (-100 \u003c= x \u003c= 100)\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eAbsolute\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-2\u003c/td\u003e\u003ctd\u003e1E-3\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003ctd\u003e1E-5\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.6%\u003c/td\u003e\u003ctd\u003e99.4%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e98.9%\u003c/td\u003e\u003ctd\u003e69.4%\u003c/td\u003e\u003ctd\u003e21.3%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003elarger_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e99.6%\u003c/td\u003e\u003ctd\u003e76.8%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e99.7%\u003c/td\u003e\u003ctd\u003e98.1%\u003c/td\u003e\u003ctd\u003e97.3%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e83.5%\u003c/td\u003e\u003ctd\u003e78.0%\u003c/td\u003e\u003ctd\u003e68.9%\u003c/td\u003e\u003ctd\u003e58.1%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eRelative\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-1\u003c/td\u003e\u003ctd\u003e1E-2\u003c/td\u003e\u003ctd\u003e1E-3\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e98.9%\u003c/td\u003e\u003ctd\u003e98.9%\u003c/td\u003e\u003ctd\u003e98.9%\u003c/td\u003e\u003ctd\u003e98.6%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e91.8%\u003c/td\u003e\u003ctd\u003e80.5%\u003c/td\u003e\u003ctd\u003e55.7%\u003c/td\u003e\u003ctd\u003e34.0%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003elarger_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e99.6%\u003c/td\u003e\u003ctd\u003e94.6%\u003c/td\u003e\u003ctd\u003e80.1%\u003c/td\u003e\u003ctd\u003e58.6%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e96.7%\u003c/td\u003e\u003ctd\u003e96.7%\u003c/td\u003e\u003ctd\u003e96.7%\u003c/td\u003e\u003ctd\u003e96.3%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e81.8%\u003c/td\u003e\u003ctd\u003e66.7%\u003c/td\u003e\u003ctd\u003e54.0%\u003c/td\u003e\u003ctd\u003e42.3%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n### CDF quantile table (0.001 \u003c= p \u003c= 0.999)\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eAbsolute\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-2\u003c/td\u003e\u003ctd\u003e1E-3\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003ctd\u003e1E-5\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e99.0%\u003c/td\u003e\u003ctd\u003e99.0%\u003c/td\u003e\u003ctd\u003e99.0%\u003c/td\u003e\u003ctd\u003e99.0%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e95.1%\u003c/td\u003e\u003ctd\u003e15.4%\u003c/td\u003e\u003ctd\u003e1.6%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003elarger_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e36.3%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.7%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e85.9%\u003c/td\u003e\u003ctd\u003e80.7%\u003c/td\u003e\u003ctd\u003e78.8%\u003c/td\u003e\u003ctd\u003e77.5%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eRelative\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-1\u003c/td\u003e\u003ctd\u003e1E-2\u003c/td\u003e\u003ctd\u003e1E-3\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e99.0%\u003c/td\u003e\u003ctd\u003e99.0%\u003c/td\u003e\u003ctd\u003e99.0%\u003c/td\u003e\u003ctd\u003e99.0%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e99.5%\u003c/td\u003e\u003ctd\u003e49.9%\u003c/td\u003e\u003ctd\u003e9.1%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003elarger_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e100.0%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.6%\u003c/td\u003e\u003ctd\u003e81.2%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e92.7%\u003c/td\u003e\u003ctd\u003e83.0%\u003c/td\u003e\u003ctd\u003e79.8%\u003c/td\u003e\u003ctd\u003e78.1%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n### Nolan CDF quantile table (0.00001 \u003c= p \u003c= 0.99999)\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eAbsolute\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-2\u003c/td\u003e\u003ctd\u003e1E-3\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003ctd\u003e1E-5\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e95.7%\u003c/td\u003e\u003ctd\u003e95.4%\u003c/td\u003e\u003ctd\u003e95.2%\u003c/td\u003e\u003ctd\u003e95.1%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e99.4%\u003c/td\u003e\u003ctd\u003e96.5%\u003c/td\u003e\u003ctd\u003e42.1%\u003c/td\u003e\u003ctd\u003e11.6%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003elarger_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e99.4%\u003c/td\u003e\u003ctd\u003e99.1%\u003c/td\u003e\u003ctd\u003e98.9%\u003c/td\u003e\u003ctd\u003e55.0%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e99.4%\u003c/td\u003e\u003ctd\u003e99.0%\u003c/td\u003e\u003ctd\u003e98.1%\u003c/td\u003e\u003ctd\u003e95.0%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e91.5%\u003c/td\u003e\u003ctd\u003e87.9%\u003c/td\u003e\u003ctd\u003e83.4%\u003c/td\u003e\u003ctd\u003e77.7%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eRelative\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-1\u003c/td\u003e\u003ctd\u003e1E-2\u003c/td\u003e\u003ctd\u003e1E-3\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e95.7%\u003c/td\u003e\u003ctd\u003e95.4%\u003c/td\u003e\u003ctd\u003e95.3%\u003c/td\u003e\u003ctd\u003e95.1%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e94.3%\u003c/td\u003e\u003ctd\u003e87.5%\u003c/td\u003e\u003ctd\u003e50.9%\u003c/td\u003e\u003ctd\u003e21.6%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003elarger_monte_carlo\u003c/td\u003e\n    \u003ctd\u003e99.4%\u003c/td\u003e\u003ctd\u003e96.6%\u003c/td\u003e\u003ctd\u003e87.2%\u003c/td\u003e\u003ctd\u003e68.7%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e97.6%\u003c/td\u003e\u003ctd\u003e97.2%\u003c/td\u003e\u003ctd\u003e97.0%\u003c/td\u003e\u003ctd\u003e96.6%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e93.7%\u003c/td\u003e\u003ctd\u003e86.0%\u003c/td\u003e\u003ctd\u003e79.5%\u003c/td\u003e\u003ctd\u003e71.5%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ca name = \"PDFAccuracy\"\u003e\u003c/a\u003e\n## PDF accuracy percentages\n\n### PDF table (-100 \u003c= x \u003c= 100)\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eAbsolute\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003ctd\u003e1E-5\u003c/td\u003e\u003ctd\u003e1E-6\u003c/td\u003e\u003ctd\u003e1E-7\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_dni\u003c/td\u003e\n    \u003ctd\u003e87.5%\u003c/td\u003e\u003ctd\u003e87.5%\u003c/td\u003e\u003ctd\u003e87.5%\u003c/td\u003e\u003ctd\u003e87.5%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e90.8%\u003c/td\u003e\u003ctd\u003e79.8%\u003c/td\u003e\u003ctd\u003e67.7%\u003c/td\u003e\u003ctd\u003e37.5%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eRelative\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-1\u003c/td\u003e\u003ctd\u003e1E-2\u003c/td\u003e\u003ctd\u003e1E-3\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e97.5%\u003c/td\u003e\u003ctd\u003e97.5%\u003c/td\u003e\u003ctd\u003e97.4%\u003c/td\u003e\u003ctd\u003e97.4%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e99.3%\u003c/td\u003e\u003ctd\u003e99.3%\u003c/td\u003e\u003ctd\u003e99.3%\u003c/td\u003e\u003ctd\u003e99.3%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_dni\u003c/td\u003e\n    \u003ctd\u003e84.0%\u003c/td\u003e\u003ctd\u003e84.0%\u003c/td\u003e\u003ctd\u003e84.0%\u003c/td\u003e\u003ctd\u003e84.0%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e79.5%\u003c/td\u003e\u003ctd\u003e48.8%\u003c/td\u003e\u003ctd\u003e24.4%\u003c/td\u003e\u003ctd\u003e11.5%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n### PDF quantile table (0.001 \u003c= p \u003c= 0.999)\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eAbsolute\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003ctd\u003e1E-5\u003c/td\u003e\u003ctd\u003e1E-6\u003c/td\u003e\u003ctd\u003e1E-7\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.7%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_dni\u003c/td\u003e\n    \u003ctd\u003e67.6%\u003c/td\u003e\u003ctd\u003e67.5%\u003c/td\u003e\u003ctd\u003e67.5%\u003c/td\u003e\u003ctd\u003e67.5%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e85.4%\u003c/td\u003e\u003ctd\u003e82.5%\u003c/td\u003e\u003ctd\u003e77.5%\u003c/td\u003e\u003ctd\u003e55.9%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eRelative\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-1\u003c/td\u003e\u003ctd\u003e1E-2\u003c/td\u003e\u003ctd\u003e1E-3\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e98.6%\u003c/td\u003e\u003ctd\u003e98.5%\u003c/td\u003e\u003ctd\u003e98.4%\u003c/td\u003e\u003ctd\u003e98.4%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_dni\u003c/td\u003e\n    \u003ctd\u003e67.0%\u003c/td\u003e\u003ctd\u003e66.7%\u003c/td\u003e\u003ctd\u003e66.6%\u003c/td\u003e\u003ctd\u003e66.5%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e85.2%\u003c/td\u003e\u003ctd\u003e79.1%\u003c/td\u003e\u003ctd\u003e77.7%\u003c/td\u003e\u003ctd\u003e75.5%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n### Nolan PDF quantile table (0.00001 \u003c= p \u003c= 0.99999)\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eAbsolute\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003ctd\u003e1E-5\u003c/td\u003e\u003ctd\u003e1E-6\u003c/td\u003e\u003ctd\u003e1E-7\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e99.9%\u003c/td\u003e\u003ctd\u003e99.8%\u003c/td\u003e\u003ctd\u003e99.7%\u003c/td\u003e\u003ctd\u003e99.6%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_dni\u003c/td\u003e\n    \u003ctd\u003e64.5%\u003c/td\u003e\u003ctd\u003e64.5%\u003c/td\u003e\u003ctd\u003e64.5%\u003c/td\u003e\u003ctd\u003e64.5%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e91.9%\u003c/td\u003e\u003ctd\u003e87.0%\u003c/td\u003e\u003ctd\u003e77.9%\u003c/td\u003e\u003ctd\u003e54.4%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003e\u003c/td\u003e\u003ctd colspan=\"4\"\u003e\u003cb\u003eRelative\u003c/b\u003e Tolerance\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003e1E-1\u003c/td\u003e\u003ctd\u003e1E-2\u003c/td\u003e\u003ctd\u003e1E-3\u003c/td\u003e\u003ctd\u003e1E-4\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003esimple_quadrature\u003c/td\u003e\n    \u003ctd\u003e92.5%\u003c/td\u003e\u003ctd\u003e91.6%\u003c/td\u003e\u003ctd\u003e91.6%\u003c/td\u003e\u003ctd\u003e91.4%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_piecewise\u003c/td\u003e\n    \u003ctd\u003e99.1%\u003c/td\u003e\u003ctd\u003e99.1%\u003c/td\u003e\u003ctd\u003e99.0%\u003c/td\u003e\u003ctd\u003e99.0%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003escipy_dni\u003c/td\u003e\n    \u003ctd\u003e58.5%\u003c/td\u003e\u003ctd\u003e57.8%\u003c/td\u003e\u003ctd\u003e57.6%\u003c/td\u003e\u003ctd\u003e57.1%\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003epylevy_miotto\u003c/td\u003e\n    \u003ctd\u003e88.3%\u003c/td\u003e\u003ctd\u003e79.8%\u003c/td\u003e\u003ctd\u003e67.5%\u003c/td\u003e\u003ctd\u003e55.3%\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ca name = \"AverageRuntimes\"\u003e\u003c/a\u003e\n## Average runtimes\n\nThe average per-call (single-threaded) runtimes on the benchmark tables are as follows.\n\n\u003ctable\u003e\n  \u003ctr\u003e\u003ctd\u003eMethod\u003c/td\u003e\u003ctd\u003eCDF average runtime\u003c/td\u003e\u003ctd\u003ePDF average runtime\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003esimple_quadrature\u003c/td\u003e\u003ctd\u003e2.18 ms\u003c/td\u003e\u003ctd\u003e1.77 ms\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003esimple_monte_carlo\u003c/td\u003e\u003ctd\u003e0.055 ms\u003csup\u003e*\u003c/sup\u003e\u003c/td\u003e\u003ctd\u003e-\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003escipy_piecewise\u003c/td\u003e\u003ctd\u003e0.70 ms\u003c/td\u003e\u003ctd\u003e1.15 ms\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003escipy_dni\u003c/td\u003e\u003ctd\u003e-\u003c/td\u003e\u003ctd\u003e4.89 ms\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003epylevy_miotto\u003c/td\u003e\u003ctd\u003e1.23 ms\u003c/td\u003e\u003ctd\u003e1.22 ms\u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\n\u003csup\u003e*\u003c/sup\u003e This method takes ~100-150 ms for single calls, but is very fast on repeated (alpha, beta) values. See [this FAQ question](#FAQ8) for more details.\n\n\u003ca name = \"FAQ\"\u003e\u003c/a\u003e\n## FAQ: notes and limitations\n\n * [How are \"accuracy percentage\" and \"composite accuracy\" defined?](#FAQ1)\n * [Where did these PDF/CDF tables come from? Are they accurate?](#FAQ2)\n * [What are some known limitations of this benchmark?](#FAQ3)\n * [Why is the range of tested absolute tolerances different for CDF vs. PDF?](#FAQ4)\n * [Where can I find the libraries tested?](#FAQ5)\n * [The literature is very inconsistent/fragmented with respect to parameterizing stable distributions. Are you sure the libraries are actually consistent in their calculations here?](#FAQ6)\n * [simple_quadrature _usually_ seems accurate. When/where is it inaccurate?](#FAQ7)\n * [simple_monte_carlo appears far slower in practice than listed here. Why?](#FAQ8)\n * [These methods vary greatly in their speed. What is a \"good\" average time per call?](#FAQ9)\n * [Some of the methods only appear in the PDF or CDF tests. Why?](#FAQ10)\n * [I know of a Python library that is missing from this benchmark. Can you add it?](#FAQ11)\n\n\u003ca name = \"FAQ1\"\u003e\u003c/a\u003e\n##### How are \"accuracy percentage\" and \"composite accuracy\" defined?\n\n\"Accuracy percentage\" is the percentage of tabulated values computed to the\ndesired tolerance.\n\nWhen listed above, the accuracy percentages are truncated (not rounded), so a\nmethod will only have 100.0% accuracy if it is within the specified tolerance\non *all* the test cases.\n\n\"Composite accuracy\" is the average accuracy percentage on all the PDF or CDF\ntables. There are three tables each for PDF and CDF, so ~33% of the weighting\ngoes to each.\n\n\u003ca name = \"FAQ2\"\u003e\u003c/a\u003e\n##### Where did these PDF/CDF tables come from? Are they accurate?\n\nThese tables are obtained from Nolan's STABLE program, which has been the gold standard for computing this distribution's functions since its release.\n\nSTABLE rounds certain inputs to avoid parameter regions where his method has difficulties. The specifics are complicated (see his README) and certain parts of his algorithm are undocumented, but can be partially reverse-engineered through his program's debug messages. Regardless, I believe these have very little effect on the overall accuracy of the tables.\n\nMore imporantly, Nolan's methods have significant inaccuracies in the distribution tails for some parameter values. For instance, alpha=1.0 with nonzero beta often has an (incorrect) discontinuity, resulting in incorrect values beyond p=1% and p=99%.\n\nWe plotted this specific issue in [a related scipy PR](https://github.com/scipy/scipy/pull/9523#issuecomment-683491485) and copy this inline below.\n\n![STABLE CDF inaccuracy near alpha=1](figures/STABLE_discontinuity_alpha_one.png)\n\nWe have also compiled [plots of all STABLE CDF errors larger than 1e-4](figures/STABLE_table_CDF_errors_larger_than_1e-4.pdf) in the tables that use the most recent (publicly available) version of the program (v3.14.02).\n\nNolan's published CDF/PDF quantiles table use a much older version of the program and contain other/larger inaccuracies.\n\nThere are more inaccuracies further in the tails for other parameter values (notably as alpha gets smaller), but they are less impactful. For such parameter choices, Nolan's integrands become very pathological and are beyond the capabilities of most general quadrature routines.\n\n\u003ca name = \"FAQ3\"\u003e\u003c/a\u003e\n##### What are some known limitations of this benchmark?\n\nOur tables lack beta \u003c 0 values, so we implicitly assume that implementations handle negative beta values correctly. This distribution has a certain \"symmetry\" in beta, so one could easily test these values as well, but it would double the running time.\n\nThe behavior **very** far out (p \u003c 0.00001) into the tails is not tested, but this is probably a minor concern in most applications\n\nThere are known errors in the tables that we use to test accuracy, but I believe this also has little effect (see the FAQ question on the PDF/CDF tables).\n\n\u003ca name = \"FAQ4\"\u003e\u003c/a\u003e\n##### Why is the range of tested absolute tolerances different for CDF vs. PDF?\n\nThis is an ad-hoc choice. It was assumed that absolute accuracies of 0.01 for the CDF and 0.0001 for the PDF are near the lower end of usefulness.\n\nIt's worth noting that simply returning the normal distribution (with e.g. `norm.pdf(1, scale=sqrt(2))`) yields composite accuracies of ~20% and ~30% at this accuracy level for the PDF and CDF, respectively.\n\n\u003ca name = \"FAQ5\"\u003e\u003c/a\u003e\n##### Where can I find the libraries tested?\n\nThere are six methods tested here. See the links below and the code in [algorithms](algorithms).\n\n * Our simple methods (relatively \"simple\" compared to existing techniques)\n   * [**simple_quadrature**](algorithms/simple_quadrature.py): direct numerical integration based on our [integrand derivation directly from the characteristic function](figures/simple_quadrature_derivation/simple_quadrature_derivation.pdf)\n   * [**simple_monte_carlo**](algorithms/simple_monte_carlo.py): monte carlo scheme based on the [Chambers-Mallows-Stuck method of simulating stable random variables](https://doi.org/10.1080%2F01621459.1976.10480344)\n   * **larger_monte_carlo**: same as simple_monte_carlo, but with a sample size of 100 million\n * Scipy's methods (tested on version 1.12.0)\n   * [**scipy_piecewise**](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.levy_stable.html) with `pdf_default_method = \"piecewise\"` and `cdf_default_method = \"piecewise\"`\n   * [**scipy_dni**](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.levy_stable.html) with `pdf_default_method = \"dni\"`\n * [**pylevy_miotto**](https://github.com/josemiotto/pylevy) (tested on commit [d6b855e](https://github.com/josemiotto/pylevy/commit/d6b855ef2ff959db7c4002c1b39b98bd968f60b0))\n * TODO: add the unofficial pystable_jones?\n\n\u003ca name = \"FAQ6\"\u003e\u003c/a\u003e\n##### The literature is very inconsistent/fragmented with respect to parameterizing stable distributions. Are you sure the libraries are actually consistent in their calculations here?\n\nMark Veillette says this well:\n\n\u003e One of the most frustrating issues in dealing with alpha-stable distribtuions is that its parameterization is not consistent across the literature (there are over half a dozen parameterizations). [...] The most common way to specify a parameterization is to look at the characteristic function of the alpha-stable random variable.\n\u003e\n\u003e One further annoyance is that the names of the 4 parameters are also inconsistent. [...] The letters alpha and beta are used almost everywhere you look, while the other two parameters are almost always different.\n\nTo this end, we've written some [tests to prove our transformations are good](tests/parameterization_tests.py). At the moment, all the libraries here appear use either the S0 or S1 parameterization (in Nolan's notation).\n\n\u003ca name = \"FAQ7\"\u003e\u003c/a\u003e\n##### simple_quadrature _usually_ seems accurate. When/where is it inaccurate?\n\nMost notably, simple_quadrature can completely fail near x = ? (TODO: this has been observed, but where and why?) and in the tails when the oscillatory components of the integrand become incredibly unwieldy.\n\nOne could probably create a hybrid scheme that uses known asymptotic tail results to improve accuracy here, though it's unclear how successful or performant such an approach would be.\n\nIf accuracy is critical, one should really use a custom integrator for computing this distribution's functions as there are some nice properties of the integrands that so far have not been exploited. To date, only general quadrature routines have been used. Indeed, most of the significant issues in Nolan's STABLE program appear to arise due to numerical quadrature failures.\n\n\u003ca name = \"FAQ8\"\u003e\u003c/a\u003e\n##### simple_monte_carlo appears far slower in practice than listed here. Why?\n\nsimple_monte_carlo is an incredibly inefficient method for computing the CDF in general. It must generate new random samples for each unique pair of (alpha, beta) values.\n\nHere, the test tables include many repeated (alpha, beta) pairs, so the method appears to run more quickly *on average* than it otherwise might.\n\n\u003ca name = \"FAQ9\"\u003e\u003c/a\u003e\n##### These methods vary greatly in their speed. What is a \"good\" average time per call?\n\nThis is highly domain specific -- some applications might need very quick calculations and can tolerate large inaccuracies. Others might require high accuracy and have speed only as a secondary consideration.\n\nThe tests here were run on a machine with a i7-9700K (8 cores, stock, up to 4.9 GHz) CPU and 16 GB DDR4 3200 MHz of RAM.\n\nNolan claims to have\n\n\u003e code to quickly approximate stable densities.  This routine is much faster than the regular density calculations: approximately 1 million density evaluations/second can be performed on a 1 GHz Pentium.\n\nwhich appears to suggest that an average time per call of \u003c1 us (!) is \"easily\" feasible on modern hardware. However, this is several orders of magnitude faster than any of the methods tested here.\n\n\u003ca name = \"FAQ10\"\u003e\u003c/a\u003e\n##### Some of the methods only appear in the PDF or CDF tests. Why?\n\nSome methods only support one or the other. In general, computing the CDF of this distribution is much easier than computing the PDF.\n\n\u003ca name = \"FAQ11\"\u003e\u003c/a\u003e\n##### I know of a Python library that is missing from this benchmark. Can you add it?\n\nYes (assuming it's publicly available), please raise an issue and I'll try to add it.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fragibson%2Flevy-stable-benchmarks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fragibson%2Flevy-stable-benchmarks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fragibson%2Flevy-stable-benchmarks/lists"}