{"id":24408021,"url":"https://github.com/ayasyrev/benchmark_utils","last_synced_at":"2025-04-12T01:16:56.203Z","repository":{"id":52642411,"uuid":"335218771","full_name":"ayasyrev/benchmark_utils","owner":"ayasyrev","description":"Simple wrapper over timeit for benchmarking.","archived":false,"fork":false,"pushed_at":"2024-07-20T08:38:48.000Z","size":929,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-12T01:16:50.632Z","etag":null,"topics":["benchmark","timeit"],"latest_commit_sha":null,"homepage":"https://ayasyrev.github.io/benchmark_utils/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ayasyrev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-02T08:27:27.000Z","updated_at":"2023-07-28T08:31:51.000Z","dependencies_parsed_at":"2023-11-12T08:24:10.420Z","dependency_job_id":"d01ff6b2-0265-463e-9b68-7a46785d3156","html_url":"https://github.com/ayasyrev/benchmark_utils","commit_stats":{"total_commits":106,"total_committers":2,"mean_commits":53.0,"dds":"0.018867924528301883","last_synced_commit":"79c08ae211557dd1a4fc0ab5113f1220a5284667"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayasyrev%2Fbenchmark_utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayasyrev%2Fbenchmark_utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayasyrev%2Fbenchmark_utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayasyrev%2Fbenchmark_utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ayasyrev","download_url":"https://codeload.github.com/ayasyrev/benchmark_utils/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248501859,"owners_count":21114684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","timeit"],"created_at":"2025-01-20T05:18:38.144Z","updated_at":"2025-04-12T01:16:56.176Z","avatar_url":"https://github.com/ayasyrev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Benchmark utils\n\nUtils for benchmark - wrapper over python timeit.\n\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/benchmark-utils)](https://pypi.org/project/benchmark-utils/)\n[![PyPI Status](https://badge.fury.io/py/benchmark-utils.svg)](https://badge.fury.io/py/benchmark-utils)  \n[![Tests](https://github.com/ayasyrev/benchmark_utils/workflows/Tests/badge.svg)](https://github.com/ayasyrev/benchmark_utils/actions?workflow=Tests)  [![Codecov](https://codecov.io/gh/ayasyrev/benchmark_utils/branch/main/graph/badge.svg)](https://codecov.io/gh/ayasyrev/benchmark_utils)  \n\nTested on python 3.8 - 3.12\n\n## Install\n\nInstall from pypi:  \n\n`pip install benchmark_utils`\n\nOr install from github repo:\n\n`pip install git+https://github.com/ayasyrev/benchmark_utils.git`\n\n## Basic use.\n\nLets benchmark some (dummy) functions.\n\n\n```python\nfrom time import sleep\n\n\ndef func_to_test_1(sleep_time: float = 0.1, mult: int = 1) -\u003e None:\n    \"\"\"simple 'sleep' func for test\"\"\"\n    sleep(sleep_time * mult)\n\n\ndef func_to_test_2(sleep_time: float = 0.11, mult: int = 1) -\u003e None:\n    \"\"\"simple 'sleep' func for test\"\"\"\n    sleep(sleep_time * mult)\n```\n\nLet's create benchmark.\n\n\n```python\nfrom benchmark_utils import Benchmark\n```\n\n\n```python\nbench = Benchmark(\n    [func_to_test_1, func_to_test_2],\n)\n```\n\n\n```python\nbench\n```\n\u003cdetails open\u003e \u003csummary\u003eoutput\u003c/summary\u003e\n\u003cpre\u003eBenchmark(func_to_test_1, func_to_test_2)\u003c/pre\u003e\n\u003c/details\u003e\nNow we can benchmark that functions.\n\n\n```python\n# we can run bench.run() or just:\nbench()\n```\n\u003cdetails open\u003e \u003csummary\u003eoutput\u003c/summary\u003e\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e Func name  | Sec \u003cspan style=\"color: #800080; text-decoration-color: #800080\"\u003e/\u003c/span\u003e run\n\u003c/pre\u003e\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_1:   \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.10\u003c/span\u003e \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.0\u003c/span\u003e%\n\u003c/pre\u003e\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_2:   \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.11\u003c/span\u003e \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e-9.1\u003c/span\u003e%\n\u003c/pre\u003e\n\u003c/details\u003e\n\n\nWe can run it again, all functions, some of it, exclude some and change number of repeats.\n\n\n```python\nbench.run(num_repeats=10)\n```\n\u003cdetails open\u003e \u003csummary\u003eoutput\u003c/summary\u003e\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e Func name  | Sec \u003cspan style=\"color: #800080; text-decoration-color: #800080\"\u003e/\u003c/span\u003e run\n\u003c/pre\u003e\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_1:   \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.10\u003c/span\u003e \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.0\u003c/span\u003e%\n\u003c/pre\u003e\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_2:   \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.11\u003c/span\u003e \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e-9.1\u003c/span\u003e%\n\u003c/pre\u003e\n\u003c/details\u003e\n\n\nAfter run, we can print results - sorted or not, reversed, compare results with best or not. \n\n\n```python\nbench.print_results(reverse=True)\n```\n\n\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e Func name  | Sec \u003cspan style=\"color: #800080; text-decoration-color: #800080\"\u003e/\u003c/span\u003e run\n\u003c/pre\u003e\n\n\n\n\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_2:   \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.11\u003c/span\u003e \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.0\u003c/span\u003e%\n\u003c/pre\u003e\n\n\n\n\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_1:   \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.10\u003c/span\u003e \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e10.0\u003c/span\u003e%\n\u003c/pre\u003e\n\n\n\nWe can add functions to benchmark as list of functions (or partial) or as dictionary: `{\"name\": function}`.\n\n\n```python\nbench = Benchmark(\n    [\n        func_to_test_1,\n        partial(func_to_test_1, 0.12),\n        partial(func_to_test_1, sleep_time=0.11),\n    ]\n)\n```\n\n\n```python\nbench\n```\n\u003cdetails open\u003e \u003csummary\u003eoutput\u003c/summary\u003e  \n\u003cpre\u003eBenchmark(func_to_test_1, func_to_test_1(0.12), func_to_test_1(sleep_time=0.11))\u003c/pre\u003e\n\u003c/details\u003e\n\n```python\nbench.run()\n```\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e Func name  | Sec \u003cspan style=\"color: #800080; text-decoration-color: #800080\"\u003e/\u003c/span\u003e run\n\u003c/pre\u003e\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_1:   \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.10\u003c/span\u003e \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.0\u003c/span\u003e%\n\u003c/pre\u003e\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e\u003cspan style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\"\u003efunc_to_test_1\u003c/span\u003e\u003cspan style=\"font-weight: bold\"\u003e(\u003c/span\u003e\u003cspan style=\"color: #808000; text-decoration-color: #808000\"\u003esleep_time\u003c/span\u003e=\u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.11\u003c/span\u003e\u003cspan style=\"font-weight: bold\"\u003e)\u003c/span\u003e:   \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.11\u003c/span\u003e \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e-9.1\u003c/span\u003e%\n\u003c/pre\u003e\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e\u003cspan style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\"\u003efunc_to_test_1\u003c/span\u003e\u003cspan style=\"font-weight: bold\"\u003e(\u003c/span\u003e\u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.12\u003c/span\u003e\u003cspan style=\"font-weight: bold\"\u003e)\u003c/span\u003e:   \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e0.12\u003c/span\u003e \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e-16.7\u003c/span\u003e%\n\u003c/pre\u003e\n\n```python\nbench = Benchmark(\n    {\n        \"func_1\": func_to_test_1,\n        \"func_2\": func_to_test_2,\n    }\n)\n```\n\n\n```python\nbench\n```\n\u003cdetails open\u003e \u003csummary\u003eoutput\u003c/summary\u003e  \n\u003cpre\u003eBenchmark(func_1, func_2)\u003c/pre\u003e\n\u003c/details\u003e\n\n\n\nWhen we run benchmark script in terminal, we got pretty progress thanks to rich. Lets run example_1.py from example folder:\n\n![example_1](images/run_example_1.gif)\n\n## BenchmarkIter\n\nWith BenchmarkIter we can benchmark functions over iterables, for example read list of files or run functions with different arguments.\n\n\n```python\ndef func_to_test_1(x: int) -\u003e None:\n    \"\"\"simple 'sleep' func for test\"\"\"\n    sleep(0.01)\n\n\ndef func_to_test_2(x: int) -\u003e None:\n    \"\"\"simple 'sleep' func for test\"\"\"\n    sleep(0.015)\n\n\ndummy_params = list(range(10))\n```\n\n\n```python\nfrom benchmark_utils import BenchmarkIter\n\nbench = BenchmarkIter(\n    func=[func_to_test_1, func_to_test_2],\n    item_list=dummy_params,\n)\n```\n\n\n```python\nbench()\n```\n\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e Func name  | Items/sec\n\u003c/pre\u003e\n\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_1:  \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e97.93\u003c/span\u003e\n\u003c/pre\u003e\n\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_2:  \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e65.25\u003c/span\u003e\n\u003c/pre\u003e\n\nWe can run it again, all functions, some of it, exclude some and change number of repeats.\nAnd we can limit number of items with `num_samples` argument:\n`bench.run(num_samples=5)`\n\n## Multiprocessing\n\nBy default we tun functions in one thread.  \nBut we can use multiprocessing with `multiprocessing=True` argument:\n`bench.run(multiprocessing=True)`\nIt will use all available cpu cores.\nAnd we can use `num_workers` argument to limit used cpu cores:\n`bench.run(multiprocessing=True, num_workers=2)`\n\n\n```python\nbench.run(multiprocessing=True, num_workers=2)\n```\n\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e Func name  | Items/sec\n\u003c/pre\u003e\n\n\n\n\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_1: \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e173.20\u003c/span\u003e\n\u003c/pre\u003e\n\n\n\n\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003efunc_to_test_2: \u003cspan style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\"\u003e120.80\u003c/span\u003e\n\u003c/pre\u003e\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fayasyrev%2Fbenchmark_utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fayasyrev%2Fbenchmark_utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fayasyrev%2Fbenchmark_utils/lists"}