{"id":19164456,"url":"https://github.com/cupy/cupy-performance","last_synced_at":"2025-05-07T12:10:49.148Z","repository":{"id":48863750,"uuid":"260392569","full_name":"cupy/cupy-performance","owner":"cupy","description":null,"archived":false,"fork":false,"pushed_at":"2022-02-28T04:26:30.000Z","size":40,"stargazers_count":5,"open_issues_count":2,"forks_count":3,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-28T23:25:36.242Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cupy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-05-01T06:14:58.000Z","updated_at":"2024-10-27T11:37:42.000Z","dependencies_parsed_at":"2022-09-05T05:31:03.297Z","dependency_job_id":null,"html_url":"https://github.com/cupy/cupy-performance","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cupy%2Fcupy-performance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cupy%2Fcupy-performance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cupy%2Fcupy-performance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cupy%2Fcupy-performance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cupy","download_url":"https://codeload.github.com/cupy/cupy-performance/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252873949,"owners_count":21817714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T09:22:08.975Z","updated_at":"2025-05-07T12:10:49.122Z","avatar_url":"https://github.com/cupy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Easy benchmark framework for cupy\n## Features\n+ Reports both, cpu and gpu time.\n+ Produces plots of the execution time, speedup or custom metrics.\n+ Saves the results in csv files.\n+ Allows automatic performance comparison with numpy or numpy API compat\nlibraries.\n+ TODO: CPU routines profiling.\n+ TODO: Profile kernels using nvprof.\n+ TODO: Performance regression detection.\n\n## How to use\n### Benchmark creation\nSimilarly to pytest, benchmarks are stored in `bench_*.py` files.\nInside a bench file `class` definitions with `Benchmark` on their name are\nconsidered as such.\n\nThere are 3 special methods in the benchmark\n`setup` used to generate the actual inputs to the routines and cupy memory allocations.\n`teardown` used to clean up the state and free the allocated memory.\n`args_key` which returns a string to name the current benchmark case (parameters combination)\n\n`setup` and `teardown` are called before and after every function in the class starting with\n`time_`. These `time_` functions are in charge of doing the actual benchmark and they will be\nexecuted multiple times to get statistically significant results. Therefore object state modification\nshould be avoided.\n\nBenchmars are parametrized by using the `params` class attribute. This attribute is a dictionary\nwhose entries are each a list of possible values for that parameter name. The  cross-product of all the\nentries is calculated before running the benchmark, and every benchmark is run for all the possible\ncombinations of the parameters. The parameters are set before calling the `setup` functions and the\ncurrent value is accessed with `self.parameter_name`.\n\n### Types of benchmarks\nBenchmarks should inherit from one of these base classes.\n+ `cupy_prof.benchmark.CupyBenchmark` Basic benchmark class for testing cupy only that plots\na logarithmic time graph of both the cpu and gpu time.\n+ `NumpyCompareBenchmark` performs a comparison within numpy and cupy for the specified routine.\nnumpy or cupy namespaces are accessed in the benchmark with `self.xp` as in chainer tests.\nIt aditionally calculates the speedup and plot it too.\n\n### Plot parameters\nBenchmarks are plotted according the `_plots` class variable in the benchmarks.\n```python\n    _plots = [{'facet': {'col': 'name', 'hue': 'backend'},\n               'plot': 'line',\n               'x': 'key',\n               'y': 'time',\n               'yscale': 'log'},\n              {'facet': {'col': 'name', 'hue': 'xp'},\n               'plot': 'bar',\n               'x': 'key',\n               'y': 'speedup'}]\n```\n`_plots` is a list with all the graphs that will be generated for the class.\nthe `facet` entry is in charge of subplotting all the `def bench_` results in \ndifferent sub-graphs. The first example says that for this facet, the column is the\nbenchmark name (method name in the class definition) and the `hue`, or the legend is the\nbackend, with is either `numpy`, `cupy-gpu`, `cupy-cpu`. For each of the graph, the x \nis the `key` values obtained by calling the benchmark `args_key` method. and the `y` is the execution time.\n\nThis values can be altered in the setup to generate different kinds of graphs.\n\n### Running benchmarks\n\n```\n$ python prof.py benchmarks\n```\nAlternatively, if a directory is specified, it will collect all the benchmarks with the file name\nstarting with `bench_` as in pytest.\n\n### Comparing different commits or branches\n\n```\npython prof.py --repo /home/ecastill/em-cupy --commits master v7 --plot -- benchmarks/bench_ufunc_cupy.py\n```\n\nIs it possible to compare different commits or branches of a repository.\nThis script will automatically checkout the branch and compile cupy, but it will install it in a virtual environment\n`virtualenv` is required for this functionaility.\n\n`--repo` can specify a common repo for all the commits, or a list of repositories, one per-commit, so that build time\ncan be reduced.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcupy%2Fcupy-performance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcupy%2Fcupy-performance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcupy%2Fcupy-performance/lists"}