{"id":18421952,"url":"https://github.com/spcl/npbench","last_synced_at":"2025-04-05T20:04:15.157Z","repository":{"id":38840427,"uuid":"362399462","full_name":"spcl/npbench","owner":"spcl","description":"NPBench - A Benchmarking Suite for High-Performance NumPy","archived":false,"fork":false,"pushed_at":"2025-03-27T21:47:37.000Z","size":22573,"stargazers_count":80,"open_issues_count":16,"forks_count":33,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-03-29T19:01:49.622Z","etag":null,"topics":["benchmarking-framework","benchmarking-suite","numpy","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spcl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-28T08:47:10.000Z","updated_at":"2025-03-16T18:02:33.000Z","dependencies_parsed_at":"2023-11-14T00:26:27.672Z","dependency_job_id":"19e764f5-953c-4b1e-9b2f-26ef0941d7d4","html_url":"https://github.com/spcl/npbench","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spcl%2Fnpbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spcl%2Fnpbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spcl%2Fnpbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spcl%2Fnpbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spcl","download_url":"https://codeload.github.com/spcl/npbench/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393551,"owners_count":20931811,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking-framework","benchmarking-suite","numpy","python"],"created_at":"2024-11-06T04:27:24.977Z","updated_at":"2025-04-05T20:04:15.150Z","avatar_url":"https://github.com/spcl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"npbench.svg\" alt=\"npbench-logo\" width=\"100\"/\u003e\n\u003ch1\u003eNPBench\u003c/h1\u003e\n\n## Quickstart\n\nTo install NPBench, simply execute:\n\n```\npython -m pip install -r requirements.txt\npython -m pip install .\n```\n\nYou can then run a subset of the benchmarks with NumPy, Numba, and DaCe and plot\nthe speedup of DaCe and Numba against NumPy:\n\n``` bash\npython -m pip install numba\npython -m pip install dace\npython quickstart.py\npython plot_results.py\n```\n\n## Supported Frameworks\n\nCurrently, the following frameworks are supported (in alphabetical order):\n- CuPy\n- DaCe\n- Dpnp\n- JAX\n- Numba\n- NumPy\n- Pythran\n\nSupport will also be added shortly for:\n- Legate\n\nPlease note that the NPBench setup only installs NumPy.\nTo run benchmarks with other frameworks, you have to install them separately.\nBelow, we provide some tips about installing each of the above frameworks:\n\n### CuPy\n\nIf you already have CUDA installed, then you can install CuPy with pip:\n```\npython -m pip install cupy-cuda\u003cversion\u003e\n```\nFor example, if you have CUDA 11.1, then you should install CuPy with:\n```\npython -m pip install cupy-cuda111\n```\nFor more installation options, consult the CuPy [installation guide](https://docs.cupy.dev/en/stable/install.html#install-cupy).\n\n### DaCe\n\nDaCe can be install with pip:\n```\npython -m pip install dace\n```\nHowever, you may want to install the latest version from the [GitHub repository](https://github.com/spcl/dace).\nTo run NPBench with DaCe, you have to select as framework (see details below)\neither `dace_cpu` or `dace_gpu`.\n\n### DPNP\n\nWith `dpnp` it is strongly recommended to use `conda` instead of `pip` for its dependency on intel packages. \nRefer to this \n[LINK](https://intelpython.github.io/dpnp/quick_start_guide.html#building-for-custom-sycl-targets) to know more \nabout building custom SYCL targets or installing `dpnp` package from the `intel` channel.\n\nUnlike the pip installation, with conda it is advisable to try installing all packages at once.\nEdit the `environment.yml` to include packages and optional dependencies (e.g. hardware-dependent frameworks\nor utilities such as `ipython`). Then type:\n\n``` bash\n$ conda env create -f environment.yml    # environment.yml contains all the right dependencies\n$ conda activate npb              # Activate the environment\n$ python -m pip install pygount          # Only dependency not distributed with conda\n```\n\nTo run NPBench with dpnp, You must select as framework, either `dpnp_cpu` or `dpnp_gpu`, depending on your hardware. See details below.\n\n_DPNP only contains a subset of the benchmarks, selected on interest and best-effort basis._\n\n### Jax\n\nJAX can be installed with pip:\n- CPU-only (Linux/macOS/Windows)\n    ```sh\n    pip install -U jax\n    ```\n- GPU (NVIDIA, CUDA 12)\n    ```sh\n    pip install -U \"jax[cuda12]\"\n    ```\n- TPU (Google Cloud TPU VM)\n  ```sh\n  pip install -U \"jax[tpu]\" -f https://storage.googleapis.com/jax-releases/libtpu_releases.html\n  ```\nFor more installation options, please consult the JAX [installation guide](https://jax.readthedocs.io/en/latest/installation.html#installation).\n\n### Numba\n\nNumba can be installed with pip:\n```\npython -m pip install numba\n```\nIf you use Anaconda on an Intel-based machine, then you can install an optimized version of Numba that uses Intel SVML:\n```\nconda install -c numba icc_rt\n```\nFor more installation options, please consult the Numba [installation guide](https://numba.readthedocs.io/en/stable/user/installing.html).\n\n### Pythran\n\nPythran can be install with pip and Anaconda. For detailed installation options, please consult the Pythran [installation guide](https://pythran.readthedocs.io/en/latest/).\n\n\n## Running benchmarks\n\nTo run individual bencharks, you can use the `run_benchmark` script:\n```\npython run_benchmark.py -b \u003cbenchmark\u003e -f \u003cframework\u003e\n```\nThe available benchmarks are listed in the `bench_info` folder.\nThe supported frameworks are listed in the `framework_info` folder.\nPlease use the corresponding JSON filenames.\nFor example, to run `adi` with NumPy, execute the following:\n```\npython run_benchmark.py -b adi -f numpy\n```\nYou can run all the available benchmarks with a specific framework using the `run_framework` script:\n```\npython run_framework.py -f \u003cframework\u003e\n```\n\n### Presets\n\nEach benchmark has four different presets; `S`, `M`, `L`, and `paper`.\nThe `S`, `M`, and `L` presets have been selected so that NumPy finishes execution\nin about 10, 100, and 1000ms respectively in a machine with two 16-core Intel Xeon\nGold 6130 processors.\nException to that are `atax`, `bicg`, `mlp`, `mvt`, and `trisolv`, which have been\ntuned for 5, 20 and 100ms approximately due to very high memory requirements.\nThe `paper` preset is the problem sizes used in the NPBench [paper](http://spcl.inf.ethz.ch/Publications/index.php?pub=412).\nBy default, the provided python scripts execute the benchmarks using the `S` preset.\nYou can select a different preset with the optional `-p` flag:\n```\npython run_benchmark.py -b gemm -f numpy -p L\n```\n\n### Visualization\n\nAfter running some benchmarks with different frameworks, you can generate plots\nof the speedups and line-count differences (experimental) against NumPy:\n```\npython plot_results.py\npython plot_lines.py\n```\n\n## Customization\n\nIt is possible to use the NPBench infrastructure with your own benchmarks and frameworks.\nFor more information on this functionality please read the documentation for [benchmarks](benchmarks.md) and [frameworks](frameworks.md).\n\n## Publication\n\nPlease cite NPBench as follows:\n\n```bibtex\n@inproceedings{\n    npbench,\n    author = {Ziogas, Alexandros Nikolaos and Ben-Nun, Tal and Schneider, Timo and Hoefler, Torsten},\n    title = {NPBench: A Benchmarking Suite for High-Performance NumPy},\n    year = {2021},\n    publisher = {Association for Computing Machinery},\n    address = {New York, NY, USA},\n    url = {https://doi.org/10.1145/3447818.3460360},\n    doi = {10.1145/3447818.3460360},\n    booktitle = {Proceedings of the ACM International Conference on Supercomputing},\n    series = {ICS '21}\n}\n```\n\n## Acknowledgements\n\nNPBench is a collection of scientific Python/NumPy codes from various domains that we adapted from the following sources:\n- Azimuthal Integration from [pyFAI](https://github.com/silx-kit/pyFAI)\n- Navier-Stokes from  [CFD Python](https://github.com/barbagroup/CFDPython)\n- Cython [tutorial](https://cython.readthedocs.io/en/latest/src/userguide/numpy_tutorial.html) for NumPy users\n- Quantum Transport simulation from [OMEN](https://nano-tcad.ee.ethz.ch/research/computational-nanoelectronics.html)\n- CRC-16-CCITT algorithm from [oysstu](https://gist.github.com/oysstu/68072c44c02879a2abf94ef350d1c7c6)\n- Numba [tutorial](https://numba.readthedocs.io/en/stable/user/5minguide.html)\n- Mandelbrot codes [From Python to Numpy](https://github.com/rougier/from-python-to-numpy)\n- N-Body simulation from [nbody-python](https://github.com/pmocz/nbody-python)\n- [PolyBench/C](http://web.cse.ohio-state.edu/~pouchet.2/software/polybench/)\n- Pythran [benchmarks](https://github.com/serge-sans-paille/numpy-benchmarks/)\n- [Stockham-FFT](http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-287731)\n- Weather stencils from [gt4py](https://github.com/GridTools/gt4py)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspcl%2Fnpbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspcl%2Fnpbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspcl%2Fnpbench/lists"}