Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/cloudmercato/awesome-benchmark


https://github.com/cloudmercato/awesome-benchmark

List: awesome-benchmark

Last synced: 4 months ago
JSON representation

Awesome Lists containing this project

README 

          
Awesome system benchmark tools

===============================



CPU

---



- `sysbench (cpu) `_ : 

  Scriptable multi-threaded benchmark tool based on LuaJIT.

- `openssl speed `_:

  Cryptography performance measurement

- `V-Ray Benchmark `_:

  Free standalone application to test how fast your system renders.

- `Phoronix Test Suite `_:

  Testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems.

- `SPEC CPU `_:

  SPEC's next-generation, industry-standardized, CPU intensive suites for measuring and comparing compute intensive performance, stressing a system's processor, memory subsystem and compiler.

- `VASP `_:

  The Vienna Ab initio Simulation Package is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.  

- `CoreMark `_: Methods for testing only a processor's core features

- `CoreMark®-PRO `_: Comprehensive, advanced processor benchmark that works with and enhances the market-proven industry-standard EEMBC CoreMark®

- `Geekbench `_ :

  Cross-platform benchmark that measures your system's performance with the press of a button.

- `db-benchmark `_: Reproducible benchmarking of database-like operations in single-node environment.

- `Renaissance Benchmark Suite `_: Aggregates common modern JVM workloads, including, but not limited to, Big Data, machine-learning, and functional programming.

- `The Computer Language Benchmarks Game `_: Toy-program performance measurements for ~24 language implementations.

- `phpbenchmarks `_: Test suite for vanilla PHP, Laravel or Symfony

- `Compute Benchmarks `_: A benchmark suite developed and maintained by NEO driver development team in order to provide a stable environment, which allows checking and reproducing performance for various parts of the driver

- `Large Text Compression Benchmark `_: Methodologies and set of results from Wikipedia's text compression through various tool

- `benchdnn `_: Extended and robust correctness verification and performance benchmarking tool for the primitives provided by oneDNN

- `mathbench-rs `_: mathbench is a suite of unit tests and benchmarks comparing the output and performance of a number of different Rust linear algebra libraries

- `CacheBench  `_: Benchmark and stress testing tool to evaluate cache performance with real hardware and real cache workloads

- `Memaccesstest  `_: Memory Access Time Benchmark - Test L1, L2,L3, Main memory access times, TLB misses and prefetch effects



RAM

---

- `STREAM `_:

- `sysbench (memory) `_ : 

  Scriptable multi-threaded benchmark tool based on LuaJIT.



Storage

-------

- `Flexible I/O Tester (FIO) `_:

  Spawns a number of threads or processes doing a particular type of I/O action as specified by the user.

- `vdbench `_:

  Disk and tape I/O workload generator for verifying data integrity and measuring performance of storage.

- `iozone `_:

  Filesystem benchmark tool

- `hdparam `_:

  Command line interface to various kernel interfaces.

- `oiping `_ Shows disk latency in the same way as ping shows network latency.



Network

-------

- `iperf `_:

  Active measurements of the maximum achievable bandwidth on IP networks.

- `tracepath `_:

  Trace path to destination discovering MTU along this path.

- `traceroute `_:

  racks the route packets taken from an IP network on their way to a given host.

- `tcptraceroute `_:

  traceroute implementation using TCP packets.

- `paris-traceroute `_: Paris traceroute is a new version of the well-known network diagnosis and measurement tool. It addresses problems caused by load balancers with the initial implementation of traceroute. 

- `dublin-traceroute `_: Dublin Traceroute is a NAT-aware multipath traceroute tool.

- `mtr `_: Combines the functionality of the 'traceroute' and 'ping' programs in a single network diagnostic tool.

- `netperf `_: CTCP/UDP/sockets performance benchmark

- `sockperf `_: Network benchmarking utility over socket API that was designed for testing performance (latency and throughput) of high-performance systems

- **ping**: Send ICMP ECHO_REQUEST packets to network hosts



GPU

---

- `V-Ray Benchmark `_:

- `Mixbench `_:

  Evaluate performance bounds of GPUs on mixed operational intensity kernels.

- `gpu-burn `_:

  Multi-GPU CUDA stress test.

- `nvbandwidth `_: A tool for bandwidth measurements on NVIDIA GPUs.



HPC

---

- `OSU Micro-Benchmarks (OMB) `_:

  Suite of benchmarks for measuring and evaluating the performance of MPI operations for point-to-point, multi-pair, and collective communications.

- `mpi-benchmarks `_:

  Intel(R) MPI Benchmarks provides a set of elementary benchmarks that conform to MPI-1, MPI-2, and MPI-3 standard.

- `FinanceBench `_:

  FinanceBench is aimed at those who work with financial code to see how certain code paths can be targeted for accelerators.

- `LAMMPS Benchmarks `_: Simple suite with Large-scale Atomic/Molecular Massively Parallel Simulator workloads

- `pyhpc-benchmark `_: Suite of benchmarks to test the sequential CPU and GPU performance of various computational backends with Python frontends



AI

--

- `AI-Benchmark `_: Deep learning workloads

- `lambda-tensorflow-benchmark `_

- `PyTorch Benchmarks `_

- `Whisper Benchmark `_:

  Simple tool to evaluate performance of `Whisper `_ models and configurations.

- `InvokeAI Benchmark `_:

  Simple tool to evaluate performance of InvokeAI.

- `DeepSparse Benchmark `_: Command-line tool for benchmarking the DeepSparse Engine with ONNX models

- `BitNet CPU `_: BitNet inference performance evaluation on a CPU

- `LLMPerf `_: A Tool for evaulation the performance of LLM APIs

- `vLLM Performance benchmark `_: Compares the performance of vllm against other LLM serving engine

- `RepoBench `_: Repository-level Code auto-completion systems evaluation

- `SWE-bench `_: Benchmark for evaluating large language models on real world software issues collected from GitHub.

- `Massive Text Embedding Benchmark `_: MTEB spans 8 embedding tasks covering a total of 58 datasets and 112 languages.

- `VBench `_: Comprehensive Benchmark Suite for Video Generative Models.



HTTP

----

- `ab (apache benchmark) `_:

  Load testing and benchmarking tool for HTTP servers.

- `cURL `_:

  Command line tool and library for transferring data with URLs.

- `wrk `_:

  Modern HTTP benchmarking tool capable of generating significant load when run on a single multi-core CPU.

- `os-benchmark `_ : Simple tool to collect metrics about all object storage.

- `s3-benchmark `_: Measure S3's performance from any location

- `warp `_: S3 benchmarking tool 

- `COSBench `_: Tool to measure the performance of Cloud Object Storage services.

- `Gosbench `_: Golang reimplementation of Cosbench. It is a distributed S3 performance benchmark tool with Prometheus exporter leveraging the official Golang AWS SDK 



Database and messaging

----------------------

- `redis-benchmark `_:

  Utility that simulates running commands done by N clients at the same time sending M total querie 

- `memtier_benchmark `_:

  Command line utility developed for load generation and bechmarking NoSQL key-value databases.

- `rpc-perf `_:

  Tool for measuring the performance of RPC services and is primarily used to benchmark caching systems.

- `sysbench (oltp) `_ : 

  Scriptable multi-threaded benchmark tool based on LuaJIT.

- `sysbench TPCC `_: 

  TPCC-like workload for sysbench 1.0.x.

- `YCSB `_:

  Framework and common set of workloads for evaluating the performance of different “key-value” and “cloud” serving stores

- `benchyou `_:

  benchyou is a benchmark tool for MySQL, similar to Sysbench.

- `Cassandra Stress `_:

  Benchmark and load-test a Cassandra cluster

- `HammerDB `_:

  Benchmarking and load testing software for the worlds most popular databases supporting Oracle Database, SQL Server, IBM Db2, MySQL, MariaDB and PostgreSQL.

- `mysqlslap `_: Emulate client load for a MySQL server and to report the timing of each stage.

- `pgbench `_: It runs the same sequence of SQL commands over and over and then calculates the average transaction rate.

- `OpenMessaging Benchmark Framework `_: Suite of tools that make it easy to benchmark distributed messaging systems in the cloud.

- `opensearch-benchmark `_: Macrobenchmarking framework for OpenSearch

- `Rally `_: Macrobenchmarking framework for Elasticsearch

- `ann-benchmarks `_: Tools to benchmark various implementations of approximate nearest neighbor (ANN)

- `big-ann-benchmarks `_: Tools to benchmark various implementations of approximate nearest neighbor (ANN)



Big data

--------

- `HiBench `_:

  The bigdata micro benchmark suite.

- `YCSB `_:

  Framework and common set of workloads for evaluating the performance of different “key-value” and “cloud” serving stores.

  

Scheduler

---------



- `Open Charge Tester (OCT) `_:

  The tools to load testing just anything at any scale.

- `Apache JMeter `_:

  Load test functional behavior and measure performance.

  

Ecology and environment

-----------------------



- `Eco benchmark `_:

  Try to compare the energy consumption, for several languages.

- `Scaphandre `_:

  Metrology agent dedicated to electrical power consumption metrics.

  

Comment

-------



Do not forget that any task/workload can be a benchmark, it always depends of your goal.