{"id":21343518,"url":"https://github.com/cloudmercato/awesome-benchmark","last_synced_at":"2026-01-30T03:03:47.978Z","repository":{"id":39031945,"uuid":"324273713","full_name":"cloudmercato/awesome-benchmark","owner":"cloudmercato","description":null,"archived":false,"fork":false,"pushed_at":"2024-08-29T17:06:25.000Z","size":188,"stargazers_count":71,"open_issues_count":1,"forks_count":7,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-11T07:02:50.017Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudmercato.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-25T02:40:57.000Z","updated_at":"2025-02-08T20:50:05.000Z","dependencies_parsed_at":"2023-12-21T02:22:49.696Z","dependency_job_id":"c4802ece-1ba1-42a8-8d97-ba8c04181002","html_url":"https://github.com/cloudmercato/awesome-benchmark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudmercato%2Fawesome-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudmercato%2Fawesome-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudmercato%2Fawesome-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudmercato%2Fawesome-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudmercato","download_url":"https://codeload.github.com/cloudmercato/awesome-benchmark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243818199,"owners_count":20352629,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T01:13:36.285Z","updated_at":"2026-01-30T03:03:47.925Z","avatar_url":"https://github.com/cloudmercato.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"Awesome system benchmark tools\n===============================\n\nCPU\n---\n\n- `sysbench (cpu) \u003chttps://github.com/akopytov/sysbench\u003e`_ : \n  Scriptable multi-threaded benchmark tool based on LuaJIT.\n- `openssl speed \u003chttps://www.openssl.org/docs/manmaster/man1/speed.html\u003e`_:\n  Cryptography performance measurement\n- `V-Ray Benchmark \u003chttps://www.chaosgroup.com/vray/benchmark\u003e`_:\n  Free standalone application to test how fast your system renders.\n- `Phoronix Test Suite \u003chttps://www.phoronix-test-suite.com/\u003e`_:\n  Testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems.\n- `SPEC CPU \u003chttps://www.spec.org/cpu2017/\u003e`_:\n  SPEC's next-generation, industry-standardized, CPU intensive suites for measuring and comparing compute intensive performance, stressing a system's processor, memory subsystem and compiler.\n- `VASP \u003chttps://www.hpc.cineca.it/content/vasp-benchmark\u003e`_:\n  The Vienna Ab initio Simulation Package is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.  \n- `CoreMark \u003chttps://github.com/eembc/coremark\u003e`_: Methods for testing only a processor's core features\n- `CoreMark®-PRO \u003chttps://github.com/eembc/coremark-pro\u003e`_: Comprehensive, advanced processor benchmark that works with and enhances the market-proven industry-standard EEMBC CoreMark®\n- `Geekbench \u003chttps://www.geekbench.com/\u003e`_ :\n  Cross-platform benchmark that measures your system's performance with the press of a button.\n- `db-benchmark \u003chttps://github.com/h2oai/db-benchmark\u003e`_: Reproducible benchmarking of database-like operations in single-node environment.\n- `Renaissance Benchmark Suite \u003chttps://github.com/renaissance-benchmarks/renaissance\u003e`_: Aggregates common modern JVM workloads, including, but not limited to, Big Data, machine-learning, and functional programming.\n- `The Computer Language Benchmarks Game \u003chttps://salsa.debian.org/benchmarksgame-team/benchmarksgame\u003e`_: Toy-program performance measurements for ~24 language implementations.\n- `phpbenchmarks \u003chttps://github.com/phpbenchmarks\u003e`_: Test suite for vanilla PHP, Laravel or Symfony\n- `Compute Benchmarks \u003chttps://github.com/intel/compute-benchmarks\u003e`_: A benchmark suite developed and maintained by NEO driver development team in order to provide a stable environment, which allows checking and reproducing performance for various parts of the driver\n- `Large Text Compression Benchmark \u003chttp://mattmahoney.net/dc/text.html\u003e`_: Methodologies and set of results from Wikipedia's text compression through various tool\n- `benchdnn \u003chttps://github.com/oneapi-src/oneDNN/blob/master/tests/benchdnn/README.md\u003e`_: Extended and robust correctness verification and performance benchmarking tool for the primitives provided by oneDNN\n- `mathbench-rs \u003chttps://github.com/bitshifter/mathbench-rs\u003e`_: mathbench is a suite of unit tests and benchmarks comparing the output and performance of a number of different Rust linear algebra libraries\n- `CacheBench  \u003chttps://cachelib.org/docs/Cache_Library_User_Guides/Cachebench_Overview\u003e`_: Benchmark and stress testing tool to evaluate cache performance with real hardware and real cache workloads\n- `Memaccesstest  \u003chttps://github.com/bigstepinc/memaccesstest\u003e`_: Memory Access Time Benchmark - Test L1, L2,L3, Main memory access times, TLB misses and prefetch effects\n\n\n\n\nRAM\n---\n- `STREAM \u003chttp://www.cs.virginia.edu/stream/\u003e`_:\n- `sysbench (memory) \u003chttps://github.com/akopytov/sysbench\u003e`_ : \n  Scriptable multi-threaded benchmark tool based on LuaJIT.\n\n\nStorage\n-------\n- `Flexible I/O Tester (FIO) \u003chttps://fio.readthedocs.io/en/latest/fio_doc.html\u003e`_:\n  Spawns a number of threads or processes doing a particular type of I/O action as specified by the user.\n- `vdbench \u003chttps://sourceforge.net/projects/vdbench/\u003e`_:\n  Disk and tape I/O workload generator for verifying data integrity and measuring performance of storage.\n- `iozone \u003chttp://iozone.org/\u003e`_:\n  Filesystem benchmark tool\n- `hdparam \u003chttps://linux.die.net/man/8/hdparm\u003e`_:\n  Command line interface to various kernel interfaces.\n- `oiping \u003chttps://github.com/koct9i/ioping\u003e`_ Shows disk latency in the same way as ping shows network latency.\n\nNetwork\n-------\n- `iperf \u003chttps://iperf.fr/\u003e`_:\n  Active measurements of the maximum achievable bandwidth on IP networks.\n- `tracepath \u003chttps://linux.die.net/man/8/tracepath\u003e`_:\n  Trace path to destination discovering MTU along this path.\n- `traceroute \u003chttps://linux.die.net/man/8/traceroute\u003e`_:\n  racks the route packets taken from an IP network on their way to a given host.\n- `tcptraceroute \u003chttps://linux.die.net/man/1/tcptraceroute\u003e`_:\n  traceroute implementation using TCP packets.\n- `paris-traceroute \u003chttps://paris-traceroute.net/index.html\u003e`_: Paris traceroute is a new version of the well-known network diagnosis and measurement tool. It addresses problems caused by load balancers with the initial implementation of traceroute. \n- `dublin-traceroute \u003chttps://github.com/insomniacslk/dublin-traceroute\u003e`_: Dublin Traceroute is a NAT-aware multipath traceroute tool.\n- `mtr \u003chttps://github.com/traviscross/mtr\u003e`_: Combines the functionality of the 'traceroute' and 'ping' programs in a single network diagnostic tool.\n- `netperf \u003chttps://github.com/HewlettPackard/netperf\u003e`_: CTCP/UDP/sockets performance benchmark\n- `sockperf \u003chttps://github.com/Mellanox/sockperf\u003e`_: Network benchmarking utility over socket API that was designed for testing performance (latency and throughput) of high-performance systems\n- **ping**: Send ICMP ECHO_REQUEST packets to network hosts\n\nGPU\n---\n- `V-Ray Benchmark \u003chttps://www.chaosgroup.com/vray/benchmark\u003e`_:\n- `Mixbench \u003chttps://github.com/ekondis/mixbench\u003e`_:\n  Evaluate performance bounds of GPUs on mixed operational intensity kernels.\n- `gpu-burn \u003chttps://github.com/wilicc/gpu-burn\u003e`_:\n  Multi-GPU CUDA stress test.\n- `nvbandwidth \u003chttps://github.com/NVIDIA/nvbandwidth\u003e`_: A tool for bandwidth measurements on NVIDIA GPUs.\n\nHPC\n---\n- `OSU Micro-Benchmarks (OMB) \u003chttp://mvapich.cse.ohio-state.edu/benchmarks/\u003e`_:\n  Suite of benchmarks for measuring and evaluating the performance of MPI operations for point-to-point, multi-pair, and collective communications.\n- `mpi-benchmarks \u003chttps://github.com/intel/mpi-benchmarks/\u003e`_:\n  Intel(R) MPI Benchmarks provides a set of elementary benchmarks that conform to MPI-1, MPI-2, and MPI-3 standard.\n- `FinanceBench \u003chttp://cavazos-lab.github.io/FinanceBench/\u003e`_:\n  FinanceBench is aimed at those who work with financial code to see how certain code paths can be targeted for accelerators.\n- `LAMMPS Benchmarks \u003chttps://lammps.org/bench.html\u003e`_: Simple suite with Large-scale Atomic/Molecular Massively Parallel Simulator workloads\n- `pyhpc-benchmark \u003chttps://github.com/dionhaefner/pyhpc-benchmarks\u003e`_: Suite of benchmarks to test the sequential CPU and GPU performance of various computational backends with Python frontends\n\n\nAI\n--\n- `AI-Benchmark \u003chttps://ai-benchmark.com/\u003e`_: Deep learning workloads\n- `lambda-tensorflow-benchmark \u003chttps://github.com/lambdal/lambda-tensorflow-benchmark\u003e`_\n- `PyTorch Benchmarks \u003chttps://github.com/lambdal/deeplearning-benchmark/tree/master/pytorch\u003e`_\n- `Whisper Benchmark \u003chttps://github.com/cloudmercato/whisper-benchmark\u003e`_:\n  Simple tool to evaluate performance of `Whisper \u003chttps://github.com/openai/whisper\u003e`_ models and configurations.\n- `InvokeAI Benchmark \u003chttps://github.com/cloudmercato/invokeai-benchmark\u003e`_:\n  Simple tool to evaluate performance of InvokeAI.\n- `DeepSparse Benchmark \u003chttps://github.com/neuralmagic/deepsparse/tree/main/src/deepsparse/benchmark\u003e`_: Command-line tool for benchmarking the DeepSparse Engine with ONNX models\n- `BitNet CPU \u003chttps://github.com/catid/bitnet_cpu\u003e`_: BitNet inference performance evaluation on a CPU\n- `LLMPerf \u003chttps://github.com/ray-project/llmperf\u003e`_: A Tool for evaulation the performance of LLM APIs\n- `vLLM Performance benchmark \u003chttps://buildkite.com/vllm/performance-benchmark/builds/3924\u003e`_: Compares the performance of vllm against other LLM serving engine\n- `RepoBench \u003chttps://github.com/Leolty/repobench\u003e`_: Repository-level Code auto-completion systems evaluation\n- `SWE-bench \u003chttps://github.com/princeton-nlp/SWE-bench\u003e`_: Benchmark for evaluating large language models on real world software issues collected from GitHub.\n- `Massive Text Embedding Benchmark \u003chttps://github.com/embeddings-benchmark/mteb\u003e`_: MTEB spans 8 embedding tasks covering a total of 58 datasets and 112 languages.\n- `VBench \u003chttps://github.com/Vchitect/VBench\u003e`_: Comprehensive Benchmark Suite for Video Generative Models.\n\n\n\nHTTP\n----\n- `ab (apache benchmark) \u003chttp://httpd.apache.org/docs/2.4/programs/ab.html\u003e`_:\n  Load testing and benchmarking tool for HTTP servers.\n- `cURL \u003chttps://curl.se/\u003e`_:\n  Command line tool and library for transferring data with URLs.\n- `wrk \u003chttps://github.com/wg/wrk\u003e`_:\n  Modern HTTP benchmarking tool capable of generating significant load when run on a single multi-core CPU.\n- `os-benchmark \u003chttps://github.com/cloudmercato/os-benchmark\u003e`_ : Simple tool to collect metrics about all object storage.\n- `s3-benchmark \u003chttps://github.com/dvassallo/s3-benchmark\u003e`_: Measure S3's performance from any location\n- `warp \u003chttps://github.com/awesome-benchmark/warp\u003e`_: S3 benchmarking tool \n- `COSBench \u003chttps://github.com/intel-cloud/cosbench\u003e`_: Tool to measure the performance of Cloud Object Storage services.\n- `Gosbench \u003chttps://github.com/mulbc/gosbench\u003e`_: Golang reimplementation of Cosbench. It is a distributed S3 performance benchmark tool with Prometheus exporter leveraging the official Golang AWS SDK \n\n\nDatabase and messaging\n----------------------\n- `redis-benchmark \u003chttps://redis.io/topics/benchmarks\u003e`_:\n  Utility that simulates running commands done by N clients at the same time sending M total querie \n- `memtier_benchmark \u003chttps://github.com/RedisLabs/memtier_benchmark\u003e`_:\n  Command line utility developed for load generation and bechmarking NoSQL key-value databases.\n- `rpc-perf \u003chttps://github.com/twitter/rpc-perf\u003e`_:\n  Tool for measuring the performance of RPC services and is primarily used to benchmark caching systems.\n- `sysbench (oltp) \u003chttps://github.com/akopytov/sysbench\u003e`_ : \n  Scriptable multi-threaded benchmark tool based on LuaJIT.\n- `sysbench TPCC \u003chttps://github.com/Percona-Lab/sysbench-tpcc\u003e`_: \n  TPCC-like workload for sysbench 1.0.x.\n- `YCSB \u003chttps://github.com/brianfrankcooper/YCSB\u003e`_:\n  Framework and common set of workloads for evaluating the performance of different “key-value” and “cloud” serving stores\n- `benchyou \u003chttps://github.com/xelabs/benchyou\u003e`_:\n  benchyou is a benchmark tool for MySQL, similar to Sysbench.\n- `Cassandra Stress \u003chttps://cassandra.apache.org/doc/latest/cassandra/tools/cassandra_stress.html\u003e`_:\n  Benchmark and load-test a Cassandra cluster\n- `HammerDB \u003chttps://www.hammerdb.com/index.html\u003e`_:\n  Benchmarking and load testing software for the worlds most popular databases supporting Oracle Database, SQL Server, IBM Db2, MySQL, MariaDB and PostgreSQL.\n- `mysqlslap \u003chttps://dev.mysql.com/doc/refman/8.0/en/mysqlslap.html\u003e`_: Emulate client load for a MySQL server and to report the timing of each stage.\n- `pgbench \u003chttps://www.postgresql.org/docs/current/pgbench.html\u003e`_: It runs the same sequence of SQL commands over and over and then calculates the average transaction rate.\n- `OpenMessaging Benchmark Framework \u003chttps://github.com/openmessaging/benchmark\u003e`_: Suite of tools that make it easy to benchmark distributed messaging systems in the cloud.\n- `opensearch-benchmark \u003chttps://github.com/opensearch-project/OpenSearch-Benchmark\u003e`_: Macrobenchmarking framework for OpenSearch\n- `Rally \u003chttps://github.com/elastic/rally\u003e`_: Macrobenchmarking framework for Elasticsearch\n- `ann-benchmarks \u003chttps://github.com/erikbern/ann-benchmarks\u003e`_: Tools to benchmark various implementations of approximate nearest neighbor (ANN)\n- `big-ann-benchmarks \u003chttps://github.com/harsha-simhadri/big-ann-benchmarks\u003e`_: Tools to benchmark various implementations of approximate nearest neighbor (ANN)\n\nBig data\n--------\n- `HiBench \u003chttps://github.com/Intel-bigdata/HiBench\u003e`_:\n  The bigdata micro benchmark suite.\n- `YCSB \u003chttps://github.com/brianfrankcooper/YCSB\u003e`_:\n  Framework and common set of workloads for evaluating the performance of different “key-value” and “cloud” serving stores.\n  \nScheduler\n---------\n\n- `Open Charge Tester (OCT) \u003chttps://github.com/TheGhouls/oct\u003e`_:\n  The tools to load testing just anything at any scale.\n- `Apache JMeter \u003chttps://jmeter.apache.org/index.html\u003e`_:\n  Load test functional behavior and measure performance.\n  \nEcology and environment\n-----------------------\n\n- `Eco benchmark \u003chttps://github.com/Boavizta/ecobenchmark-applicationweb-backend\u003e`_:\n  Try to compare the energy consumption, for several languages.\n- `Scaphandre \u003chttps://github.com/hubblo-org/scaphandre\u003e`_:\n  Metrology agent dedicated to electrical power consumption metrics.\n  \nComment\n-------\n\nDo not forget that any task/workload can be a benchmark, it always depends of your goal.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudmercato%2Fawesome-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudmercato%2Fawesome-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudmercato%2Fawesome-benchmark/lists"}