{"id":15010993,"url":"https://github.com/simonmichael/quickbench","last_synced_at":"2025-04-09T18:41:43.378Z","repository":{"id":54048421,"uuid":"71081029","full_name":"simonmichael/quickbench","owner":"simonmichael","description":"Easily time one or more commands with one or more executables and show tabular results","archived":false,"fork":false,"pushed_at":"2025-04-05T10:57:32.000Z","size":53,"stargazers_count":22,"open_issues_count":5,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-05T11:18:57.878Z","etag":null,"topics":["benchmarking","cli","haskell","shell"],"latest_commit_sha":null,"homepage":"http://hackage.haskell.org/package/quickbench","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simonmichael.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2016-10-16T22:25:17.000Z","updated_at":"2025-04-05T10:57:35.000Z","dependencies_parsed_at":"2024-02-01T17:56:27.301Z","dependency_job_id":null,"html_url":"https://github.com/simonmichael/quickbench","commit_stats":{"total_commits":33,"total_committers":1,"mean_commits":33.0,"dds":0.0,"last_synced_commit":"ca3c22e1f0261428d3f3ab959e2d8f8eb3cbc459"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonmichael%2Fquickbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonmichael%2Fquickbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonmichael%2Fquickbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonmichael%2Fquickbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simonmichael","download_url":"https://codeload.github.com/simonmichael/quickbench/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248089956,"owners_count":21045998,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking","cli","haskell","shell"],"created_at":"2024-09-24T19:38:13.342Z","updated_at":"2025-04-09T18:41:43.362Z","avatar_url":"https://github.com/simonmichael.png","language":"Haskell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# quickbench\n\nQuick \u0026 easy benchmarking of command-line programs.\n\n  [About](#about)\n| [Install](#install)\n| [Usage](#usage)\n| [Limitations](#limitations)\n| [Todo](#todo)\n\n\n## About\n\nquickbench grew from a little benchmarking tool I used in the hledger project since 2008.\nThink of it as a more powerful but still easy alternative to the unix `time` command,\nfor measuring the time taken by command-line programs,\nor for creating repeatable benchmark scripts for your projects.\nI find it very useful for quick, exploratory, and comparative measurements that can be understood at a glance.\n\nFeatures:\n\n- runs one or more commands, optionally substituting different executables\n- shows results as quickly as possible, running each command just once by default\n- shows very simple output: elapsed wall-clock seconds (with configurable precision)\n- tabulates the results, with multiple executables shown side by side\n- does not require knowledge of statistics\n- is cross platform, GPLv3+ licensed, and actively used by its maintainer(s).\n\nTimeline:\n\n- 2008: I built a benchmarking tool early in the hledger project,\n  possibly inspired by a [similar tool](https://hackage.haskell.org/package/darcs-benchmark-0.1) in Darcs\n- 2016: released it as quickbench 1.0 on [hackage](https://hackage.haskell.org/package/quickbench)\n- 9 years of quietly doing its job, with just a minor release in 2021 for latest deps\n- 2025: development revived by some pull requests; updates, 1.1 in process\n\nRelated tools:\n\n- [bench](https://github.com/Gabriel439/bench#readme) (2016) is another command line benchmarking tool written in Haskell.\n  It provides detailed statistical output and nice HTML reports. Its default output is not easy to understand.\n- [hyperfine](https://github.com/sharkdp/hyperfine) is the popular/powerful Rust alternative.\n\n## Install\n\nYou can install quickbench from source on all major platforms.\nYou'll need Haskell build tools;\nyou can get these from your packaging system,\nor [stack](https://docs.haskellstack.org/en/stable/),\nor [ghcup](https://www.haskell.org/ghcup).\nThen:\n```\ncabal install quickbench-1.1\n```\n\nor\n```\nstack install quickbench-1.1\n```\n\nor\n```\ngit clone https://github.com/simonmichael/quickbench\ncd quickbench\nstack install\n```\n\nquickbench does not seem to be packaged *anywhere* yet - please help with that if you can !\n\n\n## Usage\n\nFor help, run:\n```\n$ quickbench -h\n...\nUsage:\n  quickbench [options] [CMD...]\n...\n```\n\nYou can specify one or more commands as arguments on the command line.\nHere we time the \"echo\" and \"sleep 1\" commands.\n\"sleep 1\" is multi-word command, so it must be wrapped in quotes:\n```\n$ quickbench echo 'sleep 1'\nRunning 2 tests 1 times at 2025-04-05 08:29:53 HST:\n\nBest times:\n+---------++------+\n|         ||      |\n+=========++======+\n| echo    || 0.01 |\n| sleep 1 || 1.01 |\n+---------++------+\n```\n\nNote both of these commands run executable programs -\n`echo` is both a shell builtin and an executable in the /bin directory,\nand quickbench is running the latter:\n```\n$ which echo\necho is a shell builtin\necho is /bin/echo\n$ which sleep\nsleep is /bin/sleep\n```\n\nYou can also run commands from a file.\nLines beginning with `#` are ignored:\n```\n$ cat - \u003escript\n# important benchmark\npython3 -c \"print(2 ** 10000)\"\nruby    -e \"puts 2 ** 10000\"\nghc     -e \"print $ 2 ** 10000\"\n\n$ quickbench -f script\nRunning 3 tests 1 times at 2025-04-05 11:17:13 HST:\n\nBest times:\n+---------------------------------++------+\n|                                 ||      |\n+=================================++======+\n| python3 -c \"print(2 ** 10000)\"  || 0.05 |\n| ruby    -e \"puts 2 ** 10000\"    || 0.07 |\n| ghc     -e \"print $ 2 ** 10000\" || 0.41 |\n+---------------------------------++------+\n```\n\nA file named `bench.sh` in the current directory will be used as the default source of commands:\n```\n$ mv script bench.sh\n$ quickbench\nRunning 3 tests 1 times at 2025-04-05 11:19:43 HST:\n...\n```\n\nThe `-p DIGITS` option selects how many decimal places to display for times.\n\nSome repetition options can help show/reduce jitter, giving more confidence in the results:\n- `-n NUM` runs each test NUM times, keeping the fastest result\n- `-N NUM` runs the whole suite NUM times, displaying each run.\n\nAnd the `-v` flag shows the commands being run. Or `-V` will show them with their output.\n\n```\n$ quickbench -p3 -n5 -N2 -v\nRunning 3 tests 5 times at 2025-04-05 11:31:12 HST:\n1: python3 -c print(2 ** 10000)\n\t[0.045s]\n2: python3 -c print(2 ** 10000)\n\t[0.029s]\n3: python3 -c print(2 ** 10000)\n\t[0.027s]\n4: python3 -c print(2 ** 10000)\n\t[0.024s]\n5: python3 -c print(2 ** 10000)\n\t[0.026s]\n1: ruby -e puts 2 ** 10000\n\t[0.061s]\n2: ruby -e puts 2 ** 10000\n\t[0.060s]\n...\nBest times 1:\n+---------------------------------++-------+\n|                                 ||       |\n+=================================++=======+\n| python3 -c \"print(2 ** 10000)\"  || 0.024 |\n| ruby    -e \"puts 2 ** 10000\"    || 0.056 |\n| ghc     -e \"print $ 2 ** 10000\" || 0.315 |\n+---------------------------------++-------+\n\nBest times 2:\n+---------------------------------++-------+\n|                                 ||       |\n+=================================++=======+\n| python3 -c \"print(2 ** 10000)\"  || 0.023 |\n| ruby    -e \"puts 2 ** 10000\"    || 0.057 |\n| ghc     -e \"print $ 2 ** 10000\" || 0.316 |\n+---------------------------------++-------+\n```\n\nYou can run each command with its first word replaced by a different executable:\n```\n$ quickbench -p3 -n5 -w ghc-9.6,ghc-9.8,ghc-9.10,ghc-9.12 'ghc -e \"print $ 2 ** 1000000\"'\nRunning 1 tests 5 times with 4 executables at 2025-04-05 11:39:24 HST:\n\nBest times:\n+---------------------------++---------+---------+----------+----------+\n|                           || ghc-9.6 | ghc-9.8 | ghc-9.10 | ghc-9.12 |\n+===========================++=========+=========+==========+==========+\n| -e \"print $ 2 ** 1000000\" ||   0.319 |   0.318 |    0.309 |    0.335 |\n+---------------------------++---------+---------+----------+----------+\n```\n\nYou can convert a shell script to a benchmark suite by adding a quickbench shebang line,\nlike this (some systems will require `env -S`):\n```\n$ cat 410-times.sh\n#!/usr/bin/env quickbench -n2 -w hledger-410-before,hledger-410-8bde75c -f\nhledger -f 10k.journal print\nhledger -f 10k.journal register\nhledger -f 10k.journal balance\n\n$ ./410-times.sh\nRunning 3 tests 2 times with 2 executables at 2016-10-16 23:42:57.349721 UTC:\n\nBest times:\n+-------------------------++--------------------+---------------------+\n|                         || hledger-410-before | hledger-410-8bde75c |\n+=========================++====================+=====================+\n| -f 10k.journal print    ||               2.90 |                2.92 |\n| -f 10k.journal register ||               3.52 |                3.51 |\n| -f 10k.journal balance  ||               1.93 |                1.80 |\n+-------------------------++--------------------+---------------------+\n```\n\n\n## Quirks\n\nSome current limitations/quirks of quickbench:\n\n- It doesn't allow quickbench options to be written after the commands.\n- Some options, like `--debug`, may appear to work there, but they don't work completely and it still complains.\n- If you forget the quotes, as in `quickbench echo -n a`, it gives an unclear \"should appear before argument or is unknown\" error.\n- Commands which fail show an error message above the table, but not within it.\n\n\n## Todo\n\n- land PRs\n- update docs\n- doc deduplication/automation\n- tests\n- release 1.1\n- packaging\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonmichael%2Fquickbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonmichael%2Fquickbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonmichael%2Fquickbench/lists"}