{"id":15692026,"url":"https://github.com/jonathanhefner/benchmark-inputs","last_synced_at":"2025-11-11T20:24:59.327Z","repository":{"id":62554106,"uuid":"62646982","full_name":"jonathanhefner/benchmark-inputs","owner":"jonathanhefner","description":"Input-focused benchmarking for Ruby","archived":false,"fork":false,"pushed_at":"2023-03-18T21:01:49.000Z","size":39,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-07T23:44:34.039Z","etag":null,"topics":["benchmark","microbenchmark","ruby"],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonathanhefner.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-07-05T15:13:06.000Z","updated_at":"2024-11-05T20:50:11.000Z","dependencies_parsed_at":"2024-10-09T13:45:24.704Z","dependency_job_id":null,"html_url":"https://github.com/jonathanhefner/benchmark-inputs","commit_stats":{"total_commits":46,"total_committers":2,"mean_commits":23.0,"dds":"0.34782608695652173","last_synced_commit":"00c7b4c059146bc35b97676f3c20ac44dc65eb6c"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanhefner%2Fbenchmark-inputs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanhefner%2Fbenchmark-inputs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanhefner%2Fbenchmark-inputs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanhefner%2Fbenchmark-inputs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonathanhefner","download_url":"https://codeload.github.com/jonathanhefner/benchmark-inputs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252973616,"owners_count":21834105,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","microbenchmark","ruby"],"created_at":"2024-10-03T18:28:01.866Z","updated_at":"2025-11-11T20:24:59.295Z","avatar_url":"https://github.com/jonathanhefner.png","language":"Ruby","readme":"# benchmark-inputs\n\nInput-focused benchmarking for Ruby.  Given one or more blocks and an\narray of inputs to yield to each of them, benchmark-inputs will measure\nthe speed (in invocations per second) of each block.  Blocks which\nexecute very quickly, as in microbenchmarks, are automatically invoked\nrepeatedly to provide accurate measurements.\n\n\n## Motivation\n\nI \u003c3 [Fast Ruby][fast-ruby].  By extension, I \u003c3 [benchmark-ips].  But,\nfor some use cases, benchmark-ips doesn't let me write benchmarks the\nway I'd like.  Consider the following example, *using benchmark-ips*:\n\n```ruby\nrequire \"benchmark/ips\" ### USING benchmark-ips (NOT benchmark-inputs)\n\nSTRINGS = [\"abc\", \"aaa\", \"xyz\", \"\"]\nBenchmark.ips do |job|\n  job.report(\"String#tr\"){ STRINGS.each{|string| string.tr(\"a\", \"A\") } }\n  job.report(\"String#gsub\"){ STRINGS.each{|string| string.gsub(/a/, \"A\") } }\n  job.compare!\nend\n```\n\nThe calls to `STRINGS.each` introduce performance overhead that skews\nthe time measurements.  The less time the target function takes, the\nmore relative overhead, and thus more skew.  For a microbenchmark this\ncan be a problem.  A possible workaround is to invoke the function on\neach value individually, but that is more verbose and more error-prone:\n\n```ruby\nrequire \"benchmark/ips\" ### USING benchmark-ips (NOT benchmark-inputs)\n\nstring1, string2, string3, string4 = [\"abc\", \"aaa\", \"xyz\", \"\"]\nBenchmark.ips do |job|\n  job.report(\"String#tr\") do\n    string1.tr(\"a\", \"A\")\n    string2.tr(\"a\", \"A\")\n    string3.tr(\"a\", \"A\")\n    string4.tr(\"a\", \"A\")\n  end\n  job.report(\"String#gsub\") do\n    string1.gsub(/a/, \"A\")\n    string2.gsub(/a/, \"A\")\n    string3.gsub(/a/, \"A\")\n    string4.gsub(/a/, \"A\")\n  end\n  job.compare!\nend\n```\n\n\n## Usage\n\n*Enter benchmark-inputs*.  Here is how the same benchmark looks using\nthis gem:\n\n```ruby\nrequire \"benchmark/inputs\" ### USING benchmark-inputs\n\nBenchmark.inputs([\"abc\", \"aaa\", \"xyz\", \"\"]) do |job|\n  job.report(\"String#tr\"){|string| string.tr(\"a\", \"A\") }\n  job.report(\"String#gsub\"){|string| string.gsub(/a/, \"A\") }\n  job.compare!\nend\n```\n\nWhich prints something like the following to `$stdout`:\n\n```\nString#tr\n  1387268.0 i/s (±0.49%)\nString#gsub\n  264307.7 i/s (±1.95%)\n\nComparison:\n    String#tr:   1387268.0 i/s\n  String#gsub:    264307.7 i/s - 5.25x slower\n```\n\n\n### Benchmarking destructive operations\n\nDestructive operations also pose a challenge for microbenchmarks.  Each\ninvocation needs to operate on the same data, but `dup`ing the data\nintroduces too much overhead and skew.\n\nbenchmark-inputs' solution is to estimate the overhead incurred by each\n`dup`, and exclude that from the time measurements.  Because the\nbenchmark job already controls the input data, everything can be handled\nbehind the scenes.  To enable this, use the `dup_inputs` option:\n\n```ruby\nrequire \"benchmark/inputs\"\n\nBenchmark.inputs([\"abc\", \"aaa\", \"xyz\", \"\"], dup_inputs: true) do |job|\n  job.report(\"String#tr!\"){|string| string.tr!(\"a\", \"A\") }\n  job.report(\"String#gsub!\"){|string| string.gsub!(/a/, \"A\") }\n  job.compare!\nend\n```\n\nWhich prints out something like:\n\n```\nString#tr!\n  1793132.0 i/s (±0.46%)\nString#gsub!\n  281588.6 i/s (±0.49%)\n\nComparison:\n    String#tr!:   1793132.0 i/s\n  String#gsub!:    281588.6 i/s - 6.37x slower\n```\n\nThe above shows a slightly larger performance gap than the previous\nbenchmark.  This makes sense because the overhead of allocating new\nstrings -- previously via a non-bang method, but now via `dup` -- is now\nexcluded from the timings.  Thus, the speed of `tr!` relative to `gsub!`\nis further emphasized.\n\n\n## API\n\nSee the [API documentation](https://www.rubydoc.info/gems/benchmark-inputs).\n\n\n## Limitations\n\n`Benchmark.inputs` generates code based on the array of input values it\nis given.  Each input value becomes a local variable.  While there is\ntheoretically no limit to the number of local variables that can be\ngenerated, more than a few hundred may slow down the benchmark.  But,\nbecause input values are used to represent different scenarios rather\nthan control the number of invocations, this limitation should not pose\na problem.\n\n\n## Installation\n\nInstall the [`benchmark-inputs` gem](https://rubygems.org/gems/benchmark-inputs).\n\n\n## License\n\n[MIT License](LICENSE.txt)\n\n\n\n\n[fast-ruby]: https://github.com/JuanitoFatas/fast-ruby\n[benchmark-ips]: https://rubygems.org/gems/benchmark-ips\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonathanhefner%2Fbenchmark-inputs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonathanhefner%2Fbenchmark-inputs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonathanhefner%2Fbenchmark-inputs/lists"}