{"id":15775943,"url":"https://github.com/agentender/flexi-bench","last_synced_at":"2026-01-03T00:08:54.675Z","repository":{"id":247974888,"uuid":"827033522","full_name":"AgentEnder/flexi-bench","owner":"AgentEnder","description":"A flexible benchmarking tool controlled by javascript.","archived":false,"fork":false,"pushed_at":"2024-09-12T22:36:45.000Z","size":1154,"stargazers_count":2,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-11T17:12:03.840Z","etag":null,"topics":["benchmark","node","typescript"],"latest_commit_sha":null,"homepage":"http://craigory.dev/flexi-bench/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AgentEnder.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-10T22:17:10.000Z","updated_at":"2024-09-12T22:35:30.000Z","dependencies_parsed_at":"2024-09-13T09:55:27.369Z","dependency_job_id":"6d73f57c-b6b5-41c2-8302-cbf04409e1fa","html_url":"https://github.com/AgentEnder/flexi-bench","commit_stats":{"total_commits":19,"total_committers":2,"mean_commits":9.5,"dds":"0.052631578947368474","last_synced_commit":"aea3eff980ebefb7b2a802fdbc30095ecefb14b3"},"previous_names":["agentender/easybench"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AgentEnder%2Fflexi-bench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AgentEnder%2Fflexi-bench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AgentEnder%2Fflexi-bench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AgentEnder%2Fflexi-bench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AgentEnder","download_url":"https://codeload.github.com/AgentEnder/flexi-bench/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246655036,"owners_count":20812573,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","node","typescript"],"created_at":"2024-10-04T17:03:52.521Z","updated_at":"2026-01-03T00:08:54.646Z","avatar_url":"https://github.com/AgentEnder.png","language":"TypeScript","readme":"# FlexiBench\n\nFlexiBench is a flexible benchmarking library for JavaScript and TypeScript. It is designed to be simple to use, but also powerful and flexible. It is inspired by common testing framework APIs, and aims to provide a similar experience for benchmarking.\n\n## Installation\n\n```bash\nnpm install --save-dev flexi-bench\n```\n\n```bash\nyarn add --dev flexi-bench\n```\n\n```bash\npnpm add --save-dev flexi-bench\n```\n\n## Features\n\n- [Variations](#variations): Run the same benchmark with different configurations.\n- [Setup and Teardown](#setup-and-teardown): Run setup and teardown code before and after the benchmark.\n- [Commands](#commands): Run simple commands as benchmarks.\n\n## Usage\n\n### Runner API\n\nThe runner API is the simplest way to run benchmarks. It allows running a single benchmark or a suite of benchmarks.\n\n```javascript\nconst { suite, benchmark, setup, teardown } = require('flexi-bench');\n\n// A `suite` call at the top level will be evaluated as a whole suite.\nsuite('My Suite', () =\u003e {\n  // Nested `benchmark` calls will not be evaluated until the parent `suite` is evaluated.\n  benchmark('My Benchmark', (b) =\u003e {\n    setup(() =\u003e {\n      // some setup to run before the entire benchmark\n    });\n\n    teardown(() =\u003e {\n      // some teardown to run after the entire benchmark\n    });\n\n    b.withAction(() =\u003e {\n      // some action to benchmark\n    });\n  });\n});\n\n// Top-level `benchmark` calls will be evaluated immediately.\nbenchmark('My Benchmark', (b) =\u003e {\n  b.withIterations(10).withAction(() =\u003e {\n    // some action to benchmark\n  });\n});\n```\n\nWithin the callbacks for each `suite` or `benchmark` you can utilize the builder API for full customization of the benchmark. Variations can either be added directly via the builder API, or by nesting a `variation` call within the `benchmark` call, or at the `suite` level to apply the variation to all benchmarks in the suite.\n\n```javascript\nconst { suite, benchmark, variation } = require('flexi-bench');\n\nsuite('My Suite', () =\u003e {\n  benchmark('My Benchmark', (b) =\u003e {\n    b.withIterations(10).withAction(() =\u003e {\n      // some action to benchmark\n    });\n\n    variation('with NO_DAEMON', (v) =\u003e\n      v.withEnvironmentVariable('NO_DAEMON', 'true'),\n    );\n  });\n});\n```\n\n### Basic Benchmarks\n\nMore detailed documentation will come soon. For now, here is a simple example:\n\n```javascript\nconst { Benchmark } = require('flexi-bench');\n\nconst benchmark = new Benchmark('My Benchmark', {\n  iterations: 10,\n  action: () =\u003e {\n    // some action to benchmark\n  },\n});\n\nawait benchmark.run();\n```\n\nMost options for the benchmark can also be provided by a builder API:\n\n```javascript\nconst { Benchmark } = require('flexi-bench');\n\nconst benchmark = new Benchmark('My Benchmark')\n  .withIterations(10)\n  .withSetup(() =\u003e {\n    // some setup to run before the entire benchmark\n  })\n  .withAction(() =\u003e {\n    // some action to benchmark\n  });\n```\n\n#### Setup and Teardown\n\nSome benchmarks will require some work before the benchmark to setup the environment, and some work after the benchmark to clean up. This can be done with the `setup`/`setupEach` and `teardown`/`teardownEach` methods:\n\n```javascript\nbenchmark('My Benchmark', (b) =\u003e {\n  setup(() =\u003e {\n    // some setup to run before the entire benchmark\n  });\n\n  setupEach(() =\u003e {\n    // some setup that is ran before each iteration of the benchmark\n  });\n\n  teardown(() =\u003e {\n    // some teardown to run after the entire benchmark\n  });\n\n  teardownEach(() =\u003e {\n    // some teardown that is ran after each iteration of the benchmark\n  });\n});\n```\n\nThese can also be set using the builder API:\n\n```javascript\nconst { Benchmark } = require('flexi-bench');\n\nconst benchmark = new Benchmark('My Benchmark')\n  .withIterations(10)\n  .withSetup(() =\u003e {\n    // some setup to run before the entire benchmark\n  })\n  .withSetupEach(() =\u003e {\n    // some setup that is ran before each iteration of the benchmark\n  })\n  .withTeardown(() =\u003e {\n    // some teardown to run after the entire benchmark\n  })\n  .withTeardownEach(() =\u003e {\n    // some teardown that is ran after each iteration of the benchmark\n  });\n```\n\n#### Understanding Actions\n\nThe `action` for a benchmark provides the actual event that is being benchmarked. FlexiBench can currently benchmark 2 types of actions:\n\n- Callbacks\n- Commands\n\nAn `action` can be specified via the `action` property of the `benchmark` or `variation` constructor, or via `.withAction` on the builder API. Actions specified on the running `variation` will override the action specified on the parent `benchmark`.\n\nFor example, the following code will run the `action` specified on the `variation`, and not the `action` specified on the `benchmark`. This would result in the output `bar` being printed 10 times, instead of `foo`:\n\n```javascript\nconst { Benchmark } = require('flexi-bench');\n\nconst benchmark = new Benchmark('My Benchmark', {\n  iterations: 10,\n  action: () =\u003e {\n    console.log('foo');\n  },\n}).withVariation('with NO_DAEMON', (v) =\u003e\n  v.withAction(() =\u003e {\n    console.log('bar');\n  }),\n);\n```\n\n##### Callbacks\n\nIf the process you are benchmarking is available as a JavaScript function, you can pass it directly to the `action` property or execute it within the `action` callback. For example, if you are benchmarking a function that sorts an array, you can do the following:\n\n```javascript\nconst { benchmark, setup } = require('flexi-bench');\n\nbenchmark('My Benchmark', (b) =\u003e {\n  let array;\n\n  setup(() =\u003e {\n    array = Array.from({ length: 1000 }, () =\u003e Math.random());\n  });\n\n  b.withIterations(10).withAction(() =\u003e {\n    array.sort();\n  });\n});\n```\n\n##### Commands\n\nIf the process you are benchmarking is not available as a JavaScript function, you can run it as a command. This can be done by passing a string to the `action` property. For example, if you are benchmarking the runtime of a cli command, you can do the following:\n\n```javascript\nconst { Benchmark } = require('flexi-bench');\n\nconst benchmark = new Benchmark('My Benchmark', {\n  iterations: 10,\n  action: 'echo \"Hello, World!\"',\n});\n\nawait benchmark.run();\n```\n\nWhile it would be possible to write an action that runs a command using `child_process` methods, using the syntactic sugar provided by flexi-bench is more convenient and provides a few benefits.\n\nUtilizing the syntactic sugar also means that flexi-bench is aware that you are indeed running a command. This sounds obvious, but opens up some neat possibilities since we are running the command from within flexi-bench. For example, we can add variations based on tailoring CLI options which would not be possible if you ran your command directly using `child_process` methods on your own.\n\n```javascript\nconst { Benchmark } = require('flexi-bench');\n\nconst benchmark = new Benchmark('My Benchmark', {\n  iterations: 10,\n  action: 'echo',\n})\n  .withVariation('with argument', (v) =\u003e v.withArgument('Hello, Earth!'))\n  .withVariation('with argument', (v) =\u003e v.withArgument('Hello, Mars!'));\n\nawait benchmark.run();\n```\n\nWhen running commands via the syntactic sugar, the command is invoked with a function similar to below:\n\n```typescript\nconst child = spawn(action, variation.cliArgs, {\n  shell: true,\n  windowsHide: true,\n});\nchild.on('exit', (code) =\u003e {\n  if (code === 0) {\n    resolve();\n  } else {\n    reject(`Action failed with code ${code}`);\n  }\n});\n```\n\nFor more information on the simple command API, see the example: ./examples/simple-command.ts\n\n### Suites\n\nA suite is a collection of benchmarks that can be run together:\n\n```javascript\nconst { Benchmark, Suite } = require('flexi-bench');\n\nconst suite = new Suite('My Suite')\n  .addBenchmark(\n    new Benchmark('My Benchmark 1', {\n      iterations: 10,\n      action: () =\u003e {\n        // some action to benchmark\n      },\n    }),\n  )\n  .addBenchmark(\n    new Benchmark('My Benchmark 2', {\n      iterations: 10,\n      action: () =\u003e {\n        // some action to benchmark\n      },\n    }),\n  );\n```\n\nSuites can also be created using the runner API:\n\n```javascript\nconst { suite, benchmark } = require('flexi-bench');\n\nsuite('My Suite', () =\u003e {\n  benchmark('My Benchmark 1', (b) =\u003e {\n    b.withIterations(10).withAction(() =\u003e {\n      // some action to benchmark\n    });\n  });\n\n  benchmark('My Benchmark 2', (b) =\u003e {\n    b.withIterations(10).withAction(() =\u003e {\n      // some action to benchmark\n    });\n  });\n});\n```\n\n### Variations\n\nVariations allow running the same benchmark with different configurations:\n\n```javascript\nconst { Benchmark, Variation } = require('flexi-bench');\n\nconst benchmark = new Benchmark('My Benchmark', {\n  iterations: 10,\n  action: () =\u003e {\n    // some action to benchmark\n  },\n}).withVariation('with NO_DAEMON', (v) =\u003e\n  v.withEnvironmentVariable('NO_DAEMON', 'true'),\n);\n```\n\nVariations can do most things that the main benchmark can do, including having their own setup and teardown functions, or even a custom action.\n\nSome helper functions are provided on the `Variation` class to make it easier to set up variations:\n\n```javascript\nconst { Benchmark, Variation } = require('flexi-bench');\n\nconst benchmark = new Benchmark('My Benchmark', {\n  iterations: 10,\n  action: () =\u003e {\n    // some action to benchmark\n  },\n}).withVariations(\n  // Adds 4 variations with all possible combinations of the given environment variables\n  Variation.FromEnvironmentVariables([\n    ['NO_DAEMON', ['true', 'false']],\n    ['OTHER_VAR', ['value1', 'value2']],\n  ]),\n);\n```\n\nVariations can also be added to suites. Variations added to a suite will be applied to all benchmarks in the suite.\n\nFor example, the below suite would run each benchmark with 'NO_DAEMON' set to true, and then with 'OTHER_VAR' set to 'value1' for a total of 4 benchmark runs in the suite:\n\n```javascript\nconst { Benchmark, Suite, Variation } = require('flexi-bench');\n\nconst suite = new Suite('My Suite')\n  .addBenchmark(\n    new Benchmark('My Benchmark 1', {\n      iterations: 10,\n      action: () =\u003e {\n        // some action to benchmark\n      },\n    }),\n  )\n  .addBenchmark(\n    new Benchmark('My Benchmark 2', {\n      iterations: 10,\n      action: () =\u003e {\n        // some action to benchmark\n      },\n    }),\n  )\n  .withVariation('with NO_DAEMON', (v) =\u003e\n    v.withEnvironmentVariable('NO_DAEMON', 'true'),\n  )\n  .withVariation('with OTHER_VAR', (v) =\u003e\n    v.withEnvironmentVariable('OTHER_VAR', 'value1'),\n  );\n```\n\n## Examples\n\nSee examples folder.\n\n- ./examples/benchmark.ts is the motivation for this project. It benchmarks the performance of Nx commands with and without a daemon.\n- ./examples/performance-observer.ts is a simple example of how to use the PerformanceObserver API to measure the performance of a function.\n- ./examples/simple-command.ts demonstrates how to benchmark a simple command.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagentender%2Fflexi-bench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagentender%2Fflexi-bench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagentender%2Fflexi-bench/lists"}