{"id":14985000,"url":"https://github.com/littletof/prettybenching","last_synced_at":"2025-07-27T22:41:06.599Z","repository":{"id":53492381,"uuid":"267864042","full_name":"littletof/prettyBenching","owner":"littletof","description":":sauropod: A small lib to make your Deno benchmarking progress and results look pretty","archived":false,"fork":false,"pushed_at":"2022-01-09T13:04:44.000Z","size":774,"stargazers_count":25,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-10T23:15:29.337Z","etag":null,"topics":["action","bench","benchmarking","cli","commandline","deno","hacktoberfest","markdown","runbenchmarks"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/littletof.png","metadata":{"files":{"readme":"README.md","changelog":"history_extensions.ts","contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"ko_fi":"littletof","custom":["https://www.buymeacoffee.com/littletof","https://coindrop.to/littletof"]}},"created_at":"2020-05-29T13:25:24.000Z","updated_at":"2025-01-28T11:03:10.000Z","dependencies_parsed_at":"2022-08-19T07:40:32.233Z","dependency_job_id":null,"html_url":"https://github.com/littletof/prettyBenching","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/littletof%2FprettyBenching","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/littletof%2FprettyBenching/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/littletof%2FprettyBenching/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/littletof%2FprettyBenching/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/littletof","download_url":"https://codeload.github.com/littletof/prettyBenching/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248312134,"owners_count":21082638,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["action","bench","benchmarking","cli","commandline","deno","hacktoberfest","markdown","runbenchmarks"],"created_at":"2024-09-24T14:10:04.729Z","updated_at":"2025-04-10T23:15:36.279Z","avatar_url":"https://github.com/littletof.png","language":"TypeScript","funding_links":["https://ko-fi.com/littletof","https://www.buymeacoffee.com/littletof","https://coindrop.to/littletof"],"categories":[],"sub_categories":[],"readme":"# prettyBenching\n\nA simple Deno library, that gives you pretty benchmarking progress and results in the commandline\n\n[![version](https://img.shields.io/badge/0.3.3-brightgreen?logo=v\u0026logoColor=white\u0026labelColor=gray)](https://deno.land/x/pretty_benching@v0.3.3)\n\n[![deno version](https://img.shields.io/badge/deno%201.8.2-success?logo=deno\u0026logoColor=black\u0026labelColor=white\u0026color=black)](https://github.com/denoland/deno)\n[![deno/std version](https://img.shields.io/badge/deno/std%200.91.0-success?logo=deno\u0026logoColor=black\u0026labelColor=white\u0026color=black)](https://deno.land/std@0.91.0)\n[![documentation](https://img.shields.io/badge/docs-blue?logo=deno\u0026logoColor=black\u0026labelColor=white\u0026color=blue)](https://doc.deno.land/https/deno.land/x/pretty_benching/mod.ts)\n\n[![Build Status](https://github.com/littletof/prettyBenching/workflows/CI/badge.svg)](https://github.com/littletof/prettyBenching/actions?query=workflow%3ACI)\n[![Coverage Status](https://coveralls.io/repos/github/littletof/prettyBenching/badge.svg)](https://coveralls.io/github/littletof/prettyBenching)\n![maintained](https://img.shields.io/maintenance/yes/2022)\n[![snapper_deno](https://img.shields.io/badge/-snapper%20%F0%9F%93%B7-%230DBC79)](https://github.com/littletof/snapper)\n\n## Jump to\n\n- [prettyBenchmarkProgress](#prettybenchmarkprogress)\n- [prettyBenchmarkResults](#prettybenchmarkresults)\n- [prettyBenchmarkDown](#prettybenchmarkdown) [![deno version](https://img.shields.io/badge/Github_Action-4e4e4e?logo=github)](#as-a-github-action)\n- [prettyBenchmarkHistory](#prettybenchmarkhistory)\n- [Roadmap](#roadmap)\n\n## Try it out\n\nThis runs a short benchmark to showcase the module live.\n\n```sh\ndeno run -r --allow-hrtime https://deno.land/x/pretty_benching/example.ts\n```\n\n## Getting started\n\nAdd the following to your `deps.ts`\n\n```ts\nexport {\n  prettyBenchmarkResult,\n  prettyBenchmarkProgress,\n  prettyBenchmarkDown,\n  prettyBenchingHistory\n} from 'https://deno.land/x/pretty_benching@v0.3.3/mod.ts';\n```\n\nor just simply import it directly:\n\n```ts\nimport { prettyBenchmarkResult, prettyBenchmarkProgress, prettyBenchmarkDown, prettyBenchingHistory } from 'https://deno.land/x/pretty_benching@v0.3.3/mod.ts';\n```\n\n## Note\n\nUsing Deno's `--allow-hrtime` flag when running your code will result in a more precise benchmarking, because than float milliseconds will be used for measurement instead of integer.\n\nYou can use `nocolor` in the options of both `prettyBenchmarkProgress` and `prettyBenchmarkResult` to turn off the coloring on the output.\nIt doesn't interfere with the Deno's `fmt` color settings.\n\n# prettyBenchmarkProgress\n\nPrints the Deno `runBenchmarks()` method's `progressCb` callback values in a nicely readable format.\n\n### Usage\n\nSimply add it to `runBenchmarks()` like below and you are good to go. Using `silent: true` is encouraged, so the default logs don't interfere\n\n```ts\nawait runBenchmarks({ silent: true }, prettyBenchmarkProgress())\n```\n\nThe output would look something like this during running:\n\n![running](docs/imgs/prettyBenchingProgress_example_running.png)\n\nEnd when finished:\n\n![finished](docs/imgs/prettyBenchingProgress_example_finished.png)\n\n### Thresholds\n\nYou can define thresholds to specific benchmarks and than the times of the runs will be colored respectively\n\n```ts\nconst thresholds: Thresholds = {\n  \"for100ForIncrementX1e6\": {green: 0.85, yellow: 1},\n  \"for100ForIncrementX1e8\": {green: 84, yellow: 93},\n  \"forIncrementX1e9\": {green: 900, yellow: 800},\n  \"forIncrementX1e9x2\": {green: 15000, yellow: 18000},\n}\n\nrunBenchmarks({ silent: true }, prettyBenchmarkProgress({thresholds}))\n```\n\n![threshold](docs/imgs/prettyBenchingProgress_example_threshold.png)\n\n### Indicators\n\nYou can use indicators, which help you categorise your benchmarks. You can change the character which gets added before the benchmark. \n\n```ts\nconst indicators: BenchIndicator[] = [\n  { benches: /100/, modFn: colors.bgRed },\n  { benches: /for/, modFn: colors.red },\n  { benches: /custom/, modFn: () =\u003e colors.bgYellow(colors.black(\"%\")) }, // changes indicator char\n];\n```\n\n![indicator](docs/imgs/prettyBenchingProgress_example_indicators.png)\n\n# prettyBenchmarkResults\n\nPrints the Deno `runBenchmarks()` method's result in a nicely readable format.\n\n### Usage\n\nSimply call `prettyBenchmarkResult` with the desired settings.\n\nSetting the `nocolor` option to `true` will remove all the built in coloring. Its usefull, if you log it somewhere or save the output to a file. It won't interfere with Deno's `fmt` color settings.\n\nUse the `silent: true` flag in `runBenchmarks`, if you dont want to see the default output\n\n```ts\n// ...add benches...\n\nrunBenchmarks({silent: true})\n.then(prettyBenchmarkResult())\n.catch((e: any) =\u003e {\n  console.error(e.stack);\n});\n```\n\nThe output would look something like this:\n\n![example](docs/imgs/prettyBenchingResult_example.png)\n\n### Thresholds\n\nYou can define thresholds to specific benchmarks and than related things, like times or graph bars will be colored respectively. This can use the same thresholds object as in `prettyBenchmarkProgress`.\n\n```ts\nconst thresholds: Thresholds = {\n      \"multiple-runs\": { green: 76, yellow: 82 },\n      \"benchmark-start\": { green: 2, yellow: 3 },\n};\n\nrunBenchmarks().then(prettyBenchmarkResult({ thresholds }));\n```\n\n![threshold](docs/imgs/prettyBenchingResult_example_threshold.png)\n\n### Indicators\n\nYou can use indicators, which help you categorise your benchmarks besides just their names. You can set what color the table should have. With `modFn` you can also change what color the marker should be, or even change the indicator icon like seen below (default is `#`).\nYou can pass this object to `prettyBenchmarkProgress` too.\n\n```ts\nconst indicators: BenchIndicator[] = [\n  {\n    benches: /multiple-runs/,\n    color: colors.magenta,\n    modFn: () =\u003e \"🚀\",\n  }\n];\n\nrunBenchmarks().then(prettyBenchmarkResult({ indicators }));\n```\n\n![indicator](docs/imgs/prettyBenchingResult_example_indicators.png)\n\n### Parts\n\nYou can change what the result cards should contain with the `parts` object. Once you define it you have to set all parts you want. The default parts setting is `{ graph: true, graphBars: 5 }`.\n\nYou can define what `parts` you want to use in the options, like this:\n\n```ts\nprettyBenchmarkResult(\n  {\n    nocolor: false,\n    thresholds,\n    indicators,\n    parts: {\n      extraMetrics: true,\n      threshold: true,\n      graph: true,\n      graphBars: 10,\n    },\n  },\n)\n```\n\nUsing all options:\n\n![thresholdLine](docs/imgs/prettyBenchingResult_example_full_extra.png)\n\n##### Extra metrics `{ extraMetrics: true }`\n\nSetting this will give you an extra row, which adds extra calculated values like `min`, `max`, `mean as ((min+max)/2)` , `median`.\n\n![extraMetrics](docs/imgs/prettyBenchingResult_example_extrametrics_line.png)\n\n##### Threshold `{ threshold: true }`\n\n\u003e Need to have `thresholds` in the root of the `options` object, which have a matching threshold for the specific benchmark, otherwise it wont add it to the specific card.\n\nIt simply show what the set thresholds for the benchmark. Can be usefull if `nocolor` is set to true.\n\n![thresholdLine](docs/imgs/prettyBenchingResult_example_threshold_line.png)\n\n##### Graph `{ graph: true, graphBars: 5 }`\n\nAdds a graph, which shows the distribution of the runs of the benchmark.\n\u003e Only shows, when there are `10` or more runs set.\n\nThe graph shows the results groupped into timeframes, where the groups frame start from the value on the head of its line, and end with excluding the value on the next line.\n\nWith `graphBars` you can set how many bars it should show. Default is `5`.\n\n# prettyBenchmarkDown\n\nGenerates a summary markdown from the results of the Deno `runBenchmarks()` method's result.\n\n\u003e||Name|Runs|Total (ms)|Average (ms)|Thresholds||\n\u003e|:-:|:--|--:|--:|--:|--:|:-:|\n\u003e| |Rotating other things|1000|2143.992|2.144|-|-|\n\u003e|🎹|Rotating arrays|1000|2021.054|2.021|\u003csmall\u003e\u003c= 3.5 ✅\u003cbr\u003e\u003c= 4.4 🔶\u003cbr\u003e \u003e 4.4 🔴\u003c/small\u003e|✅|\n\u003e|%|Proving NP==P|1|4384.908|4384.908|\u003csmall\u003e\u003c= 4141 ✅\u003cbr\u003e\u003c= 6000 🔶\u003cbr\u003e \u003e 6000 🔴\u003c/small\u003e|🔶|\n\u003e|🚀|Standing out|1000|375.708|0.376|\u003csmall\u003e\u003c= 0.3 ✅\u003cbr\u003e\u003c= 0.33 🔶\u003cbr\u003e \u003e 0.33 🔴\u003c/small\u003e|🔴|\n\nA full example output: [pr_benchmark_output.md](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark_output.md)\n\n### Usage\n\nSimply call `prettyBenchmarkDown` with the desired settings.\n\n```ts\n// ...add benches...\n\nrunBenchmarks()\n.then(prettyBenchmarkDown(console.log))\n.catch((e: any) =\u003e {\n  console.error(e.stack);\n});\n```\n\nThe first parameter of this function is an output function, where you cen recieve the generated markdown's text. In the example above it just print is to `console`.\n\nWithout defining any options, it will generate one `markdown` table with one row for each benchmark.\nSomething like this:\n\n\u003e |Name|Runs|Total (ms)|Average (ms)|\n\u003e |:--|--:|--:|--:|\n\u003e |Sorting arrays|4000|1506.683|0.377|\n\u003e |Rotating arrays|1000|1935.981|1.936|\n\u003e |Proving NP==P|1|4194.431|4194.431|\n\u003e |Standing out|1000|369.566|0.370|\n\n##### Writing to a file\n\n```ts\nrunBenchmarks()\n.then(prettyBenchmarkDown(\n  (markdown: string) =\u003e { Deno.writeTextFileSync(\"./benchmark.md\", markdown); },\n  { /* ...options */ }\n))\n.catch((e: any) =\u003e {\n  console.error(e.stack);\n});\n```\n\n\u003e 🔽 Needs *--allow-write* flag to run\n\n### Options\n\nYou can fully customise the generated `markdown`. Add text, use predefined, or custom columns or group your benchmarks and define these per group.\n\nHere you can seen an example that showcases every option: [pr_benchmark_output.md](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark_output.md)\nIt was generated with: [pr_benchmarks.ts](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmarks.ts)\n\n#### Extra texts\n\n* `options.title`: Defines a level 1 title (`# MyTitle`) on the top of the generated markdown\n* `options.description`: Defines a part, that is put before all of the result tables. If defined as a function, it recieves the `runBenchmarks` result, so it can be set dynamically. It also accepts a simple string as well.\n* `options.afterTables`: Defines a part, that is put after all of the result tables. If defined as a function, it recieves the `runBenchmarks` result, so it can be set dynamically. It also accepts a simple string as well.\n\n#### Columns ```options.columns```, ```group.columns```\n\nYou can customise, what columns you want to see in each table. To see what every column type generates check out the [\u003csmall\u003eexample\u003c/small\u003e](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark_output.md)\n\n* If `not defined`, the generator uses the default columns defined by the module\n* If `defined`, you take full control, of what columns you want to see. The default columns are exported, and there are other premade columns for you to use.\n\n##### defaultColumns(columns: string[]) [\u003csmall\u003eexample\u003c/small\u003e](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark_output.md#default-columns-and-dynamic-text)\n\n```ts\ncolumns: [\n  ...defaultColumns(),\n  ...defaultColumns(['name', 'measuredRunsAvgMs'])\n]\n ```\n\nIt includes `Name`, `Runs`, `Total (ms)` and `Average (ms)` columns, these are the default values of the `BenchmarkRunResult`. Filter them with an array of propertyKeys.\n\n##### indicatorColumn(indicators: BenchIndicator[]) [\u003csmall\u003eexample\u003c/small\u003e](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark_output.md#predefiend-columns)\n\n```ts\ncolumns: [\n  indicatorColumn(indicators),\n]\n ```\n\nDefines a column, that contains the indicator for the given bench, if defined. Keep in mind, that it strips any color from the indicator.\n\n##### thresholdsColumn(thresholds: Thresholds, indicateResult?: boolean) [\u003csmall\u003eexample\u003c/small\u003e](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark_output.md#predefiend-columns)\n\n```ts\ncolumns: [\n  thresholdsColumn(thresholds), // only shows the threshold ranges\n  thresholdsColumn(thresholds, true), // shows the result in the cell too\n]\n ```\n\nDefines a column, that shows the threshold ranges for the given bench, if defined. If you set `indicateResult` to true, it shows in what range the benchmark fell, in the same cell.\n\n##### thresholdResultColumn(thresholds: Thresholds) [\u003csmall\u003eexample\u003c/small\u003e](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark_output.md#predefiend-columns)\n\n```ts\ncolumns: [\n  thresholdResultColumn(thresholds),\n]\n ```\n\nDefines a column, that show into what threhold range the benchmark fell.\n\n##### extraMetricsColumns(options?) [\u003csmall\u003eexample\u003c/small\u003e](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark_output.md#extra-metrics)\n\n```ts\ncolumns: [\n  ...extraMetricsColumns(),\n  ...extraMetricsColumns({ ignoreSingleRuns: true }), // puts '-' in cells, where bench was only run once\n  ...extraMetricsColumns({ metrics: [\"max\", \"min\", \"mean\", \"median\", \"stdDeviation\"] }),\n]\n ```\n\nDefines columns, that show extra calculated metrics like `min`, `max`, `mean`, `median`, `stdDeviation`. You can define which of these you want, in the `metrics` array. You can also tell it, to put `-` in the cells, where the benchmark was only run once with `ignoreSingleRuns`.\n\n##### Custom columns [\u003csmall\u003eexample\u003c/small\u003e](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark_output.md#custom-columns)\n\n```ts\ncolumns: [\n  {\n    title: 'CustomTotal',\n    propertyKey: 'totalMs',\n    toFixed: 5,\n    align: 'left'\n  },\n  {\n    title: 'Formatter',\n    formatter: (r: BenchmarkResult, cd: ColumnDefinition) =\u003e `${r.name}:${cd.title}`\n  },\n]\n```\n\nWhen you need something else, you can define you own columns. You can put custom `ColumnDefinitions` into the `columns` array.\n\n* The simplest way, is to give it a `propertyKey`, and than it shows that value of the `BenchmarkResult`. You can use any key here, but you will have to put these values into the results manually. If a `result[propertyKey]` is `undefined`, than it puts a `-` into that cell.\nIf your returned value is a `number`, than you can use `toFixed` to tell what precision you want to see. (It's ignored if value is not a number)\n\n* If your usecase is more complex, than you can use the `formatter` method, where you get the benchmark result, and you can return any value that you want from that. The predefined column types above use this method as well.\n\n```ts\ninterface ColumnDefinition {\n  title: string;\n  propertyKey?: string;\n  align?: \"left\" | \"center\" | \"right\";\n  toFixed?: number;\n  formatter?: (result: BenchmarkResult, columnDef: ColumnDefinition) =\u003e string;\n}\n```\n\n#### Groups ```options.groups```\n\n```ts\ngroups: [\n  {\n    include: /array/,\n    name: \"A group for arrays\",\n    description: \"The array group's description\",\n    afterTable: (gr: BenchmarkResult[], g: GroupDefinition, rr: BenchmarkRunResult) =\u003e `Dynamic ${g.name}, ${gr.length}, ${rr.results.length}`,\n    columns: [/* ... */]\n  }\n]\n```\n\nYou can group your benches, so they are separated in your generated markdown. For this, you need to define `include` RegExp. Right now, every benchmark, that doesnt fit any group will be put into one table at the bottom, so if you dont want some filter them before manually.\n\nIn each group you can define a `name` which will be a level 2 heading (`## Name`) before you group.\n\nYou can also define `description` and `afterTable`, which behave the same like the ones in the root of options.\n\nIf you want, you can have different columns in each group, if you define them in the groups `columns` array.\n\n```ts\ninterface GroupDefinition {\n  include: RegExp;\n  name: string;\n  columns?: ColumnDefinition[];\n  description?: string | ((groupResults: BenchmarkResult[], group: GroupDefinition,runResults: BenchmarkRunResult ) =\u003e string);\n  afterTable?: string | ((groupResults: BenchmarkResult[], group: GroupDefinition, runResults: BenchmarkRunResult ) =\u003e string);\n}\n```\n  \n### As a Github Action\n\nUse this in a github action, eg. comment benchmark results on PRs.\n\nYou can see an example Github Action for this [here](https://github.com/littletof/prettyBenching/blob/master/docs/prettyBenchmarkDown/pr_benchmark.yml) or see it in use in a showcase [repo](https://github.com/littletof/pretty-benching-action/pull/2).\n\n# prettyBenchmarkHistory\n\nHelps to keep track of the results of the different `runBenchmarks()` runs historically.\n\n### Usage\n\n\u003e **Note** this module doesn't handle the loading and saving of the data from/to the disk. See examples.\n\nFirst, if you already have saved historic data, you need to load it from disk (or elsewhere).\nIf no previous historicData is provided in the constructor, it starts a fresh, empty history.\n\nAfter it was initiated with the `options` and data, you can simply call `addResults` with the new results, and save them again into a file, using `getDataString()` which returns the historic data in a pretty printed JSON string. If you want to work on the data itself, call `getData()`.\n\nYou are able to set some rules in the `options`, like to only allow to add a result, if every benchmark was run a minimum of x times, or if no benchmark was added or removed or had its `runsCount` changed since the previous run.\n\nBy default it only allows to add results that were measured with `--allow-hrtime` flag, but this rule can be disabled.\n\n```ts\n// add benches, then\n\nlet historicData;\ntry {\n    historicData = JSON.parse(Deno.readTextFileSync(\"./benchmarks/history.json\"));\n} catch(e) {\n  // Decide whether you want to proceed with no history\n  console.warn(`⚠ cant read history file. (${e.message})`);\n}\n\nconst history = new prettyBenchmarkHistory(historicData, {/*options*/});\n\nrunBenchmarks().then((results: BenchmarkRunResult) =\u003e {\n    history.addResults(results {id: \"version_tag\"});\n    Deno.writeTextFileSync(\"./benchmarks/history.json\", history.getDataString());\n});\n```\n\nThe resulting historic data would look something like this, based on the options:\n\n```json\n{\n  \"history\": [\n    {\n      \"date\": \"2020-09-12T20:28:36.812Z\",\n      \"id\": \"v1.15.2\",\n      \"benchmarks\": {\n        \"RotateArrays\": {\n          \"measuredRunsAvgMs\": 0.061707600000003596,\n          \"runsCount\": 500,\n          \"totalMs\": 30.853800000001797,\n          \"extras\": {\n            \"max\": 0.45420000000001437,\n            \"min\": 0.034700000000043474,\n            \"mean\": 0.24445000000002892,\n            \"median\": 0.04179999999996653,\n            \"std\": 0.04731720894389344\n          }\n        },\n        \"x3#14\": {\n          \"measuredRunsAvgMs\": 2.6682033000000036,\n          \"runsCount\": 1000,\n          \"totalMs\": 2668.2033000000038,\n          \"extras\": {\n            \"max\": 9.25019999999995,\n            \"min\": 1.983299999999872,\n...\n```\n\n### Rules and options\n\n* **`easeOnlyHrTime`**: Allows storing low precision measurements, which where measured without `--allow-hrtime` flag\n* **`strict`**: Contains a set of rules, which are all enforced, if boolean `true` is set, but can be individually controlled if an object is provided:\n  \n  * **`noRemoval`**:  Throw an error, when previously saved benchmark is missing from the current set when calling `addResults`. Ignored on the very first set of benchmarks.\n  * **`noAddition`**: Throw an error, when previously not saved benchmark is added to the current set when calling `addResults`. Ignored on the very first set of benchmarks.\n  * **`noRunsCountChange`**: Throw an error, when the `runsCount` changes for a benchmark from the previous run's `runsCount`. Ignored on new benchmarks.\n\n* **`minRequiredRuns`**: Throw an error, when **any** benchmark has lower runsCount than the set value.\n\n* **`saveIndividualRuns`**: Saves the `measuredRunsMs` array for each benchmark.\n  **WARNING** this could result in a very big history file overtime.\n  Consider calculating necessary values before save instead with `benchExtras` or `runExtras`.\n  \n* **`benchExtras`**(result: BenchmarkResult) =\u003e T : Saves the returned `object` for each benchmark into it's `extras` property.\n\n* **`runExtras`**(runResult: BenchmarkRunResult) =\u003e K : Saves the returned `object` for each run into it's `runExtras` property.\n\n### Methods\n\n* **`addResults`**: Stores the run's result into the historic data, enforces all set rules on the results. You can specify an `id` in the options to help identify the specific historic data besides the date. It useful for example to set it to the benchmarked module's version number.\n\n* **`getDeltasFrom`**: Calls `getDeltaForBenchmark` for each benchmark in the provided `BenchmarkRunResults` and returns the values as one object.\n\n* **`getDeltaForBenchmark`**: Calculates `deltas` for given `BenchmarkResult` for each provided property key.\n\n* **`getData`**: Returns a copy of the historic data.\n\n* **`getDataString`**: Returns the historic data in a pretty-printed JSON string.\n\n* **`getBenchmarkNames`**: Returns an array of each benchmark's name, which result is present in the historic data.\n\n### Usecases\n\n* Show `deltas` in the different formats:\n\n  * **`prettyBenchmarkProgress`**: \n  ![prettyBenchingHistory_progress_delta](docs/imgs/prettyBenchingHistory_progress_delta.png)\n    \u003cdetails\u003e\n      \u003csummary\u003ecode\u003c/summary\u003e\n\n      ```ts\n      const history = new prettyBenchmarkHistory(historicData, {/*options*/});\n\n      runBenchmarks({ silent: true }, prettyBenchmarkProgress(\n        { rowExtras: deltaProgressRowExtra(history) }\n      ));\n      ```\n    \u003c/details\u003e\n\n  * **`prettyBenchmarkResults`**:\n  ![prettyBenchingHistory_result_card_delta](docs/imgs/prettyBenchingHistory_result_card_delta.png)\n    \u003cdetails\u003e\n      \u003csummary\u003ecode\u003c/summary\u003e\n\n      ```ts\n      const history = new prettyBenchmarkHistory(historicData, {/*options*/});\n\n      runBenchmarks().then(prettyBenchmarkResult(\n          { infoCell: deltaResultInfoCell(history) }\n      ));\n      ```\n    \u003c/details\u003e\n  \n  * **`prettyBenchmarkDown`**:\n  \n    |Name|Average (ms)|Change in average|\n    |:-:|:-:|:-:|\n    |x3#14|2.8319|🟢   -33% (1.3895ms)|\n    |MZ/X|5.6873|🔺   +5% (0.2468ms)|\n    |MZ/T|2.7544|-|\n\n    \u003cdetails\u003e\n      \u003csummary\u003ecode\u003c/summary\u003e\n\n      ```ts\n      const history = new prettyBenchmarkHistory(historicData, {/*options*/});\n\n      runBenchmarks().then(prettyBenchmarkDown(console.log, {\n        columns: [\n            ...defaultColumns(['name', 'measuredRunsAvgMs']),\n            deltaColumn(history),\n        ]\n      }));\n      ```\n    \u003c/details\u003e\n\n* Show each previous measurement as a column in a markdown table\n   \n  \u003e|Name|2020-09-12\u003cbr/\u003e21:54:53.706|v0.5.6|v0.8.0|Current|Change in average|\n  \u003e|:-:|--:|--:|--:|:-:|:-:|\n  \u003e|historic|0.0704|0.0740|0.0904|0.0650|🟢   -28% (0.0254ms)|\n  \u003e|x3#14|6.1675|2.9979|4.2214|3.6275|🟢   -14% (0.5939ms)|\n  \u003e|MZ/X|-|3.3095|5.4405|7.4553|🔺  +37% (2.0147ms)|\n  \u003e|MZ/T|-|-|-|3.7763|-|\n\n  \u003cdetails\u003e\n    \u003csummary\u003ecode\u003c/summary\u003e\n\n    ```ts\n      const history = new prettyBenchmarkHistory(historicData, {/*options*/});\n\n      runBenchmarks().then(prettyBenchmarkDown(console.log, {\n        columns: [\n            { title: \"Name\", propertyKey: \"name\" },\n            ...historyColumns(history),\n            { title: \"Current\", propertyKey: \"measuredRunsAvgMs\", toFixed: 4 },\n            deltaColumn(history),\n        ]\n      }));\n    ```\n  \u003c/details\u003e\n\n* Calculate thresholds from the previous results: `calculateThresholds` [docs](https://doc.deno.land/https/deno.land/x/pretty_benching/mod.ts#calculateThresholds)\n* Github Actions: Save results on version tags, report benchmarking results as a comment on PR-s.\n* Fail/warn in CI on a PR if the `delta` is too big or benchmark is in red `threshold` with: `getDeltasFrom` and `getThresholdResultsFrom`\n\n# Roadmap\n\n#### BenchmarkProgress\n\n- [x] Add `indicator` options\n- [x] Add `nocolor` option\n- [x] Unify `indicator` option types, use `color`\n- [x] Add overridable output function like in benchmark results\n\n#### BenchmarkResults\n\n- [x] Overrideable output function\n- [x] Refactor outputting result in a single call\n- [x] Add `nocolor` option\n- [x] Fix graph\n- [x] Add `indicator` options like in progress\n- [x] Tidy up current benchmark results look\n- [x] Add options to define what parts are shown in the result cards. (eg. show graph, more calculated values like mean, ...)\n- [ ] Find a place in `extraMetrics` for `standard deviation`.\n- [ ] Add option to crop outlayer results from graph (maybe with a percent limit).\n- [ ] Add an option to have a minimalist result output, that resembles the final progress output, instead of the big cards.\n  \n#### Historic data\n\n- [x] Add module to enable historic data save/read inside repo\n- [x] Make use of historic module, enable automatic calculating of thresholds from previous runs\n- [x] Option to use historic data, to tell if benchmarks got better or worse from previous runs.\n\n#### Operational\n\n- [x] Write README docs\n- [x] Separate `prettyBenchmarkResults` and `prettyBenchmarkProgress` into independently importable modules.\n- [x] Add the ability to follow the change on how the outputs look like.\n- [x] Refactor how optional `options` are handled\n- [x] Write JSDocs\n- [ ] Proper tests\n- [ ] Refactor README\n- [ ] Add showcase module, which helps to have consistent docs images\n- [ ] Make module contributor friendly\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flittletof%2Fprettybenching","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flittletof%2Fprettybenching","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flittletof%2Fprettybenching/lists"}