{"id":18006098,"url":"https://github.com/brianhicks/elm-benchmark","last_synced_at":"2025-03-26T10:32:18.816Z","repository":{"id":57674506,"uuid":"80526699","full_name":"BrianHicks/elm-benchmark","owner":"BrianHicks","description":"Benchmarking for Elm","archived":false,"fork":false,"pushed_at":"2018-05-11T15:27:41.000Z","size":703,"stargazers_count":47,"open_issues_count":0,"forks_count":5,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-21T16:07:13.411Z","etag":null,"topics":["benchmarking","performance"],"latest_commit_sha":null,"homepage":"http://package.elm-lang.org/packages/BrianHicks/elm-benchmark/latest","language":"Elm","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BrianHicks.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-31T14:18:17.000Z","updated_at":"2022-11-09T04:14:51.000Z","dependencies_parsed_at":"2022-08-29T15:41:57.505Z","dependency_job_id":null,"html_url":"https://github.com/BrianHicks/elm-benchmark","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrianHicks%2Felm-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrianHicks%2Felm-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrianHicks%2Felm-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BrianHicks%2Felm-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BrianHicks","download_url":"https://codeload.github.com/BrianHicks/elm-benchmark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245637307,"owners_count":20648127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking","performance"],"created_at":"2024-10-30T00:23:16.982Z","updated_at":"2025-03-26T10:32:18.328Z","avatar_url":"https://github.com/BrianHicks.png","language":"Elm","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Elm Benchmark\n\n## ⚠ This repo is moving to [elm-explorations/benchmark](https://github.com/elm-explorations/benchmark). Open new issues and send PRs there, please!\n\n[![Build Status](https://travis-ci.org/BrianHicks/elm-benchmark.svg?branch=master)](https://travis-ci.org/BrianHicks/elm-benchmark)\n\nRun microbenchmarks in Elm.\n\n\u003c!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc --\u003e\n**Table of Contents**\n\n- [Elm Benchmark](#elm-benchmark)\n    - [Quick Start](#quick-start)\n        - [Installing](#installing)\n        - [Running Benchmarks in the Browser](#running-benchmarks-in-the-browser)\n    - [Writing Effective Benchmarks](#writing-effective-benchmarks)\n    - [FAQ](#faq)\n        - [What Does Goodness of Fit Mean?](#what-does-goodness-of-fit-mean)\n        - [Why Are My Benchmarks Run In Parallel?](#why-are-my-benchmarks-run-in-parallel)\n        - [How Are My Benchmarks Measured?](#how-are-my-benchmarks-measured)\n    - [Inspirations and Thanks](#inspirations-and-thanks)\n    - [License](#license)\n\n\u003c!-- markdown-toc end --\u003e\n\n\n## Quick Start\n\nHere's a sample, benchmarking [`Array.Hamt`](http://package.elm-lang.org/packages/Skinney/elm-array-exploration/latest).\n\n```elm\nimport Array\nimport Arry.Hamt as Hamt\nimport Benchmark exposing (..)\n\n\nsuite : Benchmark\nsuite =\n    let\n        sampleArray =\n            Hamt.initialize 100 identity\n    in\n    describe \"Array.Hamt\"\n        [ -- nest as many descriptions as you like\n          describe \"slice\"\n            [ benchmark \"from the beginning\" \u003c|\n                \\_ -\u003e Hamt.slice 50 100 sampleArray\n            , benchmark \"from the end\" \u003c|\n                \\_ -\u003e Hamt.slice 0 50 sampleArray\n            ]\n\n        -- compare the results of two benchmarks\n        , Benchmark.compare \"initialize\"\n            \"HAMT\"\n            (\\_ -\u003e Hamt.initialize 100 identity)\n            \"core\"\n            (\\_ -\u003e Array.initialize 100 identity)\n        ]\n```\n\nThis code uses a few common functions:\n\n- `describe` to organize benchmarks\n- `benchmark` to run benchmarks\n- `compare` to compare the results of two benchmarks\n\nFor a more thorough overview, I've written an [introduction to elm-benchmark](https://www.brianthicks.com/post/2017/02/27/introducing-elm-benchmark/).\n\n### Installing\n\nYou should keep your benchmarks separate from your code since you don't want the elm-benchmark code in your production artifacts.\nThis is necessary because of how `elm-package` works; it may change in the future.\nHere are the commands (with explanation) that you should run to get started:\n\n```sh\nmkdir benchmarks                             # create a benchmarks directory\ncd benchmarks                                # go into that directory\nelm package install BrianHicks/elm-benchmark # get this project, including the browser runner\n```\n\nYou'll also need to add your main source directory (probably `../` or `../src`) to the `source-directories` list in `benchmarks/elm-package.json`.\nIf you don't do this, you won't be able to import the code you're benchmarking!\n\n### Running Benchmarks in the Browser\n\n`Benchmark.Runner` provides `program`, which takes a `Benchmark` and runs it in the browser.\nTo run the sample above, you would do:\n\n```elm\nimport Benchmark.Runner exposing (BenchmarkProgram, program)\n\n\nmain : BenchmarkProgram\nmain =\n    program suite\n```\n\nCompile and open in your browser to start the benchmarking run.\n\n## Writing Effective Benchmarks\n\nSome general principles:\n\n- Don't compare raw values from different machines.\n- When you're working on speeding up a function, keep the old implementation around and use `compare` to measure your progress.\n- \"As always, if you see numbers that look wildly out of whack, you shouldn’t rejoice that you have magically achieved fast performance—be skeptical and investigate!\" – [Bryan O'Sullivan](http://www.serpentine.com/criterion/tutorial.html)\n\n## FAQ\n\n### What Does Goodness of Fit Mean?\n\nGoodness of fit is a measurement of how well our prediction fits the measurements we have collected.\nYou want this to be as close to 100% as possible.\nIn elm-benchmark:\n\n- 99% is great\n- 95% is okay\n- 90% is not great, consider closing other programs on your computer and re-running\n- 80% and below, the result has been highly affected by outliers.\n  Please do not trust the results when goodness of fit is this low.\n  \nelm-benchmark will eventually incorporate this advice into the reporting interface.\nSee [Issue #13](https://github.com/BrianHicks/elm-benchmark/issues/13).\n\nFor more, see [Wikipedia: Goodness of Fit](https://en.wikipedia.org/wiki/Goodness_of_fit).\n\n### Why Are My Benchmarks Running In Parallel?\n\nThey're not, but they look like it because we interleave runs and only update the UI after collecting one of each.\nKeep reading for more on why we do this!\n\n### How Are My Benchmarks Measured?\n\nWhen we measure the speed of your code, we take the following steps:\n\n1. We warm up the JIT so that we can get a good measurement.\n2. We measure how many runs of the function will fit into a small but measurable timespan.\n3. We collect multiples of this amount, until we have enough to create a trend.\n   We can do this because running a benchmark twice *should* take twice as long as running it once, so we can create a reliable prediction by splitting sample sizes among a number of buckets.\n4. Once we have enough we present our prediction of runs per second *on your computer, in this configuration, now*.\n   We try to be as consistent as possible, but be aware that the environment matters a lot!\n\nIf the run contains multiple benchmarks, we interleave sampling between them.\nThis means that given three benchmarks we would take one sample of each and continue in that pattern until they were complete.\n\nWe do this because the system might be busy with other work when running the first, but give its full attention to the second and third.\nThis would make one artificially slower than the others, so we would get misleading data!\n\nBy interleaving samples, we spread this offset among all the benchmarks.\nIt sets a more even playing field for all the benchmarks, and gives us better data.\n\n## Inspirations and Thanks\n\n- [Bryan O'Sullivan's Criterion](http://www.serpentine.com/criterion/), from which we take our prediction technique.\n- [Gary Bernhardt's Readygo](https://github.com/garybernhardt/readygo#timing-methodology), from which we take interleaved runs.\n- Special thanks to Luke Westby for both the [initial implementation of the measurement code](https://gist.github.com/lukewestby/9d8e2b0816d417eae926ed86c01de0b8) and starting the conversations that lead to this library.\n\n## License\n\nelm-benchmark is licensed under a 3-Clause BSD License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrianhicks%2Felm-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrianhicks%2Felm-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrianhicks%2Felm-benchmark/lists"}