{"id":13607978,"url":"https://github.com/quasilyte/go-benchrun","last_synced_at":"2025-03-19T10:30:52.283Z","repository":{"id":85823191,"uuid":"164081202","full_name":"quasilyte/go-benchrun","owner":"quasilyte","description":"Convenience wrapper around \"go test\" + \"benchstat\".","archived":false,"fork":false,"pushed_at":"2019-11-24T18:34:41.000Z","size":7,"stargazers_count":22,"open_issues_count":1,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-28T17:57:23.262Z","etag":null,"topics":["benchmark","benchstat","go","golang","performance","tool"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quasilyte.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-01-04T08:38:06.000Z","updated_at":"2024-05-08T17:19:07.000Z","dependencies_parsed_at":"2023-03-05T14:00:23.662Z","dependency_job_id":null,"html_url":"https://github.com/quasilyte/go-benchrun","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quasilyte%2Fgo-benchrun","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quasilyte%2Fgo-benchrun/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quasilyte%2Fgo-benchrun/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quasilyte%2Fgo-benchrun/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quasilyte","download_url":"https://codeload.github.com/quasilyte/go-benchrun/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243982182,"owners_count":20378605,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","benchstat","go","golang","performance","tool"],"created_at":"2024-08-01T19:01:23.303Z","updated_at":"2025-03-19T10:30:51.990Z","avatar_url":"https://github.com/quasilyte.png","language":"Go","funding_links":[],"categories":["Benchmarks"],"sub_categories":[],"readme":"# go-benchrun\n\nConvenience wrapper around \"go test\" + [benchstat](https://godoc.org/golang.org/x/perf/cmd/benchstat).\n\nRun benchmarking in 1 simple command.\n\n## Installation \u0026 Quick start\n\nThis install `go-benchrun` binary under your `$GOPATH/bin`:\n\n```bash\ngo get github.com/Quasilyte/go-benchrun\n```\n\nIf `$GOPATH/bin` is under your system `$PATH`, `go-benchrun` command should be available after that.\u003cbr\u003e\nThis should print the help message:\n\n```bash\n$ go-benchrun --help\nUsage: go-benchrun [flags...] oldBench newBench [go test args...]\n* oldBench is a pattern for `old` benchmark (w/o `Benchmark` prefix)\n* newBench is a pattern for `new` benchmark (w/o `Benchmark` prefix)\n\nExample:\n\t# compare BenchmarkOld and BenchmarkNew from foopkg package with -count=10\n\t$ go-benchrun Old New -v -count=10 foopkg\n\nFlags and defaults:\n  -newFile string\n    \tnew benchmark results destination file (default \"./new.txt\")\n  -oldFile string\n    \told benchmark results destination file (default \"./old.txt\")\n```\n\nSee \"Workflow\" section for more usage info\".\n\n## Workflow\n\nWithout `go-benchrun`, your workflow is either of these two:\n\n1. Rely on VSC.\n\t* Store old benchmark results (run go test).\n\t* Apply optimizations.\n\t* Run benchmarks again with optimized code.\n\t* Compare results with `benchstat`.\n\t* If you need to switch between implementations, you use stash and/or branches.\n2. Rely on renaming.\n\t* Use one branch, two different benchmarks.\n\t* Collect results from both benchmarks.\n\t* Before running `benchstat`, rename benchmarks, so their name matches.\n\t\n`go-benchrun` automates (2) scheme for you.\n\n1. First, it runs `-bench=oldBench` and saves results to `oldFile`.\n2. Then it runs `-bench=newBench` and saves results to `newFile`.\n3. After that, it renames `newBench` from `newFile` to `oldBench`.\n4. Finally, it runs `benchstat -geomean oldFile newFile`.\n\nFor example, lets say that you have this test file with benchmarks:\n\n```go\npackage benchmark\n\nimport (\n\t\"testing\"\n)\n\n//go:noinline\nfunc emptySliceLit() []int {\n\treturn []int{}\n}\n\n//go:noinline\nfunc makeEmptySlice() []int {\n\treturn make([]int, 0)\n}\n\nfunc BenchmarkEmptySliceLit(b *testing.B) {\n\tfor i := 0; i \u003c b.N; i++ {\n\t\t_ = emptySliceLit()\n\t}\n}\n\nfunc BenchmarkMakeEmptySlice(b *testing.B) {\n\tfor i := 0; i \u003c b.N; i++ {\n\t\t_ = makeEmptySlice()\n\t}\n}\n```\n\nIn order to compare `BenchmarkEmptySliceLit` and `BenchmarkMakeEmptySlice` you do:\n\n```bash\n$ go-benchrun EmptySliceLit MakeEmptySlice -v -count=5 .\n  Running old benchmarks:\ngoos: linux\ngoarch: amd64\nBenchmarkEmptySliceLit-8   \t300000000\t         5.79 ns/op\nBenchmarkEmptySliceLit-8   \t300000000\t         5.66 ns/op\nBenchmarkEmptySliceLit-8   \t300000000\t         5.70 ns/op\nBenchmarkEmptySliceLit-8   \t300000000\t         5.75 ns/op\nBenchmarkEmptySliceLit-8   \t300000000\t         5.84 ns/op\nPASS\nok  \t_/home/quasilyte/CODE/go/bench\t11.595s\n  Running new benchmarks:\ngoos: linux\ngoarch: amd64\nBenchmarkMakeEmptySlice-8   \t200000000\t         6.77 ns/op\nBenchmarkMakeEmptySlice-8   \t200000000\t         6.52 ns/op\nBenchmarkMakeEmptySlice-8   \t200000000\t         6.30 ns/op\nBenchmarkMakeEmptySlice-8   \t300000000\t         5.89 ns/op\nBenchmarkMakeEmptySlice-8   \t200000000\t         7.03 ns/op\nPASS\nok  \t_/home/quasilyte/CODE/go/bench\t10.375s\n  Benchstat results:\nname             old time/op  new time/op  delta\nEmptySliceLit-8  5.75ns ± 2%  6.50ns ± 9%  +13.12%  (p=0.008 n=5+5)\n```\n\nTo skip unit tests, specify `-run` flag for `go test`, as usual.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquasilyte%2Fgo-benchrun","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquasilyte%2Fgo-benchrun","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquasilyte%2Fgo-benchrun/lists"}