{"id":13492430,"url":"https://github.com/bobheadxi/gobenchdata","last_synced_at":"2025-04-04T13:09:08.607Z","repository":{"id":37820427,"uuid":"183367253","full_name":"bobheadxi/gobenchdata","owner":"bobheadxi","description":"📉 Run Go benchmarks, publish results to an interactive web app, and check for performance regressions in your pull requests","archived":false,"fork":false,"pushed_at":"2024-10-28T18:45:31.000Z","size":8990,"stargazers_count":150,"open_issues_count":11,"forks_count":13,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-28T12:05:25.019Z","etag":null,"topics":["benchmarking","charts","continuous-benchmarking","continuous-benchmarks","github-actions","golang","typescript","vuejs"],"latest_commit_sha":null,"homepage":"https://gobenchdata.bobheadxi.dev","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bobheadxi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-25T06:06:15.000Z","updated_at":"2025-03-06T16:54:45.000Z","dependencies_parsed_at":"2024-06-12T15:42:48.267Z","dependency_job_id":"e4e8b93e-5af6-475f-bebc-8f9ab5a8a3f7","html_url":"https://github.com/bobheadxi/gobenchdata","commit_stats":{"total_commits":289,"total_committers":7,"mean_commits":"41.285714285714285","dds":0.4982698961937716,"last_synced_commit":"20682793c754a5377454d3e2b617e45a55ba0b6d"},"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobheadxi%2Fgobenchdata","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobheadxi%2Fgobenchdata/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobheadxi%2Fgobenchdata/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bobheadxi%2Fgobenchdata/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bobheadxi","download_url":"https://codeload.github.com/bobheadxi/gobenchdata/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247181909,"owners_count":20897375,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking","charts","continuous-benchmarking","continuous-benchmarks","github-actions","golang","typescript","vuejs"],"created_at":"2024-07-31T19:01:05.967Z","updated_at":"2025-04-04T13:09:08.587Z","avatar_url":"https://github.com/bobheadxi.png","language":"Go","funding_links":[],"categories":["Go","Community Resources"],"sub_categories":["Monitoring"],"readme":"# 📉 gobenchdata [![View Action](https://img.shields.io/badge/view-github%20action-yellow.svg)](https://bobheadxi.dev/r/gobenchdata) [![demo website](https://img.shields.io/website/https/gobenchdata.bobheadxi.dev.svg?down_color=grey\u0026down_message=offline\u0026label=demo%20website\u0026up_message=live)](https://gobenchdata.bobheadxi.dev/)\n\n[![publish@v1](https://github.com/bobheadxi/gobenchdata/workflows/publish@v1/badge.svg)](https://github.com/bobheadxi/gobenchdata/actions?workflow=publish%40v1)\n[![pipeline](https://github.com/bobheadxi/gobenchdata/workflows/pipeline/badge.svg)](https://github.com/bobheadxi/gobenchdata/actions?workflow=pipeline)\n[![demo](https://github.com/bobheadxi/gobenchdata/workflows/demo/badge.svg)](https://github.com/bobheadxi/gobenchdata/actions?workflow=demo)\n[![Go Reference](https://pkg.go.dev/badge/go.bobheadxi.dev/gobenchdata.svg)](https://pkg.go.dev/go.bobheadxi.dev/gobenchdata)\n[![go-recipes](https://raw.githubusercontent.com/nikolaydubina/go-recipes/main/badge.svg?raw=true)](https://github.com/nikolaydubina/go-recipes#-continuous-benchmarking-with-gobenchdata)\n\n`gobenchdata` is a tool for parsing and inspecting `go test -bench` data, and a [GitHub Action](https://github.com/features/actions) for continuous benchmarking. It was inspired by the [`deno.land` continuous benchmarks](https://deno.land/benchmarks), which aims to display performance improvements and regressions on a continuous basis.\n\n\u003ca href=\"https://gobenchdata.bobheadxi.dev/\" target=\"_blank\"\u003e\n  \u003cimg align=\"right\" width=\"500\" src=\"./.static/demo-chart.png\" alt=\"example\"\u003e\n\u003c/a\u003e\n\n- [GitHub Action](#github-action)\n  - [Setup](#setup)\n  - [Configuration](#configuration)\n    - [`inputs`](#inputs)\n      - [Publishing](#publishing)\n      - [Checks](#checks)\n    - [`env`](#env)\n  - [Pull Request Checks](#pull-request-checks)\n  - [Visualisation](#visualisation)\n- [Command Line Interface](#command-line-interface)\n- [Contributions](#contributions)\n\n\u003cbr /\u003e\n\n## GitHub Action\n\n`gobenchdata` can be used as GitHub Action for uploading Go benchmark data as\nJSON to `gh-pages` and visualizing it with a generated web app or your own web application.\n\n### Setup\n\nThe default action, `uses: bobheadxi/gobenchdata@v1`, is distributed as a [Docker container action](https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action).\nThis means that the Go version benchmarks are run with are tied to the version of Go that ships with the `gobenchdata` action.\nFor greater flexibility, such as to use custom Go versions, see [custom setup](#custom-setup).\n\nLearn more about GitHub Actions in the [official documentation](https://github.com/features/actions).\n\n#### Docker container action\n\nFor example, in `.github/workflows/push.yml`, using [the new YAML syntax for workflows](https://help.github.com/en/articles/workflow-syntax-for-github-actions), a simple benchmark [run and publish workflow](#publishing) would look like:\n\n```yml\nname: gobenchdata publish\non: push\njobs:\n  publish:\n    runs-on: ubuntu-latest\n    steps:\n    - name: checkout\n      uses: actions/checkout@v2\n    - name: gobenchdata publish\n      uses: bobheadxi/gobenchdata@v1\n      with:\n        PRUNE_COUNT: 30\n        GO_TEST_FLAGS: -cpu 1,2\n        PUBLISH: true\n        PUBLISH_BRANCH: gh-pages\n      env:\n        GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}\n```\n\n#### Custom setup\n\nFor greater flexibility and the ability to target custom Go versions, you can either:\n\n- directly use the [`entrypoint.sh`](entrypoint.sh) script, or some variant of it\n- run `gobenchdata action`, which replicates the behaviour of the GitHub Action\n\nUsing `gobenchdata action` to replicate the [Docker container action](#docker-container-action) would look like:\n\n```yml\nname: gobenchdata publish\non: push\njobs:\n  publish:\n    runs-on: ubuntu-latest\n    steps:\n    - name: checkout\n      uses: actions/checkout@v2\n    - uses: actions/setup-go@v2\n      with: { go-version: \"1.18\" }\n    - name: gobenchdata publish\n      run: go run go.bobheadxi.dev/gobenchdata@v1 action\n      env:\n        GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}\n        # Instead of 'with: ...', provide environment variables and\n        # prefix each input variable with 'INPUT'\n        INPUT_PRUNE_COUNT: 30\n        INPUT_GO_TEST_FLAGS: -cpu 1,2\n        INPUT_PUBLISH: true\n        INPUT_PUBLISH_BRANCH: gh-pages\n```\n\n### Configuration\n\n#### `inputs`\n\nInput variables are configured using\n[`jobs.\u003cjob_id\u003e.steps.with`](https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstepswith).\n\n| Variable             | Default                   | Purpose\n| -------------------- | ------------------------- | -------\n| `SUBDIRECTORY`       | `.`                       | subdirectory of project to run commands from\n| `GO_BENCHMARKS`      | `.`                       | benchmarks to run (argument for `-bench`)\n| `GO_TEST_FLAGS`      |                           | additional flags for `go test`\n| `GO_TEST_PKGS`       | `./...`                   | packages to test (argument for `go test`)\n\n##### Publishing\n\nThe following `inputs` enable publishing - this merges and publishes benchmark results to a\nrepository and branch of your choice. This is most useful in conjunction with the [`gobenchdata` web application](#visualisation).\n\n| Variable             | Default                   | Purpose\n| -------------------- | ------------------------- | -------\n| `PUBLISH`            | `false`                   | if `true`, publishes results\n| `PUBLISH_REPO`       |                           | an alternative repository to publish to\n| `PUBLISH_BRANCH`     | `gh-pages`                | branch to publish to\n| `PRUNE_COUNT`        | `0`                       | number of past runs to keep (`0` keeps everything)\n| `GIT_COMMIT_MESSAGE` | `\"add new benchmark run\"` | the commit message for the benchmark update\n| `BENCHMARKS_OUT`     | `benchmarks.json`         | destination path of benchmark data\n\n##### Checks\n\nThe following `inputs` are for enabling [Pull Request Checks](#pull-request-checks), which allow\nyou to watch for performance regressions in your pull requests.\n\n| Variable             | Default                   | Purpose\n| -------------------- | ------------------------- | -------\n| `CHECKS`             | `false`                   | if `true`, runs checks and sets JSON results to `checks-results`\n| `CHECKS_CONFIG`      | `gobenchdata-checks.yml`  | path to checks configuration\n| `PUBLISH_REPO`       |                           | repository of benchmark data to check against\n| `PUBLISH_BRANCH`     | `gh-pages`                | branch of benchmark data to check against\n| `BENCHMARKS_OUT`     | `benchmarks.json`         | path to benchmark data to check against\n\n#### `env`\n\nEnvironment variables are configured using\n[`jobs.\u003cjob_id\u003e.steps.env`](https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstepsenv).\n\n| Variable             | Recommended                   | Purpose\n| -------------------- | ----------------------------- | -------\n| `GITHUB_TOKEN`       | `${{ secrets.GITHUB_TOKEN }}` | token to provide access to repository\n| `GITHUB_ACTOR`       | set by GitHub                 | the user to make commits as\n\nNote that for `GITHUB_TOKEN`, it seems that pushes to `gh-pages` made by the default\n`secrets.GITHUB_TOKEN` might not trigger page builds. This issue can be resolved by using\na [personal access token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line)\ninstead.\n\n### Pull Request Checks\n\nInstead of publishing results, benchmark output can be used to pass and fail pull requests\nusing `CHECKS: true`. To get started, set up the checks configuration:\n\n```sh\ngo install go.bobheadxi.dev/gobenchdata@latest\ngobenchdata checks generate\n```\n\nThis will generate a file, `gobenchdata-checks.yml`, where you can configure what checks are\nexecuted. The checks are run against any benchmarks that match given `package` and `benchmarks`\nvalues, which should be provided as [regular expressions](https://regexr.com/).\n\n\u003cdetails\u003e\n\u003csummary\u003eSimple Example\u003c/summary\u003e\n\u003cp\u003e\n\n\u003c!-- copy from './gobenchdata checks generate --checks.config ./tmp/gobenchdata-checks.yml' --\u003e\n\n```yml\nchecks:\n- name: My Check\n  description: |-\n    Define a check here - in this example, we caculate % difference for NsPerOp in the diff function.\n    diff is a function where you receive two parameters, current and base, and in general this function\n    should return a negative value for an improvement and a positive value for a regression.\n  package: .\n  benchmarks: []\n  diff: (current.NsPerOp - base.NsPerOp) / base.NsPerOp * 100\n  thresholds:\n    max: 10\n```\n\n\u003c/p\u003e\n\u003c/details\u003e\n\n### Visualisation\n\nThe `gobenchdata` GitHub action eventually generates a JSON file with past benchmarks.\nYou can visualize these continuous benchmarks by creating a web app that reads\nfrom the JSON benchmarks file, or by using `gobenchdata`. An easy way to get started is:\n\n```sh\ngo install go.bobheadxi.dev/gobenchdata@latest\ngobenchdata web generate --web.config-only .\ngobenchdata web serve # opens visualization in browser\n```\n\nYou can configure the web application using `gobenchdata-web.yml`. The configuration allows\nyou to define groups of charts, where each group can be used to compare a set of benchmarks.\nBenchmarks are selected with [regular expressions](https://regexr.com/) by package and benchmark\nnames provided in the configuration.\n\nNote that in each set of compared benchmarks, every metric will get its own chart. You can\nselect which metrics to display using the `metrics` option.\n\n\u003cdetails\u003e\n\u003csummary\u003eExample\u003c/summary\u003e\n\u003cp\u003e\n\n\u003c!-- copy from './web/public/gobenchdata-web.yml' --\u003e\n\n```yml\ntitle: gobenchdata web\ndescription: Benchmarks generated using 'gobenchdata'\nrepository: https://github.com/bobheadxi/gobenchdata\nbenchmarksFile: benchmarks.json\nchartGroups:\n  - name: Demo Benchmarks\n    description: |\n      This is a demo for gobenchdata, a tool and GitHub action for setting up simple continuous\n      benchmarks to monitor performance improvements and regressions in your Golang benchmarks!\n    charts:\n      - name: specify charts by package\n        package: go.bobheadxi.dev\\/gobenchdata\\/demo\n      - name: match on specific benchmarks across packages with glob patterns\n        benchmarks: [ 'BenchmarkFib.' ]\n  - name: More Demo Benchmarks\n    description: Create multiple groups of benchmarks\n    charts:\n      - name: match by a combination of package and benchmarks\n        package: go.bobheadxi.dev\\/gobenchdata\\/.\n        benchmarks: [ 'BenchmarkPizzas.', '.FibSlow.' ]\n```\n\n\u003c/p\u003e\n\u003c/details\u003e\n\nYou can output the entire web application (to publish to Github pages, for example) using:\n\n```sh\ngobenchdata web generate ./app\n```\n\n\u003cbr /\u003e\n\n## Command Line Interface\n\n`gobenchdata`, which the GitHub Action leverages to manage benchmark data,\nis also available as a CLI:\n\n```sh\ngo install go.bobheadxi.dev/gobenchdata@latest\ngobenchdata help\n```\n\nThe easiest way to use the CLI is by piping the output of `go test -bench` to\nit - `gobenchdata` will consume the output and generate a JSON report for you.\n\n```sh\ngo test -bench . -benchmem ./... | gobenchdata --json bench.json\n```\n\nYou can use this report to create your own charts, or just use the [built-in web application](#visualisation):\n\n```sh\ngobenchdata web serve\n```\n\n`gobenchdata` can also execute checks for you to help you ensure performance\nregressions don't happen:\n\n```sh\ngobenchdata checks generate\ngobenchdata checks eval ${base benchmarks} ${current benchmarks}\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExample Report\u003c/summary\u003e\n\u003cp\u003e\n\n|    |                   BASE                   |                 CURRENT                  | PASSED CHECKS | FAILED CHECKS | TOTAL |\n|----|------------------------------------------|------------------------------------------|---------------|---------------|-------|\n| ✅ | 5aa9b7f901e770f1364bfc849aaba0cc06066336 | abfdd5c29b1aff48cb22e0cbb6f4f7526ad85604 |             2 |             0 |     2 |\n\n|    |             CHECK              |              PACKAGE              |             BENCHMARK             | DIFF  | COMMENT |\n|----|--------------------------------|-----------------------------------|-----------------------------------|-------|---------|\n| ✅ | An example NsPerOp check       | go.bobheadxi.dev/gobenchdata/demo | BenchmarkFib10/Fib()              | -2.61 |         |\n| ✅ | An example NsPerOp check       | go.bobheadxi.dev/gobenchdata/demo | BenchmarkFib10/Fib()-2            | -2.85 |         |\n| ✅ | An example NsPerOp check       | go.bobheadxi.dev/gobenchdata/demo | BenchmarkFib10/FibSlow()          | -2.47 |         |\n| ✅ | An example NsPerOp check       | go.bobheadxi.dev/gobenchdata/demo | BenchmarkFib10/FibSlow()-2        | -2.19 |         |\n| ✅ | An example NsPerOp check       | go.bobheadxi.dev/gobenchdata/demo | BenchmarkPizzas/Pizzas()          | -1.85 |         |\n| ✅ | An example NsPerOp check       | go.bobheadxi.dev/gobenchdata/demo | BenchmarkPizzas/Pizzas()-2        | -2.45 |         |\n| ✅ | An example NsPerOp check       | go.bobheadxi.dev/gobenchdata/demo | BenchmarkPizzas/PizzasSquared()   | -5.71 |         |\n| ✅ | An example NsPerOp check       | go.bobheadxi.dev/gobenchdata/demo | BenchmarkPizzas/PizzasSquared()-2 | -3.03 |         |\n| ✅ | An example custom metric check | go.bobheadxi.dev/gobenchdata/demo | BenchmarkPizzas/Pizzas()          |  8.00 |         |\n| ✅ | An example custom metric check | go.bobheadxi.dev/gobenchdata/demo | BenchmarkPizzas/Pizzas()-2        |  4.00 |         |\n| ✅ | An example custom metric check | go.bobheadxi.dev/gobenchdata/demo | BenchmarkPizzas/PizzasSquared()   |  4.00 |         |\n| ✅ | An example custom metric check | go.bobheadxi.dev/gobenchdata/demo | BenchmarkPizzas/PizzasSquared()-2 |  1.00 |         |\n\n\u003c/p\u003e\n\u003c/details\u003e\n\nFor more details on how to use checks, see the [pull request checks documentation](#pull-request-checks).\n\nThe CLI also embeds the same script that the GitHub Action offers - see [custom setup](#custom-setup) for more details.\n\n\u003cbr /\u003e\n\n## Contributions\n\nPlease report bugs and requests in the [repository issues](https://go.bobheadxi.dev/gobenchdata)!\n\nSee [CONTRIBUTING.md](./CONTRIBUTING.md) for more detailed development documentation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbobheadxi%2Fgobenchdata","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbobheadxi%2Fgobenchdata","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbobheadxi%2Fgobenchdata/lists"}