{"id":18942350,"url":"https://github.com/bugcrowd/test-summary-buildkite-plugin","last_synced_at":"2025-04-15T21:31:36.652Z","repository":{"id":40693767,"uuid":"130792515","full_name":"bugcrowd/test-summary-buildkite-plugin","owner":"bugcrowd","description":"Buildkite plugin to summarise all test failures as an annotation","archived":false,"fork":false,"pushed_at":"2023-07-12T20:04:40.000Z","size":2012,"stargazers_count":63,"open_issues_count":12,"forks_count":38,"subscribers_count":38,"default_branch":"master","last_synced_at":"2023-08-07T10:13:06.808Z","etag":null,"topics":["buildkite-plugin","junit","ruby","tap"],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bugcrowd.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-24T03:43:11.000Z","updated_at":"2023-08-07T10:13:06.809Z","dependencies_parsed_at":"2023-02-14T04:46:02.890Z","dependency_job_id":null,"html_url":"https://github.com/bugcrowd/test-summary-buildkite-plugin","commit_stats":null,"previous_names":[],"tags_count":18,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bugcrowd%2Ftest-summary-buildkite-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bugcrowd%2Ftest-summary-buildkite-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bugcrowd%2Ftest-summary-buildkite-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bugcrowd%2Ftest-summary-buildkite-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bugcrowd","download_url":"https://codeload.github.com/bugcrowd/test-summary-buildkite-plugin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223685545,"owners_count":17185873,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["buildkite-plugin","junit","ruby","tap"],"created_at":"2024-11-08T12:32:34.395Z","updated_at":"2024-11-08T12:32:35.074Z","avatar_url":"https://github.com/bugcrowd.png","language":"Ruby","readme":"# Test Summary Buildkite Plugin\n\nA [Buildkite plugin](https://buildkite.com/docs/agent/v3/plugins) that adds a single annotation\nfor all your test failures using\n[buildkite-agent annotate](https://buildkite.com/docs/agent/v3/cli-annotate).\n\nSupported formats:\n\n* JUnit\n* Checkstyle\n* [TAP](https://testanything.org)^\n* Plain text files with one failure per line\n\n\\^ Current TAP support is fairly limited. If you have an example TAP file that is not being interpreted correctly,\nfeel free to open an issue or pull request.\n\n## Example\n\nUpload test results as artifacts using any supported format. If you include the `$BUILDKITE_JOB_ID` in the path,\na link to the build will be included in the annotation.\nSome examples:\n\n```yaml\nsteps:\n  - label: rspec\n    command: rspec\n    parallelism: 10\n    # With spec_helper.rb:\n    # RSpec.configure do |config|\n    #   config.add_formatter('RspecJunitFormatter', \"artifacts/rspec-#{ENV['BUILDKITE_JOB_ID']}.xml\")\n    # end\n    artifact_paths: \"artifacts/*\"\n\n  - label: eslint\n    command: yarn run eslint -f checkstyle -o artifacts/eslint.xml\n    artifact_paths: \"artifacts/*\"\n\n  - label: ava\n    command: bash -c \"yarn --silent test --tap \u003e artifacts/ava.tap\"\n    artifact_paths: \"artifacts/*\"\n\n  - label: rubocop\n    # The emacs format is plain text with one failure per line\n    command: rubocop -f emacs -o artifacts/rubocop.txt\n    artifact_paths: \"artifacts/*\"\n```\n\nWait for all the tests to finish:\n\n```yaml\n  - wait: ~\n    continue_on_failure: true\n```\n\nAdd a build step using the test-summary plugin:\n\n```yaml\n  - label: annotate\n    plugins:\n      - bugcrowd/test-summary#v1.11.0:\n          inputs:\n            - label: rspec\n              artifact_path: artifacts/rspec*\n              type: junit\n            - label: eslint\n              artifact_path: artifacts/eslint.xml\n              type: checkstyle\n            - label: ava\n              artifact_path: artifacts/ava.tap\n              type: tap\n            - label: rubocop\n              artifact_path: artifacts/rubocop.txt\n              type: oneline\n          formatter:\n            type: details\n          context: test-summary\n```\n\nSee buildkite annotation of all the failures. There are some example annotations included below.\n\n## Configuration\n\n### Inputs\n\nThe plugin takes a list of input sources. Each input source has:\n\n* `label:` the name used in the heading to identify the test group.\n* `artifact_path:` a glob used to download one or more artifacts.\n* `type:` one of `junit`, `checkstyle`, `tap` or `oneline`.\n* `encoding:` The file encoding to use. Defaults to `UTF-8`.\n* `strip_colors:` Remove ANSI color escape sequences. Defaults to `false`.\n* `crop:` (`oneline` type only) Number of lines to crop from the start and end of the file,\n  to get around default headers and footers. Eg:\n\n```yaml\ncrop:\n  start: 3\n  end: 2\n```\n\n* `job_id_regex:` Ruby regular expression to extract the `job_id` from the artifact path. It must contain\n  a named capture with the name `job_id`. Defaults to\n  `(?\u003cjob_id\u003e[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})`.\n\n#### Junit specific options:\n\n* `summary:` Customise how the summary is generated. Includes:\n    * `format:` A ruby format string for converting the junit xml attributes\n      into a summary. All attributes are available in `\u003celement\u003e.\u003cattr-name\u003e` format.\n    * `details_regex:` A ruby regular expression, run over the body text of each failure. Any named captures\n      generated by this regex will be available to the format string. This is useful if some information is only\n      available in the contents of the failure. Eg:\n\n```yaml\nsummary:\n  format: '%{testsuites.name}: %{testsuite.name}: %{testcase.classname}: %{failure.message}%{error.message}'\n```\n\n```yaml\nsummary:\n  format: '%{location}: %{testcase.name}'\n  details_regex: '(?\u003clocation\u003e\\S+:\\d+)'\n```\n\n* `message:` Set this to false if the failure `message` attribute is not worth showing in the annotation. Defaults to `true`.\n* `details:` Set this to false if the body of the failure is not worth showing in the annotation. Defaults to `true`.\n\n### Formatter\n\nThere are two formatter types, `summary` and `details`.\n\nThe `summary` formatter includes a single line for each failure.\n\n![example summary annotation](doc/summary.png)\n\nThe `details` formatter\nincludes extra information about the failure in an accordion (if available).\nThis is the default option.\n\n![example details annotation](doc/details.png)\n\nOther formatter options are:\n\n* `show_first:` The number of failures to show before hiding the rest inside an accordion.\n  If set to zero, all failures will be hidden by default. If set to a negative number, all failures\n  will be shown. Defaults to 20.\n\n### Other options\n\n* `context:` The Buildkite annotation context. Defaults to `test-summary`.\n* `style:` Set the annotation style. Defaults to `error`.\n* `fail_on_error:` Whether the command should return non-zero exit status on failure. Defaults to `false` so failing\n  to annotate a build does not cause the entire pipeline to fail.\n* `run_without_docker:` Set the enviroment to run without docker. Defaults to `false`.\n  \n## Truncation\n\nBuildkite has a maximum annotation size of 100 kilobytes. If there are too many failures to fit within this limit, the\nplugin will truncate the failures for each input.\n\n## Developing\n\nTo run the tests:\n\n    docker-compose run --rm test rspec\n\nIf you have ruby set up, you can just run:\n\n    bundle install\n    rspec\n\nTo generate sample markdown and HTML based on the files in `spec/sample_artifacts`:\n\n    bin/run-dev\n\nNote: The sample HTML files use hardcoded references to buildkite assets. If the page is not displaying correctly,\ntry updating the css files in [templates/test_layout.html.haml](templates/test_layout.html.haml) based on what\nbuildkite is currently serving.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbugcrowd%2Ftest-summary-buildkite-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbugcrowd%2Ftest-summary-buildkite-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbugcrowd%2Ftest-summary-buildkite-plugin/lists"}