https://github.com/big-guy/sample-custom-test-task

https://github.com/big-guy/sample-custom-test-task

Last synced: 10 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/big-guy/sample-custom-test-task
• Owner: big-guy
• Created: 2024-12-11T04:46:04.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2025-02-21T20:53:35.000Z (over 1 year ago)
• Last Synced: 2025-04-05T10:41:51.910Z (about 1 year ago)
• Language: Java
• Size: 9.27 MB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
This sample shows how to create a plugin with a custom task that generates test events like the built-in `Test` task for JVM-based tests.

The plugin is packaged via an included build.

This sample is intended to demonstrate how a platform provider, like Android or Kotlin Multiplatform, would use a low-level API to generate test events.

This sample contains multiple parts.

Demonstrating custom test tasks using the Gradle Tooling API and the new test event reporter API:

* The `custom-test` plugin, which defines a custom task that generates test events.

* The `consumer` build, which creates a Tooling API client to receive the test events.

Demonstrating test report aggregation across projects:

* The `application` build, which represents an application written in an example "custom" ecosystem.

* The `library` build, which simulates an dependency of the application in the "custom" ecosystem.

* The `aggregation` build, which demonstrates custom test report aggregration across projects.

NOTE: The `consumer` requires Java 21.

### Command-line interactions

Run `gradle testPassingDebug` to run the custom test task. By default, no tests will fail and nothing will appear in the console log.

Run `gradle testFailingDebug` to run the custom test task with failing tests. The console log will show the test events.

Like the regular `Test` task, Gradle will produce an HTML report for each task. If tests fail, the link to the report will be part of the failure message.

When running multiple tasks that emit test events, Gradle will also produce an aggregated report with results from all tasks.

Run `gradle testFailingDebug testFailingRelease` to see this in action.

This should produce a clickable link to: `build/reports/aggregate-test-results/index.html`

### Tooling API interactions

Run `gradle :consumer:run` to run the Tooling API client. The client will run `gradle testPassingDebug testFailingDebug`, like above and print out any received test events.

Two types of events are captured:

* A `TestFinishEvent` that is generated by tasks that use the new test event reporter API as well as the existing `Test` task.

* A `TestMetadataEvent` that is generated by tasks that use the new test event reporter API. Eventually, the `Test` task will be able to generate these too.

The `TestMetadataEvent` includes custom metadata added by the task. These can be emitted any time between the start and end of a test or group of tests.

The custom metadata is represented by a map of key-value pairs (`String` to `Object`). Only serializable values are supported.

### Metadata Reporting

Metadata of various types is recorded during test execution and is available in the test reports located in `build/reports/tests`.

There are various types of metadata associated with at the root level, the test suite level and the individual test level.  Known types can be rendered with custom styles in the HTML report; for example, URI metadata is rendered as clickable links.

Running the failing tests, clicking the generated link, and investigating `MyTestSuite` is the easiest way to see this in action.

### Custom test suites

The custom-test plugin shows a simple "custom" ecosystem that uses test suites with multiple test suite targets and a custom test task.

For demonstration purposes, it also includes a custom application and custom library as stand-ins for Android applications and Android libraries.

### Test aggregation

The test aggregation plugin has been updated to support non-JVM test suites and their targets.

When applied to a project with an application, the aggregation plugin will attempt to aggregate test results for each test suite from all projects that the current project depends on.

This is determined by looking at the runtimeElements of the current project and following its dependencies and selecting the test results from like-named test suites in other projects.

You can see this in `application/build.gradle.kts`.

When applied to a subproject on its own, you need to explicitly list the projects you want to have aggregated and create aggregate reports manually.

You can see this in `aggregation/build.gradle.kts`.

### Final notes

If this sample is imported into IntelliJ, the IDE will automatically show the test UI pane when running the `testPassingDebug`, `testFailingDebug`, `testPassingRelease` and `testFailingRelease` tasks. Custom test tasks are not automatically supported in the IDE yet, so this is accomplished by setting an internal flag on the custom tasks.

As a platform provided for an ecosystem, you would create custom test tasks and use the `TestEventReporter` APIs to generate test events.

As a tooling integrator, like an IDE, you would use the Tooling API client to receive test events and display them as appropriate to the user.