{"id":19791983,"url":"https://github.com/xray-app/xray-junit-extensions","last_synced_at":"2025-05-01T01:33:02.330Z","repository":{"id":38452218,"uuid":"347014864","full_name":"Xray-App/xray-junit-extensions","owner":"Xray-App","description":" Enhanced JUnit 5 integration with Xray Test Management for Jira","archived":false,"fork":false,"pushed_at":"2024-08-20T10:59:05.000Z","size":362,"stargazers_count":14,"open_issues_count":5,"forks_count":7,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-08-20T12:53:43.592Z","etag":null,"topics":["jira","junit","testautomation","testing","xray"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Xray-App.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-12T09:42:51.000Z","updated_at":"2024-08-20T10:57:30.000Z","dependencies_parsed_at":"2024-03-21T17:30:30.851Z","dependency_job_id":"9ff288be-3ac8-46dc-af6f-3233ae319054","html_url":"https://github.com/Xray-App/xray-junit-extensions","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xray-App%2Fxray-junit-extensions","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xray-App%2Fxray-junit-extensions/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xray-App%2Fxray-junit-extensions/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xray-App%2Fxray-junit-extensions/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Xray-App","download_url":"https://codeload.github.com/Xray-App/xray-junit-extensions/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224231431,"owners_count":17277386,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jira","junit","testautomation","testing","xray"],"created_at":"2024-11-12T07:05:33.647Z","updated_at":"2025-05-01T01:33:02.322Z","avatar_url":"https://github.com/Xray-App.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# xray-junit-extensions\n\n[![build workflow](https://github.com/Xray-App/xray-junit-extensions/actions/workflows/CI.yml/badge.svg)](https://github.com/Xray-App/xray-junit-extensions/actions/workflows/CI.yml)\n[![Known Vulnerabilities](https://snyk.io/test/github/Xray-App/xray-junit-extensions/badge.svg)](https://snyk.io/test/github/Xray-App/xray-junit-extensions)\n![code coverage](\nhttps://raw.githubusercontent.com/Xray-App/xray-junit-extensions/main/.github/badges/jacoco.svg)\n[![license](https://img.shields.io/badge/License-EPL%202.0-green.svg)](https://opensource.org/licenses/EPL-2.0)\n[![Gitter chat](https://badges.gitter.im/gitterHQ/gitter.png)](https://gitter.im/Xray-App/community)\n[![Maven Central Version](https://img.shields.io/maven-central/v/app.getxray/xray-junit-extensions)](https://central.sonatype.com/artifact/app.getxray/xray-junit-extensions/)\n\nThis repo contains several improvements for [JUnit](https://junit.org/junit5/) that allow you to take better advantage of JUnit 5 (jupiter engine) whenever using it together with [Xray Test Management](https://getxray.app).\nThis code is provided as-is; you're free to use it and modify it at your will (see license ahead).\n\nThis is a preliminary release so it is subject to changes, at any time.\n\n## Overview\n\nResults from automated scripts implemented as `@Test` methods can be tracked in test management tools to provide insights about quality aspects targeted by those scripts and their impacts.\nTherefore, it's important to attach some relevant information during the execution of the tests, so it can be shared and analyzed later on in the test management tool (e.g. [Xray Test Management](https://getxray.app)).\n\nThis project is highly based on previous work by the JUnit team. The idea is to be able to produce a custom JUnit XML report containing additional information that Xray can take advantage of.\nThis way, testers can automate the test script and at the same time provide information such as the covered requirement, right from the test automation code. Additional information may be provided, either through new annotations or by injecting a custom reporter as argument to the test method, using a specific extension.\n\n### Features\n\n- track started and finished date timestamps for each test\n- link a test method to an existing Test issue or use auto-provisioning\n- cover a \"requirement\" (i.e. an issue in Jira) from a test method\n- specify additional fields for the auto-provisioned Test issues (e.g. summary, description, labels)\n- attach screenshots or any other file as evidence to the Test Run, right from within the test method\n- add comments ot the Test Run, right from within the test method\n- set the values for Test Run custom fields, right from within the test method\n\n### Main classes\n\nThe project consists of:\n\n- **EnhancedLegacyXmlReportGeneratingListener**: a custom TestExecutionListener implementation that is responsible for generating a custom JUnit XML with additional properties Xray can take advantage of\n- **@XrayTest**, **@Requirement**: new, optional annotations to provide additional information whenever writing the automated test methods\n- **XrayTestReporterParameterResolver**: a new, optional JUnit 5 extension that provides the means to report additional information to Xray, inside the test method flow\n\n## Installing\n\nThese extensions are available as an artifact available on (Maven) Central Repository, which is configured by default in your Maven instalation.\n\nAdd the following dependency to your pom.xml:\n\n```xml\n        \u003cdependency\u003e\n          \u003cgroupId\u003eapp.getxray\u003c/groupId\u003e\n          \u003cartifactId\u003exray-junit-extensions\u003c/artifactId\u003e\n          \u003cversion\u003e0.10.0\u003c/version\u003e\n          \u003cscope\u003etest\u003c/scope\u003e\n        \u003c/dependency\u003e\n```\n\n### Configuration\n\nIf you want, you may configure certain aspects of this extension. The defaults should be fine, otherwise please create a properties file as `src/test/resources/xray-junit-extensions.properties`, and define some settings.\n\n- `report_filename`: the name of the report, without ending .xml suffix. Default is \"TEST-junit-jupiter.xml\"\n- `report_directory`: the directory where to generate the report, in relative or absolute format. Default is \"target\"\n- `add_timestamp_to_report_filename`: add a timestamp based suffix to the report. Default is \"false\".\n- `report_only_annotated_tests`: only include tests annotated with @XrayTest or @Requirement. Default is \"false\".\n- `reports_per_class`: generate JUnit XML reports per test class instead of a single report with all results; if true, `report_filename`, and `add_timestamp_to_report_filename` are ignored. Default is \"false\".\n- `test_metadata_reader`: override the default logic responsible for reading meta-information about test methods.\n\nExample:\n\n```bash\nreport_filename=custom-report-junit\nreport_directory=reports\nadd_timestamp_to_report_filename=true\nreport_only_annotated_tests=false\nreports_per_class=false\ntest_metadata_reader=com.example.CustomTestMetadataReader\n```\n\n## How to use\n\nIn order to generate the enhanced, customized JUnit XML report we need to register the **EnhancedLegacyXmlReportGeneratingListener** listener. This can be done in [several ways](https://junit.org/junit5/docs/current/user-guide/#launcher-api-listeners-custom):\n\n- discovered automatically at runtime based on the contents of a file (e.g `src/test/resources/META-INF/services/org.junit.platform.launcher.TestExecutionListener`) \n\n```\napp.getxray.xray.junit.customjunitxml.EnhancedLegacyXmlReportGeneratingListener\n```\n\n- programmaticaly\n\n```java\nLauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder\n                                                .request().selectors(...).filters(...).build();\nLauncher launcher = LauncherFactory.create();\nlauncher.discover(request)\nEnhancedLegacyXmlReportGeneratingListener listener = new EnhancedLegacyXmlReportGeneratingListener( Paths.get(\"/tmp/reports/\"),new PrintWriter(System.out));\nlauncher.registerTestExecutionListeners(listener);\nlauncher.execute(request);\n```\n\nRegistering the listener is mandatory.\nIn order to take advantage of the capabilities of this new listener, new annotations can be used. There's also a custom ParameterResolver **XrayTestReporterParameterResolver**, which allows users to inject a **XrayTestReporter** object as argument on the test methods. This will be useful when it's needed to report/attach some additional information during the test lifecycle.\n\n\n### New annotations\n\nTwo new annotations (`@XrayTest`, `@Requirement`) can be used.\nThe annotations are optional and cannot be use more than once per test method.\n\n#### @XrayTest\n\nTest methods don't need to be annotated with `@XrayTest` unless you want to take advantage of the following enhancements.\n\nYou may use the **@XrayTest** annotation to:\n\n- enforce mapping of result to specific, existing Test identified by issue key, using the **key** attribute\n- enforce mapping of result to specific, existing Test issue identified by the Generic Test Definition, using the **id** attribute\n- enforce the summary on the corresponding Test issue, during auto-provisioning, using the **summary** attribute\n- enforce the description on the corresponding Test issue, during auto-provisioning, using the **description** attribute\n\n_Examples:_\n\n1. enforce the given test and corresponding result to be reported against the existing Test issue having the key  CALC-1000 (i.e. create a Test Run in Xray for the Test with issue key CALC-1000)  \n\n```java\n    @Test\n    @XrayTest(key = \"CALC-1000\")\n    public void CanAddNumbers()\n```\n\n2. enforce the \"Summary\" and the \"Description\" fields on the corresponding Test issue. The test will be auto-provisioned in Xray, if it doesn't exist yet, based on the class name and method name implementing the test  \n\n```java\n    @Test\n    @XrayTest(summary = \"addition of two numbers\", description = \"tests the sum of two positive integers\")\n    public void CanAddNumbers()\n```\n\n#### @Requirement\n\nYou may use the **Requirement** annotation, before your test methods in order to identify the covered requirement(s).\nIt's possible to identify one covered issue (e.g. requirement) or more, even though it's a good practice to cover just one.\n\n_Examples:_\n\n\n1. use this test to cover a requirement/user story identified by the issue key CALC-1234 (i.e create a issue link \"tests\" between the Test issue and the identified requirement);  \n\n```java\n    @Test\n    @Requirement(\"CALC-1234\")\n    public void CanAddNumbers()\n```\n\n2. use this test to cover two requirements/user stories identified by the issue keys CALC-1234, CALC-1235 (i.e create a issue link \"tests\" between the Test issue and the identified requirements).\n\n```java\n    @Test\n    @Requirement({\"CALC-1234\", \"CALC-1235\"})\n    public void CanAddNumbers()\n```\n\n\n### New Extension\n\nA new JUnit 5 compatible Extension **XrayTestReporterParameterResolver** can be used, so we can inject a **XrayTestReporter** object as argument in the test methods.\n\nIt allows to:\n\n- add one or more comments to the Test Run; if used multiple times, comments are appended;\n- define the value of some Test Run's custom field;\n- attach some file(s) as global evidence on the Test Run.\n\n_Examples:_\n\n1. add overall comments to the results (i.e. the Test Run), set the value of some Test Run custom field, and attach one (or more) evidence.\n\n```java\n@ExtendWith(XrayTestReporterParameterResolver.class)\npublic class XrayEnabledTestExamples {\n\n...\n    @Test\n    public void someTest(XrayTestReporter xrayReporter) {\n        xrayReporter.addComment(\"hello\");\n        xrayReporter.setTestRunCustomField(\"myTRcustomfield\", \"field1_value\");\n        // you can also set a multiple-value based TR custom field\n        String [] someArr = {\"Porto\", \"Lisbon\"};\n        xrayReporter.setTestRunCustomField(\"multiplevalueTRcustomfield\", someArr);\n        xrayReporter.addTestRunEvidence(\"/tmp/screenshot.png\");\n        xrayReporter.addComment(\"world\");\n    }\n```\n\n### Customizing how test metadata is read\n\nWhen generating the report, it's allowed to customize the way the test method information is read.\nBy default, test information such as id, key, summary, description and requirements are read directly from @XrayTest and @Requirements annotations.\nThis behavior can be overridden by the user when he wants to change the way these meta-information are generated, or when he wants to use their own annotations to describe the tests.\n\nTo do this, you need to create a public class with a no-argument constructor that implements the `app.getxray.xray.junit.customjunitxml.XrayTestMetadataReader` interface (or extend `app.getxray.xray.junit.customjunitxml.DefaultXrayTestMetadataReader` class).\nThen must add `test_metadata_reader` entry with the class name to the `xray-junit-extensions.properties` file.\n\n\n#### Example: Custom test metadata reader to read Jira key from custom @JiraKey annotation\n\n_JiraKey.java_\n```java\npackage com.example;\n\nimport java.lang.annotation.Retention;\nimport java.lang.annotation.RetentionPolicy;\n\n@Retention(RetentionPolicy.RUNTIME)\npublic @interface JiraKey {\n    String value();\n}\n```\n\n_CustomTestMetadataReader.java_\n```java\npackage com.example;\n\nimport app.getxray.xray.junit.customjunitxml.DefaultXrayTestMetadataReader;\nimport org.junit.platform.launcher.TestIdentifier;\n\nimport java.util.Optional;\n\npublic class CustomTestMetadataReader extends DefaultXrayTestMetadataReader {\n\n    @Override\n    public Optional\u003cString\u003e getKey(TestIdentifier testIdentifier) {\n        return getTestMethodAnnotation(testIdentifier, JiraKey.class)\n                .map(JiraKey::value)\n                .filter(s -\u003e !s.isEmpty());\n    }\n}\n```\n\n_xray-junit-extensions.properties_\n```\ntest_metadata_reader=com.example.CustomTestMetadataReader\n```\n\n_SimpleTest.java_\n```java\npackage com.example;\n\nimport org.junit.jupiter.api.DisplayName;\nimport org.junit.jupiter.api.Test;\n\nclass SimpleTest {\n\n    @Test\n    @JiraKey(\"CALC-123\")\n    @DisplayName(\"simple test\")\n    void simpleTest() {\n        // ...\n    }\n}\n```\n\n## Other features and limitations\n\n### Name of Tests\n\nThe summary of the Test issue will be set based on these rules (the first that applies):\n\n* based on the summary attribute of @XrayTest annotation (e.g. `@XrayTest(summary = \"xxx\")`); \n* based on the `@DisplayName` annotation, or the display name of dynamically created tests from a TestFactory;\n* based on the test's method name.\n\n\u003e [!TIP]\n\u003e This behavior can be changed by defining a custom test metadata reader.\n\n### Parameterized and repeated tests\n\nFor the time being, and similar to what happened with legacy JUnit XML reports produces with JUnit 4, parameterized tests (i.e. annotated with `@ParameterizedTest`) will be mapped to similar `\u003ctestcase\u003e` elements in the JUnit XML report.\nThe same happens with repeated tests (i.e annotated with `@RepeatedTest`).\n\n## Background\n\nWith JUnit 4 and surefire, legacy JUnit XML reports can be produced. These are used by many tools, including CI, and test management tools (e.g. Xray).\nHowever, the format produced by JUnit 4 is limited and cannot be extended.\n\nJunit 5 has a more flexible architecture. Even though JUnit XML format as not evolved meanwhile, it's possible to use Extensions and Test Execution Listeners to implement our own, tailored custom reporter.\n\nJUnit 5 provides a [legacy XML reporter for the jupiter engine](https://junit.org/junit5/docs/current/api/org.junit.platform.reporting/org/junit/platform/reporting/legacy/xml/LegacyXmlReportGeneratingListener.html) as a listener, which was used as basis for this implementation.\n\n\n## TO DOs\n\n...\n\n## FAQ\n\n1. Can this be used with JUnit 4?\nNo. If you're using JUnit 4 you can still generate a \"standard\" JUnit XML report but you'll miss the capabilities provided by this project. It's recommended to use JUnit 5 as JUnit 4 is an older project and much more limited.\n\n2. Is this format compatible with Jenkins and other tools?\nProbably. As there is no official JUnit XML schema, it's hard to say that in advance. However, the new information being embed on the custom JUnit XML report is done in such a way that shouldn't break other tools.\n\n3. Why use a custom JUnit XML report instead of Xray JSON report?\nFor several reasons. One of them is that it was easier to take advantage of the existing code and adapt it.\n\n4. Then enhanced JUnit XML report gets overwritten whenever I have different test classes...\nMake sure you're using a recent version of `maven-surefire-plugin`, like 3.2.5; that will fix it!  If you're using a old version (e.g., 2.22.2), surefire will generate one JUnit testplan per class, and the reporter will be invoked multiple times, overwriting the previous report. For more info see [issue #47](https://github.com/Xray-App/xray-junit-extensions/issues/47)  and [this discussion](https://github.com/Xray-App/xray-junit-extensions/discussions/48).\n\n5. I don't see the enhanced JUnit XML report or I see a report but without the Xray specific information.\nYou're probably using the legacy JUnit report and not the one generated by this plugin. Make sure you're using the `EnhancedLegacyXmlReportGeneratingListener`; see how to enable it, above in the \"How to use\" section.\n\n## Contact\n\nYou may find me on [Twitter](https://twitter.com/darktelecom).\nAny questions related with this code, please raise issues in this GitHub project. Feel free to contribute and submit PR's.\nFor Xray specific questions, please contact [Xray's support team](https://jira.getxray.app/servicedesk/customer/portal/2).\n\n## References\n\n- [JUnit 5 User Guide](https://junit.org/junit5/docs/current/user-guide/)\n- [How Xray processes JUnit XML reports](https://docs.getxray.app/display/XRAYCLOUD/Taking+advantage+of+JUnit+XML+reports)\n- [JUnit 5 legacy XML reporter](https://junit.org/junit5/docs/current/api/org.junit.platform.reporting/org/junit/platform/reporting/legacy/xml/LegacyXmlReportGeneratingListener.html)\n\n## LICENSE\n\nBased on code from [JUnit5](https://github.com/junit-team/junit5/) project.\n\n[Eclipse Public License - v 2.0](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxray-app%2Fxray-junit-extensions","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxray-app%2Fxray-junit-extensions","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxray-app%2Fxray-junit-extensions/lists"}