https://github.com/junit-team/testng-engine
TestNG Engine for the JUnit Platform
https://github.com/junit-team/testng-engine
Last synced: 9 months ago
JSON representation
TestNG Engine for the JUnit Platform
- Host: GitHub
- URL: https://github.com/junit-team/testng-engine
- Owner: junit-team
- License: epl-2.0
- Created: 2021-04-06T16:15:25.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2024-06-04T17:25:38.000Z (over 1 year ago)
- Last Synced: 2024-06-11T19:10:11.926Z (over 1 year ago)
- Language: Java
- Homepage:
- Size: 720 KB
- Stars: 25
- Watchers: 11
- Forks: 7
- Open Issues: 2
-
Metadata Files:
- Readme: README.adoc
- License: LICENSE.md
Awesome Lists containing this project
README
= TestNG Engine for the JUnit Platform
:jcommander-version: 1.72
:jquery-version: 3.5.1
:junit-platform-version: 1.8.2
:surefire-version: 3.0.0-M7
:testng-min-version: 6.14.3
:testng-max-version: 7.10.2
:testng-engine-version: 1.0.5
Allows executing https://testng.org[TestNG] tests on the JUnit Platform
== Usage
.Console Launcher
[%collapsible]
====
When running without a build tool, you need to download the following jars from Maven Central:
* https://search.maven.org/remotecontent?filepath=org/junit/platform/junit-platform-console-standalone/{junit-platform-version}/junit-platform-console-standalone-{junit-platform-version}.jar[junit-platform-console-standalone-{junit-platform-version}.jar]
* https://search.maven.org/remotecontent?filepath=org/junit/support/testng-engine/{testng-engine-version}/testng-engine-{testng-engine-version}.jar[testng-engine-{testng-engine-version}.jar]
* https://search.maven.org/remotecontent?filepath=org/testng/testng/{testng-max-version}/testng-{testng-max-version}.jar[testng-{testng-max-version}.jar]
* https://search.maven.org/remotecontent?filepath=com/beust/jcommander/{jcommander-version}/jcommander-{jcommander-version}.jar[jcommander-{jcommander-version}.jar]
* https://search.maven.org/remotecontent?filepath=org/webjars/jquery/{jquery-version}/jquery-{jquery-version}.jar[jquery-{jquery-version}.jar]
The following samples assume the above jars have been downloaded to the local `lib` folder and production and test classes to `bin/main` and `bin/test`, respectively.
[source,subs="attributes+"]
----
$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher \
-cp bin/main -cp bin/test \
--include-engine=testng --scan-classpath=bin/test
Thanks for using JUnit! Support its development at https://junit.org/sponsoring
╷
└─ TestNG ✔
└─ CalculatorTests ✔
├─ add(int, int, int) ✔
│ ├─ [0] 0, 1, 1 ✔
│ ├─ [1] 1, 2, 3 ✔
│ ├─ [2] 49, 51, 100 ✔
│ └─ [3] 1, 100, 101 ✔
└─ addsTwoNumbers ✔
2021-07-04T17:43:52.223145 description = `1 + 1 = 2`
Test run finished after 38 ms
[ 3 containers found ]
[ 0 containers skipped ]
[ 3 containers started ]
[ 0 containers aborted ]
[ 3 containers successful ]
[ 0 containers failed ]
[ 5 tests found ]
[ 0 tests skipped ]
[ 5 tests started ]
[ 0 tests aborted ]
[ 5 tests successful ]
[ 0 tests failed ]
----
====
.Gradle
[%collapsible]
====
[source,kotlin,subs="attributes+"]
.build.gradle[.kts]
----
dependencies {
testImplementation("org.testng:testng:{testng-max-version}")
testRuntimeOnly("org.junit.support:testng-engine:{testng-engine-version}") // <.>
}
tasks.test {
useJUnitPlatform() // <.>
}
----
<.> Add the engine as an extra dependency for running tests
<.> Configure the test task to use the JUnit Platform
====
.Maven
[%collapsible]
====
[source,xml,subs="attributes+"]
.pom.xml
----
org.testng
testng
{testng-max-version}
test
org.junit.support
testng-engine
{testng-engine-version}
test
maven-surefire-plugin
{surefire-version}
----
====
=== Supported versions
==== TestNG
The engine supports TestNG version {testng-min-version} and above.
==== JUnit Platform
The engine requires at least JUnit Platform 1.5.x.
=== Configuration Parameters
The following JUnit Platform https://junit.org/junit5/docs/current/user-guide/#running-tests-config-params[configuration parameters] are supported.
==== Execution
`testng.allowReturnValues` (boolean)::
whether methods annotated with `@Test` that have return values should be considered test methods (default: `false`; see https://testng.org/doc/documentation-main.html#test-methods[documentation])
+
`testng.dataProviderThreadCount` (integer)::
maximum number of threads to use for running data providers in parallel, if enabled via `@DataProvider(parallel = true)` (default: `10`; see https://testng.org/doc/documentation-main.html#parameters-dataproviders[documentation])
+
`testng.excludedGroups` (comma-separated list)::
groups to exclude (see <>)
+
`testng.groups` (comma-separated list)::
groups to be run (see <>)
+
`testng.parallel` (methods|tests|classes|instances|none)::
TestNG's parallel execution mode for running tests in separate threads (default: `"none"`; see https://testng.org/doc/documentation-main.html#parallel-tests[documentation])
+
`testng.preserveOrder` (boolean)::
whether classes and methods should be run in a predictable order (default: `true`; see https://testng.org/doc/documentation-main.html#testng-xml[documentation])
+
`testng.threadCount` (integer)::
maximum number of threads for running tests in parallel, if enabled via `testng.parallel` (default: `5`; see https://testng.org/doc/documentation-main.html#parallel-tests[documentation])
==== Reporting
`testng.listeners` (comma-separated list of fully-qualified class names)::
custom listeners that should be registered when executing tests (default: `""`; see https://testng.org/doc/documentation-main.html#testng-listeners[documentation])
+
`testng.outputDirectory` (file path)::
the output directory for reports (default: `"test-output"`; see https://testng.org/doc/documentation-main.html#running-testng[documentation])
+
`testng.useDefaultListeners` (boolean)::
whether TestNG's default report generating listeners should be used (default: `false`; see https://testng.org/doc/documentation-main.html#running-testng[documentation])
+
`testng.verbose` (integer)::
TestNG's level of verbosity for console output (default: `0`)
=== Generating TestNG reports
.Console Launcher
[%collapsible]
====
[source]
----
$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher \
-cp bin/main -cp bin/test \
--include-engine=testng --scan-classpath=bin/test \
--config=testng.useDefaultListeners=true \
--config=testng.outputDirectory=test-reports
----
====
.Gradle
[%collapsible]
====
[source,kotlin,subs="attributes+"]
.build.gradle[.kts]
----
tasks.test {
useJUnitPlatform()
systemProperty("testng.useDefaultListeners", "true")
val testNGReportsDir = layout.buildDirectory.dir("reports/testng")
outputs.dir(testNGReportsDir).withPropertyName("testng-reports")
jvmArgumentProviders += CommandLineArgumentProvider {
listOf("-Dtestng.outputDirectory=${testNGReportsDir.get().asFile.absolutePath}")
}
}
----
====
.Maven
[%collapsible]
====
[source,xml,subs="attributes+"]
----
maven-surefire-plugin
{surefire-version}
testng.useDefaultListeners = true
testng.outputDirectory = ${project.build.directory}/testng-reports
----
====
=== Registering custom listeners
.Console Launcher
[%collapsible]
====
[source]
----
$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher \
-cp bin/main -cp bin/test \
--include-engine=testng --scan-classpath=bin/test \
--config=testng.listeners=com.acme.MyCustomListener1,com.acme.MyCustomListener2
----
====
.Gradle
[%collapsible]
====
[source,kotlin,subs="attributes+"]
.build.gradle[.kts]
----
tasks.test {
useJUnitPlatform()
systemProperty("testng.listeners", "com.acme.MyCustomListener1, com.acme.MyCustomListener2")
}
----
====
.Maven
[%collapsible]
====
[source,xml,subs="attributes+"]
----
maven-surefire-plugin
{surefire-version}
testng.listeners = com.acme.MyCustomListener1, com.acme.MyCustomListener2
----
====
== Limitations
[#groups_vs_tags]
=== Groups vs. Tags
https://testng.org/doc/documentation-main.html#test-groups[Groups] declared via the `@Test` annotation on test classes and methods are exposed as tags to the JUnit Platform.
Hence, you can use https://junit.org/junit5/docs/current/user-guide/#running-tests-tag-expressions[tag filter expressions] to include or exclude certain groups.
For example, given the following test class, including the tag `included` and excluding `excluded` will run test `a` and `c` but not `b` and `d`.
[source,java]
----
import org.testng.annotations.Test;
public class TestWithGroups {
@Test(groups = "included")
public void a() {}
@Test(groups = {"included", "excluded"})
public void b() {}
@Test(groups = "included")
public void c() {}
@Test
public void d() {}
}
----
However, since tags and therefore groups are filtered rather than selected, `@BeforeGroups` and `@AfterGroups` configuration methods are not executed on some versions of TestNG.
Moreover, tag filters do not support wildcards such as `integration.*`.
Therefore, instead of using tags, you can use the `testng.groups` and `testng.excludedGroups` configuration parameters to specify the groups that should be selected.
[WARNING]
====
Ideally, you should use either tags or the group configuration parameters.
If you specify both, please ensure they match.
====
==== Using Tags
.Console Launcher
[%collapsible]
====
[source]
----
$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher \
-cp bin/main -cp bin/test \
--include-engine=testng --scan-classpath=bin/test \
--include-tag=included --exclude-tag=excluded
----
====
.Gradle
[%collapsible]
====
[source,kotlin,subs="attributes+"]
.build.gradle[.kts]
----
tasks.test {
useJUnitPlatform {
includeTags("included")
excludeTags("excluded")
}
}
----
====
.Maven
[%collapsible]
====
[source,xml,subs="attributes+"]
----
maven-surefire-plugin
{surefire-version}
included
excluded
----
====
==== Using Configuration Parameters
.Console Launcher
[%collapsible]
====
[source]
----
$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher \
-cp bin/main -cp bin/test \
--include-engine=testng --scan-classpath=bin/test \
--config=testng.groups=included --config=testng.excludedGroups=excluded
----
====
.Gradle
[%collapsible]
====
[source,kotlin,subs="attributes+"]
.build.gradle[.kts]
----
tasks.test {
useJUnitPlatform()
systemProperty("testng.groups", "included")
systemProperty("testng.excludedGroups", "excluded")
}
----
====
.Maven
[%collapsible]
====
[source,xml,subs="attributes+"]
----
maven-surefire-plugin
{surefire-version}
testng.groups = included
testng.excludedGroups = excluded
----
====
=== Suites
The engine's main intention is integration with build tools like Gradle and Maven.
Hence, custom suites specified via `testng.xml` files are not supported.