1. WildFly Testsuite Overview

This document will detail the implementation of the testsuite integration submodule as it guides you on adding your own test cases.

The WildFly integration test suite has been designed with the following goals:

  • support execution of all identified test case use cases

  • employ a design/organization which is scalable and maintainable

  • provide support for the automated measurement of test suite quality (generation of feature coverage reports, code coverage reports)

In addition, these requirements were considered:

  • identifying distinct test case runs of the same test case with a different set of client side parameters and server side parameters

  • separately maintaining server side execution results (e.g. logs, the original server configuration) for post-execution debugging

  • running the testsuite in conjunction with a debugger

  • the execution of a single test (for debugging purposes)

  • running test cases against different container modes (managed in the main, but also remote and embedded)

  • configuring client and server JVMs separately (e.g., IPv6 testing)

1.1. Prerequisites

Before building and running the WildFly testsuite, ensure the following requirements are met.

1.1.1. Java Development Kit

  • Minimum: JDK 17 or later is required to build and run the testsuite.

  • Recommended: The latest LTS or current JDK release is recommended. See the recommended.java.version property in the root pom.xml for the current recommendation.

Make sure JAVA_HOME is set to point to your JDK installation.

1.1.2. Apache Maven

A recent version of Apache Maven is required. The WildFly source tree includes mvnw (and mvnw.cmd on Windows) Maven Wrapper scripts that automatically download and use the correct Maven version. Using these scripts is recommended over a system-installed Maven.

1.1.3. Supported Operating Systems

The testsuite is regularly tested on Linux, macOS, and Windows. Linux is the primary development and CI platform.

1.1.4. Docker or Podman

Some integration tests use container images (e.g., Kafka, Keycloak, Elasticsearch). These tests require Docker or Podman to be available on the system.

If no container runtime is available, the behavior depends on how the test obtains its container:

  • Tests guarded by AssumeTestGroupUtil.assumeDockerAvailable() are skipped. Set -Dorg.wildfly.test.require.docker=true to make them fail instead.

  • Tests using Arquillian Testcontainers fail, except on Windows, where the testsuite configures them to be skipped instead.

The rest of the testsuite can still be run without a container runtime.

1.1.5. System Resources

  • Disk space: A full build with all tests requires significant disk space for the source tree, Maven local repository, and provisioned server installations.

  • Memory: At least 8 GB of RAM is recommended. Some test groups (e.g., clustering) start multiple server instances concurrently.

  • On Linux, you may need to increase the open file descriptor limit (ulimit -n) if you encounter "too many open files" errors during the testsuite run.

See Changing the container images used in integration tests for details on the container images used and how to override them.

1.2. Test Suite Organization

1.2.1. Testsuite modules

The testsuite has the following maven modules that provide test classes:

Module Description

domain

Domain management integration tests (require special framework)

galleon

Specialized tests of managing WildFly using the Galleon CLI

integration

General integration tests

layers

Validation of galleon layers provided by the wildfly-ee feature-pack

layers-expansion

Validation of galleon layers provided by the wildfly and wildfly-preview feature-packs

mixed-domain

Domain mode tests where the primary host controller is running the version under test while the secondary host controller runs a previous version

preview

Integration tests that are specific to WildFly Preview

scripts

Test various shell scripts provided by WildFly

It is expected that test contributions fit into one of these test modules.

The testsuite also includes modules that help support testsuite execution but that do not provide tests:

Module Description

aggregator-base

Controls which test modules are executed when executing the base-server-tests profile

aggregator-expansion

Controls which test modules are executed when executing the expansion-server-tests profile

aggregator-preview

Controls which test modules are executed when executing the preview-server-tests profile

build-demander-base

Ensures the server build required by the base-server-tests profile has been built, unless the maven build was started with the -Dtestsuite.only property (e.g. when running the testsuite against an external build)

build-demander-expansion

Ensures the server build required by the expansion-server-tests profile has been built, unless the maven build was started with the -Dtestsuite.only property (e.g. when running the testsuite against an external build)

build-demander-preview

Ensures the server build required by the preview-server-tests profile has been built, unless the maven build was started with the -Dtestsuite.only property (e.g. when running the testsuite against an external build)

shared

Shared utility classes that test classes can use

test-feature-pack

Feature-pack, based on wildfly-ee, that test modules can provision

test-feature-pack-preview

Feature-pack, based on wildfly-preview, that test modules can provision

test-product-conf

Produces an artifact used by an integration/manualmode test

1.2.2. Parent poms

  • testsuite/pom.xml - this has a header describing the purpose of the pom and what should and should not go in it

  • testsuite/integration/pom.xml - this has a header describing the purpose of the pom and what should and should not go in it

1.2.3. Directory structure within a testsuite module

For each logical grouping of tests X (e.g. basic integration, clustering, smoke, …​), we use the following locations:

  • src/test/java/org/jboss/as/X - holds all sources and test-specific resources for the X tests. For newer content the src/test/java/org/wildfly/X directory is used. If for a given X there is already a org/jboss/as/X directory, for new content please do not create a new src/test/java/org/wildfly/X directory; use the existing directory.

  • src/test/resources/X - holds resources shared by X tests, which include resources for creating deployments as well as Arquillian configuration files

1.2.4. Testsuite pom.xml

The pom.xml file located in the testsuite module is inherited by all submodules and is used to:

  • set defaults for common testsuite system properties (which can then be overridden on the command line)

  • define dependencies common to all tests (Arquillian, junit, and container type)

  • define a default server configuration installed into each submodule

It should not:

  • define module-specific server configuration build steps

  • define module-specific surefire executions

These elements should be defined in logical profiles associated with each logical grouping of tests; e.g., in the pom for the module which contains the tests.

1.3. Controlling testsuite execution via profiles

Maven build profiles are used to control:

  • the server variant being tested

  • testsuite modules enabled

  • test cases to be executed within a module

  • behavior of plugins used to control the tests, including how the server being tested is provisioned and launched

You should not activate profiles by -P, because that disables other profiles which are activated by default. Instead, use the activating properties described below (e.g., -DallTests, -Dts.basic).

1.3.1. Controlling the server variant being tested

The WildFly build produces four different feature-packs that can be used in five different combinations to provision servers. Use -D to activate a profile to control what variant is tested and what dependencies are used to build test deployments and on the test client side.

If none of the profiles listed below is specifically activated, the testsuite will execute as if the latest-ee-full-server-tests profile was activated.

Some testsuite modules or execution profiles provision a server using the wildfly-maven-plugin before running tests; others use pre-built installations created by the main build. The following table lists the different profiles that can be used and the feature-packs each uses for wildfly-maven-plugin provisioning and the location of the pre-built server that will be used.

Profile Feature-packs used Pre-built server used

latest-ee-only-server-tests

wildfly-ee

ee-build

latest-ee-full-server-tests

wildfly

build

legacy-ee-only-server-tests AND legacy-ee-tests

wildfly-ee-10

legacy/ee-feature-pack/ee-build

legacy-ee-full-server-tests AND legacy-ee-tests

wildfly-ee-10 + wildfly

legacy/ee-feature-pack/build

preview-server-tests

wildfly-preview

preview/build
When the legacy-ee-only-server-tests profile or the legacy-ee-full-server-tests profile is activated, the legacy-ee-tests profile must also be activated, at least if the testsuite/integration/basic module is executed. This is an unfortunate necessity due to the Maven 3 limitation of only allowing a single property to deactivate a profile. The legacy-ee-tests profile deactivates behavior that we want enabled by default.
Using an externally provisioned server

For testsuite modules/profiles that don’t logically require wildfly-maven-plugin provisioning, an externally provided pre-built server can be used. Use the jboss.dist or jboss.inst system property with a value pointing to the filesystem location of the pre-built server.

Even if an externally provided pre-built server is used, unless the default behavior is wanted one of the profiles in the table above needs to be activated to tell the testsuite what dependencies should be used for test deployments and by the test client.
Overriding dependency versions with WildFly Channels

If, instead of using servers provisioned using dependency versions defined in the WildFly pom, you wish to use versions defined in external WildFly Channels manifests, use the external.wildfly.channels system property to provide the locations of the channel manifests.

1.3.2. Controlling which modules are enabled

There are three main groupings of tests, each of which is enabled via a profile:

Profile Purpose Notes

base-server-tests

Tests functionality provided by the wildfly-ee feature-pack and the wildfly-ee-10 feature-pack

Enabled by default. Disabled if -Dpreview-server-tests is used. Otherwise, to disable use -P-base-server-tests

expansion-server-tests

Tests functionality provided by the wildfly feature-pack

Enabled by default. Disabled if -Dpreview-server-tests is used. Otherwise, to disable use -P-expansion-server-tests

preview-server-tests

Tests functionality provided by the wildfly-preview feature-pack

Disabled by default. Use -Dpreview-server-tests to activate.
The base server tests

The base server tests cover functionality provided by the wildfly-ee and wildfly-ee-10 feature-packs.

When the base-server-tests profile executes, by default the testsuite/integration/web and testsuite/integration/smoke modules execute against an unslimmed, traditional (i.e. non-bootable-jar) installation.

There are a number of other profiles that can be activated to control how tests are executed and which modules run. The following table shows the activation of these profiles and what they do.

Activation Value Effect

-DallTests

N/A

Runs nearly all of the modules that test base server functionality. Tests unslimmed traditional installations.

-Dts.basic

N/A

Runs the testsuite/integration/basic and testsuite/integration/ws modules.

-Dts.bootable

N/A

Runs all of the modules that are able to test bootable jars, configured to use the wildfly-maven-plugin to produce bootable jars.

-Dts.layers

N/A

Runs all of the modules that are able to test slimmed servers, configured to use the wildfly-maven-plugin to produce various slimmed servers and test those, instead of testing unslimmed installations.

-Dts.noSmoke

N/A

Disables execution of the testsuite/integration/web and testsuite/integration/smoke

-Djboss.test.mixed.domain.dir

Path to a directory containing older version zips.

Enables the mixed domain tests.

See also Profiles for specific testsuite modules below for how to activate execution of specific modules.

The expansion server tests

The expansion server tests cover functionality provided by the wildfly feature-pack.

When the expansion-server-tests profile executes, by default no module containing tests executes. Executing test modules requires enabling one or more other profiles.

There are a number of other profiles that can be activated to control how tests are executed and which modules run. The following table shows the activation of these profiles and what they do.

Activation Effect

-DallTests

Runs nearly all of the modules that test functionality provided by the wildfly feature-pack. Tests unslimmed traditional installations.

-Dts.bootable

Runs all of the modules that are able to test bootable jars, configured to use the wildfly-maven-plugin to produce bootable jars.

-Dts.layers

Runs all of the modules that are able to test slimmed servers, configured to use the wildfly-maven-plugin to produce various slimmed servers and test those, instead of testing unslimmed installations.

-Dts.microprofile

Runs the testsuite/integration/expansion and testsuite/integration/microprofile-tck modules

-Dts.standalone.microprofile

Runs a number of modules, including some that only test base server functionality, but using servers provisioned from the wildfly feature-pack and running the standalone-microprofile[-ha].xml configuration.
The preview server tests

The preview server tests cover functionality provided by the wildfly-preview feature-pack.

When the preview-server-tests profile executes, by default the testsuite/integration/web and testsuite/integration/smoke modules execute against an unslimmed, traditional (i.e. non-bootable-jar) installation.

There are a number of other profiles that can be activated to control how tests are executed and which modules run. The following table shows the activation of these profiles and what they do.

Activation Effect

-Dts.preview

Runs nearly all of the testsuite modules against servers provisioned from the wildfly-preview feature-pack. Tests unslimmed traditional installations.

-Dts.bootable.preview

Runs all of the modules that are able to test bootable jars, configured to use the wildfly-maven-plugin to produce bootable jars.

-Dts.noSmoke

Disables execution of the testsuite/integration/web and testsuite/integration/smoke
Profiles for specific testsuite modules

For several top-level testsuite modules, there is a profile named X.module.profile that can be enabled by -DX.module; e.g. domain.module.profile is enabled by -Ddomain.module and runs the tests in testsuite/domain. This applies to the domain, galleon, layers, layers-expansion and scripts modules.

The -Dinteg.module property enables all the child modules of testsuite/integration.

Many individual modules can also be run by executing a profile by passing a property in the form -Dts.X where X is the name of the lowest level maven module that contains the tests. For example, -Dts.domain runs the tests in testsuite/domain while -Dts.clustering runs the tests in testsuite/integration/clustering.

1.4. Integration tests

1.4.1. Smoke -Dts.smoke

Contains smoke tests.

Runs by default; use -Dts.noSmoke to prevent running.

Tests should execute quickly.

Divided into two Surefire executions:

  • One with full platform

  • Second with web profile (majority of tests).

1.4.2. Basic -Dts.basic

Basic integration tests - those which do not need a special configuration like cluster.

Divided into three Surefire executions:

  • One with full platform,

  • Second with web profile (majority of tests).

  • Third with web profile, but needs to be run after server restart to check whether persistent data are really persisted.

1.4.3. Clustering -Dts.clustering

Contains all tests relating to clustering aspects of the application server, such as:

  • web session clustering,

  • Jakarta Enterprise Beans session clustering,

  • command dispatcher,

  • web session affinity handling,

  • and other areas.

Tests should leverage shared testing logic by extending org.jboss.as.test.clustering.cluster.AbstractClusteringTestCase. The test case contract is that before executing the test method, all specified servers are started and all specified deployments are deployed. This allows Arquillian resource injection into the test case.

There are four WildFly server instances, one load-balancer (Undertow) and one datagrid (Infinispan server) available for the tests.

Maven profiles and Parallelization

There are maven profiles that might come in handy for testing:

  • ts.clustering.common.profile prepares server configurations used by test execution profiles

  • ts.clustering.cluster.ha.profile runs tests against standalone-ha.xml profile

  • ts.clustering.cluster.fullha.profile runs tests which require standalone-full-ha.xml profile; e.g. tests requiring JMS subsystem

  • ts.clustering.cluster.ha-infinispan-server.profile runs tests against standalone-ha.xml profile with Infinispan Server provisioned via @ClassRule

  • ts.clustering.single.profile runs clustering tests that are using a non-HA server profile

  • ts.clustering.single.testable.profile runs clustering tests that use testable=true deployments

  • ts.clustering.byteman.profile runs clustering tests that require installation of byteman rules

For instance, to only run tests that run against full-ha profile, activate clustering tests with -Dts.clustering and exclude the other profiles with -P:

mvn -f testsuite/integration/clustering/pom.xml clean install -P"-ts.clustering.cluster.ha.profile,-ts.clustering.cluster.fullha.profile,-ts.clustering.cluster.ha-infinispan-server.profile,-ts.clustering.single.profile,-ts.clustering.byteman.profile"

If the testsuite can be run on multiple runners in parallel, the main execution (which takes the majority of the execution time) can be split by packages using -Dts.surefire.clustering.ha.additionalExcludes property. This property feeds a regular expression to exclude sub-packages of the org.jboss.as.test.clustering.cluster package. The sub-packages at the time of writing are affinity, cdi, dispatcher, ejb, ejb2, group, jms, jpa, jsf, provider, registry, singleton, sso, web, and xsite. For instance, to parallelize testsuite execution on two machines (e.g. when using GitHub actions scripting or alike) the following commands could be used to split the clustering tests into two executions of similar execution time, the first node can run the first half of the tests in sub-packages, e.g.:

./integration-tests.sh clean install -Dts.noSmoke -Dts.clustering -P="-ts.clustering.cluster.fullha.profile,-ts.clustering.cluster.ha-infinispan-server.profile,-ts.clustering.byteman.profile,-ts.clustering.single.profile" -Dts.surefire.clustering.ha.additionalExcludes=affinity\|cdi\|dispatcher\|ejb\|ejb2\|group\|jms\|jpa

while another node can concurrently run all the other profiles and the other half of sub-packages:

./integration-tests.sh clean install -Dts.noSmoke -Dts.clustering -Dts.surefire.clustering.ha.additionalExcludes=jsf\|provider\|registry\|singleton\|sso\|web\|xsite

If the test packages get out of sync with the excludes this will result in a test running multiple times, rather than tests being omitted.

Running a single test

To run a single test, specifying -Dtest=foo is the standard way to do this. However, this overrides the includes/excludes section of the surefire maven plugin execution. So, in case of the clustering testsuite, the profile which this test belongs to, needs to be specified as well. For instance, to run a single test from the 'single' test execution, exclude the other test profiles:

./integration-tests.sh clean install -Dts.noSmoke -Dts.clustering -P="-ts.clustering.cluster.ha.profile,-ts.clustering.cluster.fullha.profile,-ts.clustering.cluster.ha-infinispan-server.profile,-ts.clustering.byteman.profile,-ts.clustering.single.profile" -Dtest=org.jboss.as.test.clustering.single.dispatcher.CommandDispatcherTestCase

1.4.4. Running Infinispan Server tests against custom distribution

To run the Infinispan Server-based tests against a custom distribution, a custom location can be specified with -Dinfinispan.server.home.override=/foo/bar and -Dinfinispan.server.profile.override=infinispan-13.0.xml to use a corresponding server profile. The distribution is then copied over to the build directories and patched with user credentials.

./integration-tests.sh clean install -Dts.noSmoke -Dts.clustering -P="-ts.clustering.cluster.ha.profile,-ts.clustering.cluster.fullha.profile,-ts.clustering.single.profile,-ts.clustering.byteman.profile,-ts.clustering.single.profile" -Dinfinispan.server.home.override=/Users/rhusar/Downloads/redhat-datagrid-8.3.0-server

Should it be required, the Infinispan Server driver version can be also overridden by -Dversion.org.infinispan.server.driver=13.0.0.Dev03.

Skipping clustering tests

To skip execution of all clustering tests use -Dts.noClustering.

1.4.5. IIOP -Dts.iiop

This section is open for contributions.

1.4.6. XTS -Dts.xts

This section is open for contributions.

1.4.7. Multinode -Dts.multinode

This section is open for contributions.

2. WildFly Integration Testsuite User Guide

See also: WildFly Testsuite Test Developer Guide

Target Audience: Those interested in running the testsuite or a subset thereof, with various configuration options.

2.1. Running the testsuite

The WildFly source code includes scripts to assist with building WildFly and running the testsuite. Using these scripts is not required. The scripts simply provide some convenience wrappers around directly calling mvn. Directly using mvn is fine and is commonly done.

Tests are run either when running the build.sh and build.bat shell scripts, or by using integration-tests.sh or integration-tests.bat. All of these scripts are found in the root directory of the source tree.

  • build.sh or build.bat, as a part of the WildFly build.

    • By default, only smoke tests are run. To run all tests, use ./build.sh install -DallTests.

  • integration-tests.sh or integration-tests.bat, a convenience script which runs all parent testsuite modules (which configure the server).

  • Direct Maven invocation, using mvn install.

The scripts are wrappers around the Maven-based build. Their arguments are passed to Maven (with few exceptions described below). This means you can use:

  • ./build.sh (defaults to install)

  • ./build.sh install

  • ./build.sh clean install

  • ./integration-tests.sh install

  • …​etc.

2.1.1. Supported Maven phases

Testsuite actions are bounds to various Maven phases up to verify. Running the build with earlier phases may fail in the submodules due to missed configuration steps. Therefore, the only Maven phases you may safely run, are:

  • clean

  • install

  • site

The test phase is not recommended to be used for scripted jobs as we are planning to switch to the failsafe plugin bound to the integration-test and verify phases. See WFLY-625 and WFLY-228.

2.1.2. Testsuite structure

testsuite/
  domain/
  galleon/
  integration/
    basic/
    clustering/
    elytron/
    elytron-oidc-client/
    expansion/
    hornetq-client/
    iiop/
    manualmode/
    manualmode-expansion/
    microprofile-tck/
    multinode/
    rbac/
    rts/
    smoke/
    vdx/
    web/
    ws/
    xts/
  layers/
  layers-expansion/
  mixed-domain/
  preview/
  scripts/

2.1.3. Test groups

To define groups of tests to be run, these properties are available:

  • -DallTests – Runs all subgroups.

  • -Dts.integration – Runs all integration tests, i.e. all child modules of testsuite/integration.

  • -Dts.basic – Basic integration tests.

  • -Dts.clustering – Clustering tests.

  • -Dts.iiop – IIOP tests.

  • -Dts.multinode – Tests with many nodes.

  • -Dts.manualmode – Tests with manual mode Arquillian containers.

  • -Dts.domain – Domain mode tests.

  • -Dts.bootable – Bootable jar tests.

  • -Dts.layers – Slimmed server / galleon layers tests.

  • -Dts.microprofile – MicroProfile tests.

  • -Dts.standalone.microprofile – Standalone MicroProfile tests.

  • -Dts.preview – Preview server tests.

  • -Dts.bootable.preview – Preview bootable jar tests.

2.2. Examples

  • ./integration-tests.sh install – Runs smoke tests.

  • ./integration-tests.sh clean install – Cleans the target directory, then runs smoke tests.

  • ./integration-tests.sh install -Dts.smoke – Same as above.

  • ./integration-tests.sh install -DallTests – Runs all testsuite tests.

  • ./integration-tests.sh install -Ddomain-tests – Runs domain tests (translated by the script to -Ddomain.module -Dts.noSmoke).

  • ./build.sh clean install -Dts.basic -Dts.domain – Builds WildFly and runs the smoke, basic integration and domain tests.

Pure maven - if you prefer not to use scripts, you may achieve the same result with:

  • mvn …​ -rf testsuite

The -rf …​ parameter stands for "resume from" and causes Maven to run the specified module and all successive.

It’s possible to run only a single module (provided the ancestor modules were already run to create the AS copies):

  • mvn …​ -pl testsuite/integration/clustering

The -pl …​ parameter stands for "project list" and causes Maven to run the specified module only.

2.2.1. Output to console

-DtestLogToFile

2.2.2. Other options

-DnoWebProfile – Run all tests with the full profile (standalone-full.xml). By default, most tests are run under web profile (standalone.xml).

-Dts.skipTests – Skip testsuite’s tests. Defaults to the value of -DskipTests, which defaults to false. To build AS, skip unit tests and run testsuite, use -DskipTests -Dts.skipTests=false.

2.2.3. Timeouts

Surefire execution timeout

Unfortunatelly, no math can be done in Maven, so instead of applying a timeout ratio, you need to specify timeout manually for Surefire.

-Dsurefire.forked.process.timeout=900
In-test timeout ratios

Ratio in percent - 100 = default, 200 = two times longer timeouts for given category.

Currently, we have five different ratios. Later, it could be replaced with just one generic, one for database and one for deployment operations.

-Dtimeout.ratio.fsio=100
-Dtimeout.ratio.netio=100
-Dtimeout.ratio.memio=100
-Dtimeout.ratio.proc=100
-Dtimeout.ratio.db=100

2.2.4. Running a single test (or specified tests)

Single test is run using -Dtest=…​ . Examples:

  • ./integration-tests.sh install -Dtest='*Clustered*' -Dts.clustering

  • ./integration-tests.sh clean install -Dtest=org/jboss/as/test/integration/ejb/async/*TestCase.java -Dts.basic

  • cd testsuite; mvn install -Dtest='*Clustered*' -Dts.basic

The same shortcuts listed in "Test groups" may be used to activate the module and group profile.

Note that -Dtest= overrides <includes> and <exludes> defined in pom.xml, so do not rely on them when using wildcards - all compiled test classes matching the wildcard will be run.

Which Surefire execution is used?

Due to Surefire’s design flaw, tests run multiple times if there are multiple surefire executions. To prevent this, if -Dtest=…​ is specified, non-default executions are disabled, and standalone-full is used for all tests. If you need it other way, you can overcome that need:

  • basic-integration-web.surefire with standalone.xml – Configure standalone.xml to be used as server config.

  • basic-integration-non-web.surefire – For tests included here, technically nothing changes.

  • basic-integration-2nd.surefire – Simply run the second test in another invocation of Maven.

2.2.5. Running against existing AS copy (not the one from

build/target/jboss-as-*)

-Djboss.dist=<path/to/jboss-as> will tell the testsuite to copy that AS into submodules to run the tests against.

For example, you might want to run the testsuite against AS located in /opt/wildfly-8 :

./integration-tests.sh -DallTests -Djboss.dist=/opt/wildfly

The difference between jboss.dist and jboss.home:

jboss.dist is the location of the tested binaries. It gets copied to testsuite submodules.

jboss.home is internally used and points to those copied AS instances (for multinode tests, may be even different for each AS started by Arquillian).

Running against a running JBoss AS instance

Arquillian’s WildFly container adapter allows specifying allowConnectingToRunningServer in arquillian.xml, which makes it check whether AS is listening at managementAddress:managementPort, and if so, it uses that server instead of launching a new one, and doesn’t shut it down at the end.

All arquillian.xml’s in the testsuite specify this parameter. Thus, if you have a server already running, it will be re-used.

2.2.6. Running with a debugger

Argument What will start with debugger Default port Port change arg.

-Ddebug

AS instances run by Arquillian

8787

-Das.debug.port=…​

-Djpda

alias for -Ddebug

-DdebugClient

Test JVMs (currently Surefire)

5050

-Ddebug.port.surefire=…​

-DdebugCLI

AS CLI

5051

-Ddebug.port.cli=…​
Examples
./integration-tests.sh install -DdebugClient -Ddebug.port.surefire=4040

...

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Listening for transport dt_socket at address: 4040
./integration-tests.sh install -DdebugClient -Ddebug.port.surefire

...

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Listening for transport dt_socket at address: 5050
./integration-tests.sh install -Ddebug
./integration-tests.sh install -Ddebug -Das.debug.port=5005
JBoss AS is started by Arquillian, when the first test which requires given instance is run. Unless you pass -DtestLogToFile=false, there’s (currently) no challenge text in the console; it will look like the first test is stuck. This is being solved in http://jira.codehaus.org/browse/SUREFIRE-781.
Depending on which test group(s) you run, multiple AS instances may be started. In that case, you need to attach the debugger multiple times.
Debugging clustering tests

With the implementation of WFLY-276 for the clustering test suite module, these tests can be run with the debugger as well. Here are some tips on how to effectively debug:

  1. Prepare your IDE with 4 remote debugger sessions templates: node1 with port 8787, node2 with port 8788, node3 with port 8789, and node4 with port 8790.

  2. Using mvn, run a single test (-Dtest=..) in the debug mode (-Ddebug / -Djpda)

  3. Connect the debugger to each node as they are starting since the JVM is configured to wait for the debugger connection to proceed.

  4. Keep connecting to starting nodes as some tests change the topology during the test.

Container start timeout in Arquillian is currently set to 60 seconds, you may need to increase the timeout depending on your swiftness with connecting the remote debugger.

2.2.7. Running tests with IPv6

-Dipv6 – Runs AS with -Djava.net.preferIPv4Stack=false -Djava.net.preferIPv6Addresses=true

and the following defaults, overridable by respective parameter:

Parameter IPv4 default IPv6 default

-Dnode0

127.0.0.1

::1

Single-node tests.

-Dnode1

127.0.0.1

::1

Two-node tests (e.g. cluster) use this for the 2nd node.

-Dmcast

230.0.0.4

ff01::1

ff01::1 is IPv6 Node-Local scope mcast addr.

-Dmcast.modcluster

224.0.1.105

ff01::3

mod_cluster multicast address.

Values are set in AS configuration XML, replaced in resources (like ejb-jar.xml) and used in tests.

2.2.8. Creating test reports

Test reports are created in the form known from EAP 5. To create them, simply run the testsuite, which will create Surefire XML files.

Creation of the reports is bound to the site Maven phase, so it must be run separately afterward. Use one of these:

./integration-tests.sh site
cd testsuite
mvn site
mvn -pl testsuite site

Note that it will take all test results under testsuite/integration/ - the pattern is **/*TestCase.xml, without need to specify -DallTests.

2.2.9. Creating coverage reports

Jira: https://redhat.atlassian.net/browse/WFLY-585

Coverage reports are created by JaCoCo.

During the integration tests, Arquillian is passed a JVM argument which makes it run with JaCoCo agent, which records the executions into ${basedir}/target/jacoco .

In the site phase, a HTML, XML and CSV reports are generated. That is done using jacoco:report Ant task in maven-ant-plugin since JaCoCo’s maven report goal doesn’t support getting classes outside target/classes.

Usage
./build.sh clean install -DskipTests
./integration-tests.sh clean install -DallTests -Dcoverage
./integration-tests.sh site -DallTests -Dcoverage ## Must run in separatedly.

Alternative:

mvn clean install -DskipTests
mvn -rf testsuite clean install -DallTests -Dcoverage
mvn -rf testsuite site -DallTests -Dcoverage

2.2.10. Cleaning the project

To have most stable build process, it should start with:

  • clean target directories

  • only central Maven repo configured

  • clean local repository or at least:

    • free of artefacts to be built

    • free of dependencies to be used (especially snapshots)

To clean the project, use the following commands:

mvn clean install -DskipTests -DallTests  ## ...to clean all testsuite modules.
mvn dependency:purge-local-repository build-helper:remove-project-artifact -Dbuildhelper.removeAll

In case the build happens in a shared environment (e.g. network disk), it’s recommended to use local repository:

cp /home/jenkins/.m2/settings.xml .
sed "s|<settings>|<settings><localRepository>/home/jenkins/jenkins-repos/$JOBNAME</localRepository>|" -i settings.xml

Or:

mvn clean install ... -Dmaven.repo.local=localrepo

See also https://redhat.atlassian.net/browse/WFLY-628.

2.3. Building server configurations

Whenever we fire a surefire execution, this causes Arquillian to start a server(s), execute tests and then shutdown the server(s). Starting a server in multiple executions kills the test logs of previous executions, and so it’s recommended to create a new named server configuration for each surefire execution.

When we build a server configuration, we need to carry out several steps:

  • Provision the server installation to be tested

  • Perhaps edit some configuration files (e.g. parametrizing the test cases for IP addresses, default databases and the like)

  • Perhaps copy in some modules

Provisioning server installations is achieved in two different ways, depending on requirements:

2.3.1. maven-resources-plugin based provisioning

In the root testsuite/pom.xml file there is a maven-resources-plugin execution ts.copy-wildfly that in the generate-test-resources phase copies most of the content of a WildFly build to the current module’s target/wildfly directory. The modules directory content is not copied, so, when using such an installation, the server’s module.path must be configured to include the modules dir of the original WildFly build.

The original WildFly build is found in either the build, ee-build, legacy/ee-feature-pack/build, legacy/ee-feature-pack/ee-build or preview/build module, or in an externally provided location, depending on the configuration of the maven build.

A module or profile that does not want a server installation to be provisioned this way can disable this provisioning by setting the phase of the ts.copy-wildfly execution to none.

2.3.2. Galleon based provisioning with the wildfly-maven-plugin

Here the wildfly-maven-plugin provision goal is used to provision a server using Galleon. The feature-packs and layers to use are controlled by the configuration of each plugin execution.

2.3.3. Customizing the management configuration of a provisioned server

The general recommendation for customizing the management configuration of a provisioned server is to implement a ServerSetupTask and configure the relevant test classes to use it via the ServerSetup annotation.

If using ServerSetupTask isn’t suitable (e.g. if the configuration settings need to be present for all tests against the installation), then JBoss CLI .cli scripts can be added to the test module’s source and executed before the test phase begins using the wildfly-maven-plugin’s execute goal.

There is some legacy code that does configuration modification via ant and xslt. This is being replaced and no new uses of this technique should be added.

2.3.4. Adding JBoss Modules modules to the provisioned server

The general recommendation for adding JBoss Modules modules that a test needs is to use the TestModule utility class from within the relevant test class.

If this approach isn’t suitable, the maven-resources-plugin can be used.

2.3.5. Other server installation customization

Addition of other content to the server installation can be done using the maven-resources-plugin. The testsuite/integration/pom.xml includes an execution that adds properties files to the standalone/configuration dir that provide users, credentials and groups that integration tests often expect.

2.4. Parameterization of testsuite runs

The testsuite/pom.xml contains a properties section which should hold all globally defined properties required to run the testsuite and their defaults (e.g. IP addresses, IP stack configs, database configs, etc). These properties will then be inherited by all of the testsuite modules.

This allows substitution of properties specified on the command line, such as:

./integration-tests.sh clean install -Dts.noSmoke -Dts.clustering -Dnode0=192.168.0.1 -Dnode1=192.168.0.2

2.5. Changing the container images used in integration tests

In integration tests, we use the following images:

  • otel/opentelemetry-collector

  • docker.elastic.co/elasticsearch/elasticsearch

  • apache/james

  • apache/kafka-native

  • quay.io/arkmq-org/activemq-artemis-broker

  • quay.io/keycloak/keycloak

It is possible to override the default images and versions by using:

  • System properties, e.g. -Dtestsuite.kafka-native.image=apache/kafka-native:3.9.0

  • Environment variables, named after the system property with . replaced by _ and converted to upper case, e.g. TESTSUITE_KAFKA-NATIVE_IMAGE=apache/kafka-native:3.9.0. Since most shells do not allow - in variable names, such variables need to be set using env, e.g. env 'TESTSUITE_KAFKA-NATIVE_IMAGE=apache/kafka-native:3.9.0' mvn …​

  • Properties file specified via testsuite.config.properties system property or TESTSUITE_CONFIG_PROPERTIES environment variable, e.g. -Dtestsuite.config.properties=/path/to/testsuite-config.properties or TESTSUITE_CONFIG_PROPERTIES=/path/to/testsuite-config.properties; the properties file format has to be the same as for system properties, e.g.:

testsuite.opentelemetry-collector.image=otel/opentelemetry-collector:0.138.0
testsuite.elasticsearch.image=docker.elastic.co/elasticsearch/elasticsearch:8.15.4
testsuite.mailserver.image=apache/james:demo-3.8.2
testsuite.kafka-native.image=apache/kafka-native:3.8.0
testsuite.activemq-artemis-broker.image=quay.io/arkmq-org/activemq-artemis-broker:artemis.2.42.0
testsuite.keycloak.image=quay.io/keycloak/keycloak:24.0.5

3. WildFly Testsuite Harness Developer Guide

The testsuite implementation was tracked by WFLY-576, which still contains a lot of useful information.

3.1. Adding a new maven plugin

The plugin version needs to be specified in jboss-parent at the <properties> section of the jboss-parent pom.xml file.

3.2. How the AS instance is built

See How the JBoss AS instance is built and configured for testsuite modules.

3.3. Properties and their propagation

Propagated to tests through arquillian.xml:
<property name="javaVmArguments">${server.jvm.args}</property>
TBD: https://redhat.atlassian.net/browse/ARQ-647

3.3.1. JBoss AS instance dir

integration/pom.xml

(currently nothing)

*-arquillian.xml

<container qualifier="jboss" default="true">
    <configuration>
        <property name="jbossHome">${basedir}/target/jbossas</property>

3.3.2. Server JVM arguments

<surefire.memory.args>-Xmx512m</surefire.memory.args>
<surefire.jpda.args></surefire.jpda.args>
<surefire.system.args>${surefire.memory.args} ${surefire.jpda.args}</surefire.system.args>

3.3.3. IP settings

  • ${ip.server.stack} used in <systemPropertyVariables>/<server.jvm.args> which is used in *-arquillian.xml.

3.3.4. Testsuite directories

  • ${jbossas.ts.integ.dir}

  • ${jbossas.ts.dir}

  • ${jbossas.project.dir}

3.3.5. Clustering properties

  • node0

  • node1

3.4. Debug parameters propagation

<surefire.jpda.args></surefire.jpda.args>       - default

<surefire.jpda.args>-Xrunjdwp:transport=dt_socket,address=${as.debug.port},server=y,suspend=y</surefire.jpda.args> - activated by -Ddebug or -Djpda


testsuite/pom.xml:        <surefire.system.args>... ${surefire.jpda.args} ...</surefire.system.args>
testsuite/pom.xml:                        <jboss.options>${surefire.system.args}</jboss.options>

testsuite/integration/pom.xml:     <server.jvm.args>${surefire.system.args} ${jvm.args.ip.server} ${jvm.args.security} ${jvm.args.timeouts} -Dnode0=${node0} -Dnode1=


integration/pom.xml:
<server.jvm.args>${surefire.system.args} ${jvm.args.ip.server} ${jvm.args.security} ${jvm.args.timeouts} -Dnode0=${node0} -Dnode1=${node1} -DudpGroup=${udpGroup} ${jvm.args.dirs}</server.jvm.args>

arquillian.xml:
<property name="javaVmArguments">${server.jvm.args} -Djboss.inst=${basedir}/target/jbossas</property>

3.5. How the WildFly is built and configured for testsuite modules

WildFly instance is copied from ${jboss.dist} to testsuite/target/jbossas. This defaults to WildFly which is built by the project (build/target/wildfly-*).

testsuite/pom.xml

From ${jboss.home} to ${basedir}/target/jbossas phase generate-test-resources, goal resource-plugin:copy-resources

testsuite/integration/pom.xml

phase process-test-resources: antrun-plugin:

<ant antfile="${basedir}/src/test/scripts/basic-integration-build.xml">
    <target name="build-basic-integration"/>
    <target name="build-basic-integration-jts"/>
</ant>

Which invokes:

<target name="build-basic-integration" description="Builds server configuration for basic-integration tests">
    <build-server-config name="jbossas"/>
    <!-- .. -->
</target>

Which invokes:

<!-- Copy the base distribution. -->
<!-- We exclude modules as they are read-only and we locate them via sys props. -->
<copy todir="@{output.dir}/@{name}">
    <fileset dir="@{jboss.dist}">
        <exclude name="**/modules/**"/>
    </fileset>
</copy>

<!-- overwrite with configs from test-configs and apply property filtering -->
<copy todir="@{output.dir}/@{name}" overwrite="true" failonerror="false">
    <fileset dir="@{test.configs.dir}/@{name}"/>
    <filterset begintoken="${" endtoken="}">
        <filter token="node0" value="${node0}"/>
        <filter token="node1" value="${node1}"/>
        <filter token="udpGroup" value="${udpGroup}"/>
        <filter-elements/>
    </filterset>
</copy>

3.5.1. Arquillian config file location

-Darquillian.xml=some-file-or-classpath-resource.xml

4. WildFly Testsuite Test Developer Guide

See also: WildFly Integration Testsuite User Guide

4.1. Pre-requisites

Please be sure to read Pre-requisites - test quality standards and follow those guidelines.

4.2. ManagementClient and ModelNode usage example

final ModelNode operation = new ModelNode();
operation.get(ModelDescriptionConstants.OP).set(ModelDescriptionConstants.READ_RESOURCE_OPERATION);
operation.get(ModelDescriptionConstants.OP_ADDR).set(address);
operation.get(ModelDescriptionConstants.RECURSIVE).set(true);

final ModelNode result = managementClient.getControllerClient().execute(operation);
Assert.assertEquals(ModelDescriptionConstants.SUCCESS, result.get(ModelDescriptionConstants.OUTCOME).asString());

ManagementClient can be obtained as described below.

4.3. Arquillian features available in tests

@ServerSetup

TBD

@ContainerResource private ManagementClient managementClient;
final ModelNode result = managementClient.getControllerClient().execute(operation);

TBD

@ArquillianResource private ManagementClient managementClient;
ModelControllerClient client = managementClient.getControllerClient();
@ArquillianResource ContainerController cc;

@Test
public void test() {
    cc.setup("test", ...properties..)
    cc.start("test")
}
<arquillian>
    <container qualifier="test" mode="manual" />
</arquillian>
// Targeted containers HTTP context.
@ArquillianResource URL url;
// Targeted containers HTTP context where servlet is located.
@ArquillianResource(SomeServlet.class) URL url;
// Targeted containers initial context.
@ArquillianResource InitialContext|Context context;
// The manual deployer.
@ArquillianResource Deployer deployer;

Note to @ServerSetup annotation: It works as expected only on non-manual containers. In case of manual mode containers it calls setup() method after each server start up which is right (or actually before deployment), but the tearDown() method is called only at AfterClass event, i.e. usually after your manual shutdown of the server. Which limits you on the ability to revert some configuration changes on the server and so on. I cloned the annotation and changed it to fit the manual mode, but it is still in my github branch :)

4.4. Properties available in tests

4.4.1. Directories

  • jbosssa.project.dir - Project’s root dir (where ./build.sh is).

  • jbossas.ts.dir - Testsuite dir.

  • jbossas.ts.integ.dir - Testsuite’s integration module dir.

  • jboss.dist - Path to AS distribution, either built (build/target/jboss-as-…​) or user-provided via -Djboss.dist

  • jboss.inst - (Arquillian in-container only) Path to the AS instance in which the test is running (until ARQ-650 is possibly done)

  • jboss.home - Deprecated as it’s name is unclear and confusing. Use jboss.dist or jboss.inst.

4.4.2. Networking

  • node0

  • node1

  • 230.0.0.4

In case some of the following causes timeouts, you may prolong the timeouts by setting value >= 100:

100 = leave as is,
150 = 50 % longer, etc.

  • timeout.ratio.gen - General ratio - can be used to adjust all timeouts. When this and specific are defined, both apply.

  • timeout.ratio.fs- Filesystem IO

  • timeout.ratio.net - Network IO

  • timeout.ratio.mem - Memory IO

  • timeout.ratio.cpu - Processor

  • timeout.ratio.db - Database

Time ratios will soon be provided by org.jboss.as.test.shared.time.TimeRatio.for*() methods.

4.5. Negative tests

Testing of invalid deployments is supposed to be leveraging the @ShouldThrowException annotation. Unfortunately, this feature cannot be used at the moment due to outstanding issue, see WFLY-673. However, the exception might be obtained using the manual deployer: 
@Deployment(name = "X", managed = false)
// ...

@Test
public void shouldFail(@ArquillianResource Deployer deployer) throws Exception {
    try {
        deployer.deploy("X");
    } catch (Exception e) {
        // do something
    }
}

4.6. Clustering tests (WFLY-616)

You need to deploy the same thing twice, so two deployment methods that just return the same thing. And then you have tests that run against each.

@Deployment(name = "deplA", testable = false)
    @TargetsContainer("serverB")
    public static Archive<?> deployment() { ... }

    @Deployment(name = "deplB", testable = false)
    @TargetsContainer("serverA")
    public static Archive<?> deployment() { ... }

    @Test
    @OperateOnDeployment("deplA")
    public void testA() { ... }

    @Test
    @OperateOnDeployment("deplA")
    public void testA() {...}

4.7. How to get the tests to main branch

  • First of all, be sure to read the "Before you add a test" section.

  • Fetch the newest main: git fetch upstream # Provided you have the wildfly/wildfly GitHub repo as a remote called 'upstream'.

  • Rebase your branch: git checkout WFLY-1234-your-branch; git rebase upstream/main

  • Run whole testsuite (integration-tests -DallTests). You may use https://ci.wildfly.org/viewType.html?buildTypeId=WF_Nightly.

    • If any tests fail and they do not fail in main, fix it and go back to the "Fetch" step.

  • Push to a new branch in your GitHub repo: git push origin WFLY-1234-new-XY-tests

  • Create a pull-request on GitHub. Go to your branch and click on "Pull Request".

    • If you have a jira, start the title with it, like - WFLY-1234 New tests for XYZ.

    • If you don’t, write some apposite title. In the description, describe in detail what was done and why should it be merged. Keep in mind that the diff will be visible under your description.

  • Keep the branch rebased daily until it’s merged (see the Fetch step). If you don’t, you’re dramatically decreasing chance to get it merged.

  • You might have someone with merge privileges to cooperate with you, so they know what you’re doing, and expect your pull request.

  • When your pull request is reviewed and merged, you’ll be notified by mail from GitHub.

  • You may also check if it was merged by the following: git fetch upstream; git cherry <branch> ## Or git branch --contains\{\{<branch> - see}} here

  • Your commits will appear in main. They will have the same hash as in your branch.

    • You are now safe to delete both your local and remote branches: git branch -D WFLY-1234-your-branch; git push origin :WFLY-1234-your-branch

5. How to Add a Test Case

Thank you for finding time to contribute to WildFly quality. Covering corner cases found by community users with tests is very important to increase stability. If you’re providing a test case to support your bug report, it’s very likely that your bug will be fixed much sooner.

5.1. Create a test case

Don’t be discouraged, it’s quite easy - a simple use case may even consist of a single short .java file.

Check WildFly test suite test cases for examples.

For more information, see WildFly Testsuite Test Developer Guide. Check the requirements for a test to be included in the testsuite.

Ask for help at WildFly Google user group or Zulip chat.

5.2. Push your test case to GitHub and create a pull request.

For information on how to create a GitHub account and push your code therein, see Hacking on WildFly.

6. Test Quality Standards

All tests must follow these quality standards.

6.1. Verify the Test Belongs in WildFly

Ensure the test validates current WildFly functionality. Do not test the internal functionality of included libraries, as those should be tested in the library’s upstream test suite. However, do test the integration of those libraries with WildFly.

6.2. Add Only Correct and Understandable Tests

Tests must be clear and verifiably correct. If a test is overly complex or uses questionable logic, refactor it to use clear logic or consult with the original author to clarify intent.

6.3. Avoid Duplicate Tests

Before adding a test, verify that the functionality is not already covered elsewhere. Use tools like git grep to search for existing test coverage.

6.4. Use Descriptive Test Names

Test class and method names should clearly describe what is being tested. Jira IDs alone are not descriptive and require external lookup to understand the test purpose. Jira references may be included in test comments or commit messages, e.g. in Javadoc @see.

6.5. Document Tests with Javadoc

All test classes and non-trivial test methods must include Javadoc explaining what functionality is being tested and why.

6.6. Expand Existing Tests When Appropriate

When adding test coverage for functionality similar to an existing test, consider adding test methods to the existing test class rather than creating a new class. Each new test class adds overhead to execution time. Only create a new class when the functionality is sufficiently different to warrant separation.

6.7. Organize Tests by Subsystem

Place integration tests in subpackages under the relevant subsystem (e.g., org.jboss.as.test.integration.ejb.async). When a test involves multiple subsystems, place it under the package for the specification that defines the primary behavior being tested.

6.8. Document Non-Obvious Specification Requirements

When testing behavior mandated by specifications but not immediately obvious, include comments explaining the requirement (e.g., "Verifies Jakarta Persistence X.Y.Z - Description of requirement").

6.9. Collocate Test Resources with Test Code

Place integration test resources (deployment descriptors, configuration files, etc.) in the same source directory as the test class. This keeps all test artifacts together and makes tests easier to understand.

6.10. Use Configurable Values for URLs and Ports

Do not hard-code URLs, hostnames, ports, or IP addresses. Use configurable values provided by Arquillian or system properties to ensure tests work with different configurations and IPv6 addresses.

If a necessary configuration property is missing, file a Jira issue with component "Testsuite".

6.11. Follow Best Practices for Commits

  • Keep changes focused on the specific issue or feature being addressed.

  • Do not include unrelated changes such as reformatting or typo fixes in feature commits. Submit these separately.

  • Prefer smaller, focused pull requests over large, difficult-to-review changes.

  • Maintain consistency across commits (e.g., when renaming, update all references).

  • Write clear commit messages that will be meaningful in the project history.

  • Include the Jira ID in commit messages when working on a tracked issue.

6.12. Avoid Blind Timeouts

Do not use Thread.sleep() without checking for the actual condition being awaited. Use active waiting with timeouts or timeout mechanisms provided by the API being tested.

Make timeouts configurable with reasonable defaults. For groups of similar tests, use a shared configurable timeout value.

6.13. Provide Descriptive Assertion Messages

Always include descriptive messages in assert*() and fail() calls. Clear messages make test failures easier to diagnose.

// Good
assertTrue("File config.xml should exist", configFile.exists());

// Bad - provides no context on failure
assertTrue(configFile.exists());

6.14. Include Configuration Context in Exceptions

When tests fail due to possible misconfiguration, include the relevant configuration property names and values in exception messages.

File jdbcJar = new File(System.getProperty("jbossas.ts.dir", "."), "integration/src/test/resources/mysql-connector-java-5.1.15.jar");
if (!jdbcJar.exists()) {
    throw new IllegalStateException("Cannot find " + jdbcJar + " using ${jbossas.ts.dir} == " + System.getProperty("jbossas.ts.dir"));
}

6.15. Clean Up Resources

  • Close all sockets, connections, streams, and file descriptors in finally blocks or use try-with-resources.

  • Avoid storing large objects in static fields. If necessary, clear them in finally blocks.

  • Do not modify server configuration unless you restore it in a finally block or @After* method.

6.16. Keep Tests Configurable

Extract configuration values such as timeouts, paths, URLs, and counts into configurable properties defined at the beginning of the test class. This improves test maintainability and flexibility.

6.17. Follow Coding Standards

Coding standards for production code also apply to test code. Please refer to Hacking on WildFly for the coding standards. Just to mention the most frequent violations, always include the copyright header:

/*
 * Copyright The WildFly Authors
 * SPDX-License-Identifier: Apache-2.0
 */

and follow our import order for the project:

import static *

import java.*
import javax.*

import others.*

7. Shared Test Classes and Resources

7.1. Among Testsuite Modules

Use the testsuite/shared module.

Classes and resources in this module are available in all testsuite modules - i.e. in testsuite/* Only use it if necessary - don’t put things "for future use" in there. Don’t split packages across modules. Make sure the java package is unique in the WildFly project.

Document your util classes* (javadoc) so they can be easily found and reused! A generated list will be put here.

7.2. Between Components and Testsuite Modules

To share component’s test classes with some module in testsuite, you don’t need to split to submodules. You can create a jar with classifier using this:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <executions>
        <execution>
            <goals>
                <goal>test-jar</goal>
            </goals>
        </execution>
    </executions>
</plugin>

This creates a jar with classifier "tests", so you can add it as dependency to a testsuite module:

    <dependency>
        <groupId>org.wildfly</groupId>
        <artifactId>wildfly-clustering-common</artifactId>
        <classifier>tests</classifier>
        <version>${project.version}</version>
        <scope>test</scope>
    </dependency>