Prospero documentation

1. Overview

Prospero is a tool designed to install and manage updates of Wildfly servers.

The tool supports:

Provisioning Wildfly servers
Updating Wildfly servers
Reverting updates to Wildfly servers
Tracking updates history
Management of update sources

Prospero includes a CLI client and a programmatic API.

Releases of the command line tool will be available at https://github.com/wildfly-extras/prospero/releases/.

To use the tool, download and unzip the release zip, then add the bin directory to your system path. Use prospero.sh or prospero.bat to launch the tool.

2. Concepts

The Wildfly servers are assembled from a large number of dependencies from various projects. If any of those components is changed, in the traditional update process, the server has to be rebuilt with updated components and distributed as a new version.

Prospero aims to simplify this process by allowing users to apply updated components to already provisioned servers.

To achieve that, Prospero utilizes two projects - Galleon and Wildfly Channels.

2.1. Feature Packs

Galleon is a provisioning tool used to assemble Wildfly distributions. In order for a piece of software to be provisioned by Galleon it needs to be packaged as a feature-pack.

Feature packs are ZIP archives usually available in Maven repositories. They contain information about installed software’s filesystem content, configuration and additional tasks needed to assemble and configure it.

In case of Wildfly server, the feature pack also contains a list of Maven coordinates for all the components included in the server distribution. During provisioning, those coordinates are used to download the components and place them in resulting server.

For more information on Galleon and the Feature Packs see the Galleon documentation

2.2. Channels

Wildfly Channels are a way to provide a curated list of compatible, up-to-date components. A channel consists of a YAML definition file and one or more underlying Maven repositories.

There are two types of Wildfly Channels supported by Prospero - open channels and manifest channels. Open channels allow access to the latest artifacts in the underlying repository, while manifest channels define a list of allowed list of components.

For more information on Wildfly Channels see the spec documentation.

2.2.1. Manifest channels

One way to specify channel content is by using manifests. Manifests are YAML file listing streams available in the channel. For example a part of wildfly-27.0.0 manifest looks like:

schemaVersion: "1.0.0"
name: Manifest for WildFly 27
description: |-
  This manifest provides updates for WildFly 27 Feature Pack.
streams:
  - groupId: "org.wildfly"
    artifactId: "wildfly-galleon-pack"
    version: "27.0.0.Final"

  - groupId: "org.wildfly"
    artifactId: "wildfly-ee-galleon-pack"
    version: "27.0.0.Final"

  - groupId: "org.wildfly.core"
    artifactId: "wildfly-core-galleon-pack"
    version: "19.0.0.Beta13"

  - groupId: "org.wildfly.core"
    artifactId: "wildfly-version"
    version: "19.0.0.Final"

  - groupId: "org.wildfly.security"
    artifactId: "wildfly-elytron-jaspi"
    version: "1.20.2.Final"

The channel using this manifest would be defined as:

schemaVersion: "2.0.0"
name: Channel for WildFly 27
manifest:
  maven:
    groupId: org.wildfly.channels
    artifactId: wildfly-27.0
repositories:
  - id: central
    url: https://repo1.maven.org/maven2/

When resolving artifacts using such channel, the versions of artifacts will be dictated by the manifest, even if newer version is available in the channel repositories.

Note	The manifest can be distributed as either a static file (referenced in a channel by a URL), or as an artifact in a Maven repository.

2.2.2. Open channels

Alternatively the channel can be defined to use the latest versions of artifacts available in its repositories.

schemaVersion: "2.0.0"
name: Channel for WildFly 27
resolve-if-no-stream: latest
repositories:
  - id: central
    url: https://repo1.maven.org/maven2/

3. General provisioning process

3.1. Installation process

Prospero delegates provisioning server to Galleon. When Galleon attempts to resolve components needed by the server, instead of using versions provided by the feature packs, Prospero intercepts that request and uses Wildfly Channels to find the correct version of the component and resolve it from the channels repositories.

Every time a server is provisioned, it will receive the latest available components.

3.2. Update process

When new versions of components are made available, Prospero can apply those to existing servers. In order to find updates following steps are executed:

Download the latest version of channel manifests from Maven repositories
Compare component versions in the server to the latest available versions in channels.

If updates are available, they will be applied to the server in a following process:

Provision a new, updated server in a temporary directory.
Compare the changes between the new and old server.
Apply changes in the new server to the base server.

3.2.1. Resolving update conflicts

Generally user modified files in the server will never be changed during the update.

Sometimes server files might be modified both in the base server and in an incoming update. An example of this might be a configuration file changed by the user to support their setup and changed in the update to modify a default configuration value.

In such case, the user changes are persisted and not modified. The new, updated file configuration is created next to the old file with a .glnew suffix, and the user is notified that there was a conflict. The user can resolve the conflict manually after the update.

The exception to this rule are files considered to be system paths. They should not be modified by the user and will always be kept in sync with the changes provided by the updates. Any user changes will be replaced during the update and stored with a .glold suffix. An example of such files is a content of modules/system folder.

User modified

Update modified

System path

Resolution

n/a

Original file is used.

n/a

Updated version is used. Original file is discarded

Updated version is used. Original file is persisted with .glold suffix

Original file is used. Updated version is persisted with .glnew suffix

3.2.2. Update operation limitations

During "apply changes" phase of updates, any processes using server’s resources need to be stopped to avoid resource locking issues. The user should manually shut down the server before performing update.

To reduce server downtime, the update operation can be performed in two steps - prepare and apply. In the first step, a candidate update server is provisioned in a new directory. The second step, compares the changes between the candidate and the base server and applies them to the base server.

If the update is split into separate steps, the prepare step can be performed while the server is running. See Working with update candidates for details.

4. Prospero server metadata

In order for Prospero to be able to update a server, the server installation has to contain some additional metadata files. Those files contain information about server’s current state and the subscribed channels. The metadata is generated when the server is installed and can be modified using Prospero commands.

The metadata is split between <SERVER_HOME>/.galleon and <SERVER_HOME>/.installation folders.

File

Description

<SERVER_HOME>/.installation/installer-channels.yaml

List of channels the server is subscribed to receive updates from

<SERVER_HOME>/.installation/manifest.yaml

Record of current versions of components installed in the server

<SERVER_HOME>/.installation/manifest_version.yaml

Record of channel manifests used to provision current server.

<SERVER_HOME>/.galleon/provisioning.xml <SERVER_HOME>/.galleon/provisioned.xml

Gallon files recording provisioned feature-packs and their customizations.

4.1. Prospero update history

The files in the .installation/ and .galleon/ folders should only be modified via the prospero tool, not manually. All metadata changes performed by prospero tool are internally versioned and can be later reviewed or rolled back by using the prospero history and prospero revert commands.

5. Usage

5.1. Installation

To provision a server, Prospero requires following information:

manifest definition
location of channel’s Maven repositories
server provisioning configuration

5.1.1. Installing predefined server profile

Prospero provides a set of predefined profiles providing a simple way to install standard servers.

$ ./prospero.sh install \
      --dir wfly-27 \
      --profile wildfly \
      --manifest /path/to/wildfly-manifest.yaml

The profile defines a feature-pack name and required repositories. It can also provide manifest information, if that manifest is published in a Maven repository.

5.1.2. Installing a feature pack

If a desired server configuration is not provided by one of the profiles, Prospero can provision a feature pack based on its Maven coordinates (groupId:artifactId). The resulting server will have a default configuration provided by that feature pack.

$ ./prospero.sh install \
      --dir wfly-27 \ # (1)
      --fpl org.wildfly:wildfly-ee-galleon-pack \ #(2)
      --manifest /path/to/wildfly-manifest.yaml \ #(3)
      --repositories https://repo1.maven.org/maven2/,https://repository.jboss.org/nexus/content/groups/public/ #(4)

directory to place provisioned server in
Maven <groupId>:<artifactId> coordinates of the feature pack to be installed
Path to the manifest file
List of Maven repositories containing components listed in the channel file.

Note	An example Wildfly manifest file is available in examples directory.

5.1.3. Installing a customized feature pack

Galleon feature packs consist of layers and packages that can be excluded or included to generate customized server.

In order to create a customized installation, the required provisioning configuration need to be described in provisioning.xml file. For example, following file adds configuration examples to the generated server.

<?xml version="1.0" ?>
<installation xmlns="urn:jboss:galleon:provisioning:3.0">
    <feature-pack location="org.wildfly:wildfly-galleon-pack::zip">
      <packages>
        <include name="docs.examples.configs"/>
      </packages>
    </feature-pack>
</installation>

The provisioning definition can then be used to create a server:

$ ./prospero.sh install \
      --dir wfly-27 \
      --definition provisioning.xml \
      --manifest /path/to/wildfly-manifest.yaml
      --repositories https://repo1.maven.org/maven2/,https://repository.jboss.org/nexus/content/groups/public/

5.1.4. Using offline or mirrored repositories

By default, any artifacts required to provision the server, will be resolved using public repositories. If the system on which the installation is performed doesn’t have access to the required repositories, alternative repositories need to provided using the --repositories argument.

The specified repositories can either be mirrors of public repositories, or local filesystem copies of the repositories. One way to generate such repository is installing the server on an online system using --local-cache argument:

$ ./prospero.sh install \ #(1)
      --dir wfly-27 \
      --profile wildfly \
      --manifest /path/to/wildfly-manifest.yaml \
      --local-cache /path/to/local_repo

$ ./prospero.sh install \ #(2)
      --dir offline-wfly-27 \
      --profile wildfly \
      --manifest /path/to/wildfly-manifest.yaml \
      --repositories /path/to/local_repo \
      --offline

Generates a repository at /path/to/local_repo
Installs the server using content of /path/to/local_repo instead of default repositories

Note	When using a <groupId>:<artifactId> to resolve manifest from a repository, the manifest artifact has to have accompanying Maven metadata (maven-metadata.xml).

5.1.5. Selecting stability level of provisioned server

Some feature packs may contain experimental or preview features not intended to be made available by default. To separate those features from stable ones, feature packs may segregate the features into stability-levels. At the provisioning time, the user is able to change the default stability level of the feature pack by using the --stability flag.

$ ./prospero.sh install \ #(1)
      --dir wfly-31 \
      --profile wildfly \
      --manifest /path/to/wildfly-manifest.yaml \
      --stability preview

For more fine-grained control, the level of installed packages and configuration can be set separately using --config-stability and --package-stability flags.

5.2. Update

Once the server is installed, it can receive updates published in the channels it is subscribed to.

Following command can be used to update the server using the subscribed channels:

$ ./prospero.sh update perform --dir wfly-27

If any updates are available, Prospero will print the update summary and prompt for confirmation to continue. If the updates are accepted, the updated server will be prepared in a way described in Update process.

Note	The server has to be stopped during `update perform` operation

5.2.1. Checking updates

prospero update list command can be used to check if any updates are available:

$ ./prospero.sh update list --dir wfly-27

5.2.2. Working with update candidates

When using prospero update perform command, the server has to be stopped during the whole update process. In order to reduce server downtime, it is possible to generate an update candidate while the server is running and apply it to a stopped server in a separate step.

$ ./wfly-27/bin/standalone.sh > /dev/null & # (1)

$ ./prospero.sh update prepare \ # (2)
       --dir wfly-27 \
       --candidate-dir candidate-dir \
       --yes

$ ./wfly-27/bin/jboss-cli.sh -c "shutdown" # (3)

$ ./prospero.sh update apply \ # (4)
       --dir wfly-27 \
       --candidate-dir candidate-dir

start the server in background
generate the candidate in candidate-dir folder
stop the server
merge the updates found in candidate-dir into wfly-27 server

Update candidate is a temporary server provisioned by prospero. It is generated using the latest updates available in the channels, but does not contain any user modifications of the original server.

Note	The candidate can only be applied to a server it was based on and only if the server has not been updated since the candidate was generated.

5.2.3. Using alternative repository

If the repositories defined in subscribed channels are not available to the server, alternative repositories can be provided using --repositories parameter. Those repositories can point to a local copy of the repository or alternate online location.

$ ./prospero.sh update perform \
       --dir wfly-27 \
       --repositories https://proxy.corp.org/maven-central # (1)

The updates will be resolved from https://proxy.corp.org/maven-central instead of the default repository.

The --repositories can also be used to resolve artifact from a local copy of the repository:

$ ./prospero.sh update perform \
       --dir wfly-27 \
       --repositories /path/to/repository

Note	At minimum, the alternative repositories have to provide all the channel manifests that the server is subscribed to and updated artifacts.

5.2.4. Subscribing existing server

Prospero relies on additional metadata files being present in the server installation to provide updates. Using update subscribe command it is possible to generate required files in a server.

$ ./prospero.sh update subscribe \
       --dir wfly-27 \
       --product=wildfly \ # (1)
       --version=29.0.1.Final # (2)

name of the installed server, or name of the feature pack used to install the server
version of installed server or feature pack

The product and version information is needed to determine currently installed software and subscribe server to correct channels.

5.3. Channel management

The server subscribes to the channels to receive updates from them. The list of subscriptions can be managed using prospero channel subcommands.

5.3.1. Listing channels

Command prospero channel list prints a summary of subscribed channels.

$ ./prospero.sh channel list --dir wfly-27
Server /tmp/wfly-27 is subscribed to following channels:

-------
# channel-0
  manifest: org.wildfly.channel:wfly-27
  repositories:
    id: central
    url: https://repo1.maven.org/maven2/
-------

5.3.2. Subscribing to a channel

Command prospero channel add subscribes the server to receive updates from additional channels. Each channel has to have a unique name, a manifest and a list of repositories. The channel’s artifacts will be used to resolve artifacts during next update.

$ ./prospero.sh channel add \
   --channel-name dev-channel \
   --manifest org.prospero.channel:wfly-27-dev \ # (1)
   --repositories file:/tmp/dev-repository \ # (2)
   --dir wfly-27
Subscribing /tmp/wfly-27 to channel dev-channel

Channel 'dev-channel' added.

The Maven coordinates (groupId:artifactId) of the new channel’s manifest
The Maven repository where the manifest and artifacts can be found in

5.3.3. Unsubscribing from a channel

Command channel remove unsubscribes the server from a selected channel, stopping it from receiving updates from that channel. The artifacts previously installed from that channel will be replaced with artifacts available in remaining channel during next update operation.

$ ./prospero.sh channel remove \
   --channel-name dev-channel \
   --dir installation-dir
Unsubscribing /tmp/wfly-27 from channel dev-channel

Channel 'dev-channel' removed.

5.4. Update history

Each time the server is updated, the previous state of the server is recorded. That state includes the channels used to perform the update and the versions of artifacts used to build the server.

This information is used to restore the server back to past state.

5.4.1. Viewing history

Command history displays a list of all server updates:

$ ./prospero.sh history --dir wfly-27
[02805caa] 2023-05-25T15:07:19Z - update [org.wildfly.channel:wfly-27::1.0.1]
[c5215591] 2023-05-25T15:05:04Z - install [org.wildfly.channel:wfly-27::1.0.0]

The server state record contains following information: a hash identifier, date of the change, type of the change (installation, update, configuration change) and a summary of channel manifests used in that update.

The details of each record can be examined using --revision parameter:

$ ./prospero.sh history --dir wfly-27 --revision 02805caa
Updates:
  [Updated artifact] io.undertow:undertow-servlet:		2.2.18.Final ==> 2.2.19.Final
  [Updated artifact] io.undertow:undertow-websockets-jsr:		2.2.18.Final ==> 2.2.19.Final
  [Updated artifact] io.undertow:undertow-core:		2.2.18.Final ==> 2.2.19.Final

5.4.2. Reverting updates

The server can be reverted to any of the recorded states. The revert commands operate in similar way to update. For example, a following command performs a full revert of a server to an earlier installation state:

$ ./prospero.sh revert perform --dir wfly-27 --revision c5215591
Reverting server /tmp/wfly-27 to state c5215591

Feature-packs resolved.
Packages installed.
Downloaded artifacts.
JBoss modules installed.
Configurations generated.

Reverted server prepared, comparing changes
Changes found:
  io.undertow:undertow-servlet                          2.2.19.Final         ==>  2.2.18.Final
  io.undertow:undertow-websockets-jsr                   2.2.19.Final         ==>  2.2.18.Final
  io.undertow:undertow-core                             2.2.19.Final         ==>  2.2.18.Final
Continue with revert [y/N]: y
Applying changes

Server reverted to state 348b9d9c.
Operation completed in 45.83 seconds.

The revert process works similar to the update process, but instead of resolving component versions from subscribed channels, the component versions recorded in the requested state will be used.

Note	The required components will be downloaded from the channel repositories the server was subscribed to at the requested state. If the channel repositories are no longer available, or do not provide the required component versions, `--repositories` can be used to overwrite them.

Note	The server has to be switched off for the duration of the `revert perform` operation.

5.4.3. Reverting updates using candidate

Similarly to update operation, the revert operation can be split into preparing the candidate and applying the changes. In such case, the server downtime is only required during the latter phase.

$ ./prospero.sh revert prepare \ #(1)
    --candidate-dir candidate \
    --revision c5215591 \
    --dir wfly-27
Building revert candidate for /tmp/wfly-27

Feature-packs resolved.
Packages installed.
Downloaded artifacts.
JBoss modules installed.
Configurations generated.

Update candidate generated in /Users/spyrkob/workspaces/set/prospero/prospero/candidate
Operation completed in 22.22 seconds.

$  ./prospero.sh revert apply \ #(2)
    --dir wfly-27 \
    --candidate-dir candidate
Reverting server /tmp/wfly-27 to state ab39b0c6

Changes found:
  io.undertow:undertow-servlet                          2.2.19.Final         ==>  2.2.18.Final
  io.undertow:undertow-websockets-jsr                   2.2.19.Final         ==>  2.2.18.Final
  io.undertow:undertow-core                             2.2.19.Final         ==>  2.2.18.Final
Continue with revert [y/N]: y
Applying changes

Server reverted to state ab39b0c6.
Operation completed in 9.40 seconds.

Prepare a revert candidate in candidate folder
Apply prepared candidate to a server in wfly-27

5.5. Replicating installation

Sometimes it might be needed to provision a copy of the server, with the same components as the original, in a different environment. To achieve that, Prospero requires following information:

a list of component versions used to provision the server,
a list of channels used during provisioning,
a list of Galleon feature packs used.

This information can be obtained from any provisioned server using clone export command:

$ ./prospero.sh clone export --dir wfly-27 --path snapshot.zip
Exporting /tmp/wfly-27 installation details to /tmp/snapshot.zip
Export complete

The generated archive contains following files:

Archive:  snapshot.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
    68301  05-26-2023 15:59   manifest.yaml
      476  05-26-2023 15:59   installer-channels.yaml
      606  05-26-2023 15:59   provisioning.xml
---------                     -------
    69383                     3 files

The archive can be used to rebuild the server:

$ ./prospero clone recreate --dir cloned --path export.zip
Recreating a server in /tmp/cloned based on /tmp/export.zip

Provisioning configuration:
 * org.wildfly:wildfly-galleon-pack:zip
Subscribed channels:
# wildfly
  manifest: org.wildfly.channel:wfly-27
  repositories:
    id: central
    url: https://repo1.maven.org/maven2/
    id: jboss-public
    url: https://repository.jboss.org/nexus/content/groups/public/
    id: mrrc
    url: https://maven.repository.redhat.com/ga/
    id: local-repo
    url: file:/Users/spyrkob/workspaces/set/prospero/prospero/test-repo

Feature-packs resolved.
Packages installed.
Downloaded artifacts.
JBoss modules installed.
Configurations generated.

Server installation was restored.
Operation completed in 28.12 seconds.

Note	The user changes made to the original server are not applied to the clone.

5.6. Installing additional Galleon Feature Packs

Galleon feature packs can be used to extend capabilities offered by base server and provide features like, for example, datasource support or alternative JSF implemetnation.

Prospero allows users to install such feature pack on top of existing server using feature-pack add operation.

5.6.1. Feature pack channels

As with all the components used to provision the server, Prospero relies on channels to supply the feature pack and any artifacts it may require. In some cases the base server channel may offer the additional feature packs, in others, an additional channel will have to be registered with the server. See Subscribing to a channel for information how to subscribe to a channel.

Sometimes feature packs may require additional artifacts whose lifecycle is separate from the feature pack itself. For example, Datasources feature pack lets users specify which version of database driver should be installed irrespective of the feature pack version. In such cases users have to create and subscribe to a new channel providing those artifacts.

For example, a channel supplying Datasources feature pack could use Maven Central repository and a following manifest:

schemaVersion: 1.0.0
streams:
  - groupId: org.wildfly
    artifactId: wildfly-datasources-galleon-pack
    version: 4.0.1.Final
  - groupId: com.mysql
    artifactId: mysql-connector-j
    version: 8.1.0

5.6.2. Installing feature pack

The feature packs are identified by GroupId:ArtifactId part of their Maven coordinates. This is known as their Feature Pack Location (fpl).

For example, assuming the server is subscribed to a datasources channel, following command will install the datasources feature pack:

$ prospero feature-pack add \
  --fpl org.wildfly:wildfly-datasources-galleon-pack \
  --dir wildfly

Note	This specific command does not in fact install any database drivers or datasources. See further examples bellow to see how to specify drivers and datasources to install or refer to wildfly-datasources documentation.

5.6.3. Customizing installed feature packs

The Galleon feature packs may provide different capabilities using layers. Users may select a set of layers to install using --layers parameter.

If the feature pack changes server configuration files, the user may also choose which server configuration should be affected using --target-config parameter. If not provided, the layer configuration will use default values. Most Wildfly feature packs provide only a single model (standalone) and the default configuration file of standalone.xml.

Note	If the affected configuration file has been modified by the user, that file will not be modified. Instead, a new configuration file with suffix `.glnew` will be created with the changes.

Note	Some feature packs do not allow customization of layers or selected configuration files. Those feature packs have to be installed without `--layers` or `--target-config` parameters.

For example to install mysql support and configure the datasource in Wildfly standalone-full.xml configuration file, a following command can be used:

$ prospero feature-pack add \
  --fpl org.wildfly:wildfly-datasources-galleon-pack \
  --layers=mysql-datasource \
  --target-config=standalone-full.xml \
  --dir wildfly

If the feature pack provides multiple configuration model, the configuration name can be prefixed with model name e.g. --target-config standalone/standalone.xml

5.6.4. Offline installation

Similarly to other operations, feature-pack add can be executed in an offline mode. To do so, users should provide local repositories with all required artifacts (both for base server and the new feature) using --repositories parameter.

5.6.5. Installation history

Installing a feature pack is recorded in the server history as feature_add event. Displaying the feature_add revision shows details of installed feature pack:

./prospero history --dir installation-dir  --revision f7488855
Updates:
  [Added artifact] org.wildfly:wildfly-datasources-galleon-pack:		[] ==> 4.0.1.Final

Installed features changes:
  [Added Feature Pack] org.wildfly:wildfly-datasources-galleon-pack::zip@maven

6. Custom channels

Sometimes a server needs to be updated with different versions of components than other servers. To achieve that, a server can subscribe to a custom channel on top of its normal channels. This channel will define modified versions of components.

For example if the main channel defines component foo:bar:1.2.3, the custom channel could contain foo:bar:1.2.3-patch0001. During any update operations, the most recent version of the foo:bar across both channels will be applied to the server.

The custom channel needs to be hosted in a Maven repository - either a local filesystem repository on the same system as the server, or a web repository shared between a group of servers. The repository is used to host a versioned channel definition and all customized artifacts.

6.1. Custom artifact versioning

When multiple versions of the same artifact are available from different channels, Prospero will always choose the latest available version of this artifact.

Therefore, the customized artifacts have to always have higher versions than artifacts they are replacing. For instance, if a server includes my:artifact:1.1.0, the customization channel can provide a version my:artifact:1.1.0-path-00001 or my:artifact:1.1.1, but not my:artifact:1.0.9.

6.2. Register custom channel

In order for the server to receive updates from a custom channel, the server needs to be subscribed to the channel and associated repositories.

The channel can be hosted either in an internal installation repository, serving a single server or in a shared repository. To configure the server with a custom channel following commands can be used:

Single server channel

prospero channel add \
  --channel-name customizations \
  --repositories file:/path/to/local/repository \
  --manifest org.wildfly.prospero:customizations

A server can use a filesystem repository and auto-generated channel to provide customizations. This command will register the server to recieve updates from a Maven repository in /path/to/local/repository folder.

The customized artifacts should be listed in a manifest file deployed to the above repository as org.wildfly.prospero:customizations:manifest:yaml:<VERSION>

Shared channel

An online repository can be used to distribute the customizations among multiple servers. Each server has to be subscribed to the repository using:

prospero channel add \
  --channel-name customizations \
  --repositories http://repository.host/path \
  --manifest=org.wildfly.prospero:customizations

Note	The repository can be used to host customizations applicable to a different group of servers. In that case each set of customizations would have a separate manifest deployed under unique Maven `groupId:artifactId` coordinates.

6.3. Distributing custom artifacts

The customized artifacts can be distributed as bundles. Each bundle is a ZIP archive containing a maven-repository directory and an optional artifact-list.yaml file.

|-artifact-list.yaml
|-maven-repository
  |- org
    |-custom
      |-artifact-one
        |-1.2.3
          |-artifact-one-1.2.3.jar
      |-artifact-two
        |-2.3.4
          |-artifact-two-2.3.4.jar
    |-wildfly
      |-prospero
        |-customizations
          |-1.0.0
            |-customizations-manifest-1.0.0.yaml
          |-maven-metadata.xml

In addition to the custom artifacts, repository has to contain a channel manifest. The manifest should list all streams available in the customization bundle.

schemaVersion: 1.0.0
name: Customizations manifest
streams:
  - groupId: org.custom
    artifactId: artifact-one
    version: 1.2.3
  - groupId: org.custom
    artifactId: artifact-two
    version: 2.3.4

Note	the Maven metadata has to be present for the manifest artifact.

The artifact-list.yaml lists all artifacts included in the bundle using following syntax:

artifacts:
  - groupId: "org.custom"
    artifactId: "artifact-one"
    version: "1.2.3"
    packaging: "jar"
    extension: ""

6.3.1. Updating custom artifacts

If a new version of customization bundle needs to be distributed, it should contain a customization manifest deployed under the same groupId:artifactId coordinates, but with an incremented version.

The repository should contain all customized artifacts required by the updated manifest. For instance, the updated manifest included a new version of artifact-one and added artifact-three, resulting in following file:

schemaVersion: 1.0.0
name: Customizations manifest
streams:
  - groupId: org.custom
    artifactId: artifact-one
    version: 1.2.4
  - groupId: org.custom
    artifactId: artifact-two
    version: 2.3.4
  - groupId: org.custom
    artifactId: artifact-three
    version: 3.4.5

The updated repository should contain three artifacts:

|-artifact-list.yaml
|-maven-repository
  |- org
    |-custom
      |-artifact-one
        |-1.2.4
          |-artifact-one-1.2.4.jar
      |-artifact-two
        |-2.3.4
          |-artifact-two-2.3.4.jar
      |-artifact-three
        |-3.4.5
          |-artifact-two-3.4.5.jar
    |-wildfly
      |-prospero
        |-customizations
          |-1.0.1
            |-customizations-manifest-1.0.1.yaml
          |-maven-metadata.xml

6.4. Promoting custom artifacts

To make the custom artifact available to registered servers, the update artifacts from the customization bundle have to be added to the existing repository.

The updated artifacts are distributed in a maven-repository folder inside the customization archive. If the server is subscribed to a local filesystem customization repository, all the artifacts can be simply copied over.

$ ./prospero.sh channel list
# customizations
  manifest: org.wildfly.prospero:customizations
  repositories:
    id: custom-repo-0
    url: file:/home/wildfly/custom/repository # (1)

$ unzip customization-1.0.1.zip # (2)
Archive:  customization-1.0.1.zip
   creating: customization/
   inflating: artifact-list.yaml
   creating: customization/maven-repository/
   [...]

$ cp -r customization/maven-repositry/* /home/wildfly/custom/repository/ #(3)

$ ./prospero.sh updates perform #(4)

Location of existing customization repository
Extracting updated repository
Adding updated content to existing repository
Apply updated customizations

If a repository manager like Nexus is used to host the customization repository, the artifacts should be deployed into it.

Warning

When promoting different versions of the same customization bundle, older versions of the bundle should never be promoted after a newer version is promoted. The customization repository contains maven-metadata.xml files associated with the customization’s manifest that would be overwritten by such operation.

6.5. Apply changes to the server

After promoting artifacts to the channel, the changes can be applied to subscribed servers.

Promoted changes are treated as any updates, so prospero update command can be used to apply them.

prospero update --dir=<PATH/TO/SERVER>
Feature-packs resolved.
Updates found:
  foo:bar                             1.2.3         ==>  1.2.3-patch0001

Note	During update, the latest artifacts available in all registered channels are chosen. If the main channel is updated with newer version of artifact then available in the custom channel, the customized artifact will be removed from the server.

7. Working with channels

More information on the Wildfly Channels can be found in https://github.com/wildfly-extras/wildfly-channel/blob/main/doc/spec.adoc. This chapter is intended to explain some concepts behind different channel types and how they can be used to provide software updates.

A channel contains a collections of artifact streams used by a server. Each stream should contain only backwards-compatible artifact updates with new versions of artifacts replacing previous ones.

There are two ways for a stream to determine the versions of its artifact:

using an artifact manifest with fixed versions, or
use the latest artifact version available in backing Maven repository.

Note	The two mechanisms can be mixed within one channel, using static list to resolve some streams and Maven metadata for others.

Channels using exclusively fixed version manifest can be more stable and make it possible to use 3rd party repositories. The artifact combination in the manifest can be tested before distributing, making sure there are no regressions.

On the other hand channels using Maven metadata can make development and testing of new component updates easier. Any component updates deployed into the Maven repository are immediately made available to the subscribed servers.

7.1. Channels with fixed version manifests

This kind of channels rely on manifests to list all available artifacts and their versions. This can be useful when the content of the repository cannot be fully controlled and might contain incompatible artifact versions.

The manifest is a YAML file, containing a list of available streams:

manifest.yaml

schemaVersion: "1.0.0"
name: "test-manifest"
streams:
  - groupId: "org.test"
    artifactId: "artifact-one"
    version: "1.2.0.Final"

The manifest has to be deployed in the channel repository and registered in the channel definition:

channel.yaml

schemaVersion: "2.0.0"
name: "test-channel"
resolve-if-no-stream: none #(1)
repositories:
  - id: "trusted"
    url: "https://trusted.repository.org/maven/"
manifest:
  maven:
    groupId: org.test.channel
    artifactId: test-manifest

this channel provides only artifacts explicitly listed in the manifest

7.1.1. Updating components in fixed version channel

Updating a component in a manifest channel requires publishing a new version of manifest.

For example, if the org.test:artifact-one is updated to 1.2.1.Final version, the new manifest would look like as follows:

manifest.yaml

schemaVersion: "1.0.0"
name: "test-manifest"
streams:
  - groupId: "org.test"
    artifactId: "artifact-one"
    version: "1.2.1.Final"

This manifest has to be published in the channel repository with a new version:

mvn deploy:deploy-file -Dfile=manifest.yaml \
-DgroupId=org.test.channels -DartifactId=test-manifest \
-Dversion=1.0.1 \ #(1)
-Dclassifier=manifest -Dpackaging=yaml \
-Durl=https://trusted.repository.org/maven/

note the updated version

There are no changes required to the channel definition. Next time components are resolved from this channel, a new version of manifest will be used providing version 1.2.1.Final of org.test:artifact-one.

7.2. Channels using backing Maven repository versions

This type of channels expose the latest available versions of artifacts found in their repositories as the component version. The versions can be curated using either version patterns or a blocklist excluding specific versions.

Channels based on Maven metadata can be useful when the content of repositories can be controlled and trusted. Any new artifact deployed into the channel repository, will automatically be made available for subscribed servers.

An example of channel exposing all underlying artifacts can be seen below:

channel.yaml

schemaVersion: "2.0.0"
name: "test-channel"
resolve-if-no-stream: latest #(1)
repositories: #(2)
  - id: "trusted"
    url: "https://trusted.repository.org/maven/"

latest strategy means that if channel cannot find a stream matching the requested artifact, it will attempt to find the latest version in its repositories
the underlying repository for the channel

7.2.1. Using version patterns to limit artifacts

Sometimes the underlying repository might contain versions of artifacts that are not compatible with the installed server. For example the server might require a 1.2.x version of a certain artifact, but the repository contains a newer version 2.0.x.

To filter incompatible versions, the channel can use a manifest file with versionPattern streams.

manifest.yaml

schemaVersion: "1.0.0"
name: "test-manifest"
streams:
  - groupId: "org.test"
    artifactId: "artifact-one"
    versionPattern: "1\\.2\\..*" #(1)

a regular expression limiting versions to ones starting with 1.2.

Note	The `versionPattern` field uses regular expressions

The above manifest file should be deployed in the channel repository with a classifier manifest and extension yaml. For example following command will deploy the file as org.test.channels:test-manifest:1.0.0

mvn deploy:deploy-file -Dfile=manifest.yaml \
  -DgroupId=org.test.channels -DartifactId=test-manifest \
  -Dversion=1.0.0 -Dclassifier=manifest -Dpackaging=yaml \
  -Durl=https://trusted.repository.org/maven/

The channel definition needs to be updated to reference the new manifest file:

channel.yaml

schemaVersion: "2.0.0"
name: "test-channel"
resolve-if-no-stream: latest
repositories:
  - id: "trusted"
    url: "https://trusted.repository.org/maven/"
manifest:
  maven:
    groupId: org.test.channel
    artifactId: test-manifest

Using this channel definition, all artifacts apart from org.test:artifact-one are still resolved to the latest versions available in the repository. The org.test:artifact-one will be resolved to the latest available "1.2.x" micro version. For example, if the repository contains versions [1.2.0.Final, 1.2.1.Final, 2.0.0.Final], the channel will pick version 1.2.1.Final.

7.2.2. Creating blocklist to exclude updates

Another option to exclude certain artifact versions is to use a blocklist. A blocklist is a YAML file deployed in the channel repository listing blocked artifact versions.

blocklist.yaml

schemaVersion: "1.0.0"
name: "test-blocklist"
blocks:
  - groupId: "org.test"
    artifactId: "artifact-one"
    versions:
    - "1.2.2.Final"

Again, the blocklist has to be deployed in the channel repository. The blocklist artifact has to use blocklist classifier and yaml extension. For example:

mvn deploy:deploy-file -Dfile=blocklist.yaml \
  -DgroupId=org.test.channels -DartifactId=test-blocklist \
  -Dversion=1.0.0 -Dclassifier=blocklist -Dpackaging=yaml \
  -Durl=https://trusted.repository.org/maven/

Finally, the channel definition has to be updated with the reference to the blocklist:

channel.yaml

schemaVersion: "2.0.0"
name: "test-channel"
resolve-if-no-stream: latest
blocklist:
  maven:
    groupId: org.test.channel
    artifactId: test-blocklist
repositories:
  - id: "trusted"
    url: "https://trusted.repository.org/maven/"
manifest:
  maven:
    groupId: org.test.channel
    artifactId: test-manifest

Resolving org.test:artifact-one from this channel will exclude any versions not matching "1.2." pattern and version 1.2.2.Final. For example, if the repository contains versions [*1.2.0.Final, 1.2.1.Final, 1.2.2.Final, 2.0.0.Final], the channel will pick version 1.2.1.Final.

7.2.3. Updating components using "open" channel

Updating component in an open channel requires only deploying the artifact into the channel repository. Neither channel definition not channel manifest has to be changed. Next time components are resolved from this channel, a new version of updated component will be used.