Testing in Docker

This guide is intended for XMPP Server developers who are building their server using a continuous integration solution that is capable of launching Docker instances.

The Docker-based approach has been created for those development environments that don’t match any of our other, more specific CI-integrations, but it none-the-less offers the full range of features.

This guide demonstrates how to add XMPP Interoperability and Standards Conformance tests integrated into a build & test pipeline. It is assumed that you already have a pipeline established, and wish to add the additional checks provided by our project.

Docker container images for the XMPP Interop Testing project are published to the GitHub Package / Container registry of our project. Using them can be as simple as this:

docker pull ghcr.io/xmpp-interop-testing/xmpp_interop_tests:latest

docker run \
    --network=host \
    ghcr.io/xmpp-interop-testing/xmpp_interop_tests:latest \
    --domain=shakespeare.lit \
    --adminAccountUsername=juliet \
    --adminAccountPassword=O_Romeo_Romeo

This example demonstrates one possible way to provision accounts. It assumes your server is running and accessible at shakespeare.lit, and that it already has an administrative account (one that is allowed to create other users, per XEP-0133) that uses the username juliet and the provided password. Other configuration methods are also available, and you can explore the full set of options in the sections below.

Configuration

Various options are available when invoking the xmpp_interop_tests, and whilst none of them are absolutely required, the defaults are unlikely to be perfect for everyone.

For the latest updates to the documentation of the configuration updates, consult the GitHub repository of the Docker image.

Provisioning Test Accounts

To be able to run the tests, the server that is being tested needs to be provisioned with test accounts. Three different mechanisms can be used for this:

• Admin Account - By configuring the username and password of a pre-existing administrative user, using the adminAccountUsername and adminAccountPassword configuration options, three test accounts will be created using XEP-0133: Service Administration functionality.
• Explicit Test Accounts - You can configure three pre-existing accounts that will be used for testing, using the accountOneUsername, accountOnePassword, accountTwoUsername, accountTwoPassword, accountThreeUsername and accountThreePassword configuration options.
• In-Band Registration - If no admin account and no explicit tests accounts are provided, in-band registration (XEP-0077) will be used to provision accounts.

For more information on provisioning accounts, consult the ‘Test Account Provisioning’ guide.

Exposing XMPP traffic logs

XMPP traffic is logged by the container in its /logs directory. To easily access the XMPP traffic logs, use a bind mount from anywhere on the host machine into the container on the /logs path.

For example, by adding -v "$(pwd)"/xmpplogs:/logs, a directory named xmpplogs (relative to the working directory) on the host is used as a mount for the /logs directory on the container:

$ docker run \
    --network=host \
    -v "$(pwd)"/xmpplogs:/logs \
    ghcr.io/xmpp-interop-testing/xmpp_interop_tests:main \
    --domain=shakespeare.lit \
    --adminAccountUsername=juliet \
    --adminAccountPassword=O_Romeo_Romeo

# Removed a lot of output for brevity.

$ ls ./xmpplogs | wc -l
255

Detecting test failures

Verbose test results are logged on standard-out. To programmatically detect test failures - for example to automatically mark a build as failed - the exit code of the ‘docker run’ execution can be used. Any non-zero exit code indicate that tests have failed:

$ docker run \
    --network=host \
    -v "$(pwd)"/xmpplogs:/logs \
    ghcr.io/xmpp-interop-testing/xmpp_interop_tests:main \
    --domain=shakespeare.lit \
    --adminAccountUsername=juliet \
    --adminAccountPassword=O_Romeo_Romeo

# Removed a lot of output for brevity, but test failures were reported.

$ echo $?
2

Next steps

Now that you have integrated the XMPP Interoperability and Standards Conformance tests into your build pipeline, it is time to start working with the tests!

We’ve compiled the following resources to help you with any run-time challenges that you may run into:

• Choosing what test to run - helps you choose how to disable specific tests (useful when you want to ignore a particular test for some reason).
• Diagnose test failures - explains what you can do to find the reason that test is failing. Did we identify an issue in your server software? Find out!

Splash image courtesy of Bernd 📷 Dittrich, Unsplash