Introduction
This guide is intended for XMPP Server developers who are building their server with Harness CI. It demonstrates how to add XMPP Interoperability and Standards Conformance tests integrated into a build & test pipeline. This assumes that you already have a pipeline established, and wish to add the additional checks.
Note that the plugin that you’ll use in Harness can be shared between Drone, Harness, Woodpecker, and possibly other continuous integration solutions. This is why you’ll see references to ‘drone’ in this document: that same plugin is re-used for various systems. The instructions provided herein are, however, specific to Harness.
The checks are executed within the pipeline by a plugin that we’re providing. The only prerequisite is that you’ve got a built XMPP server and have started it (presumably as a background service)
Assuming that you have a pre-existing pipeline that build your server and starts it for integration testing, then adding our plugin is as easy as adding one step to your pipeline, like this:
- step:
type: Plugin
name: Run XMPP Interop Framework tests against XMPP server
identifier: Run_Drone_plugin_against_XMPP_server
description: Executes the Drone Plugin against the XMPP server that has been created earlier in this Pipeline
spec:
image: ghcr.io/xmpp-interop-testing/drone-xmpp-test:latest
settings:
host: xmppserver
domain: shakespeare.lit
This example demonstrates one possible way to provision accounts. It assumes your server is running and reachable on the host xmppserver
, serving the XMPP domain shakespeare.lit
, and that it supports In-Band Registration (as defined in XEP-0077) to provision accounts. Other configuration methods are also available, and you can explore the full set of options in the sections below.
A Full Example
Harness relies heavily on containerization. The following example assumes that your pipeline is set up for building an XMPP server and making available the corresponding container image, called myorg/xmppserver
”
To execute the tests from the XMPP Integration Testing Framework, the server needs to be running. That can be achieved by running your server as a background service, as shown in this example:
- step:
type: Background
name: xmppserver
identifier: xmppserver
spec:
connectorRef: MyOrg_Container_Registry
image: myorg/xmppserver:latest
shell: Sh
entrypoint:
- /sbin/entrypoint.sh
- "-MyCustomStartupArgument 1234"
contextType: Pipeline
The host on which your XMPP service is running will match the identifier
value of the step.
Optionally, include a Run step that checks the health of your xmppserver service.
Next, you’re ready to add a step that runs the XMPP Interop Framework Testing plugin.
- step:
type: Plugin
name: Run XMPP Interop Framework tests against XMPP server
identifier: Run_Tests_against_XMPP_server
description: Executes the Drone Plugin against the XMPP server that has been created earlier in this Pipeline
spec:
image: ghcr.io/xmpp-interop-testing/drone-xmpp-test:latest
reports:
type: JUnit
spec:
paths:
- "**/*.xml"
settings:
host: xmppserver
domain: shakespeare.lit
adminAccountUsername: juliet
adminAccountPassword: O_Romeo_Romeo
enabledSpecifications: XEP-0115,XEP-0199,XEP-0352
The above assumes that your server is running and reachable on the host xmppserver
, serving the XMPP domain shakespeare.lit
and that it is provisioned with an administrative account (one that is allowed to create other users, per XEP-0133) that uses the username juliet
and the provided password. You’ll find the full range of available configuration options below.
Furthermore, the “JUnit” configuration makes available the test results in the Harness user interface, under the ‘Tests’ tab of each build result.
Harness does not make available the XMPP stanza logs that are generated by the XMPP Interop Framework plugin. To have access to these logs, which are very helpful to diagnose test failures, the logs can be uploaded as an artifact to a storage provider of your choice.
Configuration
Various options are available when calling xmpp-interop-testing/drone-xmpp-test, and whilst none of them are absolutely required, the defaults are unlikely to be perfect for everyone.
For the latest updates to the documentation of the configuration updates, consult the GitHub repository of the Drone plugin.
Option | Description | Default value |
---|---|---|
host | IP address or DNS name of the XMPP service to run the tests on. | 127.0.0.1 |
domain | the XMPP domain name of server under test. | example.org |
timeout | the amount of milliseconds after which an XMPP action (typically an IQ request) is considered timed out. | 5000 (five seconds) |
adminAccountUsername | (optional) The account name of a pre-existing user that is allowed to create other users, per XEP-0133. See: “Provisioning Test Accounts” | - |
adminAccountPassword | (optional) The password of the admin account | - |
accountOneUsername | (optional) The first account name of a set of three accounts used for testing. See: “Provisioning Test Accounts” | - |
accountOnePassword | (optional) The password of the accountOneUsername account. | - |
accountTwoUsername | (optional) The second account name of a set of three accounts used for testing. See: “Provisioning Test Accounts” | - |
accountTwoPassword | (optional) The password of the accountTwoUserName account | - |
accountThreeUsername | (optional) The third account name of a set of three accounts used for testing. See: “Provisioning Test Accounts” | - |
accountThreePassword | (optional) The password of the accountThreeUserName account | - |
disabledTests | (optional) A comma-separated list of tests that are to be skipped. For example: EntityCapsTest,SoftwareInfoIntegrationTest | - |
disabledSpecifications | (optional) A comma-separated list of specifications (not case-sensitive) that are to be skipped. For example: XEP-0045,XEP-0060 | - |
enabledTests | (optional) A comma-separated list of tests that are the only ones to be run. For example: EntityCapsTest,SoftwareInfoIntegrationTest | - |
enabledSpecifications | (optional) A comma-separated list of specifications (not case-sensitive) that are the only ones to be run. For example: XEP-0045,XEP-0060 | - |
logDir | (optional) The directory in which the test output and logs are to be stored. This directory will be created, if it does not already exist. | ./output |
Provisioning Test Accounts
To be able to run the tests, the server that is being tested needs to be provisioned with test accounts. Three different mechanisms can be used for this:
- Admin Account - By configuring the username and password of a pre-existing administrative user, using the
adminAccountUsername
andadminAccountPassword
configuration options, three test accounts will be created using XEP-0133: Service Administration functionality. - Explicit Test Accounts - You can configure three pre-existing accounts that will be used for testing, using the
accountOneUsername
,accountOnePassword
,accountTwoUsername
,accountTwoPassword
,accountThreeUsername
andaccountThreePassword
configuration options. - In-Band Registration - If no admin account and no explicit tests accounts are provided, in-band registration (XEP-0077) will be used to provision accounts.
For more information on provisioning accounts, consult the ‘Test Account Provisioning’ guide.
Next steps
Now that you have integrated the XMPP Interoperability and Standards Conformance tests into your build pipeline, it is time to start working with the tests!
We’ve compiled the following resources to help you with any run-time challenges that you may run into:
- Choosing what test to run - helps you choose how to disable specific tests (useful when you want to ignore a particular test for some reason).
- Diagnose test failures - explains what you can do to find the reason that test is failing. Did we identify an issue in your server software? Find out!
Splash image courtesy of Harness