Testing in GitLab

Introduction

This guide is intended for XMPP Server developers who are building in GitLab. It demonstrates how to add XMPP Interoperability and Standards Conformance tests integrated into a build & test pipeline. This assumes that you already have a pipeline established, and wish to add the additional checks.

Like all external components in GitLab, this one requires an include statement (with optional inputs). Since this job often requires your own dependencies and customisation, it imports as a template called .xmpp-interop-tests which is then explicit extended in a job. That job runs the tests, giving junit-esque outputs and debug logs. The only prerequisite is that you’ve got a built server, and have started it.

Assuming you already have a pipeline, and a script to launch your server, adding our GitLab component is as easy as adding a small amount YAML to your pipeline:

  include:
    - component: gitlab.com/xmpp-interop-testing/xmpp-interop-testing-gitlab-component/interop-testing@v1.6.0
      inputs:
        domain: 'shakespeare.lit'
        adminAccountUsername: 'juliet'
        adminAccountPassword: 'O_Romeo_Romeo!'

  compliance-tests:
    extends: .xmpp-interop-tests
    stage: test

This example demonstrates one possible way to provision accounts. It assumes your server is running and accessible at shakespeare.lit, and that it already has an administrative account (one that is allowed to create other users, per XEP-0133) that uses the username juliet and the provided password. Other configuration methods are also available, and you can explore the full set of options in the sections below.

A Full Example

Very generically, the job is expected to be part of such a flow in this manner:

1. Compile and build server software
2. Start server and run test job

Given the isolation of jobs in GitLab, the start of the server and the running of tests must normally be combined, usually by starting the server in the before_script.

Assume that you have a trivial pre-existing build pipeline that looks like this for compiling and storing builds of an XMPP server:

  maven-build:
    stage: build
    script: ./mvnw package
    artifacts:
      paths:
        - my_xmpp_server

Then you’d add our first example, with tweaks for dependencies and for launching, and end up with this:

  maven-build:
    stage: build
    script: ./mvnw package
    artifacts:
      paths:
        - my_xmpp_server

  include:
    - component: gitlab.com/xmpp-interop-testing/xmpp-interop-testing-gitlab-component/interop-testing@v1.6.0
      inputs:
        domain: 'shakespeare.lit'
        host: '127.0.0.1'
        adminAccountUsername: 'juliet'
        adminAccountPassword: 'O_Romeo_Romeo!'
  
  compliance-tests:
    extends: .xmpp-interop-tests
    stage: test
    dependencies:
      - maven-build
    before_script: "./my_xmpp_server/launch-my-server.sh"

In addition to the first example:

• A host option to direct the test engine to resolve shakespeare.lit as the local server
• A dependencies section in the test job to wait for the build
• A before_script to launch the built server prior to testing

Configuration

Various options are available when calling xmpp_interop_tests_action, and whilst none of them are absolutely required, the defaults are unlikely to be perfect for everyone.

For the latest updates to the documentation of the configuration updates, consult the GitLab repository of the GitLab Action.

Provisioning Test Accounts

To be able to run the tests, the server that is being tested needs to be provisioned with test accounts. Three different mechanisms can be used for this:

• Admin Account - By configuring the username and password of a pre-existing administrative user, using the adminAccountUsername and adminAccountPassword configuration options, three test accounts will be created using XEP-0133: Service Administration functionality.
• Explicit Test Accounts - You can configure three pre-existing accounts that will be used for testing, using the accountOneUsername, accountOnePassword, accountTwoUsername, accountTwoPassword, accountThreeUsername and accountThreePassword configuration options.
• In-Band Registration - If no admin account and no explicit tests accounts are provided, in-band registration (XEP-0077) will be used to provision accounts.

For more information on provisioning accounts, consult the ‘Test Account Provisioning’ guide.

Next steps

Now that you have integrated the XMPP Interoperability and Standards Conformance tests into your build pipeline, it is time to start working with the tests!

We’ve compiled the following resources to help you with any run-time challenges that you may run into:

• Choosing what test to run - helps you choose how to disable specific tests (useful when you want to ignore a particular test for some reason).
• Diagnose test failures - explains what you can do to find the reason that test is failing. Did we identify an issue in your server software? Find out!

Splash image courtesy of Joel Filipe, Unsplash