Testing in Forgejo

Introduction

This guide is intended for XMPP Server developers who are building on Forgejo Actions. It demonstrates how to add XMPP Interoperability and Standards Conformance tests integrated into a build & test pipeline. This assumes that you already have a pipeline established, and wish to add the additional checks.

The checks run as their own action within the pipeline, in which they’ll execute tests and output debug logs. The only prerequisite is that you’ve got a built server and have started it.

Assuming that you have a pre-existing pipeline that build your server and starts it for integration testing, then adding our Forgejo Action is as easy as adding one step to your pipeline, like this:

- name: Run XMPP Interoperability Tests against CI server.
  uses: https://codeberg.org/XMPP-Interop-Testing/xmpp-interop-tests-forgejo-action@v1.6.0
  with:
    domain: 'shakespeare.lit'
    adminAccountUsername: 'juliet'
    adminAccountPassword: 'O_Romeo_Romeo!'

The above assumes that your server is running and reachable on the domain shakespeare.lit and that it is provisioned with an administrative account (one that is allowed to create other users, per XEP-0133) that uses the username juliet and the provided password. You’ll find the full range of available configuration options below.

A Full Example

The following example assumes that your pipeline is set up for building an XMPP server and making available the corresponding container image, called myorg/xmppserver”

To execute the tests from the XMPP Integration Testing Framework, the server needs to be running. That can be achieved by running your server as a service, as shown in this example:

services:
  xmppserver:
    image: myorg/xmppserver:latest

The host on which your XMPP service is running will match the name of the service, xmppserver in the example above.

Next, you’re ready to add a step that runs the XMPP Interop Framework Testing plugin.

steps:
  - name: Run the XMPP Interop Tests
    uses: https://codeberg.org/XMPP-Interop-Testing/xmpp-interop-tests-forgejo-action@v1.6.0
    with:
      host: xmppserver
      domain: 'shakespeare.lit'
      adminAccountUsername: 'juliet'
      adminAccountPassword: 'O_Romeo_Romeo!'

The above assumes that your server is running and reachable on the host xmppserver, serving the XMPP domain shakespeare.lit and that it is provisioned with an administrative account (one that is allowed to create other users, per XEP-0133) that uses the username juliet and the provided password. You’ll find the full range of available configuration options below.

For completeness, here is the full pipeline, combining everything above:

on: [push]
jobs:
  test:
    runs-on: codeberg-small
    services:
      xmppserver:
        image: myorg/xmppserver:latest
    steps:
      - name: Run the XMPP Interop Tests
        uses: https://codeberg.org/XMPP-Interop-Testing/xmpp-interop-tests-forgejo-action@v1.6.0
        with:
          host: xmppserver
          domain: 'shakespeare.lit'
          adminAccountUsername: 'juliet'
          adminAccountPassword: 'O_Romeo_Romeo!'

Forgejo does not give a detailed overview of what tests succeed, and what test failed (apart from the textual output in the console) but it does make available the XMPP stanza logs that are generated by the XMPP Interop Framework plugin. These are available as an artifact of the build.

Configuration

Various options are available when calling xmpp_interop_tests_action, and whilst none of them are absolutely required, the defaults are unlikely to be perfect for everyone.

For more information on enabling and disabling tests, consult this guide.

For the latest updates to the documentation of the configuration updates, consult the Codeberg repository of the Forgejo plugin.

Next steps

Now that you have integrated the XMPP Interoperability and Standards Conformance tests into your build pipeline, it is time to start working with the tests!

We’ve compiled the following resources to help you with any run-time challenges that you may run into:

• Choosing what test to run - helps you choose how to disable specific tests (useful when you want to ignore a particular test for some reason).
• Diagnose test failures - explains what you can do to find the reason that test is failing. Did we identify an issue in your server software? Find out!

Splash image courtesy of David Revoy, Forgejo