# FATE Automated Testing Environment

# Table of Contents

# 1. Introduction

FATE provides a regression testsuite embedded within the Libav build system. It can be run locally and optionally configured to send reports to a web aggregator and viewer http://fate.libav.org.

It is advised to run FATE before submitting patches to the current codebase and provide new tests when submitting patches to add additional features.

# 2. Running FATE

## 2.1 Samples and References

In order to run, FATE needs a large amount of data (samples and references) that is provided separately from the actual source distribution.

To inform the build system about the testsuite location, pass ‘`--samples=<path to the samples>`’ to `configure` or set the SAMPLES Make variable or the FATE_SAMPLES environment variable to a suitable value.

The dataset is available through `rsync`, is possible to fetch the current sample using the straight rsync command or through a specific Makefile target.

    # rsync -aL rsync://fate-suite.libav.org/fate-suite/ fate-suite

    # make fate-rsync SAMPLES=fate-suite

# 3. Manual Run

FATE regression test can be run through `make`. Specific Makefile targets and Makefile variables are available:

## 3.1 FATE Makefile targets

‘`fate-list`’

  List all fate/regression test targets.

‘`fate-rsync`’

  Shortcut to download the fate test samples to the specified testsuite location.

‘`fate`’

  Run the FATE test suite (requires the fate-suite dataset).

## 3.2 Fate Makefile variables

‘`V`’

  Verbosity level, can be set to 0, 1 or 2.

  ‘`0`’

    show just the test arguments

  ‘`1`’

    show just the command used in the test

  ‘`2`’

    show everything

‘`SAMPLES`’

  Specify or override the path to the FATE samples at make time, it has a meaning only while running the regression tests.

‘`THREADS`’

  Specify how many threads to use while running regression tests, it is quite useful to detect thread-related regressions.

        make V=1 SAMPLES=/var/fate/samples THREADS=2 fate

# 4. Automated Tests

In order to automatically testing specific configurations, e.g. multiple compilers, `tests/fate.sh` is provided.

This shell script builds Libav, runs the regression tests and prepares a report that can be sent to fate.libav.org or directly examined locally.

## 4.1 Testing Profiles

The configuration file passed to `fate.sh` is shell scripts as well.

It must provide at least a slot identifier, the repo from which fetch the sources, the samples directory, a workdir with enough space to build and run all the tests. Optional submit command fate_recv and a comment to describe the testing profile are available.

Additional optional parameter to tune the Libav building and reporting process can be passed.

    slot=                                   # some unique identifier
    repo=git://git.libav.org/libav.git      # the source repository
    samples=/path/to/fate/samples
    workdir=                                # directory in which to do all the work
    fate_recv="ssh -T fate@fate.libav.org"  # command to submit report
    comment=                                # optional description

    # the following are optional and map to configure options
    arch=
    cpu=
    cross_prefix=
    cc=
    target_os=
    sysroot=
    target_exec=
    target_path=
    extra_cflags=
    extra_ldflags=
    extra_libs=
    extra_conf=     # extra configure options not covered above

    #make=          # name of GNU make if not 'make'
    makeopts=       # extra options passed to 'make'
    #tar=           # command to create a tar archive from its arguments on
                    # stdout, defaults to 'tar c'

## 4.2 Submitting Reports

In order to send reports you need to create an `ssh` key and send it to root@libav.org. The current server fingerprint is a4:99:d7:d3:1c:92:0d:56:d6:d5:61:be:01:ae:7d:e6