FATE provides a regression testsuite embedded within the Libav build system. It can be run locally and optionally configured to send reports to a web aggregator and viewer http://fate.libav.org.
It is advised to run FATE before submitting patches to the current codebase and provide new tests when submitting patches to add additional features.
In order to run, FATE needs a large amount of data (samples and references) that is provided separately from the actual source distribution.
To inform the build system about the testsuite location, pass
‘--samples=<path to the samples>’ to
configure or set the
SAMPLES Make variable or the LIBAV_SAMPLES environment variable
to a suitable value.
To use a custom wrapper to run the test, pass ‘--target-exec’ to
configure or set the TARGET_EXEC Make variable.
The dataset is available through
rsync, is possible to fetch
the current sample using the straight rsync command or through a specific
# rsync -aL rsync://fate-suite.libav.org/fate-suite/ fate-suite
# make fate-rsync SAMPLES=fate-suite
FATE regression test can be run through
Specific Makefile targets and Makefile variables are available:
List all fate/regression test targets.
Shortcut to download the fate test samples to the specified testsuite location.
Run the FATE test suite (requires the fate-suite dataset).
Verbosity level, can be set to 0, 1 or 2.
show just the test arguments
show just the command used in the test
Specify or override the path to the FATE samples at make time, it has a meaning only while running the regression tests.
Specify how many threads to use while running regression tests, it is quite useful to detect thread-related regressions.
Specify which threading strategy test, either slice or frame, by default slice+frame
Specify a mask to be applied to autodetected CPU flags.
Specify or override the wrapper used to run the tests.
make V=1 SAMPLES=/var/fate/samples THREADS=2 CPUFLAGS=mmx fate
In order to automatically testing specific configurations, e.g. multiple
tests/fate.sh is provided.
This shell script builds Libav, runs the regression tests and prepares a report that can be sent to http://fate.libav.org/ or directly examined locally.
The configuration file passed to
fate.sh is shell scripts as well.
It must provide at least a slot identifier, the repo from which fetch the sources, the samples directory, a workdir with enough space to build and run all the tests. Optional submit command fate_recv and a comment to describe the testing profile are available.
Additional optional parameter to tune the Libav building and reporting process can be passed.
slot= # some unique identifier repo=git://git.libav.org/libav.git # the source repository samples=/path/to/fate/samples workdir= # directory in which to do all the work fate_recv="ssh -T firstname.lastname@example.org" # command to submit report comment= # optional description # the following are optional and map to configure options arch= cpu= cross_prefix= cc= target_os= sysroot= target_exec= target_path= extra_cflags= extra_ldflags= extra_libs= extra_conf= # extra configure options not covered above #make= # name of GNU make if not 'make' makeopts= # extra options passed to 'make' #tar= # command to create a tar archive from its arguments on # stdout, defaults to 'tar c'
The TARGET_EXEC option provides a way to run FATE wrapped in
wine or on remote targets
In order to send reports you need to create an
ssh key and send it
The current server fingerprint is a4:99:d7:d3:1c:92:0d:56:d6:d5:61:be:01:ae:7d:e6