Testing

Tests included in the QEMU source

QEMU includes a test suite comprising:

• unit tests for library code
• QTest-based tests, which inject predefined stimuli into the device emulation code.
• qemu-iotests, a regression test suite for the block layer code.

make check

The unit tests and QTest-based can be run with "make check". Use "make check-help" to see a list of other available test targets and parameters (for example, you can use "make check SPEED=slow V=1" for a verbose, more thorough test run). These unit tests are used in our continuous integration systems, based on Travis and Patchew.

qemu-iotests

qemu-iotests is run from the toplevel build directory with make check-block. A full version of the testsuite, taking around half an hour to run, is run with sh ../tests/check-block.sh.

System emulation

We have a collection of links to disk images which can be used to test system emulation.

User mode emulation

Here are some links to executables that can be used to test Linux user mode emulation:

• linux-user-busyboxes-0.1.tar.xz - Collection of static busybox binaries for almost all Linux target architectures that QEMU simulates. For quick smoke testing of Linux user mode emulation.

It is also possible to run the Linux Test Project's syscall test suite under the Linux user mode emulation.

Dynamic code analysis

This includes any test to detect memory leaks, reads of uninitialised memory, buffer overflows or other forms of illegal memory access, that needs QEMU to be run, not merely compiled.

Valgrind

Typically these kind of tests are done using Valgrind on a Linux host. Any of the disk images and executables listed above can be used in such tests.

# Simple i386 boot test (BIOS only) with Valgrind.
valgrind --leak-check=full --track-origins=yes --verbose qemu-system-i386

clang UBSan

The [clang undefined behavior sanitizer] can be used to warn about accidental uses of C undefined behavior when QEMU is run. To use it you first need to configure and build QEMU with a clang compiler with the right options:

mkdir build/clang
(cd build/clang && ../../configure --cc=clang --cxx=clang++ \
   '--extra-cflags=-fsanitize=undefined -fno-sanitize=shift-base -Werror')
make -C build/clang -j8

(The -fno-sanitize=shift-base is a workaround for [LLVM bug 25552] where it did not correctly suppress some shift-related warnings when -fwrapv was in use. If you're using a clang where that bug is fixed, likely 3.9 or better, you can drop it.)

Then when you run the resulting QEMU binaries messages will be printed when UB is invoked:

hw/core/loader.c:67:15: runtime error: null pointer passed as argument 1, which is declared to never be null

See the clang documentation for more information including how to produce stack backtraces on errors.

Static code analysis

There are a number of tools which analyse C code and try to detect typical errors. None of these tools is perfect, so using different tools with QEMU will detect more bugs. Be prepared to also get lots of false warnings!

ccc-analyzer (clang)

This is an example used on Debian. It needs package clang.

# Start from the root directory with QEMU code.
mkdir -f bin/debug/ccc-analyzer
cd bin/debug/ccc-analyzer
../../../configure --enable-debug --enable-trace-backend=stderr \
     --cc=/usr/share/clang/scan-build/ccc-analyzer --disable-docs
make

At least on my Linux host (1 GiB RAM, 2 GiB swap), make hangs when ccc-analyzer analyzes target-mips/translate.c: function decode_opc is too complex for the analyzer and takes all memory. Killing the clang process helps in this situation. It's needed 6 times because there are 4 MIPS system emulations and 2 Linux MIPS user emulations.

I guess this is because target-mips/translate.c contains switches with cases covering a very large range; assuming ccc-analyzer expands these case ranges somehow, it probably blows up memory completely.

smatch

Here is a typical example using smatch (from git://repo.or.cz/smatch.git):

# Start from the root directory with QEMU code.
mkdir -f bin/debug/smatch
cd bin/debug/smatch
CHECK="smatch" ../../../configure --enable-debug --cc=cgcc --host-cc=cgcc
make

This example expects that smatch and cgcc are installed in your PATH (if not, you must add absolute paths to the example).

Coverity

Periodic scans of QEMU are done on the public Coverity Scan service (scan.coverity.com). You can request access on their website, and the administrator will grant it if you are an active participant in QEMU development.

Coverity is confused slightly by multiple definitions of functions with the same name. For this reason, Coverity scans are done as follows:

mkdir cov-int
./configure --audio-drv-list=oss,alsa,sdl,pa --disable-werror
make libqemustub.a
cov-build --dir cov-int make
tar cvf - cov-int | xz > cov-int.tar.xz

Notice that libqemustub.a is ignored by Coverity. This is because some stubs call abort() and this causes dead-code false positives. The file cov-int.tar.xz can then be uploaded to Coverity Scan's "Submit build" page. Customarily, the "project version" is set to the output of git describe HEAD and the "description/tag" is set to "commit XYZ" where XYZ is the full SHA1 hash of the commit.

Avocado and Avocado-VT

Avocado is a generic testing framework, while Avocado-VT adds support for Virtualization testing, including first level support for testing QEMU.

To get started with Avocado-VT please visit:

• http://avocado-vt.readthedocs.io
• https://github.com/avocado-framework/avocado-vt

To learn more about Avocado please visit:

• http://avocado-framework.readthedocs.io
• https://github.com/avocado-framework/avocado

After installing it, you can use Avocado-VT tests with your own build of QEMU:

avocado run boot --vt-qemu-bin /path/to/qemu-system-x86_64

Continuous Integration

There is no central point of Continuous Integration for the QEMU project. Instead various individuals and companies have set-up various automated systems that attempt to build and test QEMU to various degrees.

System	Focus	Status
Travis	Build combinations, Ubuntu 12/14.04 x86_64 hosts, MacOSX	current status
Shippable	Docker based cross-compiles (Debian Based)	current status
LAVA	Aarch64 RISU tests, ARM only	result stream
Patchew	Apply and test patches as they are sent on the mailing list.	Patchew dashboard

See Also

• All testing pages in the QEMU wiki