Testing/CI

From QEMU

Continuous Integration for the project is a distributed affair spread across a number of public CI services as well as some additional tests run on company infrastructure. There is a strong preference for tests to be integrated with our existing build system. This makes it easy for developers to run the tests directly in their own environment.

Current Status

The current status of the master branch can be included on any page by using CIStatus template:

System Focus Status
GitLab CI Majority of CI testing (builds x86 & cross), various check targets https://gitlab.com/qemu-project/qemu/badges/master/pipeline.svg [1]
Cirrus CI FreeBSD, MacOS and Windows MSYS2 compile and test https://api.cirrus-ci.com/github/qemu/qemu.svg [2]
Travis non-x86 hosts, mostly deprecated qemu.png?branch=master&file=qemu.png [3]
Coverity Static analysis https://scan.coverity.com/projects/378/badge.svg?flat=1&foo=qemu.svg [4]
Patchew Apply and test patches as they are sent on the mailing list. https://patchew.org/QEMU/badge.svg [5]
Documentation Build the RST portions of the doc/ subtree https://readthedocs.org/projects/qemu/badge/?version=latest&foo=qemu.svg [6]

There is also a custom template that you can use on your own page to track personal branches and provide a handy summary.

Troublesome Tests

Some tests seem to be particularly good at intermittently failing on our CI setup. They are likely triggered by load that is not typically seen on a developer box. If you can help in replicating and tracking down the failures then please do so. Tests the regularly fail at random tend to be disabled to avoid adding too much noise to an already noisy system.

Failure Message Test/Environment Fixed?
FAIL: test_drive_backup (__main__.TestStopWithBlockJob) GitLab - --disable-tcg, iotest 129 No
ERROR:tests/test-aio-multithread.c:365:test_multi_fair_mutex: assertion failed (counter == atomic_counter): (199186 == 199187) tests/test-aio-multithread No
Memory content inconsistency at 3374000 first_byte = 2 last_byte = 1 current = 2 hit_edge = 1 tests/migration-test (/i386/migration/precopy/tcp) old patches
ERROR:tests/migration-test.c:332:check_guests_ram: 'bad' should be FALSE variant of above old patches
ERROR:tests/prom-env-test.c:42:check_guest_memory: assertion failed (signature == MAGIC): (0x00000000 == 0xcafec0de) check-qtest-sparc64 No
ERROR:tests/vhost-user-test.c:835:test_flags_mismatch: child process (/i386/vhost-user/flags-mismatch/subprocess [4836]) failed unexpectedly QTEST_VHOST_USER_FIXME=1 tests/qos-test (virtio-net-tests/vhost-user/*) No
ERROR:tests/rcutorture.c:384:gtest_stress: assertion failed (n_mberror == 0): (1 == 0) rcutorture (various configs, not replicated on local machines) https://patchew.org/QEMU/20200213225109.13120-1-alex.bennee@linaro.org/
ERROR:tests/rtc-test.c:173:check_time: assertion failed (ABS(t - s) <= wiggle): (3 <= 2) gitlab/build-clang? No

CI pages

Bellow are all the current CI related subpages: