Testing/CI

From QEMU
Jump to: navigation, search

Continuous Integration for the project is a distributed affair spread across a number of public CI services as well as some additional tests run on company infrastructure. There is a strong preference for tests to be integrated with our existing build system. This makes it easy for developers to run the tests directly in their own environment.

Current Status

The current status of the master branch can be included on any page by using CIStatus template:

System Focus Status
Travis Build combinations, Ubuntu 12/14.04 x86_64 hosts, MacOSX qemu.png?branch=master&file=qemu.png [1]
Shippable Docker based cross-compiles (Debian Based) badge?branch=master&foo=qemu.png [2]
Cirrus CI FreeBSD compiles qemu.svg [3]
GitLab CI Linux compiles and IOTests pipeline.svg [4]
Coverity Static analysis badge.svg?flat=1&foo=qemu.svg [5]
Patchew Apply and test patches as they are sent on the mailing list. badge.svg [6]
LAVA Aarch64 RISU tests, ARM only result stream

There is also a custom template that you can use on your own page to track personal branches and provide a handy summary.

Troublesome Tests

Some tests seem to be particularly good at intermittently failing on our CI setup. They are likely triggered by load that is not typically seen on a developer box. If you can help in replicating and tracking down the failures then please do so. Tests the regularly fail at random tend to be disabled to avoid adding too much noise to an already noisy system.

Failure Message Test/Environment Fixed?
ERROR:tests/test-thread-pool.c:208:do_test_cancel: assertion failed (data[i].n == 2): (5 == 2) tests/test-thread-pool Posted
ERROR:tests/test-aio-multithread.c:365:test_multi_fair_mutex: assertion failed (counter == atomic_counter): (199186 == 199187) tests/test-aio-multithread No
Memory content inconsistency at 3374000 first_byte = 2 last_byte = 1 current = 2 hit_edge = 1 tests/migration-test (/i386/migration/precopy/tcp) old patches
ERROR:tests/migration-test.c:332:check_guests_ram: 'bad' should be FALSE variant of above old patches
ERROR:tests/prom-env-test.c:42:check_guest_memory: assertion failed (signature == MAGIC): (0x00000000 == 0xcafec0de) check-qtest-sparc64 No
ERROR:tests/vhost-user-test.c:835:test_flags_mismatch: child process (/i386/vhost-user/flags-mismatch/subprocess [4836]) failed unexpectedly QTEST_VHOST_USER_FIXME=1 tests/qos-test (virtio-net-tests/vhost-user/*) No
ERROR:tests/rcutorture.c:384:gtest_stress: assertion failed (n_mberror == 0): (1 == 0) rcutorture (CONFIG="--with-coroutine=sigaltstack --disable-tcg"/CONFIG=xcode9.4) No
ERROR:tests/rtc-test.c:173:check_time: assertion failed (ABS(t - s) <= wiggle): (3 <= 2) gitlab/build-clang? No

CI pages

Bellow are all the current CI related subpages:

Testing/CI/CirrusCITesting/CI/GitLabCITesting/CI/Patchew
Testing/CI/ShippableTesting/CI/TravisTesting/CI/TroublesomeTests