FlakyTests: Difference between revisions
(Created page with "This is a page to document tests which currently seem to be flaky, either in CI or locally. If you have information/possible causes/fixes for any of these, please feel free to update. == ast2500_evb_sdk avocado test == This one is flaky for me on local 'make check-avocado' builds; I haven't seen it in the CI. <pre> (41/69) tests/avocado/machine_aspeed.py:AST2x00MachineSDK.test_arm_ast2500_evb_sdk: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeo...") |
No edit summary |
||
| Line 1: | Line 1: | ||
This is a page to document tests which currently seem to be flaky, either in CI or locally. If you have information/possible causes/fixes for any of these, please feel free to update. | This is a page to document tests which currently seem to be flaky, either in CI or locally. If you have information/possible causes/fixes for any of these, please feel free to update. If particular tests seem flaky to you, please note that also. Try to include dates when flakiness was observed so we have some hope of distinguishing old-and-fixed from current problems... | ||
== ast2500_evb_sdk avocado test == | == ast2500_evb_sdk avocado test == | ||
Seen: 2023-02-21 (and for months at least before that) | |||
This one is flaky for me on local 'make check-avocado' builds; I haven't seen it in the CI. | This one is flaky for me on local 'make check-avocado' builds; I haven't seen it in the CI. | ||
| Line 10: | Line 12: | ||
== migration-test == | == migration-test == | ||
Seen: 2023-02-21 and at least back into December of 2022 | |||
Fails a lot for me on my local macos x86 box, but not when run individually -- possibly it's bad when the host is under heavy load? Also fails elsewhere, but much more intermittently. | Fails a lot for me on my local macos x86 box, but not when run individually -- possibly it's bad when the host is under heavy load? Also fails elsewhere, but much more intermittently. | ||
| Line 58: | Line 62: | ||
== segfaults in check-tcg == | == segfaults in check-tcg == | ||
Seen: 2023-02-21 | |||
Here's a failed job on aarch64 host, aarch64 guest, segfault on bti-3 in tcg-tests: | Here's a failed job on aarch64 host, aarch64 guest, segfault on bti-3 in tcg-tests: | ||
Revision as of 14:34, 21 February 2023
This is a page to document tests which currently seem to be flaky, either in CI or locally. If you have information/possible causes/fixes for any of these, please feel free to update. If particular tests seem flaky to you, please note that also. Try to include dates when flakiness was observed so we have some hope of distinguishing old-and-fixed from current problems...
ast2500_evb_sdk avocado test
Seen: 2023-02-21 (and for months at least before that)
This one is flaky for me on local 'make check-avocado' builds; I haven't seen it in the CI.
(41/69) tests/avocado/machine_aspeed.py:AST2x00MachineSDK.test_arm_ast2500_evb_sdk: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n{'name': '41-tests/avocado/machine_aspeed.py:AST2x00MachineSDK.test_arm_ast2500_evb_sdk', 'logdir': '/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/arm-clang/tests... (240.87 s)
migration-test
Seen: 2023-02-21 and at least back into December of 2022
Fails a lot for me on my local macos x86 box, but not when run individually -- possibly it's bad when the host is under heavy load? Also fails elsewhere, but much more intermittently.
https://gitlab.com/qemu-project/qemu/-/jobs/3806090216 (a FreeBSD job)
32/648 ERROR:../tests/qtest/migration-helpers.c:205:wait_for_migration_status: assertion failed: (g_test_timer_elapsed() < MIGRATION_STATUS_WAIT_TIMEOUT) ERROR
on a local macos x86 box:
▶ 34/621 ERROR:../../tests/qtest/migration-helpers.c:151:migrate_query_not_failed: assertion failed: (!g_ str_equal(status, "failed")) ERROR 34/621 qemu:qtest+qtest-i386 / qtest-i386/migration-test ERROR 168.12s killed by signal 6 SIGABRT ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― stderr: qemu-system-i386: Failed to peek at channel query-migrate shows failed migration: Unable to write to socket: Broken pipe ** ERROR:../../tests/qtest/migration-helpers.c:151:migrate_query_not_failed: assertion failed: (!g_str_equal(status, "failed")) (test program exited with status code -6) ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ▶ 37/621 ERROR:../../tests/qtest/migration-helpers.c:151:migrate_query_not_failed: assertion failed: (!g_str_equal(status, "failed")) ERROR 37/621 qemu:qtest+qtest-x86_64 / qtest-x86_64/migration-test ERROR 174.37s killed by signal 6 SIGABRT ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― stderr: query-migrate shows failed migration: Unable to write to socket: Broken pipe ** ERROR:../../tests/qtest/migration-helpers.c:151:migrate_query_not_failed: assertion failed: (!g_str_equal(status, "failed")) (test program exited with status code -6) ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
I've seen this on other CI jobs as well, but Gitlab's UI makes it pretty much impossible to re-find failed jobs, since you can't search for them by failure reason at all.
I've also seen this fail on the OpenBSD vm build.
I've seen the migration-test hang on the s390 private CI runner in such a way that even though the CI job has timed out, the stale QEMU and migration-test processes are still lying around on the host.
segfaults in check-tcg
Seen: 2023-02-21
Here's a failed job on aarch64 host, aarch64 guest, segfault on bti-3 in tcg-tests: https://gitlab.com/qemu-project/qemu/-/jobs/3806772144
TEST bti-3 on aarch64 Segmentation fault make[1]: *** [Makefile:170: run-bti-3] Error 139
Seems to be intermittent, didn't happen on re-run of the job.