ToDo/QMPTesting

QMP Testing TODO

This page describes the work to be done on improving QMP test coverage and quality.

We have a few tools/frameworks available for this. The following sections describe them, more or less in importance order. Please, note that a number of items can be addressed in parallel.

kvm-autotest

Page: http://www.linux-kvm.org/page/KVM-Autotest.

kvm-autotest is going to be used to do all client-side QMP testing.

This work can be split in three parts:

1. Testing that the basic protocol works as specified. That is, ensuring that the greeting message, success responses and error messages contain the basic information the spec says they do. More importantly, several errors conditions at the protocol level should be tested
2. Test that each available command behaves as specified by its *own* specification. This is a big effort, as each command should have its own test suite
3. Asynchronous messages testing. Like command testing, each asynchronous message should be tested separately

TODO

1. Introduce the basic protocol suite as explained above (Luiz)
2. Not all existing kvm-autotest tests work under QMP, fix them (eg. migration exec)
3. Add tests for all commands available in QMP. Each command should have its own test suite, this is a big effort and should be done continuously
4. Add tests for asynchronous messages

unit-tests

C level unit-testing using check. This is used to check QEMU's internal APIs.

Currently we have good coverage of QObjects types (QInt, QString, etc) and the JSON parser. We do need to expand this to the Monitor code.

There are two difficulties associated with this, though:

1. Monitor code is not modular and has not been designed with unit-tests in mind
2. It's unlikely that it will be possible to test handlers code this way, as most of them depend on other subsystems

Item 1 will be solved when the current monitor redesign work is finished.

TODO

1. Introduce a unit-test tree structure in QEMU
2. Add a 'make test' target, which runs *all* QEMU's unit-tests
3. Add unit-tests for the new monitor code

libvirt-TCK

Git repository: git://libvirt.org/libvirt-tck.git

This is libvirt's automated test framework. Although this goes beyond QMP testing, we should ensure that any libvirt API which uses QMP gets properly covered.

TODO

1. Check what's the current status wrt libvirt APIs coverage using QMP
2. Help libvirt guys expanding the suite