ToDo/QMPTesting: Difference between revisions
(→TODO) |
m (moved QMPTestingTodo to ToDo/QMPTesting) |
||
| (6 intermediate revisions by one other user not shown) | |||
| Line 19: | Line 19: | ||
=== TODO === | === TODO === | ||
# Introduce the basic protocol suite as explained above (Luiz | # <strike>Introduce the basic protocol suite as explained above</strike> (Luiz) | ||
# Not all existing kvm-autotest tests work under QMP, fix them (eg. migration exec) | # Not all existing kvm-autotest tests work under QMP, fix them (eg. migration exec) | ||
# Add tests for all commands available in QMP. Each command should have its own test suite, this is a big effort and should be done continuously | # Add tests for all commands available in QMP. Each command should have its own test suite, this is a big effort and should be done continuously | ||
| Line 26: | Line 26: | ||
== unit-tests == | == unit-tests == | ||
C level testing | C level unit-testing using [http://check.sourceforge.net check]. This is used to check QEMU's internal APIs. | ||
Currently we have | Currently we have good coverage of QObjects types (QInt, QString, etc) and the JSON parser. We do need to expand this to the Monitor code. | ||
There are two difficulties associated with this, though: | |||
# Monitor code is not modular and has not been designed with unit-tests in mind | # Monitor code is not modular and has not been designed with unit-tests in mind | ||
# It's unlikely that it will be possible to test handlers code this way, as most of them | # It's unlikely that it will be possible to test handlers code this way, as most of them depend on other subsystems | ||
Item 1 will be solved when the current monitor redesign work is finished. | Item 1 will be solved when the current monitor redesign work is finished. | ||
TODO | === TODO === | ||
# Introduce a unit-test tree structure in QEMU | # Introduce a unit-test tree structure in QEMU | ||
# Add a 'make test' target, which runs *all* QEMU's unit-tests | # Add a 'make test' target, which runs *all* QEMU's unit-tests | ||
# Add unit-tests for new monitor code | # Add unit-tests for the new monitor code | ||
== libvirt-TCK == | == libvirt-TCK == | ||
Git repository: git://libvirt.org/libvirt-tck.git | |||
This is libvirt's automated test framework. Although this goes beyond QMP testing, we should ensure that any libvirt API which uses QMP gets properly covered. | This is libvirt's automated test framework. Although this goes beyond QMP testing, we should ensure that any libvirt API which uses QMP gets properly covered. | ||
=== TODO === | |||
# Check what's the current status wrt libvirt APIs coverage using QMP | |||
# Help libvirt guys expanding the suite | |||
Latest revision as of 08:17, 12 October 2016
QMP Testing TODO
This page describes the work to be done on improving QMP test coverage and quality.
We have a few tools/frameworks available for this. The following sections describe them, more or less in importance order. Please, note that a number of items can be addressed in parallel.
kvm-autotest
Page: http://www.linux-kvm.org/page/KVM-Autotest.
kvm-autotest is going to be used to do all client-side QMP testing.
This work can be split in three parts:
- Testing that the basic protocol works as specified. That is, ensuring that the greeting message, success responses and error messages contain the basic information the spec says they do. More importantly, several errors conditions at the protocol level should be tested
- Test that each available command behaves as specified by its *own* specification. This is a big effort, as each command should have its own test suite
- Asynchronous messages testing. Like command testing, each asynchronous message should be tested separately
TODO
Introduce the basic protocol suite as explained above(Luiz)- Not all existing kvm-autotest tests work under QMP, fix them (eg. migration exec)
- Add tests for all commands available in QMP. Each command should have its own test suite, this is a big effort and should be done continuously
- Add tests for asynchronous messages
unit-tests
C level unit-testing using check. This is used to check QEMU's internal APIs.
Currently we have good coverage of QObjects types (QInt, QString, etc) and the JSON parser. We do need to expand this to the Monitor code.
There are two difficulties associated with this, though:
- Monitor code is not modular and has not been designed with unit-tests in mind
- It's unlikely that it will be possible to test handlers code this way, as most of them depend on other subsystems
Item 1 will be solved when the current monitor redesign work is finished.
TODO
- Introduce a unit-test tree structure in QEMU
- Add a 'make test' target, which runs *all* QEMU's unit-tests
- Add unit-tests for the new monitor code
libvirt-TCK
Git repository: git://libvirt.org/libvirt-tck.git
This is libvirt's automated test framework. Although this goes beyond QMP testing, we should ensure that any libvirt API which uses QMP gets properly covered.
TODO
- Check what's the current status wrt libvirt APIs coverage using QMP
- Help libvirt guys expanding the suite