Features/QAPI: Difference between revisions

From QEMU
No edit summary
Line 12: Line 12:
QMP has evolved from the human monitor code and has inherited a number of the deficiencies of the human monitor as a result.  In particular, error reporting and asynchronous commands really demonstrate the historic flaws in the human monitor.  This spec describes the types of changes that are needed to decouple QMP from the human monitor and fix the deficiencies that are currently present.
QMP has evolved from the human monitor code and has inherited a number of the deficiencies of the human monitor as a result.  In particular, error reporting and asynchronous commands really demonstrate the historic flaws in the human monitor.  This spec describes the types of changes that are needed to decouple QMP from the human monitor and fix the deficiencies that are currently present.


== Issues ==
== Implementation ==


=== Error Reporting ===
QAPI uses a schema/IDL written in JSON to automatically generate the types and
marshalling information for QMP commands.  It is used to generate both the
server side marshalling interfaces and a client library.


Error reporting works reasonably well when dealing with synchronous commands and when the errors are generated within the actual function implementing the command.  Today we have two mechanisms to support errors.  The first is QError which provides a QObject based error.  The second mechanism is qerror_report() which combines the creation of a QError with the association of the error with the current monitor state.  The current monitor state is defined as the monitor that is currently executing a synchronous command.
A typical function definition looks like this:


The reason qerror_report() behaves this way is to minimize change in non-monitor code within QEMU.  Since most of these functions simply fprintf() an error, they provide no way to return a rich error and have no context associated with them that an rich error could be associated with. Usually these functions will return an integer error but it's usually just -1 and it's very often not propagated reliably.
##
# @change:
#
# This command is multiple commands multiplexed together.  Generally speaking,
# it should not be used in favor of the single purpose alternatives such as
# @change-vnc-listen, @change-vnc-password, and @change-blockdev.
  #
# @device: This is normally the name of a block device but it may also be 'vnc'.
#          when it's 'vnc', then sub command depends on @target
#
# @target: If @device is a block device, then this is the new filename.
#          If @device is 'vnc', then if the value 'password' selects the vnc
#          change password command.  Otherwise, this specifies a new server URI
#          address to listen to for VNC connections.
#
# @arg:    If @device is a block device, then this is an optional format to open
#          the device with.
#          If @device is 'vnc' and @target is 'password', this is the new VNC
#          password to set.  If this argument is an empty string, then no future
#          logins will be allowed.
#
# Returns: Nothing on success.
#          If @device is not a valid block device, DeviceNotFound
#          If @format is not a valid block format, InvalidBlockFormat
#          If the new block device is encrypted, DeviceEncrypted.  Note that
#          if this error is returned, the device has been opened successfully
  #          and an additional call to @block_passwd is required to set the
#          device's password.  The behavior of reads and writes to the block
#          device between when these calls are executed is undefined.
#
# Notes:  It is strongly recommended that this interface is not used especially
#        for changing block devices.
#
# Since: 0.14.0
##
[ 'change', {'device': 'str', 'target': 'str'}, {'arg': 'str'}, 'none' ]


The nature of qerror_report() makes it impossible to implement asynchronous commands.  As an example, if you have to asynchronous commands in flight at once, say A and B, if A calls a function do_foo() as part of it's processing that calls qerror_report(), that error will be associated with the cur_mon state.  However, when processing B's completion, it's not possible to tell whether that error was as a result of processing on behalf of command A or B.
The comments above the function are written in gtk-doc format and meant to be
extracted to generate both protocol documentation and libqmp documentation.


Another related albeit less entangled problem is reporting errors in asynchronous events.  qerror_report() has a clear behavior of associating an error with the cur_mon which in turn associates an error with whatever command is being processed.  Under some circumstances, qerror_report() will actually output to standard error.  There is no obvious way to have certain errors result in asynchronous events.
The first element of the list is the command name, in this case, 'change'.


=== Relationship to the Human Monitor Protocol ===
The second element of the list is a dictionary containing the required arguments
for the function.  The key is the argument name and the value is the type.  The
type can be a string or a complex type (see section on Typing).


Right now, QMP and HMP are implemented as a single dispatch table.  HMP functions are implemented in terms of QMP functions with additional pretty printing.  However, because errors are not returned by QMP functions, the HMP pretty printing functions are not able to add additional contextual information to the error output.  The result is that we tend to have very specific error types because we need each error type to output a message based on specific context.  In other words, we tend to have duplicate error types for every monitor command because we often want the same error to be printed differently for each command.
The third element is a dictionary of optional arguments that follows the same
rules as the required arguments.


Additionally, the single dispatch table makes it very difficult to introduce HMP functions without a direct QMP mapping and vice versa.  While there is significant overlap between QMP and HMP, some QMP commands don't make sense for HMP (like capabilities) and some HMP commands (like x/xp) don't make sense for QMP.
The final list element is the return type of the function.  'none' is a special
type representing a void return value.


== Potential Solutions ==
== QMP Server ==


QErrors should be treated like normal QObjects and should be propagated as a return value.  For functions that only return success or failure, their signatures can be converted to return a QError or NULL.  For commands that return values upon success, an additional parameter can be added as a QError ** whereas an error can be propagated that way.
This definition will generate the following signature in qemu/qmp.h:


Additionally, the QMP signature should be changed to reflect this.  An example would be:
void qmp_change(const char *device, const char *target, bool has_arg,
                const char *arg, Error **errp);


  QObject *qmp_command(const QDict *args);
Which is then implemented in the appropriate place in QEMU. The optional
arguments always are prefixed with a boolean argument to indicate whether the
option has been specified.  The final argument is a pointer to an Error object
in the fashion of GError.  The caller of this function is will check for a
non-NULL error object to determine if the function failed.  errp can be set to
NULL if the caller does not care about failure.


Upon success, the QObject return would be a non-QError type.  Upon failure, a QError would be returned.
Currently, Error is roughly identical to QError with the exception that it only
supports simple key/value arguments.


For asynchronous commands, an appropriate signature would be:
== Complex Types ==


  void qmp_async_command(const QDict *args, QMPCompletionSlot *cb);
Some commands take or return complex types. It's usually a good idea to define
complex types outside of the function definition.  An example of a command
that returns a complex type is below:


Where QMPCompletionSlot was a structure of the following form:
##
# @VersionInfo:
#
# A description of QEMU's version.
#
# @qemu.major:  The major version of QEMU
#
# @qemu.minor:  The minor version of QEMU
#
# @qemu.micro:  The micro version of QEMU.  By current convention, a micro
#              version of 50 signifies a development branch.  A micro version
#              greater than or equal to 90 signifies a release candidate for
#              the next minor version.  A micro version of less than 50
#              signifies a stable release.
#
# @package:    QEMU will always set this field to an empty string.  Downstream
#              versions of QEMU should set this to a non-empty string.  The
#              exact format depends on the downstream however it highly
#              recommended that a unique name is used.
#
# Since: 0.14.0
##
{ 'VersionInfo': {'qemu': {'major': 'int', 'minor': 'int', 'micro': 'int'},
                  'package': 'str'} }
##
# @query-version:
#
# Returns the current version of QEMU.
#
# Returns:  A @VersionInfo object describing the current version of QEMU.
#
# Since: 0.14.0
##
[ 'query-version', {}, {}, 'VersionInfo' ]
 
The syntax the use of a dictionary instead of a list is an indication of a
typedef.  Within QEMU,  This will generate the following code in
qemu/qmp-types.h:
 
typedef struct VersionInfo
{
    struct {
        int64_t major;
        int64_t minor;
        int64_t micro;
    } qemu;
    const char *package;
    struct VersionInfo *next;
} VersionInfo;
 
The use of a next pointer is to enable support for returning lists of complex
types.  The query-version command will generate the following signature:
 
// qemu/qmp-types.h
VersionInfo *qmp_alloc_version_info(void);
void qmp_free_version_info(VersionInfo *obj);
// qemu/qmp.h
VersionInfo *qmp_query_version(Error **errp);
 
A typical implementation might look something like:
 
VersionInfo *qmp_query_version(Error **errp)
{
    VersionInfo *info = qmp_alloc_version_info();
    info->qemu.major = 0;
    info->qemu.minor = 14;
    info->qemu.micro = 92;
    info->package = qemu_strdup(" (qemu-kvm)");
    return info;
}
 
== Optional Structure Members ==


typedef struct QMPCompletionSlot {
Optional structure members can be specified by using a '*' as a prefix to the
    void (*complete)(struct QMPCompletionSlot *slot, QObject *result);
member name. For example:
  };


When the asynchronous command completes, it passes either a QError to cb->complete on failure or it passes the data generated by the command.
##
# @BlockDeviceInfo:
#
# Information about the backing device for a block device.
#
# @file: the filename of the backing device
#
# @ro: true if the backing device was open read-only
#
# @drv: the name of the block format used to open the backing dvice
#
# @backing_file: #optional the name of the backing file (for copy-on-write)
#
# @encrypted: true if the backing device is encrypted
#
# Since: 0.14.0
#
# Notes: This interface is only found in @BlockInfo.
##
{ 'BlockDeviceInfo': { 'file': 'str', 'ro': 'bool', 'drv': 'str',
                        '*backing_file': 'str', 'encrypted': 'bool' } }


QError should also expose a function to interact with the QError information such as the error type.  This allows callers to determine what the QError is and present an appropriate error.  For instance, the following could implement the HMP monitor command memory_save
A typical implementation may look like:


  void hmp_memory_save(Monitor *mon, uint32_t size, const char *filename, target_long addr)
  BlockDeviceInfo *qmp_query_block_device_info(const char *device, Error **errp)
  {
  {
    QObject *ret;
    BlockDeviceInfo *info;
    BlockDriverState *bs;
    Error *local_err = NULL;
    bs = bdrv_find(device, &local_err);
    if (local_err) {
        error_propagate(errp, local_err);
        return NULL;
    }
   
   
    ret = qmp_command("memory_save", "{'size': %d, 'filename': %s, 'val': %d}",
    info->file = qemu_strdup(bs->filename);
                      size, filename, addr);
    info->ro = bs->readonly;
    if (qobject_type(ret) == QOBJ_ERROR) {
    info->drv = qemu_strdup(bs->drv);
      QError *err = qobject_to_qerror(ret);
    info->encrypted = bs->encrypted;
      const char *type = qerror_get_type(err);
    if (bs->backing_file[0]) {
      if (strcmp(type, "FileNotFound") == 0) {
        info->has_backing_file = true;
          hmp_printf(mon, "Failed to open file\n");
        info->backing_file = qemu_strdup(info->backing_file);
      } else if (strcmp(type, "IoError") == 0) {
    }
          hmp_printf(mon, "An I/O error occurred while writing to memory file\n");
      } else {
    return info;
          hmp_printf(mon, "Unexpected error: %s\n", qerror_to_str(err));
      }
    }
  }
  }


This also addresses how to decouple QMP from the monitor.  The individual QMP functions should be stand alone with a separate dispatch table.  A HMP function should be implemented mostly in terms of invoking the QMP function. In this case, a helper is used to make QMP invocation slightly easier.
== Enumeration Types ==
 
QAPI also supports enumeration types.  The following syntax is used:
 
{ 'VirtioNetTxStrategy': ['bh', 'timer'] }
 
A list of strings signifies a enumeration type.  The enumeration type will
generate the following code in qemu/qmp-types.h:
 
typedef enum VirtioNetTxStrategy
{
    VNTS_BH = 0,
    VNTS_TIMER,
} VirtioNetTxStrategy;
 
Any use of the type in QEMU or in libqmp will use the enumeration.  At the
moment, the integer values of the enumeration are sent over the wire in order to
better support languages like Python that don't have enumeration types.
 
String to enumeration conversion functions will also be generated.
 
// qemu/qmp-types.h
VirtioNetTxStrategy qmp_virtio_net_tx_strategy_from_str(const char *str,
                                                        Error **errp);
  char *qmp_virtio_net_tx_strategy_to_str(VirtioNetTxStrategy value,
                                        Error **errp);
 
== Client Library ==
 
A client library will also be generated that makes use of qmp-types.h.  The
client library functions are very similar to the QEMU functions except they
reside in libqmp.h, use a different prefix, and take a QmpSession argument.
For instance, 'query-version' will generate:


== Status ==
VersionInfo *libqmp_query_version(QmpSession *sess, Error **errp);


A very very rough sketch of this conversion is at the following location:
== QMP Server Discovery ==


* http://repo.or.cz/w/qemu/aliguori-queue.git qmp2
QAPI has a standard mechanism to discover QMP servers.  By default, a QMP
session is always created in ~/.qemu/name-$name.sock or ~/.qemu/pid-$pid.sock.
libqmp provides functions to enumerate the running guests and connect to a guest
by name or pid.

Revision as of 16:00, 15 February 2011

Summary

Introduce a number of changes to the internal implementation of QMP to simplify maintainability and enable features such as asynchronous command completion and rich error reporting. Minimize protocol-visible changes as much as possible. Provide a C library interface to QMP such that can be used to write thorough in-tree unit tests.

Owner

Summary

QMP has evolved from the human monitor code and has inherited a number of the deficiencies of the human monitor as a result. In particular, error reporting and asynchronous commands really demonstrate the historic flaws in the human monitor. This spec describes the types of changes that are needed to decouple QMP from the human monitor and fix the deficiencies that are currently present.

Implementation

QAPI uses a schema/IDL written in JSON to automatically generate the types and marshalling information for QMP commands. It is used to generate both the server side marshalling interfaces and a client library.

A typical function definition looks like this: 

##
# @change:
#
# This command is multiple commands multiplexed together.  Generally speaking,
# it should not be used in favor of the single purpose alternatives such as
# @change-vnc-listen, @change-vnc-password, and @change-blockdev.
#
# @device: This is normally the name of a block device but it may also be 'vnc'.
#          when it's 'vnc', then sub command depends on @target
#
# @target: If @device is a block device, then this is the new filename.
#          If @device is 'vnc', then if the value 'password' selects the vnc
#          change password command.   Otherwise, this specifies a new server URI
#          address to listen to for VNC connections.
#
# @arg:    If @device is a block device, then this is an optional format to open
#          the device with.
#          If @device is 'vnc' and @target is 'password', this is the new VNC
#          password to set.  If this argument is an empty string, then no future
#          logins will be allowed.
#
# Returns: Nothing on success.
#          If @device is not a valid block device, DeviceNotFound
#          If @format is not a valid block format, InvalidBlockFormat
#          If the new block device is encrypted, DeviceEncrypted.  Note that
#          if this error is returned, the device has been opened successfully
#          and an additional call to @block_passwd is required to set the
#          device's password.  The behavior of reads and writes to the block
#          device between when these calls are executed is undefined.
#
# Notes:  It is strongly recommended that this interface is not used especially
#         for changing block devices.
#
# Since: 0.14.0
##
[ 'change', {'device': 'str', 'target': 'str'}, {'arg': 'str'}, 'none' ]

The comments above the function are written in gtk-doc format and meant to be extracted to generate both protocol documentation and libqmp documentation.

The first element of the list is the command name, in this case, 'change'.

The second element of the list is a dictionary containing the required arguments for the function. The key is the argument name and the value is the type. The type can be a string or a complex type (see section on Typing).

The third element is a dictionary of optional arguments that follows the same rules as the required arguments.

The final list element is the return type of the function. 'none' is a special type representing a void return value.

QMP Server

This definition will generate the following signature in qemu/qmp.h: 

void qmp_change(const char *device, const char *target, bool has_arg,
                const char *arg, Error **errp);

Which is then implemented in the appropriate place in QEMU. The optional arguments always are prefixed with a boolean argument to indicate whether the option has been specified. The final argument is a pointer to an Error object in the fashion of GError. The caller of this function is will check for a non-NULL error object to determine if the function failed. errp can be set to NULL if the caller does not care about failure.

Currently, Error is roughly identical to QError with the exception that it only supports simple key/value arguments.

Complex Types

Some commands take or return complex types. It's usually a good idea to define complex types outside of the function definition. An example of a command that returns a complex type is below: 

##
# @VersionInfo:
#
# A description of QEMU's version.
#
# @qemu.major:  The major version of QEMU
#
# @qemu.minor:  The minor version of QEMU
#
# @qemu.micro:  The micro version of QEMU.  By current convention, a micro
#               version of 50 signifies a development branch.  A micro version
#               greater than or equal to 90 signifies a release candidate for
#               the next minor version.  A micro version of less than 50
#               signifies a stable release.
#
# @package:     QEMU will always set this field to an empty string.  Downstream
#               versions of QEMU should set this to a non-empty string.  The
#               exact format depends on the downstream however it highly
#               recommended that a unique name is used.
#
# Since: 0.14.0
##
{ 'VersionInfo': {'qemu': {'major': 'int', 'minor': 'int', 'micro': 'int'},
                  'package': 'str'} } 

##
# @query-version:
#
# Returns the current version of QEMU.
#
# Returns:  A @VersionInfo object describing the current version of QEMU.
#
# Since: 0.14.0
##
[ 'query-version', {}, {}, 'VersionInfo' ]

The syntax the use of a dictionary instead of a list is an indication of a typedef. Within QEMU, This will generate the following code in qemu/qmp-types.h: 

typedef struct VersionInfo
{
    struct {
        int64_t major;
        int64_t minor;
        int64_t micro;
    } qemu;
    const char *package;
    struct VersionInfo *next;
} VersionInfo;

The use of a next pointer is to enable support for returning lists of complex types. The query-version command will generate the following signature: 

// qemu/qmp-types.h
VersionInfo *qmp_alloc_version_info(void);
void qmp_free_version_info(VersionInfo *obj);

// qemu/qmp.h
VersionInfo *qmp_query_version(Error **errp);

A typical implementation might look something like: 

VersionInfo *qmp_query_version(Error **errp)
{
    VersionInfo *info = qmp_alloc_version_info();
    info->qemu.major = 0;
    info->qemu.minor = 14;
    info->qemu.micro = 92;
    info->package = qemu_strdup(" (qemu-kvm)");
    return info;
}

Optional Structure Members

Optional structure members can be specified by using a '*' as a prefix to the member name. For example: 

##
# @BlockDeviceInfo:
#
# Information about the backing device for a block device.
#
# @file: the filename of the backing device
#
# @ro: true if the backing device was open read-only
#
# @drv: the name of the block format used to open the backing dvice
#
# @backing_file: #optional the name of the backing file (for copy-on-write)
#
# @encrypted: true if the backing device is encrypted
#
# Since: 0.14.0
#
# Notes: This interface is only found in @BlockInfo.
##
{ 'BlockDeviceInfo': { 'file': 'str', 'ro': 'bool', 'drv': 'str',
                       '*backing_file': 'str', 'encrypted': 'bool' } }

A typical implementation may look like: 

BlockDeviceInfo *qmp_query_block_device_info(const char *device, Error **errp)
{
    BlockDeviceInfo *info;
    BlockDriverState *bs;
    Error *local_err = NULL;

    bs = bdrv_find(device, &local_err);
    if (local_err) {
        error_propagate(errp, local_err);
        return NULL;
    }

    info->file = qemu_strdup(bs->filename);
    info->ro = bs->readonly;
    info->drv = qemu_strdup(bs->drv);
    info->encrypted = bs->encrypted;
    if (bs->backing_file[0]) {
        info->has_backing_file = true;
        info->backing_file = qemu_strdup(info->backing_file);
    }

    return info;
}

Enumeration Types

QAPI also supports enumeration types. The following syntax is used: 

{ 'VirtioNetTxStrategy': ['bh', 'timer'] }

A list of strings signifies a enumeration type. The enumeration type will generate the following code in qemu/qmp-types.h: 

typedef enum VirtioNetTxStrategy
{
    VNTS_BH = 0,
    VNTS_TIMER,
} VirtioNetTxStrategy;

Any use of the type in QEMU or in libqmp will use the enumeration. At the moment, the integer values of the enumeration are sent over the wire in order to better support languages like Python that don't have enumeration types.

String to enumeration conversion functions will also be generated. 

// qemu/qmp-types.h
VirtioNetTxStrategy qmp_virtio_net_tx_strategy_from_str(const char *str,
                                                        Error **errp);
char *qmp_virtio_net_tx_strategy_to_str(VirtioNetTxStrategy value,
                                        Error **errp);

Client Library

A client library will also be generated that makes use of qmp-types.h. The client library functions are very similar to the QEMU functions except they reside in libqmp.h, use a different prefix, and take a QmpSession argument. For instance, 'query-version' will generate: 

VersionInfo *libqmp_query_version(QmpSession *sess, Error **errp);

QMP Server Discovery

QAPI has a standard mechanism to discover QMP servers. By default, a QMP session is always created in ~/.qemu/name-$name.sock or ~/.qemu/pid-$pid.sock. libqmp provides functions to enumerate the running guests and connect to a guest by name or pid.

Retrieved from "https://wiki.qemu.org/index.php?title=Features/QAPI&oldid=943"