Features/Modules

Summary

Introduce a mechanism to support loadable modules in QEMU similar to the Linux kernel. It should integrate into the existing build system. Currently, we allow features to be specified in for inclusion in a boolean fashion (either 'y' or 'n' to include or exclude). This proposal would add a third state to allow building as a loadable module ('m').

The most obvious reason why we may want such a feature is to allow downstreams to package different parts of qemu in separate binary packages, reducing runtime dependencies of the main package. For example, on a headless server there's no need to install sdl or gtk display types which require all the client-side X stack and all surrounding support packages.

Owner

Name: Anthony Liguori
Email: anthony@codemonkey.ws

What this is not

A mechanism to support third party extensions to QEMU or out of tree drivers/features
A stable interface
A GPL barrier

This system should not be (ab)used to allow 3rd-party modules to be loaded into qemu, especially to "work around" GPL restrictions. In order to ensure this, the modules system should be built in a way to allow loading only modules which were built together with qemu, by adding, for example, hashes of current build to the main exported symbols.

Usage

The build system already allows features to be specified by setting appropriate flags in <target-subdir>/config-devices.mak. Initially, configuring modules would involve setting build options to 'm' instead of 'y' as below:

   # Default configuration for x86_64-softmmu
   
   CONFIG_VGA=y
   # Build QXL as a loadable module
   CONFIG_QXL=m
   CONFIG_VGA_PCI=y
   CONFIG_VGA_ISA=y
   CONFIG_VGA_CIRRUS=y
   # Build VMware devices as loadable modules
   CONFIG_VMWARE_VGA=m
   CONFIG_VMMOUSE=m
   CONFIG_SERIAL=y
   ...

The build system will install all generated modules in ${prefix}/lib/qemu/libqemu-<modulename>.so. When built as a module, the module_init() function will be defined as a public function with a fixed name instead of a static function marked as a constructor.

Loading Modules

All modules in ${prefix}/lib/qemu will be scanned at early startup and loaded. When loaded, the public function will be invoked to register the module init functions. Note that this must happens before any module init functions are invoked.

[mjt: there's no actual reason in scanning and loading everything at startup, but this leads to surprizes - for example, we are unlikely to be able to load some gui/display module from qemu-img which does not export the necessary gui-related functions. On the other hand, it is very convinient to try to load some thing from a module of the same name when that thing is requested but is not listed in a list of corresponding objects. For example, when we specified -hda qed:/some/where, and qed block format isn't registered, we may try to load ${prefix}/lib/qemu/qed.so and retry one more time.]

QEMU must *not* dlclose the library, since that would unload it!

In the long term, we should also support a QMP/HMP command to load a module after startup. We will need to consider how module init ordering should work here. Any easy solution would be to force all modules to use type_init() and call all new MODULE_TYPE init functions any time we load a module.

Unloading Modules

While not strictly needed at first, we should consider implementing module unloading in the long run.

Rough implementation idea

Only devices that are compiled once can be modularized.

For a single-file module:

  module-obj-$(CONFIG_LSI_SCSI_PCI) += lsi53c895a.o

For a multi-file module (actually, one could use two single-file modules for this particular example):

  modules-$(CONFIG_XIO3130) += xio3130.so
  common-obj-$(CONFIG_XIO3130) += xio3130_upstream.o xio3130_downstream.o
  $(obj)/xio3130.so: $(obj)/xio3130_upstream.o $(obj)/xio3130_downstream.o

In Makefile.target:

  common-obj-y += $(module-obj-y)

In Makefile:

  # Add single-file modules to modules-m
  modules-m += $(patsubst %.o,%.so,$(module-obj-m))

  # Build modules
  all: $(modules-m)

  # Auto-create dependencies for single-file modules
  $(foreach M, $(module-obj-m), $(eval $(M:.o=.so): $M))

  # Rule for building modules
  $(modules-m): %.so:
         libtool --mode=link ...

  ifneq ($(obj-m),)
  $(error obj-m must be empty!)
  endif

Note that modules-y and common-obj-m are ignored.

Individual libs from libs_softmmu have to be split into their own variables, then:

  $(obj)/pulseaudio.so: LDFLAGS += $(LIBS_PULSEAUDIO)
  libs-$(CONFIG_PULSEAUDIO) += $(LIBS_PULSEAUDIO)

where again, libs-m is ignored.

Status

No code exists for this feature yet.

Another implementation idea

For single-file modules, the idea is the same, just listed in a different variable:

 obj-$(CONFIG_CURL) += curl.o

This goes to either $(obj-y), which should be linked into executable statically, or to $(obj-m), which will be built as modules. For each module listed in $(obj-m), a corresponding .so file will be generated.

Each module may specify their own libraries, using this syntax:

 $(obj)/curl.libs = -lcurl

so all the required libs will be lined to either executable, or to the .so file.

For modules which consists of more than one file, we introduce a new fake file extension, .mod:

 obj-$(CONFIG_QED) += qed.mod
 $(obj)/qed.mod: qed-snapshot.o qed-gencb.o qed-table.o
 $(obj)/qed.libs = -lz

This will be expanded at link time into proper set of files/libs for either static or modular build.

When a module is specified using .mod construct like this, the .libs variable can be used only for the module itself, not for its individual objects. In the above example, specifying $(obj)/qed-table.libs will have no effect, only $(obj)/qed.libs will be used.

For symmetry, similar syntax can be used to specify addidional CFLAGS:

 $(obj)/curl.cflags = -DNO_OLD_VERSION_SUPPORT

(which is supported for every .o file being compiled).

Status

Some code for this exists in mjt's repository, http://git.corpit.ru/?p=qemu.git;a=shortlog;h=refs/heads/mjt-buildsys