Features/Modules

Summary

Introduce a mechanism to support loadable modules in QEMU similar to the Linux kernel. It should integrate into the existing build system. Currently, we allow features to be specified in for inclusion in a boolean fashion (either 'y' or 'n' to include or exclude). This proposal would add a third state to allow building as a loadable module ('m').

The most obvious reason why we may want such a feature is to allow downstreams to package different parts of qemu in separate binary packages, reducing runtime dependencies of the main package. For example, on a headless server there's no need to install sdl or gtk display types which require all the client-side X stack and all surrounding support packages.

Owner

• Name: Anthony Liguori
• Email: anthony@codemonkey.ws

What this is not

• A mechanism to support third party extensions to QEMU or out of tree drivers/features
• A stable interface
• A GPL barrier

This system should not be (ab)used to allow 3rd-party modules to be loaded into qemu, especially to "work around" GPL restrictions. In order to ensure this, the modules system should be built in a way to allow loading only modules which were built together with qemu, by adding, for example, hashes of current build to the main exported symbols.

Usage

The build system already allows features to be specified by setting appropriate flags in <target-subdir>/config-devices.mak. Initially, configuring modules would involve setting build options to 'm' instead of 'y' as below:

   # Default configuration for x86_64-softmmu
   
   CONFIG_VGA=y
   # Build QXL as a loadable module
   CONFIG_QXL=m
   CONFIG_VGA_PCI=y
   CONFIG_VGA_ISA=y
   CONFIG_VGA_CIRRUS=y
   # Build VMware devices as loadable modules
   CONFIG_VMWARE_VGA=m
   CONFIG_VMMOUSE=m
   CONFIG_SERIAL=y
   ...

The build system will install all generated modules in ${prefix}/lib/qemu/libqemu-<modulename>.so.

Loading Modules

All modules in ${prefix}/lib/qemu will be scanned at early startup and loaded. When loaded, the public function will be invoked to register the module init functions. Note that this must happens before any module init functions are invoked.

[mjt: there's no actual reason in scanning and loading everything at startup, but this leads to surprizes - for example, we are unlikely to be able to load some gui/display module from qemu-img which does not export the necessary gui-related functions. On the other hand, it is very convinient to try to load some thing from a module of the same name when that thing is requested but is not listed in a list of corresponding objects. For example, when we specified -hda qed:/some/where, and qed block format isn't registered, we may try to load ${prefix}/lib/qemu/qed.so and retry one more time.]

[clord: there's now a set of patches pending which allow modules to be loaded dynamically at runtime rather than being unconditionally loaded as mentioned above. Of course modules could still be loaded unconditionally at startup, but it's no longer the only option.]

QEMU must *not* dlclose the library, since that would unload it!

In the long term, we should also support a QMP/HMP command to load a module after startup. We will need to consider how module init ordering should work here. Any easy solution would be to force all modules to use type_init() and call all new MODULE_TYPE init functions any time we load a module.

Unloading Modules

While not strictly needed at first, we should consider implementing module unloading in the long run.

Rough implementation idea

Only devices that are compiled once can be modularized.

For a single-file module:

  module-obj-$(CONFIG_LSI_SCSI_PCI) += lsi53c895a.o

For a multi-file module (actually, one could use two single-file modules for this particular example):

  modules-$(CONFIG_XIO3130) += xio3130.so
  common-obj-$(CONFIG_XIO3130) += xio3130_upstream.o xio3130_downstream.o
  $(obj)/xio3130.so: $(obj)/xio3130_upstream.o $(obj)/xio3130_downstream.o

In Makefile.target:

  common-obj-y += $(module-obj-y)

In Makefile:

  # Add single-file modules to modules-m
  modules-m += $(patsubst %.o,%.so,$(module-obj-m))

  # Build modules
  all: $(modules-m)

  # Auto-create dependencies for single-file modules
  $(foreach M, $(module-obj-m), $(eval $(M:.o=.so): $M))

  # Rule for building modules
  $(modules-m): %.so:
         libtool --mode=link ...

  ifneq ($(obj-m),)
  $(error obj-m must be empty!)
  endif

Note that modules-y and common-obj-m are ignored.

Individual libs from libs_softmmu have to be split into their own variables, then:

  $(obj)/pulseaudio.so: LDFLAGS += $(LIBS_PULSEAUDIO)
  libs-$(CONFIG_PULSEAUDIO) += $(LIBS_PULSEAUDIO)

where again, libs-m is ignored.

Status

No code exists for this feature yet.

Another implementation idea

For single-file modules, the idea is the same, just listed in a different variable:

 obj-$(CONFIG_CURL) += curl.o

This goes to either $(obj-y), which should be linked into executable statically, or to $(obj-m), which will be built as modules. For each module listed in $(obj-m), a corresponding .so file will be generated.

Each module may specify their own libraries, using this syntax:

 $(obj)/curl.libs = -lcurl

so all the required libs will be linked either to executable or to the .so file.

For modules which consists of more than one file, we introduce a new fake file extension, .mod:

 obj-$(CONFIG_QED) += qed.mod
 $(obj)/qed.mod: qed-snapshot.o qed-gencb.o qed-table.o
 $(obj)/qed.libs = -lz

This will be expanded at link time into proper set of files/libs for either static or modular build.

When a module is specified using .mod construct like this, the .libs variable can be used only for the module itself, not for its individual objects. In the above example, specifying $(obj)/qed-table.libs will have no effect, only $(obj)/qed.libs will be used.

When we link things, we add all $(obj-y) (expanding .mod variables in there and adding corresponding .libs) to the link lines of corresponding executables, so things works like it is now.

And for every item listed in $(obj-m), we build a .so file -- either from a single .o file, or from the files from which $(foo.mod) depends.

When compiling .o files, we always use -fPIC -DPIC (or whatever), to always create position-independed code, -- this is good for both the modules (which require to be built as pic), and for the statically linked code too, -- in the latter case it lets the runtime linker to randomly relocate executable for security reasons. So we don't really need different compiler flags for modular/static builds, so don't need to distinguish them by the suffix and don't need to watch for rebuilds when modular/static status changes.

For symmetry, a syntax similar to $(obj)/foo.libs can be used to specify addidional CFLAGS:

 $(obj)/curl.cflags = -DNO_OLD_VERSION_SUPPORT

(which is supported for every .o file being compiled). Another way to specify the same is to use

 $(obj)/curl.o: CFLAGS += -DNO_OLD_VERSION_SUPPORT

which is used currently for a few objects.

Internal details

In Linux kbuild system, per-object variables are specified without the infamous $(obj)/ prefix. This is because kbuild makefiles are recursive, so that subdirs are processed by sub-makes, while qemu makefiles are read into one large makefile.

Due to the way how Makefile.objs fragments are processed (by including them recursively into the main makefile), $(obj) always has the "./" prefix. So when expanding *.mod, *.libs and *.cflags, we have to add this prefix before dereferencing a variable. This is a bit ugly.

Status

Some code for this exists in mjt's repository, http://git.corpit.ru/?p=qemu.git;a=shortlog;h=refs/heads/mjt-buildsys

Modularization Status

Status as of 2016-08 - Work has been done to modularize a few of the block drivers which link to external libraries. These are generally single file modules, for example:

 block-obj-$(CONFIG_LIBISCSI) += iscsi.o

Module libraries and flags can also be set:

 iscsi.o-cflags := $(LIBISCSI_CFLAGS)
 iscsi.o-libs   := $(LIBISCSI_LIBS)

Multi-file modules can be made as well, and are defined with the .mo suffix in makefiles, for example:

 common-obj-$(CONFIG_GTK) += gtk.mo
 
 gtk.mo-objs   := gtk.o gtk-egl.o
 gtk.mo-cflags := $(GTK_CFLAGS)
 gtk.mo-libs   := $(GTK_LIBS)

A number of block drivers have been modularized so far. This includes iscsi, nfs (pending approval), curl, rbd, gluster, archipelago, and ssh. Also pending approval, these will be loaded dynamically at runtime rather than unconditionally loaded at startup. The remaining block drivers were decided to be not good candidates for modularization due to the large amount of code movement and additional source/header files that it would create with no performance benefit since the remaining block drivers don't tend to link to external libraries.

Some attempts have been made to modularize GTK and SDL, but may need some serious work before being accepted (see patches for SDL and GTK modularization attempts).