Features/Documentation

QEMU's documentation needs some reorganization.

Ultimately there should be five manuals:

QEMU user mode emulation (docs/qemu-user, Features/Documentation/qemu-user)

currently in qemu-doc.texi

QEMU full-system emulation user's guide (docs/qemu-system, Features/Documentation/qemu-system)

Category:User documentation

the largest part of qemu-doc.texi

also covers qemu-img, qemu-io

parts of docs/

QEMU full-system emulation management guide (docs/qemu-system-mgmt, Features/Documentation/qemu-system-mgmt)

qmp-commands.txt

qemu-ga.texi

more pieces of docs/

docs/specs/vhost-user.txt

probably should include file format specs in docs/specs/

QEMU full-system emulation guest hardware specifications (docs/qemu-system-hw, Features/Documentation/qemu-system-hw)

currently in docs/specs/

QEMU developer's guide (docs/qemu-devel, Features/Documentation/qemu-devel)

Category:Developer documentation

the rest of docs/

the implementation notes in qemu-doc.texi

tcg/README

doc comments in the source code

automatically generated docs for Python classes in qemu-iotests and scripts/qmp

Choices

Based on experience from the Linux kernel, QEMU's docs pipeline is going to be based on Sphinx (sample developer doc, sample user doc. Sphinx is extensible and it is easy to add new input formats and input directives.

Currently, QEMU documentation is written in a mix of Texinfo and text files roughly based on Markdown. Sphinx's native format is reStructuredText (rST). Texinfo input is not supported by Sphinx, but Paolo has a docutils (Sphinx-compatible) parser for Docbook; Texinfo is able to convert .texi input files into Docbook. This parser was used to produce the sample user doc above.

Currently, QEMU doc comments have never been actually used together with a actual documentation generator in mind, but are roughly based on gtk-doc syntax. We will probably end up keeping a copy of the Linux kernel's kernel-doc script. kernel-doc currently supports rST and Docbook output.

As a first step, we should decide how to evolve this into something more structured.

For text, the two possible choices should ultimately be:

1. Convert everything to reStructuredText

   Advantages: Flexible enough, native format for Sphinx, Linux is using it

   Disadvantages: Less "obvious" than Markdown (though a basic conversion seems to be simple enough), looks somewhat weird when you use more advanced features, expect some hand editing after Texinfo conversion (makeinfo --docbook + either Pandoc or db2rst.py).

2. Convert everything to Texinfo

   Advantages: qemu-doc.texi and command-line doc comments are already written in Texinfo, QAPI automatically generated docs also use Texinfo in the current version of the patches; more traditional/"precise" markup

   Disadvantages: Most different from basic ASCII text, markup not supported by kernel-doc

Markdown (i.e. Sphinx's reCommonMark extension) is not going to be used.

For C doc comments, the three possible choices are:

1. Use reStructuredText markup

   Advantages: Flexible enough, native format for Sphinx, Linux is using it

   Disadvantages: Would use rST for doc comments but not for the rest of docs/, or for command-line doc comments?

2. Extend kernel-doc to support Texinfo markup, and convert that to Docbook

   Advantages: qemu-doc.texi and command-line doc comments are already written in Texinfo

   Disadvantages: More work to do, Rube Goldberg would be proud

3. Use no markup, apart from annotating fields, functions etc. using sigils (e.g. kernel-doc's &struct Foo or %Foo).

   Advantages: Simplest solution, can use any kernel-doc backend (DocBook and rST), doesn't conflict with chioce of format for docs/

   Disadvantages: none?

We should also choose whether to place comments close to the definition or the declaration of functions. Linux seems to place comments close to the definition—the disadvantage being that struct comments must obviously be placed in headers.

Sample command-line invocation

The build-time workflow for Texinfo-based documentation is:

• convert Texinfo to docbook4 using makeinfo
• convert docbook4 to docbook5 using xsltproc
• split the huge docbook output using xsltproc again

makeinfo --docbook -o qemu-doc.xml ../qemu-doc.texi
xsltproc db4-upgrade.xsl qemu-doc.xml | \
  xsltproc --stringparam chunk.section.depth 1 \
           --stringparam assembly.filename index.xml \
           --stringparam base.dir topics/ \
           /usr/share/sgml/docbook/xsl-ns-stylesheets-1.79.1/assembly/topic-maker-chunk.xsl

For docs/-based documentation, each .txt file becomes one .texi file, included by a "master" document with lines like

@chapter ACPI interface
@include acpi-cpu-hotplug.texi

@chapter Registries
@include pci-ids.texi

where each .texi file defines a @section.

Sphinx can invoke kernel-doc through the same extension that Linux is using (if using the rST backend) or a similar one (if using the docbook backend). The Texinfo markup would look like the following:

@macro kerneldoc{file}
@docbook
<?rst kernel-doc: \file\ ?>
@end docbook
@end macro

@kerneldoc{include/exec/memory.h}

(not yet implemented).

Things to do

For now in no particular order:

• Merge Marc-André's patches to automatically generate QAPI documentation
• Prepare five wiki pages to work collaboratively on the manuals' table of contents, populate them with current qemu-doc.texi ToC and a list of docs/ files.
• Split qemu-doc.texi so that the bulk of its contents is included from small files in docs/
• Annotate developer documentation in docs/ with the corresponding C source code, so as to prioritize where to add doc comments

Things missing in the Sphinx docbook frontend:

• support for rst and kernel-doc
• needs to be tested against Marc-André's QAPI generator