Features/Documentation

QEMU's documentation needs some reorganization.

Ultimately there should be five manuals:

QEMU user mode emulation (docs/user, Features/Documentation/user): currently in qemu-doc.texi
QEMU full-system emulation user's guide (docs/system, Features/Documentation/system): Category:User documentation; the largest part of qemu-doc.texi; also covers qemu-img, qemu-io; docs/ (not subdirectories)
QEMU full-system emulation management and interoperability guide (docs/interop, Features/Documentation/interop): qemu-ga.texi; docs/interop/ (see this patch
QEMU full-system emulation guest hardware specifications (docs/specs, Features/Documentation/specs)
QEMU developer's guide (docs/devel, Features/Documentation/devel): Category:Developer documentation; docs/devel/ (see this patch; the implementation notes in qemu-doc.texi; tcg/README; doc comments in the source code; automatically generated docs for Python classes in qemu-iotests and scripts/qmp

Choices

Based on experience from the Linux kernel, QEMU's docs pipeline is going to be based on Sphinx (sample developer doc, sample user doc. Sphinx is extensible and it is easy to add new input formats and input directives.

Currently, QEMU documentation is written in a mix of Texinfo and text files roughly based on Markdown. Sphinx's native format is reStructuredText (rST). Texinfo input is not supported by Sphinx, but Paolo has a docutils (Sphinx-compatible) parser for Docbook; Texinfo is able to convert .texi input files into Docbook. This parser was used to produce the sample user doc above.

Currently, QEMU doc comments have never been actually used together with a actual documentation generator in mind, but they are roughly based on gtk-doc syntax. We will probably end up keeping a copy of the Linux kernel's kernel-doc script. kernel-doc's main supported backends are rST and Docbook.

As a first step, we should decide how to evolve this into something more structured.

For text, the two possible choices should ultimately be:

Convert everything to reStructuredText
Advantages: Flexible enough, native format for Sphinx, Linux is using it

Disadvantages: Less "obvious" than Markdown (looks somewhat weird when you use more advanced features, though a basic conversion seems to be simple enough), expect some hand editing after Texinfo->rST conversion (makeinfo --docbook + either Pandoc or db2rst.py).
Convert everything to Texinfo
Advantages: qemu-doc.texi and command-line doc comments are already written in Texinfo, QAPI automatically generated docs also use Texinfo in the current version of the patches; more traditional/"precise" markup

Disadvantages: Most different from basic ASCII text, markup not supported by kernel-doc

Even though docs/ is currently written in Markdown-like format and there is a Sphinx extension to parse Markdown, it's unlikely to be a third contender.

For C doc comments, the three possible choices are:

Use reStructuredText markup
Advantages: Flexible enough, native format for Sphinx, Linux is using it

Disadvantages: Would use rST for doc comments but not for the rest of docs/, or for command-line doc comments?
Extend kernel-doc to support Texinfo markup, and convert that to Docbook
Advantages: qemu-doc.texi and command-line doc comments are already written in Texinfo

Disadvantages: More work to do, Rube Goldberg would be proud
Use no markup, apart from annotating fields, functions etc. using sigils (e.g. kernel-doc's &struct Foo or %Foo).
Advantages: Simplest solution, can use any kernel-doc backend (DocBook and rST), doesn't conflict with chioce of format for docs/

Disadvantages: none?

We should also choose whether to place comments close to the definition or the declaration of functions. Linux seems to place comments close to the definition—the disadvantage being that struct comments must obviously be placed in headers.

Sample command-line invocation

The build-time workflow for Texinfo-based documentation is:

convert Texinfo to docbook4 using makeinfo
convert docbook4 to docbook5 using xsltproc
split the huge docbook output using xsltproc again

makeinfo --docbook -o qemu-doc.xml ../qemu-doc.texi
xsltproc db4-upgrade.xsl qemu-doc.xml | \
  xsltproc --stringparam chunk.section.depth 1 \
           --stringparam assembly.filename index.xml \
           --stringparam base.dir topics/ \
           /usr/share/sgml/docbook/xsl-ns-stylesheets-1.79.1/assembly/topic-maker-chunk.xsl

For docs/-based documentation, each .txt file becomes one .texi file, included by a "master" document with lines like

@chapter ACPI interface
@include acpi-cpu-hotplug.texi

@chapter Registries
@include pci-ids.texi

where each .texi file defines a @section.

Sphinx can invoke kernel-doc through the same extension that Linux is using (if using the rST backend) or a similar one (if using the docbook backend). The Texinfo markup would look like the following:

@macro kerneldoc{file}
@docbook
<sphinx:directive xmlns:sphinx="https://pypi.python.org/pypi/db4sphinx"
        sphinx:name="kernel-doc" sphinx:arg="\file\"/>
@end docbook
@end macro

@kerneldoc{include/exec/memory.h}

Things to do

For now in no particular order:

Prepare five wiki pages to work collaboratively on the manuals' table of contents, populate them with current qemu-doc.texi ToC and a list of docs/ files.
Split qemu-doc.texi so that the bulk of its contents is included from small files in docs/
Annotate developer documentation in docs/ with the corresponding C source code, so as to prioritize where to add doc comments

Things missing in the Sphinx docbook frontend:

needs to be tested against Marc-André's QAPI generator