Features/Documentation: Difference between revisions

From QEMU
Line 66: Line 66:
* split the huge docbook output using [http://doccookbook.sourceforge.net/html/en/dbc.structure.splitting-into-topics.html xsltproc again]
* split the huge docbook output using [http://doccookbook.sourceforge.net/html/en/dbc.structure.splitting-into-topics.html xsltproc again]


  makeinfo --docbook -o - ../qemu-doc.texi | \
  makeinfo --docbook -o qemu-doc.xml ../qemu-doc.texi
  xsltproc db4-upgrade.xsl - | \
xsltproc db4-upgrade.xsl qemu-doc.xml | \
   xsltproc --stringparam chunk.section.depth 0 \
   xsltproc --stringparam chunk.section.depth 1 \
             --stringparam assembly.filename index.xml \
             --stringparam assembly.filename index.xml \
             --stringparam base.dir topics/ \
             --stringparam base.dir topics/ \
Line 75: Line 75:
For docs/-based documentation, each .txt file becomes one .texi file, included by a "master" document with lines like
For docs/-based documentation, each .txt file becomes one .texi file, included by a "master" document with lines like


  @title ACPI CPU hotplug
  @chapter ACPI interface
  @include acpi-cpu-hotplug.texi
  @include acpi-cpu-hotplug.texi
  @title PCI id assignment
 
  @chapter Registries
  @include pci-ids.texi
  @include pci-ids.texi
where each .texi file defines a @section.


Sphinx can invoke kernel-doc through the same extension that Linux is using (if using the rST backend) or a similar one (if using the docbook backend).  The Texinfo markup would look like the following:
Sphinx can invoke kernel-doc through the same extension that Linux is using (if using the rST backend) or a similar one (if using the docbook backend).  The Texinfo markup would look like the following:

Revision as of 16:51, 3 January 2017

QEMU's documentation needs some reorganization.

Ultimately there should be five manuals:

QEMU user mode emulation (docs/qemu-user, Features/Documentation/qemu-user)
currently in qemu-doc.texi
QEMU full-system emulation user's guide (docs/qemu-system, Features/Documentation/qemu-system)
Category:User documentation
the largest part of qemu-doc.texi
also covers qemu-img, qemu-io
parts of docs/
QEMU full-system emulation management guide (docs/qemu-system-mgmt, Features/Documentation/qemu-system-mgmt)
qmp-commands.txt
qemu-ga.texi
more pieces of docs/
docs/specs/vhost-user.txt
probably should include file format specs in docs/specs/
QEMU full-system emulation guest hardware specifications (docs/qemu-system-hw, Features/Documentation/qemu-system-hw)
currently in docs/specs/
QEMU developer's guide (docs/qemu-devel, Features/Documentation/qemu-devel)
Category:Developer documentation
the rest of docs/
the implementation notes in qemu-doc.texi
tcg/README
doc comments in the source code
automatically generated docs for Python classes in qemu-iotests and scripts/qmp

Choices

Based on experience from the Linux kernel, QEMU's docs pipeline is going to be based on Sphinx (sample developer doc, sample user doc. Sphinx is extensible and it is easy to add new input formats and input directives.

Currently, QEMU documentation is written in a mix of Texinfo and text files roughly based on Markdown. Sphinx's native format is reStructuredText (rST). Texinfo input is not supported by Sphinx, but Paolo has a docutils (Sphinx-compatible) parser for Docbook; Texinfo is able to convert .texi input files into Docbook. This parser was used to produce the sample user doc above.

Currently, QEMU doc comments have never been actually used together with a actual documentation generator in mind, but are roughly based on gtk-doc syntax. We will probably end up keeping a copy of the Linux kernel's kernel-doc script. kernel-doc currently supports rST and Docbook output.

As a first step, we should decide how to evolve this into something more structured.

For text, the two possible choices should ultimately be:

  1. Convert everything to reStructuredText
    Advantages: Flexible enough, native format for Sphinx, Linux is using it
    Disadvantages: Less "obvious" than Markdown (though a basic conversion seems to be simple enough), looks somewhat weird when you use more advanced features, expect some hand editing after Texinfo conversion (makeinfo --docbook + either Pandoc or db2rst.py).
  2. Convert everything to Texinfo
    Advantages: qemu-doc.texi and command-line doc comments are already written in Texinfo, QAPI automatically generated docs also use Texinfo in the current version of the patches; more traditional/"precise" markup
    Disadvantages: Most different from basic ASCII text, markup not supported by kernel-doc

Markdown (i.e. Sphinx's reCommonMark extension) is not going to be used.

For C doc comments, the three possible choices are:

  1. Use reStructuredText markup
    Advantages: Flexible enough, native format for Sphinx, Linux is using it
    Disadvantages: Would use rST for doc comments but not for the rest of docs/, or for command-line doc comments?
  2. Extend kernel-doc to support Texinfo markup, and convert that to Docbook
    Advantages: qemu-doc.texi and command-line doc comments are already written in Texinfo
    Disadvantages: More work to do, Rube Goldberg would be proud
  3. Use no markup, apart from annotating fields, functions etc. using sigils (e.g. kernel-doc's &struct Foo or %Foo).
    Advantages: Simplest solution, can use any kernel-doc backend (DocBook and rST), doesn't conflict with chioce of format for docs/
    Disadvantages: none?

We should also choose whether to place comments close to the definition or the declaration of functions. Linux seems to place comments close to the definition—the disadvantage being that struct comments must obviously be placed in headers.

Sample command-line invocation

The build-time workflow for Texinfo-based documentation is:

  • convert Texinfo to docbook4 using makeinfo
  • convert docbook4 to docbook5 using xsltproc
  • split the huge docbook output using xsltproc again
makeinfo --docbook -o qemu-doc.xml ../qemu-doc.texi
xsltproc db4-upgrade.xsl qemu-doc.xml | \
  xsltproc --stringparam chunk.section.depth 1 \
           --stringparam assembly.filename index.xml \
           --stringparam base.dir topics/ \
           /usr/share/sgml/docbook/xsl-ns-stylesheets-1.79.1/assembly/topic-maker-chunk.xsl

For docs/-based documentation, each .txt file becomes one .texi file, included by a "master" document with lines like

@chapter ACPI interface
@include acpi-cpu-hotplug.texi
@chapter Registries
@include pci-ids.texi

where each .texi file defines a @section.

Sphinx can invoke kernel-doc through the same extension that Linux is using (if using the rST backend) or a similar one (if using the docbook backend). The Texinfo markup would look like the following:

@macro kerneldoc{file}
@docbook
<?rst kernel-doc: \file\ ?>
@end docbook
@end macro
@kerneldoc{include/exec/memory.h}

(not yet implemented).

Things to do

For now in no particular order:

  • Merge Marc-André's patches to automatically generate QAPI documentation
  • Prepare five wiki pages to work collaboratively on the manuals' table of contents, populate them with current qemu-doc.texi ToC and a list of docs/ files.
  • Split qemu-doc.texi so that the bulk of its contents is included from small files in docs/
  • Annotate developer documentation in docs/ with the corresponding C source code, so as to prioritize where to add doc comments

Things missing in the Sphinx docbook frontend:

  • support for docbook5 assembly (convert them to Sphinx toctree)