Features/qtest driver framework: Difference between revisions

From QEMU
No edit summary
No edit summary
Line 14: Line 14:


   (machine)          (driver)            (driver)          (interface)      (driver)
   (machine)          (driver)            (driver)          (interface)      (driver)
      pc  ────────→ i440FX-host ───────→ QPCIBusPC  ───────→ QPCIBus ───────→ QSDHCI_PCI
  x86_64/pc  ────────→ i440FX-host ───────→ QPCIBusPC  ───────→ QPCIBus ───────→ QSDHCI_PCI
          contains            contains            produces        consumed by  │ produces
              contains            contains            produces        consumed by  │ produces
                                                                                 
                                                                                   
                                                                                 
                                                                                   
                                                        (driver)            (interface)
                                                            (driver)            (interface)
                                                        QSDHCI_MM ────────────→ QSDHCI
                                                            QSDHCI_MM ────────────→ QSDHCI
                                                        produces                 │ consumed by
                                                            produces                 │ consumed by
                                                                                 
                                                                                   
                                                                              register-test
                                                                                  register-test


The above is a subset in a potentially very large graph. The relations are hard-coded in the classes and in the tests, and even include some unused classes such as QSDHCI_MM (a memory-mapped SD-HCI device).
The above is a subset of a potentially very large graph. The relations are hard-coded and even include some unused classes such as QSDHCI_MM (a memory-mapped SD-HCI device).  That's okay: if there is no incoming edge, simply there will be no way to test the device.  Another machine can add the required node and then QSDHCI_MM will be tested too.


Let's look at the path from pc to register-test.  register-test is a unit test for the SD-HCI device and it would add itself to the graph like this:
After the graph is built, tests can be discovered by walking the graph: each test to run corresponds to a path from the root of the graphs to a unit test. Let's look at the path from pc to register-test, a unit test for the SD-HCI device.  The above path could represent a test like "/x86_64/pc/i440FX-host/pcibus-pci/sdhci-pci/sdhci/registers-test":
 
  qos_add_test("register-test", "sdhci", sdhci_register_test_func);
                    │            │            ╰─────── Function invoked to the run test
                    │            ╰──────────────────── Interface consumed by the test
                    ╰────────────────────────────────── Test name
 
and likewise all the edges in the graph would be hard coded:
 
  pc = qos_node_create_machine("x86_64", "pc", qos_create_machine_x86_pc);
  qos_node_contains(pc, "i440FX-host");
 
  qos_node_create_driver("i440FX-host");
  qos_node_create_driver("pci-bus-pc");
  qos_node_contains("i440FX-host", "pci-bus-pc");
  qos_node_create_interface("pci-bus");
  qos_node_produces("pci-bus-pc", "pci-bus");
  qos_node_create_driver("sdhci-pci");
  qos_node_consumes("sdhci-pci", "pci-bus");  // edge from pci-bus to sdhci-pci
  qos_node_produces("sdhci-pci", "sdhci");
  qos_node_create_driver("sdhci-mm");
  qos_node_produces("sdhci-mm", "sdhci");
 
Here is how another machine could be added:
 
  raspi3 = qos_node_create_machine("arm", "raspi3", qos_create_machine_arm_raspi3);
  qos_node_contains(raspi3, "sdhci-mm");
 
After the graph is built, tests can be discovered by walking the graph: each test to run corresponds to a path from the root of the graphs to a unit test. The above path could represent a test like "/x86_64/pc/i440FX-host/pcibus-pci/sdhci-pci/sdhci/registers-test":


   /x86_64/pc/i440FX-host/pcibus-pc/pci-bus/sdhci-pci/sdhci/registers-test
   /x86_64/pc/i440FX-host/pcibus-pc/pci-bus/sdhci-pci/sdhci/registers-test
Line 79: Line 51:


At run-time, a lot of steps are hidden in the framework, and are realized by walking the path:
At run-time, a lot of steps are hidden in the framework, and are realized by walking the path:
* QOSMachine walks the path to the device under test
* All the drivers and tests register themselves into the graph.
** the machine itself adds "-M pc" to the command line
* libqos runs QEMU to detect machine types
** QPCIBusPC need not add anything to the command line because the device is embedded
* libqos takes machine types that it knows about and adds them to the graph.
** QSDHCI_PCI adds "-device sdhci-pci" to the command line
* libqos walks the graph and registers a testcase for each path (from machine to test).
* QOSMachine starts QEMU
* for each test:
* QOSMachine finally executes the code for the unit test
** libqos walks the path and builds the QEMU command line
** The test asks QSDHCI_PCI to start the device
** libqos starts QEMU
*** QSDHCI_PCI sets up the PCI device using the methods of QPCIBus
** libqos creates the machine object
** The test runs and tests the SDHCI device
** starting from the machine object, libqos walks the path to obtain the object needed for the test function
** libqos passes the interface to the test function
 
When register-test runs, for example, the command line is built like this:
* the machine itself adds "-M pc" to the command line
* QPCIBusPC need not add anything to the command line because the device is embedded
* QSDHCI_PCI adds "-device sdhci-pci" to the command line
 
And then when the test function runs:
* It asks QSDHCI_PCI to start the device
** QSDHCI_PCI sets up the PCI device using the methods of QPCIBus
* The function tests the SDHCI device
 
== Creating the graph ==
 
The test would add itself to the graph like this:
 
  qos_add_test("register-test", "sdhci", sdhci_register_test_func);
                    │            │            ╰─────── Function invoked to the run test
                    │            ╰──────────────────── Interface consumed by the test
                    ╰────────────────────────────────── Test name
 
and likewise all the edges in the graph would be hard coded:
 
  qos_node_create_machine("x86_64/pc", qos_create_machine_x86_pc);
  qos_node_contains("x86_64/pc", "i440FX-host");
  qos_node_create_driver("i440FX-host");
  qos_node_create_driver("pci-bus-pc");
  qos_node_contains("i440FX-host", "pci-bus-pc");
  qos_node_create_interface("pci-bus");
  qos_node_produces("pci-bus-pc", "pci-bus");
  qos_node_create_driver("sdhci-pci");
  qos_node_consumes("sdhci-pci", "pci-bus");  // "consumed by" edge from pci-bus to sdhci-pci
  qos_node_produces("sdhci-pci", "sdhci");
  qos_node_create_driver("sdhci-mm");
  qos_node_produces("sdhci-mm", "sdhci");
 
Here is how another machine could be added:
 
  qos_node_create_machine("arm/raspi", qos_create_machine_arm_raspi3);
  qos_node_contains(raspi3, "sdhci-mm");
 


== Possible milestones ==
== Possible milestones ==

Revision as of 22:36, 21 March 2018

Through a driver framework, libqos can expose a description of QEMU's supported machine types and a set of drivers; unit tests can request a driver, and the framework takes care of starting QEMU with options that provide that driver.

For example, the framework could provide:

  1. an interface for SD-HCI (SD Host Controller Interface) devices (abstract class QSDHCI)
  2. a driver for PCI SD-HCI devices (a subclass of QSDHCI, for example QSDHCI_PCI)
  3. a description of how QEMU's "sdhci-pci" device maps to QSDHCI_PCI (for simplicity this can be part of QSDHCI_PCI)
  4. an interface for a PCI bus (abstract class QPCIBus)
  5. a driver for the PCI bus in an x86 PC machine (a subclass of QPCIBus, for example QPCIBusPC)
  6. a description of QEMU's "pc" machine (a subclass of QOSMachine) and its embedded devices (in this case a QPCIBusPC)

(Right now libqos provides items 4 and 5 only).

You can construct a graph where the nodes are interfaces, drivers and unit tests, connected by relations such as "X produces Y" or "X consumes Y":

 (machine)           (driver)             (driver)          (interface)       (driver)
 x86_64/pc  ────────→ i440FX-host ───────→ QPCIBusPC  ───────→ QPCIBus ───────→ QSDHCI_PCI
             contains             contains            produces        consumed by   │ produces
                                                                                    │
                                                                                    ↓
                                                           (driver)             (interface)
                                                           QSDHCI_MM ────────────→ QSDHCI
                                                           produces                 │ consumed by
                                                                                    ↓
                                                                                 register-test

The above is a subset of a potentially very large graph. The relations are hard-coded and even include some unused classes such as QSDHCI_MM (a memory-mapped SD-HCI device). That's okay: if there is no incoming edge, simply there will be no way to test the device. Another machine can add the required node and then QSDHCI_MM will be tested too.

After the graph is built, tests can be discovered by walking the graph: each test to run corresponds to a path from the root of the graphs to a unit test. Let's look at the path from pc to register-test, a unit test for the SD-HCI device. The above path could represent a test like "/x86_64/pc/i440FX-host/pcibus-pci/sdhci-pci/sdhci/registers-test":

  /x86_64/pc/i440FX-host/pcibus-pc/pci-bus/sdhci-pci/sdhci/registers-test
        │  │       │          │         │     │      │       ╰────── Test name
        │  │       │          │         │     │      ╰────────────── Interface provided by "sdhci-pci"
        │  │       │          │         │     ╰───────────────────── Device on the PCI bus
        │  │       │          │         ╰─────────────────────────── Interface provided by "pcibus-pc"
        │  │       │          ╰───────────────────────────────────── Driver provided by "i440FX-host"
        │  │       ╰──────────────────────────────────────────────── Device embedded by "pc"
        │  ╰──────────────────────────────────────────────────────── QEMU machine
        ╰─────────────────────────────────────────────────────────── Target architecture

or reading it in the other direction (right to left):

  /x86_64/pc/i440FX-host/pcibus-pc/pci-bus/sdhci-pci/sdhci/registers-test
        │  │       │          │       │      │        │       ╰────── Test name
        │  │       │          │       │      │        ╰────────────── Driver or interface consumed by the test
        │  │       │          │       │      ╰─────────────────────── Driver providing the "sdhci" interface
        │  │       │          │       ╰────────────────────────────── Interface consumed by "sdhci-pci"
        │  │       │          ╰────────────────────────────────────── Driver providing the "pcibus" interface
        │  │       ╰───────────────────────────────────────────────── Driver embedding a "pcibus-pc" device
        │  ╰───────────────────────────────────────────────────────── QEMU machine embedding an "i440FX-host" device
        ╰──────────────────────────────────────────────────────────── Target architecture

At run-time, a lot of steps are hidden in the framework, and are realized by walking the path:

  • All the drivers and tests register themselves into the graph.
  • libqos runs QEMU to detect machine types
  • libqos takes machine types that it knows about and adds them to the graph.
  • libqos walks the graph and registers a testcase for each path (from machine to test).
  • for each test:
    • libqos walks the path and builds the QEMU command line
    • libqos starts QEMU
    • libqos creates the machine object
    • starting from the machine object, libqos walks the path to obtain the object needed for the test function
    • libqos passes the interface to the test function

When register-test runs, for example, the command line is built like this:

  • the machine itself adds "-M pc" to the command line
  • QPCIBusPC need not add anything to the command line because the device is embedded
  • QSDHCI_PCI adds "-device sdhci-pci" to the command line

And then when the test function runs:

  • It asks QSDHCI_PCI to start the device
    • QSDHCI_PCI sets up the PCI device using the methods of QPCIBus
  • The function tests the SDHCI device

Creating the graph

The test would add itself to the graph like this:

 qos_add_test("register-test", "sdhci", sdhci_register_test_func);
                    │             │            ╰─────── Function invoked to the run test
                    │             ╰──────────────────── Interface consumed by the test
                    ╰────────────────────────────────── Test name

and likewise all the edges in the graph would be hard coded:

 qos_node_create_machine("x86_64/pc", qos_create_machine_x86_pc);
 qos_node_contains("x86_64/pc", "i440FX-host");
 qos_node_create_driver("i440FX-host");
 qos_node_create_driver("pci-bus-pc");
 qos_node_contains("i440FX-host", "pci-bus-pc");
 qos_node_create_interface("pci-bus");
 qos_node_produces("pci-bus-pc", "pci-bus");
 qos_node_create_driver("sdhci-pci");
 qos_node_consumes("sdhci-pci", "pci-bus");   // "consumed by" edge from pci-bus to sdhci-pci
 qos_node_produces("sdhci-pci", "sdhci");
 qos_node_create_driver("sdhci-mm");
 qos_node_produces("sdhci-mm", "sdhci");

Here is how another machine could be added:

 qos_node_create_machine("arm/raspi", qos_create_machine_arm_raspi3);
 qos_node_contains(raspi3, "sdhci-mm");


Possible milestones

  1. Create graph framework and write unit tests
  2. Convert all PC-based PCI device tests to use the graph framework
  3. Convert all PCI device tests to use the graph framework, so that all PC-based PCI device tests can work on more than one machine
  4. Create more drivers (e.g. SD-HCI)