Features/qtest driver framework
Status: qgraph was merged in QEMU 4.0
Through a driver framework, libqos can expose a description of QEMU's supported machine types and a set of drivers; unit tests can request a driver, and the framework takes care of starting QEMU with options that provide that driver.
For example, the framework could provide:
- an interface for SD-HCI (SD Host Controller Interface) devices (abstract class QSDHCI)
- a driver for PCI SD-HCI devices (a subclass of QSDHCI, for example QSDHCI_PCI)
- a description of how QEMU's "sdhci-pci" device maps to QSDHCI_PCI (for simplicity this can be part of QSDHCI_PCI)
- an interface for a PCI bus (abstract class QPCIBus)
- a driver for the PCI bus in an x86 PC machine (a subclass of QPCIBus, for example QPCIBusPC)
- a description of QEMU's "pc" machine (a subclass of QOSMachine) and its embedded devices (in this case a QPCIBusPC)
(Right now libqos provides items 4 and 5 only).
You can construct a graph where the nodes are interfaces, drivers and unit tests, connected by relations such as "X produces Y" or "X consumes Y":
(machine) (driver) (driver) (interface) (driver)
x86_64/pc ────────→ i440FX-pcihost ───────→ QPCIBusPC ───────→ QPCIBus ───────→ QSDHCI_PCI
contains contains produces consumed by │ produces
│
↓
(driver) (interface)
QSDHCI_MM ────────────→ QSDHCI
produces │ consumed by
↓
register-test
The above is a subset of a potentially very large graph. The relations are hard-coded and even include some unused classes such as QSDHCI_MM (a memory-mapped SD-HCI device). That's okay: if there is no incoming edge, simply there will be no way to test the device. Another machine can add the required node and then QSDHCI_MM will be tested too.
After the graph is built, tests can be discovered by walking the graph: each test to run corresponds to a path from the root of the graphs to a unit test. Let's look at the path from pc to register-test, a unit test for the SD-HCI device. The above path could represent a test like "/x86_64/pc/i440FX-pcihost/pcibus-pci/sdhci-pci/sdhci/registers-test":
/x86_64/pc/i440FX-pcihost/pcibus-pc/pci-bus/sdhci-pci/sdhci/registers-test
│ │ │ │ │ │ │ ╰────── Test name
│ │ │ │ │ │ ╰────────────── Interface provided by "sdhci-pci"
│ │ │ │ │ ╰───────────────────── Device on the PCI bus
│ │ │ │ ╰─────────────────────────── Interface provided by "pcibus-pc"
│ │ │ ╰───────────────────────────────────── Driver provided by "i440FX-pcihost"
│ │ ╰─────────────────────────────────────────────────── Device embedded by "pc"
│ ╰─────────────────────────────────────────────────────────── QEMU machine
╰────────────────────────────────────────────────────────────── Target architecture
or reading it in the other direction (right to left):
/x86_64/pc/i440FX-pcihost/pcibus-pc/pci-bus/sdhci-pci/sdhci/registers-test
│ │ │ │ │ │ │ ╰────── Test name
│ │ │ │ │ │ ╰────────────── Driver or interface consumed by the test
│ │ │ │ │ ╰─────────────────────── Driver providing the "sdhci" interface
│ │ │ │ ╰────────────────────────────── Interface consumed by "sdhci-pci"
│ │ │ ╰────────────────────────────────────── Driver providing the "pcibus" interface
│ │ ╰──────────────────────────────────────────────────── Driver embedding a "pcibus-pc" device
│ ╰──────────────────────────────────────────────────────────── QEMU machine embedding an "i440FX-pcihost" device
╰─────────────────────────────────────────────────────────────── Target architecture
At run-time, a lot of steps are hidden in the framework, and are realized by walking the path:
- All the drivers and tests register themselves into the graph.
- libqos runs QEMU to detect machine types
- libqos takes machine types that it knows about and adds them to the graph.
- libqos walks the graph and registers a testcase for each path (from machine to test).
- for each test:
- libqos walks the path and builds the QEMU command line
- libqos starts QEMU
- libqos creates the machine object
- starting from the machine object, libqos walks the path to obtain the object needed for the test function
- libqos passes the interface to the test function
When register-test runs, for example, the command line is built like this:
- the machine itself adds "-M pc" to the command line
- QPCIBusPC need not add anything to the command line because the device is embedded
- QSDHCI_PCI adds "-device sdhci-pci" to the command line
And then when the test function runs:
- It asks QSDHCI_PCI to start the device
- QSDHCI_PCI sets up the PCI device using the methods of QPCIBus
- The function tests the SDHCI device
Creating the graph
The test would add itself to the graph like this:
qos_add_test("register-test", "sdhci", sdhci_register_test_func);
│ │ ╰─────── Function invoked to the run test
│ ╰──────────────────── Interface consumed by the test
╰────────────────────────────────── Test name
and likewise all the edges in the graph would be hard coded:
qos_node_create_machine("x86_64/pc", qos_create_machine_x86_pc);
qos_node_contains("x86_64/pc", "i440FX-pcihost");
qos_node_create_driver("i440FX-pcihost", NULL); // created via "contains" only
qos_node_create_driver("pci-bus-pc", NULL);
qos_node_contains("i440FX-pcihost", "pci-bus-pc");
qos_node_create_interface("pci-bus");
qos_node_produces("pci-bus-pc", "pci-bus");
qos_node_create_driver("sdhci-pci", sdhci_pci_create);
qos_node_consumes("sdhci-pci", "pci-bus"); // "consumed by" edge from pci-bus to sdhci-pci
qos_node_produces("sdhci-pci", "sdhci");
qos_node_create_driver("generic-sdhci", NULL);
qos_node_produces("generic-sdhci", "sdhci");
Here is how another machine could be added:
qos_node_create_machine("arm/raspi2", qos_create_machine_arm_raspi2);
qos_node_contains("arm/raspi2", "generic-sdhci");
Sample data structures
- QOSGraphNode
- contains the name of a driver/machine/test/interface and possibly extra data depending on the kind of object (e.g. the "constructor" function if needed---machines need it because they're the root, and also anything that is reached by a "consumes" edge). QOSGraphNode could also take care of creating the command line. We'll get to the command line later but (for now) suffice to say that for example QSDHCI_PCI will add "-device sdhci-pci" to the command line.
- QOSGraphEdge
- joins two QOSGraphNode with a "produces" or "consumed by" relationship
- QOSGraphObject
- a struct that is common to all "instances" created during the walk of a graph path. The most important function pointer in the struct is the one that returns the sub-objects (pci-device, sdhci, etc.) of the QOSGraphObject.
Note the asymmetry between "producer" (outgoing "produces" edge) and "consumer" (incoming "consumed by" edge). Whenever there is a "produces" relationship, an object is asked to return the next step in the graph. This is a method in QOSGraphObject. Whenever there is a "consumed by" relationship, the parent doesn't know what the next step in the graph will be. It can be a test, a driver, whatever. So it is up to the framework to create the object, using the constructor in QOSGraphNode itself.
QOSGraphObject example
In QEMU we have right now the following code (tests/sdhci-test.c):
static uint16_t sdhci_readw(QSDHCI *s, uint32_t reg)
{
uint16_t val;
if (s->pci.dev) {
val = qpci_io_readw(s->pci.dev, s->mem_bar, reg);
} else {
val = qtest_readw(global_qtest, s->addr + reg);
}
return val;
}
The two arms of the "if" basically correspond to two drivers, respectively "sdhci-pci" and "sdhci-memory-mapped". Both of them implements QSDHCI which looks like this:
struct QSDHCI {
...
uint16_t (*sdhci_readw)(QSDHCI *s, uint32_t reg);
...
}
The first would have an implementation like:
struct QSDHCI_PCI {
QOSGraphObject obj;
QPCIDevice dev;
QSDHCI sdhci;
...
};
static uint16_t sdhci_pci_readw(QSDHCI *s, uint32_t reg)
{
QSDHCI_PCI *spci = container_of(s, QSDHCI, sdhci);
return = qpci_io_readw(&spci->dev, spci->mem_bar, reg);
}
static void *sdhci_pci_get_driver(void *object,
const char *interface)
{
QSDHCI_PCI *spci = obj;
if (!strcmp(interface, "pci-device")) {
return &spci->dev;
}
if (!strcmp(interface, "sdhci")) {
return &spci->sdhci;
}
abort();
}
and the constructor (pointed to by the QOSGraphNode) initializes the function pointers when it creates the object:
void *sdhci_pci_create(void *pci_bus)
{
QPCIBus *bus = pci_bus;
qpci_device_init(&spci->dev, bus, QPCI_DEVFN(4, 0));
spci->obj.get_driver = sdhci_pci_get_driver;
spci->sdhci.readw = sdhci_pci_readw;
}
The second would have:
struct QSDHCI_MemoryMapped {
QOSGraphObject obj;
QSDHCI sdhci;
...
};
static uint16_t sdhci_mm_readw(QSDHCI *s, uint32_t reg)
{
QSDHCI_MemoryMapped *smm = container_of(smm, QSDHCI, sdhci);
return qtest_readw(global_qtest, smm->addr + reg);
}
static void *sdhci_mm_get_driver(void *object,
const char *interface)
{
QSDHCI_MemoryMapped *spci = obj;
if (!strcmp(interface, "sdhci")) {
return &spci->sdhci;
}
}
with similar initialization (a bit different because this device would be "contained" in the machine, rather than a "consumer" of a bus:
void qos_create_sdhci_mm(QSDHCI_MemoryMapped *sdhci, uint32_t addr)
{
sdhci->obj.get_driver = sdhci_mm_get_driver;
sdhci->sdhci.readw = sdhci_mm_readw;
sdhci->addr = addr;
}
static QOSGraphObject *raspi2_get_device(void *obj, const char *device)
{
QRaspi2Machine *machine = obj;
if (!strcmp(device, "generic-sdhci")) {
return &machine->sdhci.obj;
}
abort();
}
QOSGraphObject *qos_create_machine_arm_raspi2(void)
{
QRaspi2Machine *machine = calloc(sizeof(QRaspi2Machine), 1);
machine->obj.get_device = raspi2_get_device;
qos_create_sdhci_mm(&machine->sdhci, 0x300000);
return &machine->obj;
}
Possible milestones
- Create graph framework and write unit tests
- Convert all PC-based PCI device tests to use the graph framework
- Convert all PCI device tests to use the graph framework, so that all PC-based PCI device tests can work on more than one machine
- Create more drivers (e.g. SD-HCI)