Features/VirtioVhostUser
The virtio-vhost-user device lets guests act as vhost device backends so that virtual network switches and storage appliance VMs can provide virtio devices to other guests.
virtio-vhost-user is currently under development and is not yet ready for production.
Use cases
Appliances for cloud environments
In cloud environments everything is a guest. It is not possible for users to run vhost-user processes on the host. This precludes high-performance vhost-user appliances from running in cloud environments.
virtio-vhost-user allows vhost-user appliances to be shipped as virtual machine images. They can provide I/O services directly to other guests instead of going through an extra layer of device emulation like a host network switch:
Traditional Appliance VMs virtio-vhost-user Appliance VMs +-------------+ +-------------+ +-------------+ +-------------+ | VM1 | | VM2 | | VM1 | | VM2 | | Appliance | | Consumer | | Appliance | | Consumer | | ^ | | ^ | | <------+---+------> | +------|------+---+------|------+ +-------------+---+-------------+ | +-----------------+ | | | | Host | | Host | +-------------------------------+ +-------------------------------+
Exitless VM-to-VM communication
Once the vhost-user session has been established all vring activity can be performed by poll mode drivers in shared memory. This eliminates vmexits in the data path so that the highest possible VM-to-VM communication performance can be achieved.
Even when interrupts are necessary, virtio-vhost-user can use lightweight vmexits thanks to ioeventfd instead of exiting to host userspace. This ensures that VM-to-VM communication bypasses device emulation in QEMU.
How it works
Virtio devices were originally emulated inside the QEMU host userspace process. Later on, vhost allowed a subset of a virtio device, called the vhost device backend, to be implement inside the host kernel. vhost-user then allowed vhost device backends to reside in host userspace processes instead.
virtio-vhost-user takes this one step further by moving the vhost device backend into a guest. It works by tunneling the vhost-user protocol over a new virtio device type called virtio-vhost-user.
The following diagram shows how two guests communicate:
+-------------+ +-------------+ | VM1 | | VM2 | | | | | | vhost | shared memory | | | device | +-----------------> | | | backend | | | | | | virtio-net | +-------------+ +-------------+ | | | | | virtio- | vhost-user socket | | | vhost-user | <-----------------> | vhost-user | | QEMU | | QEMU | +-------------+ +-------------+
VM2 sees a regular virtio-net device. VM2's QEMU uses the existing vhost-user feature as if it were talking to a host userspace vhost-user backend.
VM1's QEMU tunnels the vhost-user protocol messages from VM1's QEMU to the new virtio-vhost-user device so that guest software in VM1 can act as the vhost-user backend.
It is possible to reuse existing vhost-user backend software with virtio-vhost-user since they use the same vhost-user protocol messages. A driver is required for the virtio-vhost-user PCI device that carries the message instead of the usual vhost-user UNIX domain socket. The driver can be implemented in a guest userspace process using Linux vfio-pci but guest kernel driver implementation would also be also possible.
The vhost device backend vrings are accessed through shared memory and do not require vhost-user message exchanges in the data path. No vmexits are taken when poll mode drivers are used. Even when interrupts are used, QEMU is not involved in the data path because ioeventfd lightweight vmexits are taken.
All vhost device types work with virtio-vhost-user, including net, scsi, and blk.
Code
- QEMU: stefanha's qemu.git repo
- DPDK: stefanha's dpdk.git repo
- VIRTIO device specification: stefanha's virtio.git repo