Features/Channel I/O Passthrough
Channel I/O is a high-performance input/output (I/O) architecture that is implemented (especially) on s390 (see the wikipedia article).
In the past, a guest virtualized via QEMU/KVM on s390 only sees paravirtualized virtio devices via the "Virtio Over Channel I/O (virtio-ccw)" transport. This makes virtio devices discoverable via standard operating system algorithms for handling channel devices.
However this is not enough. To make the majority of devices on s390, which use the standard Channel I/O based mechanism, usable in a QEMU virtual machine, a passthrough mechanism is needed. This includes devices that don't have a virtio counterpart (e.g. tape drives) or that have specific characteristics which guests want to exploit.
For passing a device to a guest, this uses the same interface as everybody else, namely vfio. Thus, the vfio support for channel devices is introduced, and this new vfio device is named "vfio-ccw".
vfio-ccw is realized with a mdev implementation. It has two drivers for two types of devices in the kernel:
- The vfio_ccw driver for the physical subchannel device.
- The vfio_mdev driver for the mediated vfio ccw device.
The QEMU part introduces a basic Channel I/O passthrough infrastructure based on vfio.
- Focus on supporting dasd-eckd (cu_type/dev_type = 0x3990/0x3390) as the target device currently.
- Support new QEMU parameters in the style of:
-machine s390-ccw-virtio(,s390-squash-mcss=on|off) \ -device vfio-ccw,sysfsdev=$MDEV_PATH
The new machine option s390-squash-mcss is added to squash e.g. passed-through channel devices from their real css (0-3, or 0 for hosts not activating MCSS-E) into the default css, as all virtio-ccw devices are in css 0xfe (and show up in the default css 0 for guests not activating MCSS-E).
This example setup is done for a Linux guest running in an s390x-ccw-virtio machine.
- Kernel Configuration
CONFIG_S390_CCW_IOMMU=m CONFIG_VFIO=m CONFIG_VFIO_MDEV=m CONFIG_VFIO_MDEV_DEVICE=m CONFIG_VFIO_CCW=m
- Modules Required
modprobe vfio.ko modprobe mdev.ko modprobe vfio_mdev.ko modprobe vfio_iommu_type1.ko modprobe vfio_ccw.ko
- You need to have a DASD/ECKD device as the target device.
Find the subchannel (0."$ssid"."$schid") of your target DASD/ECKD device and bind the subchannel to the vfio_ccw driver.
#find the DASD you can use with lsdasd on your host, e.g.: #devno="7e52" #schid="16ca" #ssid="0" #for device 0.0.7e52 on subchannel 0.0.16ca. #unbind the CCW device from its subchannel echo 0."$ssid"."$devno" > /sys/bus/ccw/devices/0."$ssid"."$devno"/driver/unbind #unbind the subchannel from the I/O subchannel driver echo 0."$ssid"."$schid" > /sys/bus/css/devices/0."$ssid"."$schid"/driver/unbind #bind the subchannel to the vfio_ccw driver echo 0."$ssid"."$schid" > /sys/bus/css/drivers/vfio_ccw/bind
- Create a Mediated Device for the physical device
#generate a uuid with uuidgen. e.g.: #uuid="6dfd3ec5-e8b3-4e18-a6fe-57bc9eceb920" echo "$uuid" > /sys/bus/css/devices/0."$ssid"."$schid"/mdev_supported_types/vfio_ccw-io/create
- Add a vfio-ccw device to your QEMU command line (machine needs to be
-M s390-ccw-virtio,s390-squash-mcss=on \ -device vfio-ccw,devno=0.0.1234,sysfsdev=/sys/bus/mdev/devices/$uuid \ ... ...
- Start QEMU. Your guest will be presented with a real Channel I/O device, with the example above, to be more specific, a DASD/ECKD device.
Try the device on your Guest
- The guest will see the DASD/ECKD as a channel-attached device. With the example above, the device will show up as 0.0.1234 (verify e.g. via the lscss tool).
- You can online the device by calling
chccwdev -e 0.0.1234and use it as a block device.
- Only target I/O subchannels.
- Only basic commands (read/write) have been tested.
- Some commands may need special handling in the future, for example, anything related to path grouping.