Features/Channel I/O Passthrough: Difference between revisions

From QEMU
No edit summary
(Typos, minor wording changes, mention ssid)
Line 8: Line 8:
standard operating system algorithms for handling channel devices.
standard operating system algorithms for handling channel devices.


However this is not enough. On s390 for the majority of devices, which use the standard Channel I/O based mechanism, it also needs to provide the functionality of passing through them to a QEMU virtual machine. This includes devices that don't have a virtio counterpart (e.g. tape drives) or that have specific characteristics which guests want to exploit.
However this is not enough. To make the majority of devices on s390, which use the standard Channel I/O based mechanism, usable in a QEMU virtual machine, a passthrough mechanism is needed. This includes devices that don't have a virtio counterpart (e.g. tape drives) or that have specific characteristics which guests want to exploit.


For passing a device to a guest, this uses the same interface as everybody else, namely vfio. Thus, the vfio support for channel devices is introduced , and this new vfio device is named "vfio-ccw".
For passing a device to a guest, this uses the same interface as everybody else, namely vfio. Thus, the vfio support for channel devices is introduced, and this new vfio device is named "vfio-ccw".


== Implementation ==
== Implementation ==
Line 29: Line 29:
== Setup ==
== Setup ==


This example setup is done with a Linux guest running in an s390x-ccw-virtio machine.
This example setup is done for a Linux guest running in an s390x-ccw-virtio machine.


=== Host ===
=== Host ===
Line 48: Line 48:


* You need to have a [https://en.wikipedia.org/wiki/Direct-access_storage_device DASD/ECKD] device as the target device.
* You need to have a [https://en.wikipedia.org/wiki/Direct-access_storage_device DASD/ECKD] device as the target device.
Find the subchannel (0.0."%schid") of your target DASD/ECKD device and bind the subchannel to the vfio_ccw driver.
Find the subchannel (0."%ssid"."%schid") of your target DASD/ECKD device and bind the subchannel to the vfio_ccw driver.
   #find the DASD you can use with lsdasd on your host. e.g.:
   #find the DASD you can use with lsdasd on your host, e.g.:
   devno="7e52"
   #devno="7e52"
   schid="16ca"
   #schid="16ca"
  #ssid="0"
  #for device 0.0.7e52 on subchannel 0.0.16ca.
   #unbind the CCW device from its subchannel
   #unbind the CCW device from its subchannel
   echo 0.0."$devno" > /sys/bus/ccw/devices/0.0."$devno"/driver/unbind
   echo 0."%ssid"."$devno" > /sys/bus/ccw/devices/0."%ssid"."$devno"/driver/unbind
   #unbind the subchannel from the I/O subchannel driver
   #unbind the subchannel from the I/O subchannel driver
   echo 0.0."$schid" > /sys/bus/css/devices/0.0."$schid"/driver/unbind
   echo 0."%ssid"."$schid" > /sys/bus/css/devices/0."%ssid"."$schid"/driver/unbind
   #bind the subchannel to the vfio_ccw driver
   #bind the subchannel to the vfio_ccw driver
   echo 0.0."$schid" > /sys/bus/css/drivers/vfio_ccw/bind
   echo 0."%ssid"."$schid" > /sys/bus/css/drivers/vfio_ccw/bind


* Create a Mediated Device for the phsical device
* Create a Mediated Device for the physical device
   #generate a uuid with uuidgen. e.g.:
   #generate a uuid with uuidgen. e.g.:
   uuid="6dfd3ec5-e8b3-4e18-a6fe-57bc9eceb920"
   uuid="6dfd3ec5-e8b3-4e18-a6fe-57bc9eceb920"
   echo "$uuid" > /sys/bus/css/devices/0.0."$schid"/mdev_supported_types/vfio_ccw-io/create
   echo "$uuid" > /sys/bus/css/devices/0."%ssid"."$schid"/mdev_supported_types/vfio_ccw-io/create


=== Starting QEMU ===
=== Starting QEMU ===
Line 74: Line 76:
=== Try the device on your Guest ===
=== Try the device on your Guest ===


* The guest will see the DASD/ECKD as a channel-attached device. With the example above, the device will show up as 0.0.1234.
* The guest will see the DASD/ECKD as a channel-attached device. With the example above, the device will show up as 0.0.1234 (verify e.g. via the lscss tool).


* You can online the device by calling <code>chccwdev -e 0.0.1234</code> and use it as a block device.
* You can online the device by calling <code>chccwdev -e 0.0.1234</code> and use it as a block device.
Line 80: Line 82:
== Restrictions ==
== Restrictions ==


* Only target the I/O subchannels.
* Only target I/O subchannels.
* Only basic commands (read/write) are supported.
* Only basic commands (read/write) have been tested.
* Some commands may need special handling in the future, for example, anything related to path grouping.
* Some commands may need special handling in the future, for example, anything related to path grouping.

Revision as of 11:16, 22 May 2017

Notice: this page is still under construction

Channel I/O is a high-performance input/output (I/O) architecture that is implemented (especially) on s390 (see the wikipedia article).

Motivation

In the past, a guest virtualized via QEMU/KVM on s390 only sees paravirtualized virtio devices via the "Virtio Over Channel I/O (virtio-ccw)" transport. This makes virtio devices discoverable via standard operating system algorithms for handling channel devices.

However this is not enough. To make the majority of devices on s390, which use the standard Channel I/O based mechanism, usable in a QEMU virtual machine, a passthrough mechanism is needed. This includes devices that don't have a virtio counterpart (e.g. tape drives) or that have specific characteristics which guests want to exploit.

For passing a device to a guest, this uses the same interface as everybody else, namely vfio. Thus, the vfio support for channel devices is introduced, and this new vfio device is named "vfio-ccw".

Implementation

vfio-ccw is realized with a mdev implementation. It has two drivers for two types of devices in the kernel:

  • The vfio_ccw driver for the physical subchannel device.
  • The vfio_mdev driver for the mediated vfio ccw device.

The QEMU part introduces a basic Channel I/O passthrough infrastructure based on vfio.

  • Focus on supporting dasd-eckd (cu_type/dev_type = 0x3990/0x3390) as the target device currently.
  • Support new QEMU parameters in the style of:
   -machine s390-ccw-virtio(,s390-squash-mcss=on|off) \
   -device vfio-ccw,sysfsdev=$MDEV_PATH

The new machine option s390-squash-css is added to squash e.g. passed-through channel devices from their real css (0-3, or 0 for hosts not activating MCSS-E) into the default css, as all virtio-ccw devices are in css 0xfe (and show up in the default css 0 for guests not activating MCSS-E).

Setup

This example setup is done for a Linux guest running in an s390x-ccw-virtio machine.

Host

  • Kernel Configuration
 CONFIG_S390_CCW_IOMMU=m
 CONFIG_VFIO=m
 CONFIG_VFIO_MDEV=m
 CONFIG_VFIO_MDEV_DEVICE=m
 CONFIG_VFIO_CCW=m
  • Modules Required
 modprobe vfio.ko
 modprobe mdev.ko
 modprobe vfio_mdev.ko
 modprobe vfio_iommu_type1.ko
 modprobe vfio_ccw.ko
  • You need to have a DASD/ECKD device as the target device.

Find the subchannel (0."%ssid"."%schid") of your target DASD/ECKD device and bind the subchannel to the vfio_ccw driver.

 #find the DASD you can use with lsdasd on your host, e.g.:
 #devno="7e52"
 #schid="16ca"
 #ssid="0"
 #for device 0.0.7e52 on subchannel 0.0.16ca.
 #unbind the CCW device from its subchannel
 echo 0."%ssid"."$devno" > /sys/bus/ccw/devices/0."%ssid"."$devno"/driver/unbind
 #unbind the subchannel from the I/O subchannel driver
 echo 0."%ssid"."$schid" > /sys/bus/css/devices/0."%ssid"."$schid"/driver/unbind
 #bind the subchannel to the vfio_ccw driver
 echo 0."%ssid"."$schid" > /sys/bus/css/drivers/vfio_ccw/bind
  • Create a Mediated Device for the physical device
 #generate a uuid with uuidgen. e.g.:
 uuid="6dfd3ec5-e8b3-4e18-a6fe-57bc9eceb920"
 echo "$uuid" > /sys/bus/css/devices/0."%ssid"."$schid"/mdev_supported_types/vfio_ccw-io/create

Starting QEMU

  • Add a vfio-ccw device to your QEMU command line (machine needs to be s390x-ccw-virtio, with s390-squash-css=on). Example:
 -M s390-ccw-virtio,s390-squash-css=on \
 -device vfio-ccw,devno=0.0.1234,sysfsdev=/sys/bus/mdev/devices/$uuid \
 ... ...
  • Start QEMU. Your guest will be presented with a real Channel I/O device, with the example above, to be more specific, a DASD/ECKD device.

Try the device on your Guest

  • The guest will see the DASD/ECKD as a channel-attached device. With the example above, the device will show up as 0.0.1234 (verify e.g. via the lscss tool).
  • You can online the device by calling chccwdev -e 0.0.1234 and use it as a block device.

Restrictions

  • Only target I/O subchannels.
  • Only basic commands (read/write) have been tested.
  • Some commands may need special handling in the future, for example, anything related to path grouping.