Features/Channel I/O Passthrough: Difference between revisions

From QEMU
(Created page with "'''Notice: this page is still under construction''' The patch series introduce a basic channel I/O passthrough infrastructure based on vfio. * Focus on supporting dasd-eckd(c...")
 
No edit summary
Line 1: Line 1:
'''Notice: this page is still under construction'''
'''''Notice: this page is still under construction'''''


The patch series introduce a basic channel I/O passthrough infrastructure based on vfio.
Channel I/O is a high-performance input/output (I/O) architecture that is implemented (especially) on s390 (see [https://en.wikipedia.org/wiki/Channel_I/O the wikipedia article]).
* Focus on supporting dasd-eckd(cu_type/dev_type = 0x3990/0x3390) as the target device.  
 
* Support new qemu parameters in the style of:
== Motivation ==
 
In the past, a guest virtualized via QEMU/KVM on s390 only sees paravirtualized virtio devices via the "Virtio Over Channel I/O (virtio-ccw)" transport. This makes virtio devices discoverable via
standard operating system algorithms for handling channel devices.
 
However this is not enough. On s390 for the majority of devices, which use the standard Channel I/O based mechanism, it also needs to provide the functionality of passing through them to a QEMU virtual machine. This includes devices that don't have a virtio counterpart (e.g. tape drives) or that have specific characteristics which guests want to exploit.
 
For passing a device to a guest, this uses the same interface as everybody else, namely vfio. Thus, the vfio support for channel devices is introduced , and this new vfio device is named "vfio-ccw".
 
== Implementation ==
 
vfio-ccw is realized with a mdev implementation. It has two drivers for two types of
devices in the kernel:
* The vfio_ccw driver for the physical subchannel device.
* The vfio_mdev driver for the mediated vfio ccw device.
 
The QEMU part introduces a basic Channel I/O passthrough infrastructure based on vfio.
* Focus on supporting dasd-eckd (cu_type/dev_type = 0x3990/0x3390) as the target device currently.  
* Support new QEMU parameters in the style of:
     -machine s390-ccw-virtio(,s390-squash-mcss=on|off) \
     -machine s390-ccw-virtio(,s390-squash-mcss=on|off) \
     -device vfio-ccw,sysfsdev=$MDEV_PATH
     -device vfio-ccw,sysfsdev=$MDEV_PATH
We want to support real (i.e. not virtual) channel devices even for guests that do not support MCSS-E (where guests may see devices from any channel subsystem image at once). As all virtio-ccw devices are in css 0xfe (and show up in the default css 0 for guests not activating MCSS-E), we need an option to squash e.g. passed-through channel devices from their real css (0-3, or 0 for hosts not activating MCSS-E) into the default css, that is what the new machine option s390-squash-css is added.
 
The new machine option s390-squash-css is added to squash e.g. passed-through channel devices from their real css (0-3, or 0 for hosts not activating MCSS-E) into the default css, as all virtio-ccw devices are in css 0xfe (and show up in the default css 0 for guests not activating MCSS-E).


== Setup ==
== Setup ==
Line 12: Line 31:
This example setup is done with a Linux guest running in an s390x-ccw-virtio machine.
This example setup is done with a Linux guest running in an s390x-ccw-virtio machine.


=== Prereqs ===
=== Host ===
 
==== Kernel Configuration ====


* Kernel Configuration
   CONFIG_S390_CCW_IOMMU=m
   CONFIG_S390_CCW_IOMMU=m
   CONFIG_VFIO=m
   CONFIG_VFIO=m
Line 22: Line 40:
   CONFIG_VFIO_CCW=m
   CONFIG_VFIO_CCW=m


==== Modules Required ====
* Modules Required
 
   modprobe vfio.ko
   modprobe vfio.ko
   modprobe mdev.ko
   modprobe mdev.ko
Line 30: Line 47:
   modprobe vfio_ccw.ko
   modprobe vfio_ccw.ko


==== Prepare a Subchannel ====
* You need to have a [https://en.wikipedia.org/wiki/Direct-access_storage_device DASD/ECKD] device as the target device.
 
Find the subchannel (0.0."%schid") of your target DASD/ECKD device and bind the subchannel to the vfio_ccw driver.
Find a subchannel(0.0."%schid") of a DASD-ECKD device and bind it to vfio_ccw driver.
   #find the DASD you can use with lsdasd on your host. e.g.:
   #find the dasd you can use with lsdasd on your host. e.g.:
   devno="7e52"
   devno="7e52"
   schid="16ca"
   schid="16ca"
   #unbind the ccw device from the subchannel
   #unbind the CCW device from its subchannel
   echo 0.0."$devno" > /sys/bus/ccw/devices/0.0."$devno"/driver/unbind
   echo 0.0."$devno" > /sys/bus/ccw/devices/0.0."$devno"/driver/unbind
   #unbind the subchannel from io_subchannel driver
   #unbind the subchannel from the I/O subchannel driver
   echo 0.0."$schid" > /sys/bus/css/devices/0.0."$schid"/driver/unbind
   echo 0.0."$schid" > /sys/bus/css/devices/0.0."$schid"/driver/unbind
   #bind the subchannel with vfio_ccw driver
   #bind the subchannel to the vfio_ccw driver
   echo 0.0."$schid" > /sys/bus/css/drivers/vfio_ccw/bind
   echo 0.0."$schid" > /sys/bus/css/drivers/vfio_ccw/bind


 
* Create a Mediated Device for the phsical device
==== Create a Mediated Device ====
 
   #generate a uuid with uuidgen. e.g.:
   #generate a uuid with uuidgen. e.g.:
   uuid="6dfd3ec5-e8b3-4e18-a6fe-57bc9eceb920"
   uuid="6dfd3ec5-e8b3-4e18-a6fe-57bc9eceb920"
   echo "$uuid" > \
   echo "$uuid" > /sys/bus/css/devices/0.0."$schid"/mdev_supported_types/vfio_ccw-io/create
  /sys/bus/css/devices/0.0."$schid"/mdev_supported_types/vfio_ccw-io/create
 


=== Starting QEMU ===
=== Starting QEMU ===


Pass-through this device to a vm:
* Add a vfio-ccw device to your QEMU command line (machine needs to be <code>s390x-ccw-virtio</code>, with <code>s390-squash-css=on</code>). Example:
   -M s390-ccw-virtio,s390-squash-css=on \
   -M s390-ccw-virtio,s390-squash-css=on \
   -device vfio-ccw,sysfsdev=/sys/bus/mdev/devices/$uuid \
   -device vfio-ccw,devno=0.0.1234,sysfsdev=/sys/bus/mdev/devices/$uuid \
   ... ...
   ... ...
* Start QEMU. Your guest will be presented with a real Channel I/O device, with the example above, to be more specific, a DASD/ECKD device.
=== Try the device on your Guest ===
* The guest will see the DASD/ECKD as a channel-attached device. With the example above, the device will show up as 0.0.1234.
* You can online the device by calling <code>chccwdev -e 0.0.1234</code> and use it as a block device.
== Restrictions ==
* Only target the I/O subchannels.
* Only basic commands (read/write) are supported.
* Some commands may need special handling in the future, for example, anything related to path grouping.

Revision as of 01:47, 22 May 2017

Notice: this page is still under construction

Channel I/O is a high-performance input/output (I/O) architecture that is implemented (especially) on s390 (see the wikipedia article).

Motivation

In the past, a guest virtualized via QEMU/KVM on s390 only sees paravirtualized virtio devices via the "Virtio Over Channel I/O (virtio-ccw)" transport. This makes virtio devices discoverable via standard operating system algorithms for handling channel devices.

However this is not enough. On s390 for the majority of devices, which use the standard Channel I/O based mechanism, it also needs to provide the functionality of passing through them to a QEMU virtual machine. This includes devices that don't have a virtio counterpart (e.g. tape drives) or that have specific characteristics which guests want to exploit.

For passing a device to a guest, this uses the same interface as everybody else, namely vfio. Thus, the vfio support for channel devices is introduced , and this new vfio device is named "vfio-ccw".

Implementation

vfio-ccw is realized with a mdev implementation. It has two drivers for two types of devices in the kernel:

  • The vfio_ccw driver for the physical subchannel device.
  • The vfio_mdev driver for the mediated vfio ccw device.

The QEMU part introduces a basic Channel I/O passthrough infrastructure based on vfio.

  • Focus on supporting dasd-eckd (cu_type/dev_type = 0x3990/0x3390) as the target device currently.
  • Support new QEMU parameters in the style of:
   -machine s390-ccw-virtio(,s390-squash-mcss=on|off) \
   -device vfio-ccw,sysfsdev=$MDEV_PATH

The new machine option s390-squash-css is added to squash e.g. passed-through channel devices from their real css (0-3, or 0 for hosts not activating MCSS-E) into the default css, as all virtio-ccw devices are in css 0xfe (and show up in the default css 0 for guests not activating MCSS-E).

Setup

This example setup is done with a Linux guest running in an s390x-ccw-virtio machine.

Host

  • Kernel Configuration
 CONFIG_S390_CCW_IOMMU=m
 CONFIG_VFIO=m
 CONFIG_VFIO_MDEV=m
 CONFIG_VFIO_MDEV_DEVICE=m
 CONFIG_VFIO_CCW=m
  • Modules Required
 modprobe vfio.ko
 modprobe mdev.ko
 modprobe vfio_mdev.ko
 modprobe vfio_iommu_type1.ko
 modprobe vfio_ccw.ko
  • You need to have a DASD/ECKD device as the target device.

Find the subchannel (0.0."%schid") of your target DASD/ECKD device and bind the subchannel to the vfio_ccw driver.

 #find the DASD you can use with lsdasd on your host. e.g.:
 devno="7e52"
 schid="16ca"
 #unbind the CCW device from its subchannel
 echo 0.0."$devno" > /sys/bus/ccw/devices/0.0."$devno"/driver/unbind
 #unbind the subchannel from the I/O subchannel driver
 echo 0.0."$schid" > /sys/bus/css/devices/0.0."$schid"/driver/unbind
 #bind the subchannel to the vfio_ccw driver
 echo 0.0."$schid" > /sys/bus/css/drivers/vfio_ccw/bind
  • Create a Mediated Device for the phsical device
 #generate a uuid with uuidgen. e.g.:
 uuid="6dfd3ec5-e8b3-4e18-a6fe-57bc9eceb920"
 echo "$uuid" > /sys/bus/css/devices/0.0."$schid"/mdev_supported_types/vfio_ccw-io/create

Starting QEMU

  • Add a vfio-ccw device to your QEMU command line (machine needs to be s390x-ccw-virtio, with s390-squash-css=on). Example:
 -M s390-ccw-virtio,s390-squash-css=on \
 -device vfio-ccw,devno=0.0.1234,sysfsdev=/sys/bus/mdev/devices/$uuid \
 ... ...
  • Start QEMU. Your guest will be presented with a real Channel I/O device, with the example above, to be more specific, a DASD/ECKD device.

Try the device on your Guest

  • The guest will see the DASD/ECKD as a channel-attached device. With the example above, the device will show up as 0.0.1234.
  • You can online the device by calling chccwdev -e 0.0.1234 and use it as a block device.

Restrictions

  • Only target the I/O subchannels.
  • Only basic commands (read/write) are supported.
  • Some commands may need special handling in the future, for example, anything related to path grouping.