ToDo/Block/old: Difference between revisions

From QEMU
No edit summary
Line 1: Line 1:
=Material for next QEMU release=
= QEMU 1.0 =


====Coroutines in the block layer [Kevin]====
====Coroutines in the block layer [Kevin]====
* Programming model to simplify block drivers without blocking QEMU threads
* Programming model to simplify block drivers without blocking QEMU threads
* All synchronous drivers converted to asynchronous
====VMDK enhancements [Fam, GSoC 2011]====
* Implement latest VMDK specs to support modern image files
* Patches currently being reviewed and merged
====iSCSI block device integration====
* Enable userspace-only remote access to disk images
= QEMU 1.1 =


====Generic copy-on-read [Stefan]====
====Generic copy-on-read [Stefan]====
Line 10: Line 20:
* Make block_stream commands available for all image formats that support backing files
* Make block_stream commands available for all image formats that support backing files


====Live block copy====
====Block I/O limits [Zhi Yong]====
* Resource control for guest I/O bandwidth/iops consumption
* Usable with virtio on QEMU 1.0
 
====snapshot_blkdev and Backup API [Jeff]====
* Support for consistent disk snapshots
 
====NBD asynchronous I/O [Paolo]====
* Improved performance
 
====virtio-scsi [Paolo/Stefan]====
* The next step after virtio-blk, full SCSI command set and appears as SCSI HBA in guest
* Real /dev/sda devices in guest
* No more modifying guest drivers to add simple storage protocol features
 
=Material for next QEMU release=
 
====Live block operations====
* Copy the contents of an image file while a guest is using it
* Copy the contents of an image file while a guest is using it
* Mostly a libvirt feature based on QEMU's blockdev-snapshot-sync and block_stream
** Various implementations: active/synchronous (guest sees I/O completion when data reaches both source and destination), passive/asynchronous (guest sees I/O completion when data reaches source only)
* Improved error handling, similar to -drive rerror/werror
** Errors can pause or cancel the job (with possibly separate handling for ENOSPC)


====In-place qcow2 <-> qed conversion [Devin, GSoC 2011]:====
====In-place qcow2 <-> qed conversion [Devin, GSoC 2011]:====
Line 18: Line 47:
* Patches currently being reviewed and merged
* Patches currently being reviewed and merged


====VMDK enhancements [Fam, GSoC 2011]====
====Runtime WCE toggling [Paolo, mst?]====
* Implement latest VMDK specs to support modern image files
* wce=none|on|off and run-time guest toggling support
* Patches currently being reviewed and merged
** Is wce=... an option for guest devices or block devices?
* Prerequisite to switching away from cache=writethrough by default
* Idea: replace O_DSYNC with manual bdrv_flush calls after each write
** Possible to get some speedups on qcow2 metadata updates too (certainly with respect to O_SYNC; not sure how much with O_DSYNC)
 
====Block migration [?]====
* Block migration working with a separate migration thread
* Perhaps just drop it.


====Block I/O limits [Zhi Yong]====
====IDE CD-ROM passthrough====
* Resource control for guest I/O bandwidth/iops consumption
* Track host tray state


====Runtime WCE toggling [?]====
====virtio-scsi [Paolo/Stefan]====
* wce=none|on|off and run-time guest toggling support
* multiqueue (in the kernel)
* Prerequisite to switching away from cache=writethrough by default
* MODE SELECT (WCE enable, perhaps something else?)


=Future changes=
=Future changes=
Line 33: Line 69:
====Cow overlay [Dong Xu "Robert"]====
====Cow overlay [Dong Xu "Robert"]====
* Allow live block copy and image streaming to raw destination files
* Allow live block copy and image streaming to raw destination files
====snapshot_blkdev and Backup API [Jes, Jagane]====
* Support for consistent disk snapshots and dirty block tracking
* Allow backup software to integrate with QEMU


====-blockdev [Markus?]====
====-blockdev [Markus?]====
Line 49: Line 81:
** luks-like key scheme that allows changing passphrase without re-encrypting data
** luks-like key scheme that allows changing passphrase without re-encrypting data


====iSCSI/NBD/Remote block device integration====
====NBD server for block device migration====
* Enable remote access to disk images for live migration and other tasks
* Enable remote access to live disk images for external backup software
* Still unclear what will be done in this area
 
====Pre/post block copy based on image streaming====
* Working block migration that works with a separate migration thread


====Avoid blocking QEMU threads====
====Avoid blocking QEMU threads====
Line 62: Line 90:
* All blocking operations must be done asynchronously or in a worker thread
* All blocking operations must be done asynchronously or in a worker thread


====virtio-scsi [Paolo/Stefan]====
====tcm_vhost [Zhi Yong]====
* The next step after virtio-blk, full SCSI command set and appears as SCSI HBA in guest
* Real /dev/sda devices in guest
* No more modifying guest drivers to add simple storage protocol features
 
====tcm_vhost [Stefan]====
* Directly connect virtio-scsi with Linux in-kernel SCSI target
* Directly connect virtio-scsi with Linux in-kernel SCSI target
* Pass-through of host SCSI devices
* Pass-through of host SCSI devices
Line 77: Line 100:
====qed online resize [Zhi Yong]====
====qed online resize [Zhi Yong]====
* Support shrinking
* Support shrinking
====qemu_aio_* shouldn't return NULL====
* Some callers don't handle NULL properly and it provides no error code
* Block drivers can use a BH to invoke the callback with an error return value
=Future changes=
Cow overlay [Dong Xu "Robert"]
* Allow live block copy and image streaming to raw destination files
snapshot_blkdev and Backup API [Jes, Jagane]
* Support for consistent disk snapshots and dirty block tracking
* Allow backup software to integrate with QEMU
-blockdev [Markus?]
* Explicit user control over block device trees
* Perhaps base this on QEMU Object Model right away
QCOW3
* Extend qcow2 format to address current and future image format challenges
  * Feature bits for fine-grained file format extensions
  * Sub-clusters to reduce metadata size and fragmentation
  * Zero clusters for efficient sparse images and copy-on-read
  * luks-like key scheme that allows changing passphrase without re-encrypting data
iSCSI/NBD/Remote block device integration
* Enable remote access to disk images for live migration and other tasks
* Still unclear what will be done in this area
Pre/post block copy based on image streaming
* Working block migration that works with a separate migration thread
Avoid blocking QEMU threads
* Today loss of NFS connectivity can hang guests
* It's critical never to block the vcpu thread
* The iothread should also not block while the qemu mutex is held
* All blocking operations must be done asynchronously or in a worker thread
virtio-scsi [Paolo/Stefan]
* The next step after virtio-blk, full SCSI command set and appears as SCSI HBA in guest
* Real /dev/sda devices in guest
* No more modifying guest drivers to add simple storage protocol features
tcm_vhost [Stefan]
* Directly connect virtio-scsi with Linux in-kernel SCSI target
* Pass-through of host SCSI devices
qcow2 online resize [Zhi Yong]
* Handle snapshots
* Support shrinking
qed online resize [Zhi Yong]
* Support shrinking

Revision as of 15:09, 18 April 2012

QEMU 1.0

Coroutines in the block layer [Kevin]

  • Programming model to simplify block drivers without blocking QEMU threads
  • All synchronous drivers converted to asynchronous

VMDK enhancements [Fam, GSoC 2011]

  • Implement latest VMDK specs to support modern image files
  • Patches currently being reviewed and merged

iSCSI block device integration

  • Enable userspace-only remote access to disk images

QEMU 1.1

Generic copy-on-read [Stefan]

  • Populate image file to avoid fetching same block from backing file again later

Generic image streaming [Stefan]

  • Make block_stream commands available for all image formats that support backing files

Block I/O limits [Zhi Yong]

  • Resource control for guest I/O bandwidth/iops consumption
  • Usable with virtio on QEMU 1.0

snapshot_blkdev and Backup API [Jeff]

  • Support for consistent disk snapshots

NBD asynchronous I/O [Paolo]

  • Improved performance

virtio-scsi [Paolo/Stefan]

  • The next step after virtio-blk, full SCSI command set and appears as SCSI HBA in guest
  • Real /dev/sda devices in guest
  • No more modifying guest drivers to add simple storage protocol features

Material for next QEMU release

Live block operations

  • Copy the contents of an image file while a guest is using it
    • Various implementations: active/synchronous (guest sees I/O completion when data reaches both source and destination), passive/asynchronous (guest sees I/O completion when data reaches source only)
  • Improved error handling, similar to -drive rerror/werror
    • Errors can pause or cancel the job (with possibly separate handling for ENOSPC)

In-place qcow2 <-> qed conversion [Devin, GSoC 2011]:

  • Fast conversion between qcow2 and qed image formats without copy all data
  • Patches currently being reviewed and merged

Runtime WCE toggling [Paolo, mst?]

  • wce=none|on|off and run-time guest toggling support
    • Is wce=... an option for guest devices or block devices?
  • Prerequisite to switching away from cache=writethrough by default
  • Idea: replace O_DSYNC with manual bdrv_flush calls after each write
    • Possible to get some speedups on qcow2 metadata updates too (certainly with respect to O_SYNC; not sure how much with O_DSYNC)

Block migration [?]

  • Block migration working with a separate migration thread
  • Perhaps just drop it.

IDE CD-ROM passthrough

  • Track host tray state

virtio-scsi [Paolo/Stefan]

  • multiqueue (in the kernel)
  • MODE SELECT (WCE enable, perhaps something else?)

Future changes

Cow overlay [Dong Xu "Robert"]

  • Allow live block copy and image streaming to raw destination files

-blockdev [Markus?]

  • Explicit user control over block device trees
  • Perhaps base this on QEMU Object Model right away

QCOW3

  • Extend qcow2 format to address current and future image format challenges
    • Feature bits for fine-grained file format extensions
    • Sub-clusters to reduce metadata size and fragmentation
    • Zero clusters for efficient sparse images and copy-on-read
    • luks-like key scheme that allows changing passphrase without re-encrypting data

NBD server for block device migration

  • Enable remote access to live disk images for external backup software

Avoid blocking QEMU threads

  • Today loss of NFS connectivity can hang guests
  • It's critical never to block the vcpu thread
  • The iothread should also not block while the qemu mutex is held
  • All blocking operations must be done asynchronously or in a worker thread

tcm_vhost [Zhi Yong]

  • Directly connect virtio-scsi with Linux in-kernel SCSI target
  • Pass-through of host SCSI devices

qcow2 online resize [Zhi Yong]

  • Handle snapshots
  • Support shrinking

qed online resize [Zhi Yong]

  • Support shrinking