|
|
| (60 intermediate revisions by 8 users not shown) |
| Line 1: |
Line 1: |
| This page contains block layer and storage features that have been proposed. These features may not be in active development and questions about them should be addressed to the QEMU mailing list at qemu-devel@nongnu.org. | | This page contains block layer and storage features that have been proposed. These features may not be in active development and questions about them should be addressed to the QEMU mailing list at qemu-devel@nongnu.org. |
|
| |
|
| == op blockers == | | == Core block layer == |
| * node-name allows starting operations in the middle of the chain; we need to protect against incompatible concurrent operations | | * Implement request cancellation in block/io_uring.c and util/thread-pool.c so that device reset and guest cancel commands promptly cancel hung requests. Today QEMU waits for requests to complete. Linux io_uring offers a cancel command that may be able to abort hung NFS requests (the liburing API is io_uring_prep_cancel()). QEMU's thread pool could be extended to use a signal to interrupt blocking system calls. Linux AIO does not seem to offer useful cancel semantics because only drivers/usb/gadget/ implements the kernel's kiocb_set_cancel_fn() API. |
| ** In fact, we even used paths before node-name (e.g. for live commit), so this has existed for a while
| |
| * bs->backing_blocker already forbids almost everything on backing files
| |
| ** Except live commit, which needs to be forbidden only when another job runs on the same chain
| |
| * Plan for 2.1 was to block all nodes recursively
| |
| ** bdrv_swap() during block job completion turns out to be nasty, especially for live commit of active layer:
| |
| *** Need to clean up blockers on the removed subchain
| |
| *** Which blockers should the newly swapped in node have?
| |
| * Alternative plan for 2.1:
| |
| ** Keep checking blockers on the requested node (for bs->backing_blockers to be effective)
| |
| ** But also check in the active layer because this is where block jobs do their blocking
| |
| *** bottommost node might work as well
| |
| **** As Kevin pointed out on IRC, in the current code blockers exist on backing files that don't exist on the active layer
| |
| * Long term (2.2+): Block categories of operations
| |
| ** Some thoughts, in the form of code example for block-commit: http://fpaste.org/113005/14036975/
| |
| * blockdev-add probably shouldn't be able to reference a node that has blockers
| |
|
| |
|
| == Basic infrastructure for blockdev-add [Kevin, Markus] == | | == virtio-blk == |
| * Allow libvirt to override backing file options (filename, format, cache mode) | | * Add a cancel command to the virtio-blk device so that running requests can be aborted. This requires changing the VIRTIO spec, extending QEMU's device emulation, and implementing blk_mq_ops->timeout() in Linux virtio_blk.ko. This task depends on first implementing real request cancellation in QEMU. |
| ** Convert bdrv_open() flags to QDict options (or handle in drive_init)
| |
| ** Implemented: filename, format, driver-specific options
| |
| ** Missing: Cache mode, AIO mode, discard mode, copy on read (basically everything that is contained in bs->open_flags)
| |
| ** Investigating possible QemuOpts misuse killing drive_del [Markus]
| |
| * Convert remaining drivers to make use of "QDict options" argument
| |
| * bdrv_reopen() needs to use "QDict options" instead of only flags
| |
|
| |
|
| == raw-posix: Error out on aio=native with cache.direct=off instead of falling back to aio=threads [Kevin] == | | == Old == |
| | | * [[ToDo/Block/old|Old block layer roadmap]] |
| == blockdev-add + blockdev-del QMP interface ==
| | * [[ToDo/Block/Qcow2PerformanceRoadmap|Old QCOW2 performance roadmap]] |
| * By default, return an error for blockdev-del if reference count > 1 | |
| * But have a force option that closes the image file, even if it breaks the remaining users (e.g. uncooperative guest that doesn't release its PCI device)
| |
| * Prevent mixing blockdev-add with drive_del and vice versa
| |
| ** Ideally drive_add BDSes are exactly those with a DriveInfo
| |
| | |
| == Split BlockBackend from BlockDriverState [armbru] ==
| |
| * armbru got a bit of unfinished exploration here
| |
| ** I'd like to try to flush obsolete parts (if any), post rest as RFC, no ETA yet
| |
| ** Still no progress; need to start somewhere to get anywhere, so let's split it with a big axe:
| |
| *** clone the bdrv interface to blk, implemented as trivial wrapper around bdrv
| |
| *** convert appropriate users from bdrv to blk
| |
| *** clean up the silly wrappers as we go
| |
| * Separate lifecycles for BlockDriverStates and guest devices (e.g. keep block job running after removing the guest device)
| |
| ** Need QMP command to disconnect BDS from guest device (already have blockjob-cancel)
| |
| ** blockdev-del only works with refcount = 1
| |
| * s/bs->device_name/bs->id/ after splitting out BlockBackend fields
| |
| ** BDS is no longer associated with a DeviceState, don't call it "device_name"
| |
| ** Careful, device_name[0] == '\0' has special meaning: internal BDS chain node
| |
| *** BDSes with a name are in bdrv_states and can be accesses e.g. by monitor commands
| |
| * Once BlockBackend is implemented, convert from bs->opaque to extending the struct and using container_of
| |
| * BlockBackend
| |
| ** Device emulation uses only BlockBackend, no direct BDS reference
| |
| ** Builtin NBD server uses BB (?)
| |
| ** Fields to move:
| |
| *** inserted
| |
| *** dev_*
| |
| *** enable_write_cache (?)
| |
| **** Separate field/flag for block jobs
| |
| *** on_read/write_error
| |
| *** BDRV_O_UNMAP (?)
| |
| ** To be moved to BlockFilters later (stay in BDS for now; BlockFilters implemented as BlockDriver):
| |
| *** bps_limits
| |
| *** copy_on_read
| |
| | |
| == BlockFilter and dynamic reconfiguration of the BDS graph ==
| |
| * Add/remove (e.g. filter) BDSes at runtime
| |
| * Ability to implement light-weight block drivers that play together with snapshots (e.g. block debug, active-mirroring, copy-on-read, I/O throttling, multipath, etc)
| |
| ** Converting current I/O throtltling code to a block filter should be simple, mostly a mechanical task.
| |
| * Requires BlockBackend split
| |
| ** Keep filters on top even after taking snapshots
| |
| * filters implement ops normally, and call out to their child BDS explicitly, no before- or after-ops-magic
| |
| * Benoît's customer may want I/O throttling in arbitrary places in the graph
| |
| * Be careful to never add cycles to the graph!
| |
| | |
| == New design for better encryption support [armbru] ==
| |
| * Requires BlockFilter
| |
| * Getting rid of qcow2 encryption, make it modular using luks or something, make it stackable (a block filter)
| |
| * Why? It's amateur encryption, should not be used for anything! Besides, usability is horrible.
| |
| * Deprecate, except for qemu-img convert?
| |
| | |
| == Block jobs ==
| |
| * node-name support for block-commit, block-stream
| |
| ** will be needed by libvirt for mixed local name / relative protocol (e.g. gluster) pathname support
| |
| * custom backing file string for block-commit, block-stream, standalone
| |
| ** needed by libvirt for relative pathname / protocol support
| |
| * 'top' optional for block-commit
| |
| ** silly, but needed by libvirt as a way to probe that active layer commit is available
| |
| * Live commit of the active layer [fam]
| |
| * Live streaming of intermediate layers (using block-stream) [benoit]
| |
| ** Ask Eric if libvirt will use this functionality [jeff] | |
| ** Benoît needs this
| |
| | |
| == Remove bs->job field and allow multiple jobs on a BDS [mreitz] ==
| |
| * allows more than one blockjob at a block device at a time
| |
| * infra-structure, refactoring work
| |
| * Careful not to break QMP API, need wrapper
| |
| * Allow non-block jobs (long-running operations outside block layer)
| |
| | |
| == Test Infrastructure ==
| |
| * Desires?
| |
| * support for testing AIO requests
| |
| ** No design yet, but we need some way to label I/O requests in blkdebug
| |
| * QMP support in qtest
| |
| ** widely useful, not just for block work
| |
| ** Responses are handled now, events still missing
| |
| | |
| == Active mirroring: just like mirroring, but live, on the fly, skip the bitmap ==
| |
| * similar to drive-backup
| |
| * security (virus-scan or some sort of inspection)
| |
| * should be implemented as a block filter
| |
| | |
| == qcow2 ==
| |
| * Cluster allocation performance: qcow2 delayed COW [kevin]
| |
| * qcow2 backing file validation (parent modification invalidates children) [jeff]
| |
| ** similar to vmdk and vhdx
| |
| * qcow2 internal snapshot read-only BlockDriverState
| |
| ** Allows accessing snapshots while guest accesses disk image
| |
| ** Tricky, we should probably drop it from our backlog
| |
| * subclusters
| |
| * Implement refcount_bits for smaller metadata
| |
| * Allow cache size configuration with a runtime option
| |
| ** Should use a default in bytes, not in number of tables (also for the default)
| |
| | |
| == Image fleecing [stefan, fam] ==
| |
| * Image creation with existing BlockDriverState as backing file (BlockDriverState ref count)
| |
| * Patches: http://lists.gnu.org/archive/html/qemu-devel/2013-11/msg03692.html
| |
| ** not merged yet
| |
| | |
| == Incremental backup [fam, jsnow] ==
| |
| * Backup applications need a dirty block bitmap so they can read only blocks that changed
| |
| * Two approaches discussed:
| |
| ** Dirty bitmap file
| |
| ** Write changed data through NBD:
| |
| *** See also: http://lists.gnu.org/archive/html/qemu-devel/2013-11/msg03035.html
| |
| ** Merkle tree - hash tree allows efficient syncing of image files between hosts
| |
| * QMP API exposed via libvirt
| |
| ** tricky?
| |
| * Series on list for drive-backup posted by Fam, needs review
| |
| | |
| == Image formats ==
| |
| * Gluster fixes / support [jeff]
| |
| ** memory leakage (in libglusterfs)
| |
| | |
| == virtio-blk data plane [stefanha] ==
| |
| * Protect BlockDriverState:
| |
| ** Miscellaneous monitor commands
| |
| *** commit
| |
| *** blockdev-snapshot-sync
| |
| *** blockdev-snapshot-internal-sync
| |
| *** blockdev-snapshot-delete-sync
| |
| *** transaction
| |
| *** block-passwd
| |
| ** Block jobs
| |
| | |
| == virtio-scsi data plane [fam] ==
| |
| | |
| == Multiqueue block layer ==
| |
| * Depends on: virtio-blk dataplane
| |
| * Single BlockDriverState, multiple independent threads accessing in parallel
| |
| * Allows us to extend Linux multiqueue block layer up into guest
| |
| * For maximum SMP scalability and performance with high IOPS SSDs
| |
| | |
| == Make qemu-img use QMP command implementations internally (e.g. use mirroring for qemu-img convert) [Max] ==
| |
| * Ensures that live operations provide the same functionality as we have offline
| |
| | |
| == Tests for -drive discard= ==
| |
| * Currently the discard feature is not well-tested in qemu-iotests
| |
| | |
| == AHCI emulation [jsnow] ==
| |
| * Enable migration
| |
| * qtests
| |
| * Sort out -hda and -cdrom with q35
| |
| ** Add machine option that determines what if=ide means (IDE or AHCI; pc defaults to IDE, q35 to AHCI)
| |
| * rerror=/werror=
| |
| ** Patches waiting for post/review for non-queued I/O
| |
| ** To do for queued I/O
| |
| | |
| == Block job I/O throttling ==
| |
| * Reuse Benoît's throttling implementation
| |
| * Handle large buffer sizes used by block jobs
| |
| ** block jobs like to work in bulk for efficiency but throttling doesn't like big, bursty requests (note from Benoît: I think the solution would be to make the clock used for the throttling computation coarser)
| |
| | |
| == I/O accounting (for query-blockstats) ==
| |
| * Driving it was made device model's responsibility (commit a597e79 in 2011)
| |
| ** Most of them still don't
| |
| ** The ones that do are inconsistent
| |
| ** Consequently, query-blockstats is better treated as wild guess, not data
| |
| * Need to take a step back
| |
| ** Use cases
| |
| *** Better pick libvirt and libguestfs brains
| |
| *** picked up a cloud provider using KVM brain
| |
| **** measuring in the device model is good for billing
| |
| **** some metrics are missing
| |
| **** It would be good to collect the same data everywhere in the BDS graph for metrology purpose (seeking hidden costs)
| |
| **** having a dedicated JSON socket to output accounting data would be good
| |
| * Working on revamping the I/O accounting infrastructure (Benoît)
| |
| | |
| == Add guard page to bottom of coroutine stacks in order to detect stack overflows ==
| |
| | |
| == iotests.py - Python module for writing qemu-iotests [stefan] ==
| |
| * Extract qemu.py generic QEMU interaction code
| |
| * Document qemu.py and iotests.py so it meets standard Python module conventions
| |
| * Port live migration qemu-iotest to Python to see if it's preferrable to the shell version
| |
| | |
| == Quorum ==
| |
| * Add/remove/replace children at runtime
| |
| * children mirroring and replacing done and merged
| |
| | |
| == I/O throttling groups ==
| |
| * BDSes already have multiple children, do they need multiple parents as well? (And be able to distinguish them)
| |
| * Action:
| |
| ** Add bpsgroup=<name> property to -drive [Benoit]
| |
| *** First drive using bpsgroup= can set the initial group bps/iops values
| |
| *** Successive drives will get EBUSY if they try to set group bps/iops values
| |
| *** throttle_set_limits on drive0 linked to a group will update the group bps/iops values
| |
| *** When last drive using group is deleted, the group is destroyed too
| |
| ** Patchs written an on the list for review
| |
| | |
| == Dynamic graph reconfiguration (e.g. adding block filters, taking snapshots, etc.) ==
| |
| * Where does the new node get inserted and how to specify how it is linked up with the existing nodes?
| |
| ** On a given "arrow" between two nodes (only works with 1 child, 1 parent)
| |
| ** On a given set of arrows (possibly more complex than what is really needed?)
| |
| * How does removing a node work with more than one child of the deleted node?
| |
| * Keep using the existing QMP command for I/O throttling for now, until we understand the general problem reasonably well
| |
| * Action:
| |
| ** Figure out the general problem
| |
| ** Split I/O throttling off into own BDS [Benoît]
| |
| *** Requires some care with snapshots etc.
| |
| == Proper specification for blockdev-add ==
| |
| * What does -drive add?
| |
| * Filename parsing and protocol detectioin
| |
| * Format probing
| |
| * Desugaring -drive
| |
| * What does BDRV_O_PROTOCOL mean?
| |
| ** Disable format probing
| |
| ** Parse protocol name from filename (but not from options QDict)
| |
| *** Put filename then into options QDict
| |
| ** Set bs->growable
| |
| ** Disable adding of a bs->file layer
| |
| ** Ignore BDRV_O_SNAPSHOT
| |
| ** Which callers need which of these properties?
| |
| * Action:
| |
| ** Convert network block drivers to QDict options (keep legacy filename parsing for compatibility) [volunteer?]
| |
| ** Add network block drivers to blockdev-add [volunteer?]
| |
| ** Translate bdrv_open() arguments into options qdict, if appropriate [Kevin]
| |
| *** Translate legacy "filename" to qdict
| |
| ** Specify bdrv_open() behavior (especially magic) [Kevin]
| |
| == BDS graph rules and manipulating arbitrary nodes ==
| |
| * A proper design: iterate children, safely manipulate graph
| |
| ** Action:
| |
| *** Get rid of bdrv_swap() and update child/parent pointers instead (depends on BlockBackend) [Markus?]
| |
| **** Add notifier list to BDS so users can get updated when pointer changes
| |
| ***** bdrv_register_bs_pointer(bs, &mystruct->bs) /* automatically refresh pointer */
| |
| *** Add parents and child list to BlockDriverState (could be realloc array or just a function interface that operates on ->file/->backing_hd) [nice to have]
| |
| **** Audit drivers
| |
| ***** Especially VMDK and quorum
| |
| ***** Make them use generic child interface
| |
| ***** Use child list where generic block layer currently hardcodes ->backing_hd and ->file
| |
| * Mutual exclusion of operations/background jobs (bs->in_use / BlockOpType)
| |
| ** Streaming in two different parts of the backing chain - allowed? (Benoît though that not, but does anything break?)
| |
| ** Does streaming only require that streamed images stay read-only (i.e. backing chain segment on which the operation is performed)
| |
| ** Live commit in the opposite direction at the same time?
| |
| ** Action:
| |
| *** Draw up matrix of operations (mirror, stream, resize, etc)
| |
| *** Make op blocker mechanism use matrix as data instead of code (define an array)
| |
| *** Enforce that new QMP/QAPI commands and block jobs add themselves to the matrix
| |
| *** Recursively add blockers to child nodes (driver method?) [Benoit]
| |
| **** Done and pass the test: up for review
| |
| * Arbitrary nodes
| |
| ** drive-mirror of arbitrary node
| |
| ** block-stream of arbitrary node
| |
| ** Jeff Cody's block-commit of arbitrary node patch series
| |
| ** Action:
| |
| *** Add base-nodename argument to block-stream command [Jeff]
| |
| *** Add top-nodename argument to block-stream command [Benoit]
| |
| **** If command can modify part of a backing chain, need to add option to update the parent's backing filename field on disk! [Jeff]
| |
| **** Add optional backing-filename argument (since libvirt may use fd passing and QEMU's filename is useless) [Jeff]
| |
| **** Add boolean whether to update backing file (for users who don't need to override backing filename)
| |
| *** drive-mirror (block-mirror) of arbitrary node [Benoit]
| |
| *** Deprecate filename references in QMP commands in favour of node names (e.g. streaming base) [Jeff?]
| |
| | |
| == BlockBackend split ==
| |
| * Action:
| |
| ** Split BlockBackend from BlockDriverState [Markus]
| |
| ** bdrv_new+bdrv_open and bdrv_close+bdrv_unref should be same, eliminate ENOMEDIUM semantics
| |
| ** Make block driver private embedded in BlockDriverState instead of opaque pointer
| |
| == Dataplane programming model recap ==
| |
| * What do block drivers need to be careful of?
| |
| * Any comments on new docs/dataplane.txt documentation?
| |
| * Action:
| |
| ** AioContext assertions to prevent callbacks in wrong event loop [Stefan]
| |
| == geometry and block size passthrough for host devices ==
| |
| * See the patches from Ekaterina Tumanova. https://lists.nongnu.org/archive/html/qemu-devel/2014-07/msg03791.html
| |
| * On s390 we have a stricter requirement of passing through geometry and block size for DASD disk devices as this geometry is more than a imaginary number triplet. It is required for proper partition detection and is actually a property of the real (or better emulated on storage servers like DS8x000) disk.
| |
| == Adding QMP to qemu-nbd ==
| |
| Patches are on the list
| |
| == Adding TLS to NBD ==
| |
| NBD does not have support encryption. Something similar to VNC TLS is needed for NBD since the network may not be trusted. Both qemu-nbd and the run-time NBD server must support TLS. The NBD client also needs TLS support.
| |
| == Export QEMU volumes as ISCSI or FCOE ==
| |
| * Andy Grover is working on implementing a preliminary tcmu-runner plugin using the block layer
| |
| * a QMP socket would be needed to make this usable in a cloud context
| |
| | |
| == Dependency graph ==
| |
| (paste on http://yuml.me/diagram/scruffy/class/draw)
| |
| <code>
| |
| [Image fleecing]
| |
| [Incremental backup]
| |
| [qcow2 improvements]
| |
| [Make qemu-img use QMP command implementations internally]
| |
| [Recursive Op Blocker]->[Category Op Blocker]
| |
| [Category Op Blocker]->[intermediate layers live streaming]
| |
| [Category Op Blocker]->[Basic infrastructure for blockdev-add]
| |
| [Category Op Blocker]->[drive-mirror (block-mirror) of arbitrary node]
| |
| [Category Op Blocker]->[Jeff Cody's block-commit of arbitrary node]
| |
| [Proper specification for blockdev-add]->[Basic infrastructure for blockdev-add]
| |
| [Basic infrastructure for blockdev-add]->[blockdev-add + blockdev-del QMP interface]
| |
| [Image formats]
| |
| [virtio-blk data plane]->[Multiqueue block layer]
| |
| [virtio-scsi data plane]
| |
| [Split BlockBackend from BlockDriverState]->[Get rid of bdrv_swap]
| |
| [Split BlockBackend from BlockDriverState]->[BlockFilter and dynamic reconfiguration of the BDS graph]
| |
| [BlockFilter and dynamic reconfiguration of the BDS graph]->[New design for better encryption support]
| |
| [BlockFilter and dynamic reconfiguration of the BDS graph]->[Active mirroring]
| |
| [BlockFilter and dynamic reconfiguration of the BDS graph]->[Throttle as a filter ?]
| |
| [Remove bs->job field and allow multiple jobs on a BDS]
| |
| [iotests.py - Python module for writing qemu-iotests]->[Test Infrastructure]
| |
| [Tests for -drive discard]
| |
| [Adding TLS to NBD]
| |
| [AHCI emulation]
| |
| [Block job I/O throttling]
| |
| [I/O accounting]
| |
| [Add guard page to bottom of coroutine stack]
| |
| [I/O throttling groups]
| |
| [QMP added to qemu-nbd] -> [Export QEMU volumes as ISCSI or FCOE with QMP]
| |
| </code>
| |
| (result: http://yuml.me/c2e9edb8)
| |
