Features/ImageStreamingAPI: Difference between revisions

From QEMU
No edit summary
 
(8 intermediate revisions by 2 users not shown)
Line 1: Line 1:
=Changelog=
=Changelog=
v7:
* Existing libvirt code uses offset/len instead of the position/end as specified in v3.  Revert back to offset/len names for compatibility.
v6:
* Note about 'NotSupported' versus 'Unsupported' QMP errors: the 'Unsupported' error was introduced while the Image Stream API was being developed with its own 'NotSupported' error.  libvirt has already merged and released with the 'NotSupported' error so we really need to stick with that.
v5:
* Change non-existent 'NotSupported' error to 'Unsupported'
v4:
* Rename block_stream_set_speed to block_job_set_speed, slight semantics change
* Add 'speed' value to stream info object
* Clarify BLOCK_JOB_COMPLETED 'error' field semantics
v3:
v3:
* Rename cancel and info operations to make a reusable block job API
* Rename cancel and info operations to make a reusable block job API
Line 32: Line 46:
retains its backing file.  A new streaming operation can be started at a later
retains its backing file.  A new streaming operation can be started at a later
time.
time.
The block_job_set_speed command adjusts the rate limit for a streaming
operation.  This can be used to limit the impact on storage performance during
streaming.


The command synopses are as follows:
The command synopses are as follows:
Line 69: Line 87:
  fields:
  fields:
   
   
- type:    job type ("stream" for image streaming, json-string)
  - device:  device name (json-string)
  - device:  device name (json-string)
  - end:      maximum progress value (json-int)
  - len:      maximum progress value (json-int)
  - position: current progress value (json-int)
  - offset:   current progress value (json-int)
- speed:    rate limit, bytes per second (json-int)
  - error:    error message (json-string, only on error)
  - error:    error message (json-string, only on error)
   
   
  The completion event is raised both on success and on failure.  On
  The completion event is raised both on success and on failure.  On
  success position is equal to end.  On failure position and end can be
  success offset is equal to len.  On failure offset and len can be
  used to indicate at which point the operation failed.
  used to indicate at which point the operation failed.
On failure the error field contains a human-readable error message.  There are
no semantics other than that streaming has failed and clients should not try
to interpret the error string.
   
   
  Examples:
  Examples:
Line 83: Line 107:
  <- { "return":  {} }
  <- { "return":  {} }
   
   
  block_stream_set_speed
  block_job_set_speed
  ----------------------
  -------------------
   
   
  Set maximum speed for a block streaming operations.
  Set maximum speed for a background block operation.
   
   
  This is a per-block device setting and also affect the active image
  This command can only be issued when there is an active block job.
streaming operation (if any).
Throttling can be disabled by setting the speed to 0.
   
   
  Arguments:
  Arguments:
Line 95: Line 120:
  - device: device name (json-string)
  - device: device name (json-string)
  - value:  maximum speed, in bytes per second (json-int)
  - value:  maximum speed, in bytes per second (json-int)
Errors:
NotSupported:    job type does not support speed setting
DeviceNotActive: streaming is not active on this device
   
   
  Example:
  Example:
   
   
  -> { "execute": "block_stream_set_speed",
  -> { "execute": "block_job_set_speed",
      "arguments": { "device": "virtio0", "value": 1024 } }
    "arguments": { "device": "virtio0", "value": 1024 } }
<- { "return": {} }
   
   
  block_job_cancel
  block_job_cancel
Line 141: Line 169:
  - type:    job type ("stream" for image streaming, json-string)
  - type:    job type ("stream" for image streaming, json-string)
  - device:  device name (json-string)
  - device:  device name (json-string)
  - end:      maximum progress value (json-int)
  - len:      maximum progress value (json-int)
  - position: current progress value (json-int)
  - offset:   current progress value (json-int)
- speed:    rate limit, bytes per second (json-int)
   
   
  Progress can be observed as position increases and it reaches end when
  Progress can be observed as offset increases and it reaches len when
  the operation completes.  Position and end have undefined units but can be
  the operation completes.  Offset and len have undefined units but can be
  used to calculate a percentage indicating the progress that has been made.
  used to calculate a percentage indicating the progress that has been made.
   
   
  Example:
  Example:
   
   
  -> { "execute": "query-block-stream" }
  -> { "execute": "query-block-jobs" }
  <- { "return":[
  <- { "return":[
       { "type": "stream", "device": "virtio0",
       { "type": "stream", "device": "virtio0",
         "end": 10737418240, "position": 709632}
         "len": 10737418240, "offset": 709632,
        "speed": 0 }
     ]
     ]
   }
   }
Line 163: Line 193:
# Create and switch to the destination file: snapshot_blkdev virtio-blk0 destination.$fmt $fmt
# Create and switch to the destination file: snapshot_blkdev virtio-blk0 destination.$fmt $fmt
# Stream the base into the image file: block_stream -a virtio-blk0
# Stream the base into the image file: block_stream -a virtio-blk0
[[Category:Completed feature pages]]

Latest revision as of 14:56, 11 October 2016

Changelog

v7:

  • Existing libvirt code uses offset/len instead of the position/end as specified in v3. Revert back to offset/len names for compatibility.

v6:

  • Note about 'NotSupported' versus 'Unsupported' QMP errors: the 'Unsupported' error was introduced while the Image Stream API was being developed with its own 'NotSupported' error. libvirt has already merged and released with the 'NotSupported' error so we really need to stick with that.

v5:

  • Change non-existent 'NotSupported' error to 'Unsupported'

v4:

  • Rename block_stream_set_speed to block_job_set_speed, slight semantics change
  • Add 'speed' value to stream info object
  • Clarify BLOCK_JOB_COMPLETED 'error' field semantics

v3:

  • Rename cancel and info operations to make a reusable block job API
  • Rename offset/size to position/end and make their units undefined for block job API
  • Add block_stream_set_speed, similar to migrate_set_speed

v2:

  • Remove iteration interface where management tool drives individual copy iterations
  • Add block_stream_cancel command (like migrate_cancel)
  • Add 'base' common backing file argument to block_stream
  • Replace QError object in BLOCK_STREAM_COMPLETED with an error string
  • Add error documentation

Image streaming API

The stream commands populate an image file by streaming data from its backing file. Once all blocks have been streamed, the dependency on the original backing image is removed. The stream commands can be used to implement post-copy live block migration and rapid deployment.

The block_stream command starts streaming the image file. Streaming is performed in the background while the guest is running.

When streaming completes successfully or with an error, the BLOCK_JOB_COMPLETED event is raised.

The progress of a streaming operation can be polled using query-block-jobs. This returns information regarding how much of the image has been streamed for each active streaming operation.

The block_job_cancel command stops streaming the image file. The image file retains its backing file. A new streaming operation can be started at a later time.

The block_job_set_speed command adjusts the rate limit for a streaming operation. This can be used to limit the impact on storage performance during streaming.

The command synopses are as follows:

block_stream
------------

Copy data from a backing file into a block device.

The block streaming operation is performed in the background until the entire
backing file has been copied.  This command returns immediately once streaming
has started.  The status of ongoing block streaming operations can be checked
with query-block-jobs.  The operation can be stopped before it has completed
using the block_job_cancel command.

If a base file is specified then sectors are not copied from that base file and
its backing chain.  When streaming completes the image file will have the base
file as its backing file.  This can be used to stream a subset of the backing
file chain instead of flattening the entire image.

On successful completion the image file is updated to drop the backing file.

Arguments:

- device: device name (json-string)
- base:   common backing file (json-string, optional)

Errors:

DeviceInUse:    streaming is already active on this device
DeviceNotFound: device name is invalid
NotSupported:   image streaming is not supported by this device

Events:

On completion the BLOCK_JOB_COMPLETED event is raised with the following
fields:

- type:     job type ("stream" for image streaming, json-string)
- device:   device name (json-string)
- len:      maximum progress value (json-int)
- offset:   current progress value (json-int)
- speed:    rate limit, bytes per second (json-int)
- error:    error message (json-string, only on error)

The completion event is raised both on success and on failure.  On
success offset is equal to len.  On failure offset and len can be
used to indicate at which point the operation failed.

On failure the error field contains a human-readable error message.  There are
no semantics other than that streaming has failed and clients should not try
to interpret the error string.

Examples:

-> { "execute": "block_stream", "arguments": { "device": "virtio0" } }
<- { "return":  {} }

block_job_set_speed
-------------------

Set maximum speed for a background block operation.

This command can only be issued when there is an active block job.

Throttling can be disabled by setting the speed to 0.

Arguments:

- device: device name (json-string)
- value:  maximum speed, in bytes per second (json-int)

Errors:
NotSupported:    job type does not support speed setting
DeviceNotActive: streaming is not active on this device

Example:

-> { "execute": "block_job_set_speed",
    "arguments": { "device": "virtio0", "value": 1024 } }

block_job_cancel
----------------

Stop an active block streaming operation.

This command returns once the active block streaming operation has been
stopped.  It is an error to call this command if no operation is in progress.

The image file retains its backing file unless the streaming operation happens
to complete just as it is being cancelled.

A new block streaming operation can be started at a later time to finish
copying all data from the backing file.

Arguments:

- device: device name (json-string)

Errors:

DeviceNotActive: streaming is not active on this device
DeviceInUse:     cancellation already in progress

Examples:

-> { "execute": "block_job_cancel", "arguments": { "device": "virtio0" } }
<- { "return":  {} }

query-block-jobs
----------------

Show progress of ongoing block device operations.

Return a json-array of all block device operations.  If no operation is
active then return an empty array.  Each operation is a json-object with the
following data:

- type:     job type ("stream" for image streaming, json-string)
- device:   device name (json-string)
- len:      maximum progress value (json-int)
- offset:   current progress value (json-int)
- speed:    rate limit, bytes per second (json-int)

Progress can be observed as offset increases and it reaches len when
the operation completes.  Offset and len have undefined units but can be
used to calculate a percentage indicating the progress that has been made.

Example:

-> { "execute": "query-block-jobs" }
<- { "return":[
      { "type": "stream", "device": "virtio0",
        "len": 10737418240, "offset": 709632,
        "speed": 0 }
   ]
 }

How live block copy works

Live block copy does the following:

  1. Create and switch to the destination file: snapshot_blkdev virtio-blk0 destination.$fmt $fmt
  2. Stream the base into the image file: block_stream -a virtio-blk0