Features/SnapshottingImprovements: Difference between revisions

From QEMU
No edit summary
No edit summary
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Summary ==
== Summary ==


Refactor the '''savevm''', '''loadvm''' and '''info snapshots''' commands to facilitate the transtion to QMP. This page is being maintained by the GSoC student Miguel Di Ciurcio Filho as part the [[Google_Summer_of_Code_2010/QMP|QMP project]]. All feedback is appreciated.
Adjust the '''savevm''', '''loadvm''' and '''info snapshots''' commands to facilitate the transtion to QMP.
 
This page is being maintained by the GSoC student Miguel Di Ciurcio Filho as part the [[Google_Summer_of_Code_2010/QMP|QMP project]]. All feedback is appreciated.


== Issues ==
== Issues ==


{| cellpadding="4" cellspacing="0" border="1"
{| cellpadding="4" cellspacing="0" border="1"
! Index
! Problem
! Problem
! Difficulty
! Difficulty
Line 11: Line 14:
! Status
! Status
|-
|-
| 1
| [[#Snapshot existence is not verified before bdrv_snapshot_goto() | Snapshot existence is not verified before bdrv_snapshot_goto()]]
| Easy
|
* Improve the resilience of the load_vmstate() function, doing further and better ordered tests.
|
* Patch applied to the block branch: http://repo.or.cz/w/qemu/kevin.git/commit/d98644fe6fc05ce75c203c761920a6948b0ad853
|-
| 2
| [[#Monitor shows wrong information about snapshots | Monitor shows wrong information about snapshots]]
| [[#Monitor shows wrong information about snapshots | Monitor shows wrong information about snapshots]]
| Easy
| Easy
|
|
* Consolidate the snapshot information to show only snapshots fully available
* Consolidate the snapshot information to show only snapshots fully available
* Do not display block device information anymore
* Do not display any block device information anymore
| rowspan=3 |
* Patch v2: http://lists.gnu.org/archive/html/qemu-devel/2010-08/msg00346.html
|-
| 3
| [[#Do not allow empty names |Do not allow empty names]]
| Easy
|
|
* If the user does not specify a name, we must create a name from a template, using a timestamp like: '''vm-20100801121212'''.
|-
|-
| 4
| [[#Snapshot overwriting|Snapshot overwriting]]
| [[#Snapshot overwriting|Snapshot overwriting]]
| Easy
| Easy
Line 23: Line 43:
* Don't save the snapshot if the name given by the user already exists.
* Don't save the snapshot if the name given by the user already exists.
* If the user really wants to overwrite it, add a '''-f''' parameter to '''savevm'''.
* If the user really wants to overwrite it, add a '''-f''' parameter to '''savevm'''.
* Do we really need to support ''savevm <ID>''? Today this overwrites a VM with the specified ID.
* In case of overwriting, do not reuse the ID, just the name.
| Patch sent to qemu-devel: http://lists.gnu.org/archive/html/qemu-devel/2010-07/msg01701.html
|-
| 5
| [[#snapshot ID collision|snapshot ID collision]]
| Easy
|
* Use an stronger ID numbering system instead of simple sequential decimal numbers
* Take the ID generation out of the block drivers
* UUID seams a good mechanism, needs to depend on libuuid-dev
| Patch v1: http://lists.gnu.org/archive/html/qemu-devel/2010-08/msg00204.html
|-
|-
| 6
| [[#ID and TAG confusion|ID and TAG confusion]]
| [[#ID and TAG confusion|ID and TAG confusion]]
| Medium
| Medium
|
|
* When specifying an snapshot to be loaded or saved, only use names/tags.
* When specifying an snapshot to be loaded or saved, only use names/tags.
* The ID information must not be exposed to the user, unless when using '''qemu-img''' to inspect a disk image.
* Have separate commands to load by ID and name, or add parameters to explicitly distinguish one from another in '''loadvm'''.
* If the user does not specify a name, we must create a name from a template, like: ''vm-20100801121212''.
* Do not support '''savevm <ID>''' anymore. We are trying not to expose ID information to the user.
|
|
|-
| [[#snapshot ID collision|snapshot ID collision]]
| Hard
|
* Verify the availability of IDs before loading anything
* Use an stronger ID numbering system instead of simple sequential decimal numbers
* UUID seams good enough
|
* Patch to check all devices before loading sent and acked by Kevin Wolf: http://lists.gnu.org/archive/html/qemu-devel/2010-07/msg01065.html
|-  
|-  
| 7
| [[#inheritance|inheritance]]
| [[#inheritance|inheritance]]
| Hard
| Hard
Line 50: Line 71:
* When loading and saving snapshots, verify if extra_data_size > 0 and feed the information info QEMUSnapshotInfo.parent_id_str
* When loading and saving snapshots, verify if extra_data_size > 0 and feed the information info QEMUSnapshotInfo.parent_id_str
* libvirt wants this feature
* libvirt wants this feature
* Update the '''info snapshots''' output to show the parent_id_str?
* Update the '''info snapshots''' output to hide ID and PARENT_ID information
|
* Add another command to show full details of snapshots
* The ID information must not be exposed to the user, unless when using '''qemu-img''' to inspect a disk image.
| Patch v1: http://lists.gnu.org/archive/html/qemu-devel/2010-08/msg00204.html
|}
|}
== Snapshot existence is not verified before bdrv_snapshot_goto() ==
This patch improves the resilience of the load_vmstate() function, doing
further and better ordered tests.
In load_vmstate(), if there is any error on bdrv_snapshot_goto(), except if the
error is on VM state device, load_vmstate() will return zero and the VM will be
started with major corruption chances.


== Monitor shows wrong information about snapshots ==
== Monitor shows wrong information about snapshots ==
Line 93: Line 124:
  3                                1.5M 2010-07-26 17:26:49  00:00:13.245
  3                                1.5M 2010-07-26 17:26:49  00:00:13.245
  4                                1.5M 2010-07-26 19:01:00  00:00:46.763  
  4                                1.5M 2010-07-26 19:01:00  00:00:46.763  
== Do not allow empty names ==
When running:
(qemu) savevm
The snapshot is saved without a name and can only be loaded using the generated ID. The idea is to not have IDs exposed, so every snapshot must have a name.


== Snapshot overwriting ==
== Snapshot overwriting ==


* Is this the expected behavior?
Is this the expected behavior? Don't think so.
** Don't think so.
* What other projects do?
** VirtualBox does not allow specifying an ID, just a name and description of an snapshot. http://www.virtualbox.org/manual/ch01.html#snapshots
** VMware does not allow specifying and ID, just a name and description. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1015180
** XenServer does not allow specifying an ID, just a name. http://support.citrix.com/servlet/KbServlet/download/23735-102-646135/Administrators_Guide.pdf - page 127


=== By id ===
=== By id ===
Line 119: Line 153:
  1                                3.9M 2010-07-21 17:08:31  00:00:03.696
  1                                3.9M 2010-07-21 17:08:31  00:00:03.696
  2                                3.9M 2010-07-21 17:08:53  00:00:25.918 <-- 17:08:53
  2                                3.9M 2010-07-21 17:08:53  00:00:25.918 <-- 17:08:53
Ideas for improvements:
* Do not allow the user to specify an ID when saving a VM, just a TAG.


=== By tag ===
=== By tag ===
Line 142: Line 173:
Ideas for improvements:
Ideas for improvements:
* Do not allow the user to overwrite an snapshot, even if saving with the same name (will have different IDs anyway)
* Do not allow the user to overwrite an snapshot, even if saving with the same name (will have different IDs anyway)
== snapshot ID collision ==
qemu-img create -f qcow2 diskA.qcow2 10G
qemu-img create -f qcow2 diskB.qcow2 10G
qemu -hda diskA.qcow2
(qemu) savevm a
(qemu) savevm b
(qemu) savevm c
qemu -hda diskB.qcow2
(qemu) savevm a
(qemu) savevm b
qemu -hda diskB.qcow2 -hdb diskA.qcow2
(qemu) loadvm b
diskA.qcow2 and diskB.qcow2 have both the same snapshot identification '''b''' and '''2''', but they were taken at different moments and must not be loaded. We need an stronger ID mechanism to identify snapshots among different disks. It is not a common case, but we reduce the chance of anyone having trouble.


== ID and TAG confusion ==
== ID and TAG confusion ==
What other projects do?
* VirtualBox does not allow specifying an ID, just a name and description of an snapshot. http://www.virtualbox.org/manual/ch01.html#snapshots
* VMware does not allow specifying an ID, just a name and description. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1015180
* XenServer does not allow specifying an ID, just a name. http://support.citrix.com/servlet/KbServlet/download/23735-102-646135/Administrators_Guide.pdf - page 127


Starting a VM with no previous snapshot on the disks.
Starting a VM with no previous snapshot on the disks.
Line 171: Line 226:
  (qemu) loadvm_byid 3
  (qemu) loadvm_byid 3
  (qemu) loadvm_bytag xxx
  (qemu) loadvm_bytag xxx
== snapshot ID collision ==
A VM is loaded with two devices, one device with three previous snapshots and
the other device with two previous snapshots.
I take a new snapshot of the VM without a name: the first device will
save the snapshot as id_str 4 and the second device with id_str 3.
I play around for a while and I want to restore the VM to my latest
state. When 'info snapshots' is run, only the first device is show
with the VM state, and the latest ID shown will be 4.
Ideas for improvements:
* Use an stronger ID model, not sequential and large enough to be virtually unique.
=== loading without checking all devices ===
So, I fire up loadvm 4. qemu will try to load by id or name anyway,
will succeed loading the VM state from the first device, but will fail
to load an non-existent snapshot id from the second device and will
still be running with the second device out of sync with the VM state.
=== equal IDs on different devices with different states (id collision) ===
Running again:
(qemu) savevm 4
Continuing on the same case.
The previously saved state identified by id_str 4 will be deleted and
replaced by a new one, but only in the first device of my example
above. The second device will get a new snapshot, since it does not
have a id_str equal to 4.
If I run loadvm 4 again, the snapshot id 3 in the second device will be left
there in an state diferent from the snapshot id 3 from the first device.
Ideas for improvements:
* Use an stronger ID model, not sequential and large enough to be virtually unique.


== inheritance ==
== inheritance ==
Line 228: Line 245:
  5        e                      3.9M 2010-07-21 17:33:28  00:00:15.548
  5        e                      3.9M 2010-07-21 17:33:28  00:00:15.548


Snapshot '''e''' is newer than '''d''', but it is an state saved right after '''b''', and not '''d''' as implied by the ids and dates.
Snapshot '''e''' is newer than '''d''', but it is an state saved right after '''b''', and not '''d''' as implied by the ids and dates. libvirt wants this.
 
Ideas for improvements:
* Store the previous snapshot taken as parent_id of a new snapshot.
 


== QMP ==
== QMP ==


Here is a list of suggested commands to manipulate snapshots information on a QEMU VM.
Here is a list of suggested commands to manipulate snapshots information on a QEMU VM.
== query-snapshot-status ==
=== query-snapshot-status ===
Returns '''true''' when the VM can be snapshotted, '''false''' otherwise.
Returns '''true''' when the VM can be snapshotted, '''false''' otherwise.


Line 317: Line 330:
       }
       }
   ]
   ]
[[Category:Obsolete feature pages]]

Latest revision as of 15:26, 11 October 2016

Summary

Adjust the savevm, loadvm and info snapshots commands to facilitate the transtion to QMP.

This page is being maintained by the GSoC student Miguel Di Ciurcio Filho as part the QMP project. All feedback is appreciated.

Issues

Index Problem Difficulty Proposed solution Status
1 Snapshot existence is not verified before bdrv_snapshot_goto() Easy
  • Improve the resilience of the load_vmstate() function, doing further and better ordered tests.
2 Monitor shows wrong information about snapshots Easy
  • Consolidate the snapshot information to show only snapshots fully available
  • Do not display any block device information anymore
3 Do not allow empty names Easy
  • If the user does not specify a name, we must create a name from a template, using a timestamp like: vm-20100801121212.
4 Snapshot overwriting Easy
  • Don't save the snapshot if the name given by the user already exists.
  • If the user really wants to overwrite it, add a -f parameter to savevm.
  • In case of overwriting, do not reuse the ID, just the name.
5 snapshot ID collision Easy
  • Use an stronger ID numbering system instead of simple sequential decimal numbers
  • Take the ID generation out of the block drivers
  • UUID seams a good mechanism, needs to depend on libuuid-dev
Patch v1: http://lists.gnu.org/archive/html/qemu-devel/2010-08/msg00204.html
6 ID and TAG confusion Medium
  • When specifying an snapshot to be loaded or saved, only use names/tags.
  • Have separate commands to load by ID and name, or add parameters to explicitly distinguish one from another in loadvm.
  • Do not support savevm <ID> anymore. We are trying not to expose ID information to the user.
7 inheritance Hard
  • Add char parent_id_str[128] to QEMUSnapshotInfo and char *parent_id_str to QCowSnapshot
  • Use the extra_data_size field of the qcow2 header to store the parent_id_str
  • When loading and saving snapshots, verify if extra_data_size > 0 and feed the information info QEMUSnapshotInfo.parent_id_str
  • libvirt wants this feature
  • Update the info snapshots output to hide ID and PARENT_ID information
  • Add another command to show full details of snapshots
  • The ID information must not be exposed to the user, unless when using qemu-img to inspect a disk image.
Patch v1: http://lists.gnu.org/archive/html/qemu-devel/2010-08/msg00204.html

Snapshot existence is not verified before bdrv_snapshot_goto()

This patch improves the resilience of the load_vmstate() function, doing further and better ordered tests.

In load_vmstate(), if there is any error on bdrv_snapshot_goto(), except if the error is on VM state device, load_vmstate() will return zero and the VM will be started with major corruption chances.

Monitor shows wrong information about snapshots

The output generated by 'info snapshots' shows only snapshots that exist on the block device that saves the VM state. This output can cause an user to erroneously try to load an snapshot that is not available on all block devices.

$ qemu-img snapshot -l xxtest.qcow2
Snapshot list:
ID        TAG                 VM SIZE                DATE       VM CLOCK
1                                1.5M 2010-07-26 16:51:52   00:00:08.599
2                                1.5M 2010-07-26 16:51:53   00:00:09.719
3                                1.5M 2010-07-26 17:26:49   00:00:13.245
4                                1.5M 2010-07-26 19:01:00   00:00:46.763
$ qemu-img snapshot -l xxtest2.qcow2
Snapshot list:
ID        TAG                 VM SIZE                DATE       VM CLOCK
3                                   0 2010-07-26 17:26:49   00:00:13.245
4                                   0 2010-07-26 19:01:00   00:00:46.763 

Current output:

$ qemu -hda xxtest.qcow2 -hdb xxtest2.qcow2 -monitor stdio -vnc :0
QEMU 0.12.4 monitor - type 'help' for more information
(qemu) info snapshots
Snapshot devices: ide0-hd0
Snapshot list (from ide0-hd0):
ID        TAG                 VM SIZE                DATE       VM CLOCK
1                                1.5M 2010-07-26 16:51:52   00:00:08.599
2                                1.5M 2010-07-26 16:51:53   00:00:09.719
3                                1.5M 2010-07-26 17:26:49   00:00:13.245
4                                1.5M 2010-07-26 19:01:00   00:00:46.763 

Snapshots 1 and 2 do not exist on xxtest2.qcow, but they are displayed anyway.

New output:

(qemu) info snapshots
ID        TAG                 VM SIZE                DATE       VM CLOCK
3                                1.5M 2010-07-26 17:26:49   00:00:13.245
4                                1.5M 2010-07-26 19:01:00   00:00:46.763 


Do not allow empty names

When running:

(qemu) savevm

The snapshot is saved without a name and can only be loaded using the generated ID. The idea is to not have IDs exposed, so every snapshot must have a name.

Snapshot overwriting

Is this the expected behavior? Don't think so.

By id

(qemu) savevm 
(qemu) savevm 
(qemu) info snapshots 
Snapshot devices: ide0-hd0
Snapshot list (from ide0-hd0):
ID        TAG                 VM SIZE                DATE       VM CLOCK
1                                3.9M 2010-07-21 17:08:31   00:00:03.696
2                                3.9M 2010-07-21 17:08:33   00:00:05.419 <-- 17:08:33
(qemu) savevm 2 <overwrittes the snapshot with id 2>
(qemu) info snapshots 
Snapshot devices: ide0-hd0
Snapshot list (from ide0-hd0):
ID        TAG                 VM SIZE                DATE       VM CLOCK
1                                3.9M 2010-07-21 17:08:31   00:00:03.696
2                                3.9M 2010-07-21 17:08:53   00:00:25.918 <-- 17:08:53

By tag

(qemu) savevm test1
(qemu) savevm test2
 (qemu) info snapshots 
Snapshot devices: ide0-hd0
Snapshot list (from ide0-hd0):
ID        TAG                 VM SIZE                DATE       VM CLOCK
1         test1                  3.9M 2010-07-21 17:12:11   00:00:08.126 <-- 17:12:11
2         test2                  3.9M 2010-07-21 17:12:19   00:00:16.053
(qemu) savevm test1
(qemu) info snapshots 
Snapshot devices: ide0-hd0
Snapshot list (from ide0-hd0):
ID        TAG                 VM SIZE                DATE       VM CLOCK
2         test2                  3.9M 2010-07-21 17:12:19   00:00:16.053
1         test1                  3.9M 2010-07-21 17:12:33   00:00:29.221 <-- 17:12:33

Ideas for improvements:

  • Do not allow the user to overwrite an snapshot, even if saving with the same name (will have different IDs anyway)

snapshot ID collision

qemu-img create -f qcow2 diskA.qcow2 10G
qemu-img create -f qcow2 diskB.qcow2 10G
qemu -hda diskA.qcow2
(qemu) savevm a
(qemu) savevm b
(qemu) savevm c
qemu -hda diskB.qcow2
(qemu) savevm a
(qemu) savevm b
qemu -hda diskB.qcow2 -hdb diskA.qcow2
(qemu) loadvm b

diskA.qcow2 and diskB.qcow2 have both the same snapshot identification b and 2, but they were taken at different moments and must not be loaded. We need an stronger ID mechanism to identify snapshots among different disks. It is not a common case, but we reduce the chance of anyone having trouble.

ID and TAG confusion

What other projects do?

Starting a VM with no previous snapshot on the disks.

(qemu) savevm 2
(qemu) savevm
(qemu) info snapshots 
Snapshot devices: ide0-hd0
Snapshot list (from ide0-hd0):
ID        TAG                 VM SIZE                DATE       VM CLOCK
1         2                      1.5M 2010-07-21 16:55:48   00:00:15.581
2                                1.5M 2010-07-21 16:55:55   00:00:21.501
(qemu) loadvm 2 <loads the VM with the id 2>

It is not clear witch snapshot is going to be loaded.

Ideas for improvements:

  • savevm tests the argument to check if it is an ID or a TAG. Make it accept only a TAG and do not allow the user to specify and ID.
  • loadvm does the same thing. The user must specify if an ID or TAG is wanted. Suggestions:
(qemu) loadvm id=3
(qemu) loadvm tag=xxx
(qemu) loadvm xxx <error>
(qemu) loadvm 3 <error>

In case loadvm cannot be changed due to retro-compatibility, introduce a new command. Suggestions:

(qemu) load_vm id=3
(qemu) load_vm tag=xxx
(qemu) loadvm_byid 3
(qemu) loadvm_bytag xxx

inheritance

The only way to know witch snapshot came before of after another is to look at the date, but it is not possible to fully deduct the relationship. e.g.:

(qemu) savevm a
(qemu) savevm b
(qemu) savevm c
(qemu) savevm d
(qemu) loadvm b
(qemu) savevm e
(qemu) info snapshots 
Snapshot devices: ide0-hd0
Snapshot list (from ide0-hd0):
ID        TAG                 VM SIZE                DATE       VM CLOCK
1         a                      3.9M 2010-07-21 17:32:55   00:00:06.144
2         b                      3.9M 2010-07-21 17:33:00   00:00:11.103
3         c                      3.9M 2010-07-21 17:33:13   00:00:22.953
4         d                      3.9M 2010-07-21 17:33:16   00:00:26.289
5         e                      3.9M 2010-07-21 17:33:28   00:00:15.548

Snapshot e is newer than d, but it is an state saved right after b, and not d as implied by the ids and dates. libvirt wants this.

QMP

Here is a list of suggested commands to manipulate snapshots information on a QEMU VM.

query-snapshot-status

Returns true when the VM can be snapshotted, false otherwise.

-> {"execute": "query-snapshot-status"}
<- {"return": true }
  • Not sure if QMP allow returning just a boolean.

snapshot-create

Runs savevm.

-> {"execute": "snapshot-create", "arguments": {"name": "vm_before_upgrade"} }
<- { "return": {} }
  • "name" is mandatory and must be unique.
  • Needs to return any errors.

snapshot-delete

Runs delvm.

-> {"execute": "snapshot-delete", "arguments": {"name": "vm_before_upgrade"} }
<- { "return": {} }
  • "name" is mandatory and must be unique.
  • Needs to return any errors.

snapshot-load

Runs loadvm.

-> {"execute": "snapshot-load", "arguments": {"name": "vm_before_upgrade"} }
<- { "return": {} }
  • "name" is mandatory and must be unique.
  • Needs to return any errors.

query-snapshots

-> {"execute": "query-snapshots"}
<- {
"return": [
   {           
     "name": "vm_before_upgrade",
     "parent_name": "",
     "id": "3fab28c9-a6bd-4659-b9bc-683780d8a2d5",
     "parent_id": "00000000-0000-0000-0000-000000000000",
     "date": 20100415164856
   },
   {           
     "name": "vm_before_upgrade-2",
     "parent_name": "vm_before_upgrade",
     "id": "e693dce2-6e12-4474-9bbf-2b3f97323423",
     "parent_id": "00000000-0000-0000-0000-000000000000",
     "date": 20100415100000
   }
  ]        
}
  • Do we really need to expose IDs or names should be enough?
  • Does JSON has a date type of just using a integer like YYYYMMDDHHMMSS would be better?

Instead of a list, how about this:

-> {"execute": "query-snapshots"}
<- {
"return": [
   {           
      "vm_before_upgrade": { 
		      "id": "3fab28c9-a6bd-4659-b9bc-683780d8a2d5", 
		      "date": 20100415164856, 
		      "child": { "vm_before_upgrade-2": 
  					"id": "e693dce2-6e12-4474-9bbf-2b3f97323423",
 				        "date": 20100415100000 
			       }
                           }
        "random-backup": {
 		      "id": "abcdefcc-a6bd-4659-b9bc-683780d8a2d5",
 		      "date": 20111111111111
                         }
     }
  ]