Features/AutoconvergeLiveMigration

Summary

This feature provides a method for dynamically throttling guest cpus during autoconverge migrations in order to force migration of very busy guests to complete.

Author

• Name: Jason J. Herne
• Email: jjherne _AT_ linux.vnet.ibm.com

Current Status

• Last Updated: 2015-09
• Released In: QEMU 2.5

Scope

• Changes in Qemu only, no OS support required
• Effective on all architectures
• Requires no Guest OS support

Detailed Description

With extreme memory write intensive workloads, normal live migration will never complete because the guest is writing to memory faster than Qemu can transfer the memory changes to the destination system. In this case normal migration will continue forever, not making enough progress to stop the guest and proceed to the non-live "finishing up" phase of migration.

This feature provides a method for slowing down guest execution speed, thus hopefully, also slowing down guest memory write speed. As time advances autoconverge will continually increase the amount of guest cpu throttling until guest memory write speed slows enough to allow the guest to be stopped and migration to finish.

It should be noted that some workloads may not benefit from autoconverge. There may be workloads where memory write speed is not significantly dependent on cpu execution speed. Autoconverge will likely be ineffective for these workloads as it relies on the assumption that throttling cpu will also effectively throttle memory write speed.

Autoconverge was originally introduced in Qemu 1.6 to deal with the above situation however the mechanism used to slow guest cpus was not dynamic and ineffective for many memory write intensive workloads. As of Qemu 2.5 dynamic throttling has been added to autoconverge dramatically increasing its effectiveness.

Benefits to QEMU and upper layers

Many previously non-migratable workloads become migratable.

Added commands / state

Migration parameters:

• x-cpu-throttle-initial
• x-cpu-throttle-increment

How to use

From the Qemu monitor before starting your migration:

 migrate_set_capability auto-converge on

Now start your migration as you normally would:

 migrate -d tcp:destination:port

If auto-converge activates throttling then the throttling percentage will be displayed on the info migrate command:

 cpu throttle percentage: 20

The initial throttling percentage defaults to 20%. This value is a migration parameter and can be changed with migrate_set_parameter:

 migrate_set_parameter x-cpu-throttle-initial 30

If after a period of time the migration has still not completed then throttling is incremented. This process continues until migration completes or we reach 99% throttled. By default the throttling rate is always incremented by 10%. This increment is a migration parameter and can be changed with the migrate_set_parameter:

 migrate_set_parameter x-cpu-throttle-increment 15

Both x-cpu-throttle-initial and x-cpu-throttle-increment can be queried with info migrate_parameters:

 info migrate_parameters

List of commits

• dc3256272cf70b2152279b013a8abb16e0f6fe96 migration: Disambiguate MAX_THROTTLE
• 4782893e099cde24c0b10a48d0412eea575ddcb3 qmp/hmp: Add throttle ratio to query-migrate and info migrate
• 070afca258f973c704dcadf2769aa1ca921209a1 migration: Dynamic cpu throttling for auto-converge
• 1626fee3bdbb295d5e8aff800f7621357bb376d6 migration: Parameters for auto-converge cpu throttling
• 2adcc85d407c1ab985f5abed808c78dbb84f4773 cpu: Provide vcpu throttling interface

Supporting documentation

• Qemu mailing List Conversation about improving autoconverge