Features/RDMALiveMigration

From QEMU
Revision as of 22:07, 11 February 2013 by Michael Hines (talk | contribs)

summary

Live migration using RDMA instead of TCP.

contact

description

Uses standard OFED software stack, which supports both RoCE and Infiniband.

design

  1. In order to provide maximum cross-device compatibility, we use the librdmacm library, which abstracts out the RDMA capabilities of each individual type of RDMA device, including infiniband, iWARP, as well as RoCE. This patch has been tested on both RoCE and infiniband devices from Mellanox.
  2. A new file named "migration-rdma.c" contains the core code required to perform librdmacm connection establishment and the transfer of actual RDMA contents.
  3. Files "arch_init.c" and "savevm.c" have been modified to transfer the VM's memory in the standard live migration path using RMDA memory instead of using TCP.
  4. Currently, the XBZRLE capability and the detection of zero pages (dup_page()) significantly slow down the empircal throughput observed when RDMA is activated, so the code path skips these capabilities when RDMA is enabled. Hopefully, we can stop doing this in the future and come up with a way to preserve these capabilities simultaneously with the use of RDMA.

usage

Command-line:

  • Start like this:

performance

Perf.png

TODO

  • Integrate with XOR-based run-length encoding (if possible)
  • Stop skipping the zero-pages check

links