Features/RDMALiveMigration
summary
Live migration using RDMA instead of TCP.
contact
- Name: Michael Hines
- Email: mrhines@us.ibm.com
description
Uses standard OFED software stack, which supports both RoCE and Infiniband.
design
- In order to provide maximum cross-device compatibility, we use the librdmacm library, which abstracts out the RDMA capabilities of each individual type of RDMA device, including infiniband, iWARP, as well as RoCE. This patch has been tested on both RoCE and infiniband devices from Mellanox.
- A new file named "migration-rdma.c" contains the core code required to perform librdmacm connection establishment and the transfer of actual RDMA contents.
- Files "arch_init.c" and "savevm.c" have been modified to transfer the VM's memory in the standard live migration path using RMDA memory instead of using TCP.
- Currently, the XBZRLE capability and the detection of zero pages (dup_page()) significantly slow down the empircal throughput observed when RDMA is activated, so the code path skips these capabilities when RDMA is enabled. Hopefully, we can stop doing this in the future and come up with a way to preserve these capabilities simultaneously with the use of RDMA.
usage
Command-line:
- Start like this:
performance
TODO
- Integrate with XOR-based run-length encoding (if possible)
- Stop skipping the zero-pages check