Features/Migration-Multiple-fds

Feature Name

Use multiple FD's for migration

owner

• Name: Juan Quintela
• Email: quintela@redhat.com
• IRC: quintela (oftc)

Current Status

WIP

Summary

Currently there is a single stream for migration, this causes several problems:

• CPU that does the reception is a bottleneck on 10Gigabit and faster
• We copy all the pages for migration, even when we can send directly
• We make Transparent HugePages more difficult to use

So, the idea here is to split the migration stream in two, one for the current code, and the other for the content of the RAM pages. This avoids the copies altogether.

Scope

Only needs changes in QEMU. It can be configured if used or not for the cases where the destination don't know about it.

How to use

To enable the feature:

 migrate_set_capability x-multiple-fds on

And it would be use from there. Notice that it needs to be enabled in both sides.

Added commands / state

• Migration capability: x-multiple-fds

Design

Current migration code uses a single migration stream, so for memory pages, we are writting things like:

[page header 1][4k page 1][page header 2][4k page 2]

Idea is to change this to two streams:

• migration stream

 [page header 1][page header 2]

• additional fd

 [4k page 1][4 page 2]

This allows us:

• not use a single copy in either sending or receive. We just send the page directly and receive it in place
• We don't modify the migration stream. The only difference is that pages payload gets received on a side channel
• We can do like send/receive HUGE pages in one go, making life for Transparent Huge pages easier

We could also adapt this code for RDMA.

Future Enhancements

• Enable the use of more than one fd
• Use a different fd for postcopy page faults
• Use a different fd for each compression/decompression cpu