Internships/ProjectIdeas/NVMePerformance

NVMe Emulation Performance Optimization

Summary: QEMU's NVMe emulation uses the traditional trap-and-emulation method to emulate I/Os, thus the performance suffers due to frequent VM-exits. It is possible to run emulation in a dedicated thread called an IOThread using the "ioeventfd" mechanism to receive notifications of Submission Queue Tail Doorbell writes. Emulating NVMe in a separate thread allows the vcpu thread to continue execution and results in better performance.

The NVMe Specification 1.3 defines a new feature to update doorbell registers using a Shadow Doorbell Buffer. This can be utilized to enhance performance of emulated controllers by reducing the number of Submission Queue Tail Doorbell writes.

Finally, it is possible for the emulation code to watch for changes to the queue memory instead of waiting for doorbell writes. This technique is called polling and reduces notification latency at the expense of an another thread consuming CPU to detect queue activity.

The goal of this project is to add implement these optimizations so QEMU's NVMe emulation performance becomes comparable to virtio-blk performance.

Tasks include:

• Add Submission Queue Tail Doorbell register ioeventfd support (see existing patch linked below)
• Add Shadow Doorbell Buffer support to reduce doorbell writes
• Add Submission Queue polling
• Add IOThread support so emulation can run in a dedicated thread

Links:

• https://nvmexpress.org/wp-content/uploads/NVM_Express_Revision_1.3.pdf
• http://ucare.cs.uchicago.edu/pdf/fast18-femu.pdf
• https://github.com/ucare-uchicago/femu
• https://vmsplice.net/~stefan/stefanha-kvm-forum-2017.pdf
• https://patchwork.kernel.org/patch/10259575/

Details:

• Skill level: intermediate-advanced
• Language: C
• Mentor: Paolo Bonzini <pbonzini@redhat.com> ("bonzini" on IRC), Stefan Hajnoczi <stefanha@redhat.com> ("stefanha" on IRC)
• Suggested by: Huaicheng Li <huaicheng@cs.uchicago.edu>, Paolo Bonzini