Internships/ProjectIdeas/VirtiofsdAio

Asynchronous request handling for virtiofsd

Expected outcome: Make virtiofsd’s request handling asynchronous, allowing single-threaded parallel request processing.

virtiofsd is a virtio-fs device implementation, i.e. grants VM guests access to host directories. In its current state, it processes guest requests one by one, which means operations of long duration will block processing of others that could be processed more quickly.

With asynchronous request processing, longer-lasting operations could continue in the background while other requests with lower latency are fetched and processed in parallel. This should improve performance especially for mixed workloads, i.e. one guest process executing longer-lasting filesystem operations, while another runs random small read requests on a single file.

Your task is to:

• Get familiar with a Linux AIO interface, preferably io_uring
• Have virtiofsd make use of that interface for its operations
• Make the virtiofsd request loop process requests asynchronously, so requests can be fetched and processed while others are continuing in the background
• Evaluate the resulting performance with different workloads

How you make the request loop asynchronous will largely be left to your discretion. You can use Rust async together with a runtime like tokio, some other runtime, or an entirely custom one; or keep the code synchronous, but allow deferring operations to the background, and when they complete, return the virtio descriptors to the guest then. That said, we assume that using async would provide the better long-term solution, as long as you’re comfortable to use/learn it.

Links:

• virtiofsd repository: https://gitlab.com/virtio-fs/virtiofsd
• virtiofsd’s filesystem operations: https://gitlab.com/virtio-fs/virtiofsd/-/blob/main/src/passthrough/mod.rs#L1490
• virtiofsd’s request processing loop: https://gitlab.com/virtio-fs/virtiofsd/-/blob/main/src/vhost_user.rs#L244

Details:

• Project size: 350 hours
• Skill level: intermediate
• Language: Rust
• Mentors: Hanna Czenczek (hreitz@redhat.com), German Maglione (gmaglione@redhat.com)