Internships/ProjectIdeas/VirtiofsdAio: Difference between revisions
m (Stefanha moved page Internships/VirtiofsdAio to Internships/ProjectIdeas/VirtiofsdAio without leaving a redirect) |
|||
| (One intermediate revision by one other user not shown) | |||
| Line 1: | Line 1: | ||
== Asynchronous request handling for virtiofsd == | == Asynchronous request handling for virtiofsd == | ||
''' | '''Expected outcome:''' Make virtiofsd’s request handling asynchronous, allowing single-threaded parallel request processing. | ||
single-threaded parallel request processing. | |||
virtiofsd is a virtio-fs device implementation, i.e. grants VM guests | virtiofsd is a virtio-fs device implementation, i.e. grants VM guests | ||
| Line 21: | Line 20: | ||
* Make the virtiofsd request loop process requests asynchronously, so requests can be fetched and processed while others are continuing in the background | * Make the virtiofsd request loop process requests asynchronously, so requests can be fetched and processed while others are continuing in the background | ||
* Evaluate the resulting performance with different workloads | * Evaluate the resulting performance with different workloads | ||
How you make the request loop asynchronous will largely be left to your discretion. You can use Rust <code>async</code> together with a runtime like tokio, some other runtime, or an entirely custom one; or keep the code synchronous, but allow deferring operations to the background, and when they complete, return the virtio descriptors to the guest then. That said, we assume that using <code>async</code> would provide the better long-term solution, as long as you’re comfortable to use/learn it. | |||
'''Links:''' | '''Links:''' | ||
Latest revision as of 14:28, 10 February 2025
Asynchronous request handling for virtiofsd
Expected outcome: Make virtiofsd’s request handling asynchronous, allowing single-threaded parallel request processing.
virtiofsd is a virtio-fs device implementation, i.e. grants VM guests access to host directories. In its current state, it processes guest requests one by one, which means operations of long duration will block processing of others that could be processed more quickly.
With asynchronous request processing, longer-lasting operations could continue in the background while other requests with lower latency are fetched and processed in parallel. This should improve performance especially for mixed workloads, i.e. one guest process executing longer-lasting filesystem operations, while another runs random small read requests on a single file.
Your task is to:
- Get familiar with a Linux AIO interface, preferably io_uring
- Have virtiofsd make use of that interface for its operations
- Make the virtiofsd request loop process requests asynchronously, so requests can be fetched and processed while others are continuing in the background
- Evaluate the resulting performance with different workloads
How you make the request loop asynchronous will largely be left to your discretion. You can use Rust async together with a runtime like tokio, some other runtime, or an entirely custom one; or keep the code synchronous, but allow deferring operations to the background, and when they complete, return the virtio descriptors to the guest then. That said, we assume that using async would provide the better long-term solution, as long as you’re comfortable to use/learn it.
Links:
- virtiofsd repository: https://gitlab.com/virtio-fs/virtiofsd
- virtiofsd’s filesystem operations: https://gitlab.com/virtio-fs/virtiofsd/-/blob/main/src/passthrough/mod.rs#L1490
- virtiofsd’s request processing loop: https://gitlab.com/virtio-fs/virtiofsd/-/blob/main/src/vhost_user.rs#L244
Details:
- Project size: 350 hours
- Skill level: intermediate
- Language: Rust
- Mentors: Hanna Czenczek (hreitz@redhat.com), German Maglione (gmaglione@redhat.com)