Google Summer of Code 2023: Difference between revisions

From QEMU
Line 62: Line 62:
{{:Internships/ProjectIdeas/RustBindingsForLibNBD}}
{{:Internships/ProjectIdeas/RustBindingsForLibNBD}}
{{:Internships/ProjectIdeas/RDPServer}}
{{:Internships/ProjectIdeas/RDPServer}}
{{:Internships/ProjectIdeas/VirtioSound}}
{{:Internships/ProjectIdeas/SVQOptimization}}
{{:Internships/ProjectIdeas/SVQOptimization}}
{{:Internships/ProjectIdeas/SVQCVQSupport}}
{{:Internships/ProjectIdeas/SVQCVQSupport}}

Revision as of 16:47, 21 February 2023

Introduction

QEMU is applying for Google Summer of Code 2023. This page contains our ideas list and information for applicants and mentors. Google Summer of Code is an open source internship program offering paid remote work.

Status: Google will announce accepted organizations on February 22 at 18:00 UTC. Applicants may get in touch with mentors now but please don't invest too much time until Google officially announces participating organization.

Application Process

1. Discuss the project idea with the mentor(s)

Read the project ideas list and choose one you are interested in. Read the links in the project idea description and start thinking about how you would approach this. Ask yourself:

  • Do I have the necessary technical skills to complete this project?
  • Will I be able to work independently without the physical presence of my mentor?

If you answer no to these questions, choose another project idea and/or organization that fits your skills.

Once you have identified a suitable project idea, email the mentor(s) your questions about the idea and explain your understanding of the project idea to them to verify that you are on the right track.

2. Fill out the application form

The application form asks for a problem description and outline of how you intend to implement a solution. You will need to do some background research (looking at source code, browsing relevant specifications, etc) in order to decide how to tackle the project. The form asks for an initial project schedule which you should create by breaking down the project into tasks and estimating how long they will take. The schedule can be adjusted during the summer so don't worry about getting everything right ahead of time.

3. Interview including a coding exercise

You may be invited to an interview consisting of a 30-minute coding exercise, followed by technical discussion and a chance to ask questions you have about the project idea, QEMU, and GSoC. The coding exercise is designed to show fluency in the programming language for your project idea (QEMU projects are typically in C but could also be in Python or Rust).

Here is a C coding exercise we have used in previous years when interviewing applicants: 2014 coding exercise

Try it and see if you can complete the sample coding exercise comfortably. We cannot answer questions about the previous coding exercise but hopefully it should be self-explanatory.

If you find the exercise too challenging, think about applying to other organizations where you have a stronger technical background and will be more competitive compared with other candidates.

Key Dates

From the timeline:

  • February 22 18:00 UTC - Organizations and project ideas announced
  • March 20 - April 4 18:00 UTC - Application period
  • May 4 18:00 UTC - Accepted applicants announced
  • May 29 - August 28 - Standard coding period (an extended timeline is possible depending on your project)

Find Us

For general questions about QEMU in GSoC, please contact the following people:

Project Ideas

This is the listing of suggested project ideas. Students are free to suggest their own projects, see #How to propose a custom project idea below.

Native Library Calls

Summary: Implement a native bypass for when a program running under linux-user calls a well known library call

When running under linux-user we translate the whole program which includes all the library calls. A number of well understood library functions are usually fairly heavily optimised for the processor they run on. For example the semantics of memcpy are very well understood. We could consider instead of translating those library functions call to native versions of the code. As the runtime of library functions is generally biased to a few core functions only a small subset would need to be hooked to be useful (mem*, str*).

Links:

Details:

  • Project size: 350 hours
  • Skill level: intermediate with a good understanding of how linkers link to library calls
  • Language: C
  • Mentor: Alex Bennée (alex.bennee@linaro.org)
  • Suggested by: Alex Bennée

Rust bindings for libnbd

Summary: Add Rust bindings for libnbd

Network block device (NBD) is a protocol for exporting block devices (disk-like things) between processes, either across the network or between processes on the same machine. libnbd is a client library and some tools for talking to NBD servers. This library has bindings in various languages such as Python, but not in Rust, and the project is about writing those bindings.

Note that the bindings in libnbd are generated. You must add Rust bindings to the generator so that they are updated automatically as the libnbd API evolves in future. Simply addng Rust bindings for the current API will not be enough to complete the project. The Rust bindings must also be of high quality and natural for Rust developers to use. Documentation and tests are also required.

Links:

Details:

  • Project size: 350 hours
  • Skill level: intermediate or advanced
  • Language: C, Rust, OCaml
  • Mentor: rjones@redhat.com rwmjones on IRC
  • Suggested by: Richard Jones

RDP server

Summary: An out-of-process RDP server using `-display dbus`

QEMU can export its display via D-Bus since 7.0. The interface is close to the QEMU-internal API and offers display, audio, inputs, chardev, usb... Among other projects using it, a gtk4 client is in development (qemu-rdw), helping some applications such as GNOME Boxes, to port their code.

It's also interesting to use the interface to export the display on the network, using existing protocols, like VNC, Spice, or RDP. For example, and as a starting point perhaps, there is already an experimental VNC server written 100% Rust.

RDP being the most supported protocol, with more features and efficiency, this project idea is to develop a RDP server to export `-display dbus` over the network. Ideally, the project should be written in Rust. Unfortunately, IronRDP, the RDP crate, is lacking some server support. Investments in IronRDP is thus required. Alternatively, the project could be written in C (or other languages) with FreeRDP.

Links:

Details:

  • Project size: 350 hours
  • Skill level: intermediate (familiarity with RDP would be a big plus)
  • Languages: C, Rust
  • Mentor: Marc-André Lureau - marcandre.lureau@redhat.com - elmarco
  • Suggested by: Marc-André Lureau

Shadow Virtqueue performance optimization

Summary: Implement multithreading and mmap Queue Notifier optimizations for Shadow Virtqueues

To perform a virtual machine live migration, QEMU needs to know when devices modify memory so that the memory contents can be migrated each time they have been modified. Otherwise the virtual machine would resume with outdated memory and probably crash.

This is especially difficult with passthrough hardware devices because QEMU does not intercept the memory writes. As a method to overcome this for virtio devices, QEMU can present an emulated virtqueue to the device, called a Shadow Virtqueue (SVQ), instead of allowing the device to access guest memory directly. SVQ will then forward the writes to the guest, being the effective writer of the guest memory and knowing when a portion of it needs to be migrated again.

As this effectively breaks passthrough and adds extra steps in the communication, this comes with a performance penalty in several forms: Context switches, more memory reads and writes increasing cache pressure, etc.

At this moment the SVQ code is not optimized. It cannot forward buffers in parallel using multiqueue and multithreading, and it does not use the mmap Queue Notify mechanism to notify the device for available buffers efficiently, so these notifications need to perform an extra host kernel context switch.

The SVQ code requires modifications for multithreading. There are examples of multithreaded devices already, like virtio-blk, which can be used as a template. Proposals for a command-line syntax to map virtqueues to threads have been sent to QEMU mailing list already and you can use them.

Regarding the mmap Queue Notify optimization, QEMU already has code to set up the Queue Notify address for a vDPA device in vhost_vdpa_host_notifier_init(). At the moment the SVQ code doesn't make use of this yet. The SVQ code needs to write into the mmap region to avoid the more expensive event_notifier_set() call in vhost_svq_kick().

Your goal is the complete the following tasks:

  • Measure SVQ performance compared to non-SVQ with standard profiling tools like netperf (TCP_STREAM & TCP_RR) or iperf equivalent + DPDK's testpmd with AF_PACKET.
  • Add multithreading to SVQ, extracting the code from the Big QEMU Lock (BQL)
  • Add mmap Queue Notify support to vhost_svq_kick().

Links:

Details:

  • Project size: 350 hours
  • Skill level: intermediate
  • Languages: C
  • Mentor: Eugenio Perez Martin <eperezma@redhat.com> (eperezma on IRC)
  • Suggested by: Eugenio Perez Martin <eperezma@redhat.com>

Add virtio-net Control Virtqueue state restore support

Summary: Implement virtio-net Control Virtqueue state restore in Shadow Virtqueue mode

The virtio-net device has a control virtqueue (CVQ) for changing device parameters like the Ethernet MAC address or the number of active queues. These parameter changes are inherently hard to track on vDPA net devices if the CVQ passed through, because QEMU is not aware of CVQ activity. In this situation, QEMU is not able to migrate these devices, as it is not able to tell the actual state of the device.

Shadow Virtqueue (SVQ) allows QEMU to offer an emulated queue to the device, effectively forwarding the descriptors of that communication, tracking the device internal state, and being able to migrate it to a new destination QEMU.

To restore that state in the destination, SVQ is able to send these messages as regular CVQ commands. The code to understand and parse virtio-net CVQ commands is already in QEMU as part of its emulated device, but the code to send some of the new state is not, and some features are missing. There is already code to restore basic commands like MAC [1] or multiqueue [2], and it is easy to use them as a template.

Your goal is to implement these missing virtio-net CVQ commands:

  • VIRTIO_NET_CTRL_RX family, to control receive mode.
  • VIRTIO_NET_CTRL_GUEST_OFFLOADS
  • VIRTIO_NET_CTRL_VLAN family
  • VIRTIO_NET_CTRL_MQ_HASH config
  • VIRTIO_NET_CTRL_MQ_RSS config

Links:

Details:

  • Project size: 175 hours (small)
  • Skill level: intermediate
  • Language: C
  • Mentor: Eugenio Perez Martin <eperezma@redhat.com> (eperezma on IRC)
  • Suggested by: Eugenio Perez Martin <eperezma@redhat.com>

Zoned device support for libblkio

Summary: Add support for zoned block devices to the libblkio library.

Zoned block devices are special kinds of disks that are split into several regions called zones, where each zone may only be written sequentially and data can't be updated without resetting the entire zone.

libblkio is a library that provides an API for efficiently accessing block devices using modern high-performance block I/O interfaces like Linux io_uring.

The goal is to extend libblkio so users can use it to access zoned devices properly. This will require adding support for more request types, expanding its API to expose additional metadata about the device, and making the appropriate changes to each libblkio "driver".

The project consists of the following tasks:

  • Add libblkio properties that identify zoned devices and their capabilities (zone size, max active zones, whether zone append is supported, etc).
  • Add zone append, report zones, and zone management to the libblkio API.
  • Extend the io_uring driver to support zoned storage. Zone append may be tricky to implement because Linux has a no userspace API for it and can be considered a stretch goal if there's time.
  • Extend the virtio-driver crate to implement zoned APIs.
  • (Stretch goal) Modify QEMU's blkio block driver to use the new libblkio zoned APIs.

This is important for QEMU since it will soon support zoned devices too and several of its BlockDrivers rely on libblkio. In particular, this project would enable QEMU to access zoned vhost-user-blk and vhost-vdpa-blk devices.

Links:

Details:

  • Project size: 175 hours (small)
  • Skill level: intermediate
  • Language: Rust, C
  • Mentor: Alberto Faria <afaria@redhat.com> (afaria on IRC), Stefan Hajnoczi <stefanha@redhat.com> (stefanha on IRC)
  • Suggested by: Alberto Faria <afaria@redhat.com>

FreeBSD user emulation improvements

Summary: Get "Hello World" compiling and running in a FreeBSD/armv7 chroot

QEMU's user emulator (bsd-user) runs BSD executables. This is often used in cross-compilation environments where bsd-user enables programs for other CPU architectures to run. FreeBSD support is not yet complete in QEMU and this project will make progress towards this.

Your task is to get "Hello World" compiling with clang 14.0 in a FreeBSD 13.2/armv7 jail running on FreeBSD/amd64 and the resulting "Hello World" executable should also run successfully. In order to do this, the following 35 system calls need to be added: __realpathat, __sysctl, _umtx_op, cpuset_getaffinity, execve, fcntl, fstat, fstatat, ftruncate, getcontext, getdirentries, geteuid, getpid, getrandom, getrlimit, getuid, ioctl, issetugid, lseek, madvise, minherit, mmap, mprotect, munmap, rfork, rtprio_thread, sigaction, sigaltstack, sigfastblock, sigprocmask, sysarch, thr_new, thr_self, wait4.

Code for these system calls exists in Warner Losh's "blitz" git branch and consists of a few thousand lines. This code is working, but not polished enough to be merged into QEMU without additional work. Your tasks will involve:

  • Verify that the code achieves the goal and debug it when necessary
  • Build patch series (git commits) that extract just the necessary code and send them to the qemu-devel mailing list
  • Update the patches to address code review feedback from the mailing list

Each individual commit must compile and should not break anything that was working before the commit. Unit tests need to be written for subsets of the system calls that are needed to help with the development process (or existing QEMU test cases that fail need to have that failure corrected). A plan for incremental testing as the code develops is needed.

This project will expose you to FreeBSD system calls and emulation. You will also gain experience with porting and maintaining patches.

Links:

Details:

  • Project size: 350 hours
  • Skill level: intermediate
  • Language: C
  • Mentor: Warner Losh <imp@bsdimp.com>
  • Suggested by: Warner Losh <imp@bsdimp.com>

Sibling VM communication in vhost-user-vsock

Summary: Extend the existing vhost-user-vsock Rust application to support sibling VM communication

During GSoC 2021, we developed vhost-user-vsock application in Rust. It leveraged the vhost-user protocol to emulate a virtio-vsock device in an external process. It provides the hybrid VSOCK interface over AF_UNIX introduced by Firecracker.

The current implementation supports a single virtual machine (VM) per process instance. The idea of this project is to extend the vhost-user-vsock crate available in the rust-vmm/vhost-device workspace to support multiple VMs per instance and allow communication between sibling VMs.

vsock is usually used as a communication channel between the host and the guest running in a VM. Recently it has been proposed to extend vsock to also allow communication between VMs running on the same host (sibling VMs).

Usually struct sockaddr_vm { .svm_cid = 42, .svm_port = 1234 } address inside a VM is used to communicate with a nested VM, but a new flag is recently added to allow sibling VM communication. So by setting .svm_flags = VMADDR_FLAG_TO_HOST, the connection will be forwarded to the host, allowing communication with the sibling VM that has CID 42 (if the device supports it).

AF_VSOCK in Linux already supports it, but the vhost-user-vsock application still does not support it, so we want to extend it.

This project will allow you to learn more about the virtio-vsock specification, rust-vmm crates, and vhost-user protocol to interface with QEMU.

This work will be done in Rust, but we may need to patch the virtio-vsock driver or the AF_VSOCK core in Linux if we will find some issues.

Goals:

  • Understand how a virtio-vsock device works
  • Refactor vhost-user-vsock code to allow multiple virtio-vsock device instances
  • Extend the vhost-user-vsock CLI to define sibling VMs
  • Implement sibling VM communication
  • (optional) Support adding new VMs at runtime

Links:

Details:

  • Project size: 350 hours
  • Skill level: intermediate (knowledge of Rust and virtualization)
  • Language: Rust
  • Mentors: Stefano Garzarella <sgarzare@redhat.com>, German Maglione <gmaglione@redhat.com>
    • IRC: sgarzare / Matrix: @sgarzare:matrix.org
  • Suggested by: Stefano Garzarella <sgarzare@redhat.com>

A sandboxing tool for virtiofsd

Summary: Create a tool that runs virtiofsd in a sandboxed environment

Virtiofs is a shared file system that lets virtual machines access a directory tree on the host. Unlike existing approaches, it is designed to offer local file system semantics and performance.

Currently, virtiofsd integrates the sandboxing code and the server code in a single binary. The goal is to extract that code and create an external tool that creates a sandbox environment and runs virtiofsd in it. In addition, that tool should be extended to be able to run virtiofsd in a restricted environment with Landlock.

This will allow greater flexibility when integrating virtiofsd into a VMM or running it inside a container.

Goals:

  • Understand how to setup a restricted environment using chroot, namespaces, and Landlock
  • Refactor virtiofsd to extract the sandbox code to its own crate
  • Create an external sandboxing tool for virtiofsd

Links:

Details:

  • Project size: 175 hours
  • Skill level: intermediate (knowledge of Rust and C)
  • Language: Rust
  • Mentor: German Maglione <gmaglione@redhat.com>, Stefano Garzarella <sgarzare@redhat.com>
  • Suggested by: German Maglione <gmaglione@redhat.com>

How to add a project idea

  1. Create a new wiki page under "Internships/ProjectIdeas/YourIdea" and follow #Project idea template.
  2. Add a link from this page like this: {{:Internships/ProjectIdeas/YourIdea}}

This is the listing of suggested project ideas. Students are free to suggest their own projects, see #How to propose a custom project idea below.

Project idea template

=== TITLE ===
 
 '''Summary:''' Short description of the project
 
 Detailed description of the project.
 
 '''Links:'''
 * Wiki links to relevant material
 * External links to mailing lists or web sites
 
 '''Details:'''
 * Skill level: beginner or intermediate or advanced
 * Language: C
 * Mentor: Email address and IRC nick
 * Suggested by: Person who suggested the idea

How to propose a custom project idea

Applicants are welcome to propose their own project ideas. The process is as follows:

  1. Email your project idea to qemu-devel@nongnu.org. CC Stefan Hajnoczi <stefanha@gmail.com> and regular QEMU contributors who you think might be interested in mentoring.
  2. If a mentor is willing to take on the project idea, work with them to fill out the "Project idea template" above and email Stefan Hajnoczi <stefanha@gmail.com>.
  3. Stefan will add the project idea to the wiki.

Note that other candidates can apply for newly added project ideas. This ensures that custom project ideas are fair and open.

How to get familiar with our software

See what people are developing and talking about on the mailing lists:

Grab the source code or browse it:

Build QEMU and run it: QEMU on Linux Hosts

Links

Information for mentors

Mentors are responsible for keeping in touch with their intern and assessing progress. GSoC has evaluations where both the mentor and intern assess each other.

The mentor typically gives advice, reviews the intern's code, and has regular communication with the intern to ensure progress is being made.

Being a mentor is a significant time commitment, plan for 5 hours per week. Make sure you can make this commitment because backing out during the summer will affect the intern's experience.

The mentor chooses their intern by reviewing application forms and conducting IRC interviews with applicants. Depending on the number of candidates, this can be time-consuming in itself. Choosing the right intern is critical so that both the mentor and the intern can have a successful experience.

Retrieved from "https://wiki.qemu.org/index.php?title=Google_Summer_of_Code_2023&oldid=11176"