Outreachy 2015 MayAugust

From QEMU

Introduction

QEMU is seeking funding to participate as a mentoring organization in Outreachy 2015 May-August. This page contains our ideas list and information for candidates and mentors.

Note that QEMU is also participating in Google Summer of Code. We ask all Outreachy applicants who are students and coders to also apply for Google Summer of Code. You can apply for both programs with the same project proposal text. The program administrators will coordinate the selection process to ensure that we accept the top applicants through one of the programs.

Find Us

  • IRC: #qemu-outreachy on irc.oftc.net
  • IRC (development):
    • QEMU: #qemu on irc.oftc.net
    • libvirt: #virt on irc.oftc.net
    • KVM: #kvm on chat.freenode.net

Please contact the mentor for the project idea you are interested in. IRC is usually the quickest way to get an answer.

For general questions about QEMU in Outreachy, please contact the following people:

How to get familiar with our software

See what people are developing and talking about on the mailing lists:

Grab the source code or browse it:

Build QEMU and run it: QEMU on Linux Hosts

Project Ideas

This is the listing of suggested project ideas. Students are free to suggest their own projects by emailing qemu-devel@nongnu.org and (optionally) CCing potential mentors.

QEMU/KVM projects

Userspace Interrupt Handling

Summary: Move some of the interrupt controller devices out of the kernel, for improved flexibility and security

On all architectures, KVM places some of the interrupt controller devices in the kernel. In many cases, these devices are intimately linked to the processor and it is both simpler and faster to implement this in the kernel. x86 however places all the interrupt controllers in the kernel, even if most of them are not used anymore except during boot or by legacy guests (for which performance is not of utmost importance). This is inflexible, because the properties of these interrupt controllers can vary for different emulated chipsets, and also insecure, because several recent KVM vulnerabilities were located in interrupt emulation.

The purpose of this project is to decouple the local APIC device in the kernel from the IOAPIC and PIC, so that it is possible to run a virtual machine with an in-kernel local APIC while emulating the IOAPIC and PIC inside QEMU. The goal is to do so with minimal (<1%) performance impact on modern hardware and operating systems.

If there is time, it is possible to follow up with extra work in the same areas:

  • VT-d interrupt emulation with KVM support: QEMU gained basic Intel IOMMU support (VT-d) during last year's GSoC; DMA remapping (DMAR) works already, but interrupt remapping is still missing. There are patches available that use the userspace IOAPIC to implement interrupt remapping, but they were not merged. because they suffer from the limited performance of the userspace local APIC. After the above project is completed, it should be possible to integrate interrupt remapping into the QEMU's IOAPIC device model.
  • Relocating more hypervisor code (for example MSR handling) to userspace, where vulnerabilities have fewer consequences and are easier to mitigate (i.e. userspace code execution bugs only result in non-privileged execution, and exploit mitigations are more common in userspace).

Links:

Details:

  • Skill level: advanced
  • Language: C
  • Mentor: Steve Rutherford <srutherford@google.com> (srutherford on IRC), Paolo Bonzini <pbonzini@redhat.com>, Jan Kiszka <jan.kiszka@web.de>
  • Suggested by: Steve Rutherford, Andrew Honig <ahonig@google.com>, Paolo Bonzini <pbonzini@redhat.com>, Jan Kiszka <jan.kiszka@web.de>

VT-d interrupt emulation with KVM support

Summary: Extend the existing VT-d model with support for interrupt remapping, including KVM irqchip support

QEMU gained basic Intel IOMMU support (VT-d) during last year's GSoC. DMA remapping (DMAR) works already, but interrupt remapping is still missing. There are some hacks that pleases Linux as well as the Jailhouse hypervisor (both shall serve as test cases here), but they are not upstream, lack error handling and only work if the KVM in-kernel irqchip is disabled.

This project shall overcome the limitations. It can start with addressing the open issues in -no-kvm-irqchip mode and get the changes upstream. Then it shall look into extending KVM to support interrupt remapping emulation as well. The primary challenge here is to ensure the interrupts of the in-kernel IOAPIC are properly remapped according to the user space VT-d model and its programming by the guest. This likely involves designing a new KVM user space interface, getting design and implementation reviewed and accepted by the KVM community and then make use of it in QEMU. Other in-kernel IRQ delivery paths will need a look as well as far as they are IOMMU compatible (virtio is not yet, thus can be skipped; vfio may be considered).

Possibly not much code needs to be written for this, but intensive interaction with KVM maintainers will be required because sensitive code will have to be touched. So this will be a unique chance to get deeply involved in the QEMU/KVM development process and work on both kernel as well as user space components.

Links:

Details:

  • Skill level: advanced
  • Language: C
  • Mentor: Jan Kiszka <jan.kiszka@web.de>
  • Suggested by: Jan Kiszka

Nested Virtualization hardening

Summary: Extend kvm-unit-tests with security and sanity checks

With Nested Virtualization, guest operating systems can behave as hypervisors and run their own guests utilizing host hardware extensions. In this context, the hypervisor that runs on baremetal is L0, the hypervisor/guest that L0 runs is L1 and so forth. A major challenge in improving the stability of this feature in KVM is to test various configurations with different hypervisors coupled with various guests. However, a more practical approach is to write unit tests that mimic the nested hypervisor's basic behavior. We already have a number of unit tests as part of the kvm-unit-tests framework. [0][1]

The scope of this project is to write unit tests for conditions that potentially let the level 2 guest kill or hang the level 1 guest or perhaps less importantly, the level 2 guest itself. First, we could visit known cases which have caused crashes/hangs in the past, and investigate whether we could write unit tests so we don't inadvertently introduce the same problems again when adding newer features and/or bug fixes. For example, a bug was introduced a while back that incorrectly injected L1's interrupts to L2 [2]. We could write a unit test that attempts to inject a few interrupts to L1(with help from L0) and verify that L1 indeed received all the interrupts. There are also cases[3] where emulate_invalid_guest_state=0 avoids hangs; which probably means that we could write tests that force certain paths during instruction emulation. This will probably also help with checks for BUG() at various places (and whether we should consider removing them with something less sinister :)) Another idea is to run nested unit tests with various values for kvm_intel module parameters - there have been cases where new commits break only if a certain feature is enabled/disabled.

Here's a tentative list of key hardening areas - 

 1. Writing unit tests for missing VMX features tests; eg- invvpid,
    load/store, posted interrupts. [1]
 2. Unit tests for emulated instructions[4] - can we write a test that 
    can hit a BUG() during instruction emulation ?
 3. A framework that can enable/disable various module parameters and run
    1. and 2.
 4. Unit tests for interrupt handling paths - can we write a test
    that times an interrupt injection such that results in an incorrect
     injection.
 5. Anything else that comes up as we start tackling these areas.

We should primarily focus on 1, 2 and 3 and get to the others if time permits.

Links:

Details:

  • Skill level: advanced
  • Language: C
  • Mentor: Bandan Das <bsd@redhat.com>
  • Suggested by: Bandan Das

Instruction Emulation Security Improvements

Summary: The purpose of this project is to carefully relocate the x86 instruction emulator into userspace (into QEMU), where vulnerabilities have fewer consequences and are easier to mitigate (i.e. userspace code execution bugs only result in non-privileged execution, and exploit mitigations are more common in userspace). The goal is to do so with minimal (<1%) performance impact on modern hardware and operating systems.

Andrew Honig and I have written KVM patches which make it possible to handle instruction emulation in QEMU (or any other userspace VMM), but have not updated QEMU to take advantage of these patches (we currently only have this working for a proprietary VMM). The project will require gaining familiarity with x86 instruction emulation in QEMU/KVM.


Links:

Details:

  • Skill level: advanced
  • Language: C
  • Mentor: Steve Rutherford <srutherford@google.com> (srutherford on IRC)
  • Suggested by: Steve Rutherford and Andrew Honig <ahonig@google.com>

QEMU projects

Integrate IDE ATAPI and SCSI CD-ROM emulation

Summary: Unify IDE ATAPI CD-ROM emulation with SCSI CD-ROM emulation to reduce code duplication and squash bugs

Currently the IDE ATAPI and SCSI CD-ROM emulated devices are two distinct implementations, and have different bugs/features. This leads to the situation that things which work when using the IDE emulation don't work when using the SCSI emulation and vice versa. So this project is for implementing a virtual ATA-to-SCSI bridge in QEMU, use this for emulating an IDE ATAPI drive, and merging the missing features from the IDE implementation into the SCSI one.

This project will allow you to expore IDE and SCSI storage protocols and learn how CD-ROM drives are operated.

Links:

Details:

  • Skill level: intermediate
  • Language: C
  • Mentor: Hannes Reinecke <hare@suse.de>, John Snow <jsnow@redhat.com> (jsnow on QEMU IRC)
  • Suggested by: Hannes Reinecke <hare@suse.de>

AMD IOMMU emulation

Summary: Rework existing AMD IOMMU emulation patches according to new IOMMU model of QEMU

There was already a proposal to add AMD IOMMU emulation in 2011, but QEMU was lacking a proper layer to integrate this effort smoothly. This is now possible like the successful integration of basic VT-d emulation demonstrated during last year's GSoC. So this task is a little bit simpler: study the old patches as well as the Intel version, port the old ones over the new interfaces, add ACPI tables (missing in the old version), ensure that the resulting IOMMU model works according to the AMD specification - and that guests like Linux/KVM can use it correctly. The model should also support interrupt remapping, but only for the user-space irqchip mode (kernel interface support is part of a different project proposal).

Links:

Details:

  • Skill level: intermediate to advanced
  • Language: C
  • Mentor: Jan Kiszka <jan.kiszka@web.de>
  • Suggested by: Jan Kiszka

Implement support for Mac OS 9 in QEMU

Summary: QEMU has gone a long way in emulating a Macintosh. But we can still improve. Adding support for Mac OS 9 would be a great imporovement. This would allow everyone who misses their older applications to be reacquainted with them. It would also expand QEMU's abilities.


Links:

Details:

  • Skill level: Advanced
  • Language: C
  • Mentor: Alexander Graf <agraf@suse.de>
  • Suggested by: John Arbuckle <programmingkidx@gmail.com>

Move SLOF Device-tree Generation into QEMU

Summary: Move device-tree generation out of SLOF firmware and into QEMU when running pSeries/PowerPC guests (PowerKVM)

When emulating/virtualizing the pSeries/PowerPC architecture, QEMU relies on SLOF (Slim-line Open Firmware), which runs as part of the guest's early boot process, to generate an in-memory device-tree description that the guest operating system can load/parse to configure devices.

QEMU has already taken over much of this task in generating an initial device-tree with descriptions of physical host-bridges, VirtualIO devices, CPU, memory, etc. beforehand using the libfdt flat device-tree library, which QEMU's version of SLOF knows to parse beforehand prior to generating the remainder of the device-tree for things like individual PCI devices.

SLOF is written in Forth, which makes it a challenging code-base to maintain, and recent developments like support for PCI/CPU/MEM hotplug for pSeries/PowerPC guests have brought about a need to modify or duplicate some of the device-tree generation that it currently handles.

The goal of this project is to move these aspects of device-tree generation out of SLOF, and better leverage QEMU's libfdt-based device-tree generation instead, which will improve QEMU's maintainability and give you experience with some of the more obscure languages and low-level software that are critical to running guests on new PowerKVM/PowerPC servers.

Links:

Details:

  • Skill level: advanced
  • Language: Forth, C
  • Mentor: Michael Roth <mdroth@linux.vnet.ibm.com>
  • Suggested by: David Gibson <david@gibson.dropbear.id.au>

Implement Unit-tests for the QEMU Guest Agent

Summary: For some time now QEMU has provided it's own agent, the QEMU Guest Agent (QGA), to run inside of guests to handle requests on behalf of QEMU and guest management software running on the host. This agent is being being used increasingly more to enable support for things like data-consistent disk snapshots and guest password/resource management.

It continues to lack any sort of unit-testing framework with the QEMU source tree, however.

This is an open-ended project to design/implement a unit-testing framework to address this. This could potentially be done using python, or developing a C library that integrates well with QEMU's existing unit-test framework, and ideally would involve evaluating these numerous approaches and working closely with the community to present/determine/implement the best approach.

Links:

Details:

  • Skill level: beginner - intermediate
  • Language: Python, C
  • Mentor: Michael Roth <mdroth@linux.vnet.ibm.com>

Libvirt projects

Introducing job control to the storage driver

Summary: Currently, libvirt support job cancellation and progress reporting on domains. That is, if there's a long running job on a domain, e.g. migration, libvirt reports how much data has already been transferred to the destination and how much still needs to be transferred. However, libvirt lacks such information reporting in storage area, to which libvirt developers refer to as the storage driver. The aim is to report progress on several storage tasks, like volume wiping, file allocation an others.

  • Component: libvirt
  • Skill level: intermediate
  • Language: C
  • Mentor: Michal Privoznik <mprivozn@redhat.com>, mprivozn on IRC (#virt OFTC)
  • Suggested by: Michal Privoznik <mprivozn@redhat.com>

Enhancing libvirt-designer

Summary: The project is in its very early stage of life. The libvirt-designer tries to ease generation of libvirt XML with coworking with libosinfo project. Contact me and we can find something suitable.

  • Component: libvirt
  • Skill level: beginer
  • Language: C
  • Mentor: Michal Privoznik <mprivozn@redhat.com>, mprivozn on IRC (#virt OFTC)
  • Suggested by: Michal Privoznik <mprivozn@redhat.com>

Making virsh more bash like

Summary: If you have ever used virsh, you certainly reached the point where you stuggle with its user friendliness. Or unfriendliness I should rather say. Virsh is missing a lot of bash functionality that users consider natural: from automatic completion of object names, through redirecting command outputs through piping commands together. The aim would be to make these functions available in virsh and thus make user experience better.

  • Component: libvirt
  • Skill level: Advanced
  • Language: C
  • Mentor: Michal Privoznik <mprivozn@redhat.com>, mprivozn on IRC (#virt OFTC)
  • Suggested by: Michal Privoznik <mprivozn@redhat.com>

Abstracting device address allocation

Summary: There are many types of addresses that devices can have in libvirt's XML description. Not all address types are properly assigned and checked for duplicates. The goal of this is to have an abstract data structure that would handle assigning all kinds of addresses, handle duplicates, etc.

  • Component: libvirt
  • Skill level: beginner
  • Language: C
  • Mentor: Michal Privoznik <mprivozn@redhat.com>, mprivozn on IRC (#virt OFTC)
  • Suggested by: Martin Kletzander <mkletzan@redhat.com>

Admin interface APIs

Summary: We are currently working on an administrative interface for the libvirt daemon. This interface will be able to probe and change daemon settings live. Since this is going to be a new interface, there needs to be new APIs implemented. These APIs may do various interesting things strarting from showing the current logging filters up to force-disconnecting clients with certain properties. It is up to you what you would like to add, but of course this needs to be discussed and agreed on with the community.

  • Component: libvirt
  • Skill level: intermediate
  • Language: C
  • Mentor: Michal Privoznik <mprivozn@redhat.com>, mprivozn on IRC (#virt OFTC)
  • Suggested by: Martin Kletzander <mkletzan@redhat.com>

Your own idea

Just catch me (Michal Privoznik) on IRC and we can discuss what interests you.

  • Component: libvirt
  • Skill level: subject to negotiation
  • Language: C
  • Mentor: Michal Privoznik <mprivozn@redhat.com>, mprivozn on IRC (#virt OFTC)
  • Suggested by: Michal Privoznik <mprivozn@redhat.com>

Links:

Details:

  • Component: libvirt
  • Skill level: (see description to each item)
  • Language: C
  • Mentor: Michal Privoznik <mprivozn@redhat.com>, mprivozn on IRC (#virt OFTC)
  • Suggested by: Michal Privoznik <mprivozn@redhat.com>

Running docker containers using virt-sandbox

Summary: The basic idea is to be able to grab container images from the docker registry and run them using virt-sandbox. The application could then run as either a KVM or LXC guest.

Daniel already has a proof of concept to download the docker images, but there is still quite some work to be done, like:

  • transforming the docker image into a qcow2 image,
  • getting libvirt to set the username of the process to run,
  • getting libvirt to set environment variables

Links:

Details:

  • Skill level: advanced
  • Language: C, possibly python too
  • Mentor: Cedric Bosdonnat <cbosdonnat@suse.com>; cbosdonnat on IRC (OFTC and freenode)
  • Suggested by: Cedric Bosdonnat

Project idea template

=== TITLE ===
 
 '''Summary:''' Short description of the project
 
 Detailed description of the project.
 
 '''Links:'''
 * Wiki links to relevant material
 * External links to mailing lists or web sites
 
 '''Details:'''
 * Skill level: beginner or intermediate or advanced
 * Language: C
 * Mentor: Email address and IRC nick
 * Suggested by: Person who suggested the idea

Information for mentors

Mentors are responsible for keeping in touch with their candidate and assessing the candidate's progress.

The mentor typically gives advice, reviews the candidate's code, and has regular communication with the candidate to ensure progress is being made.

Being a mentor is a significant time commitment, plan for 5 hours per week. Make sure you can make this commitment because backing out during the summer will affect the candidate's experience.

The mentor chooses their candidate by reviewing candidate application forms and conducting IRC interviews with candidates. Depending on the number of candidates, this can be time-consuming in itself. Choosing the right candidate is critical so that both the mentor and the candidate can have a successful experience.

Retrieved from "https://wiki.qemu.org/index.php?title=Outreachy_2015_MayAugust&oldid=5402"