Outreachy 2016 DecemberMarch: Difference between revisions

From QEMU
No edit summary
 
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= Introduction =
= Introduction =


QEMU is considering participating in [https://www.gnome.org/outreachy/ Outreachy 2016 December-March]. This page contains our ideas list and information for mentors.
QEMU is participating in [https://www.gnome.org/outreachy/ Outreachy 2016 December-March]. This page contains our ideas list and information for applicants and mentors.


= Find Us =
= Find Us =
Line 17: Line 17:
== QEMU projects ==
== QEMU projects ==


=== Event loop profiling tool ===
=== Reorganizing the QEMU documentation ===


'''Summary:''' Develop a top(1)-like tool to monitor event loop dispatching
'''Summary:''' Produce beautiful user and developer documentation for QEMU


A running QEMU process can have a number of different types of threadsAn I/O thread (either the main thread, or a custom iothread for dataplane devices) is a thread that runs an poll based event loop.
QEMU's documentation currently consists of:
* a manual written in the Texinfo language, whose source code is in part embedded in QEMU's program source code
* several files written as markdown or text documents, stored in the docs/ folderThese include user documentation, developer documentation, and hardware specifications.
* automatically generated documentation for the QMP protocol (in progress)


The event loop dispatches I/O events that come from user interface (e.gmonitor fd), guest OS (e.g. ioeventfd), or program's internal sources (e.g.  bottom halves or timers). Their occupation of host CPU time is often very useful debug/diagnostic information. Ideally the profiling code in QEMU would be in a dedicated thread so it is still usable even when the event loops are stuck.
This project will look into the following tasks:
# reorganizing the Texinfo documentation and extracting its contents to more easily editable markdown or reStructuredText filespandoc can be used to convert md and rst back to Texinfo until the next steps are performed (pandoc can read rst and md, and it can write all of rst/md/Texinfo).<br>At this point, the bulk of the Texinfo documentation is already built from files residing in docs/.
# converting the QMP protocol documentation generator to emit reStructuredText instead of Texinfo.
# incorporating the contents of the docs/ folder into three manuals (user, developer, hardware).
# using sphinx to build beautiful documentation directly from markdown or reStructuredText.
# automating the publishing of this documentation on readthedocs.org.


In this project you will develop a tool for QEMU that is like the top(1) utility for Linux, to monitor QEMU's event loops. As a prerequisite, you need to modify QEMU to expose necessary data that will be collected by the new tool to generate the profiling output.
'''Links:'''
* http://pandoc.org/
* https://github.com/davidmalcolm/texi2rst
 
'''Details:'''
* Skill level: beginner-medium
* Languages: Makefile, Python, Perl
* Mentor: Paolo Bonzini <pbonzini@redhat.com> (bonzini on IRC)
 
=== qtest-os: a mini operating system written in Python ===
'''Summary:''' Write a Python library to interact with QEMU's qtest, and then as much as possible of a "mini-OS" written in Python
 
QEMU uses "qtest" as a mechanism for tests to interact with devices.  qtest unit tests are currently written in C, using the GTest framework from glib and glue libraries called "libqtest" and "libqos". libqtest implements the qtest socket protocol, while libqos provides utility functions to deal with e.g. guest memory allocation and PCI devices. However, the functionality of libqos is limited, and using a high-level language like Python will make it easier to prototype and build more complex functionality in libqos.
 
This project will investigate using qtest from Python, including:
* writing a Python library with the same functionality as libqtest
* converting some of the existing tests from C to Python
<!-- * using the existing Python bindings to ACPICA (the ACPI reference implementation) to write ACPI unit tests -->
* extending qtest with a driver model ("qtest-os").


You must be familiar with (n)curses library and multi-threaded programming. You can write the tool in either C or Python.
The last bullet splits a unit test in three parts: a description of QEMU's supported machine types, a set of drivers, and the unit test code proper.  For example, given:
* a SCSI unit test
* a description of the machine type X saying that X a PCI bus
* a driver for X's PCI host bridge
* a driver for virtio-scsi
qtest would infer that the unit test can run by starting X with a virtio-scsi device.


'''Links:'''
'''Links:'''
* [https://lists.nongnu.org/archive/html/qemu-devel/2011-12/msg00139.html An old prototype of a Python qtest library]
'''Details:'''
* Skill level: medium
* Language: Python
* Mentor: Paolo Bonzini <pbonzini@redhat.com> (bonzini on IRC)
=== VIRTIO 1.0 support in libqos ===
'''Summary:''' Add support for the latest VIRTIO paravirtualized device standard to the libqos test framework.
The VIRTIO 1.0 specification defines a family of paravirtualized devices,
including virtio-net and virtio-blk, which are commonly used in QEMU guests.
Paravirtualized devices are "hardware" interfaces designed specifically for
emulation or virtualization. 
QEMU has a low-level device test suite that uses a framework called libqos for
common functionality like PCI and VIRTIO device access.  These tests can
exercise hardware registers to verify that emulated devices are implemented
correctly by QEMU.
Currently libqos only supports legacy VIRTIO devices.  The recent VIRTIO 1.0
standard introduces more flexible and performant memory layouts and hardware
register interfaces.
The goal of this project is to add VIRTIO 1.0 support to libqos alongside the
existing legacy VIRTIO support.  This way both VIRTIO 1.0 and legacy code paths
can be tested.  If you complete this project early there are plenty of related
tasks that you could tackle with the VIRTIO knowledge you will gain!
This project is suitable for candidates with an interest in device driver
programming and hardware interfaces.  You must be fluent in C programming.


* https://en.wikipedia.org/wiki/Event_loop
'''Links:'''
* [https://docs.python.org/2/library/curses.html Python curses module]
* [http://git.qemu-project.org/?p=qemu.git;a=blob;f=tests/libqos/virtio.h;h=0250842bf2d56bcfa51b1a4297154231b24ea450;hb=HEAD libqos virtio.h]
* [http://git.qemu-project.org/?p=qemu.git;a=blob;f=tests/virtio-blk-test.c;h=811cf756c8d0d6234168950241a6e0b6a4317e56;hb=HEAD virtio-blk test using libqos]
* [https://docs.oasis-open.org/virtio/virtio/v1.0/virtio-v1.0.html VIRTIO 1.0 specification]
* [http://vmsplice.net/~stefan/virtio-devconf-2014.pdf VIRTIO 1.0 overview (pdf)]
* [https://www.youtube.com/watch?v=5QIE0F7nU3U Presentation on latest VIRTIO status from KVM Forum 2016]


'''Details:'''
'''Details:'''
* Skill level: advanced
* Skill level: intermediate
* Language: C, (optional) Python
* Language: C
* Mentor: Fam Zheng <famz@redhat.com> (fam on IRC), Stefan Hajnoczi <stefanha@redhat.com> (stefanha on IRC)
* Mentor: Stefan Hajnoczi <stefanha@gmail.com> ("stefanha" on IRC)


== Project idea template ==
== Project idea template ==

Latest revision as of 16:57, 12 October 2016

Introduction

QEMU is participating in Outreachy 2016 December-March. This page contains our ideas list and information for applicants and mentors.

Find Us

  • IRC: #qemu-outreachy on irc.oftc.net

For general questions about QEMU in Outreachy, please contact the following people:

Project Ideas

This is the listing of suggested project ideas.

QEMU projects

Reorganizing the QEMU documentation

Summary: Produce beautiful user and developer documentation for QEMU

QEMU's documentation currently consists of:

  • a manual written in the Texinfo language, whose source code is in part embedded in QEMU's program source code
  • several files written as markdown or text documents, stored in the docs/ folder. These include user documentation, developer documentation, and hardware specifications.
  • automatically generated documentation for the QMP protocol (in progress)

This project will look into the following tasks:

  1. reorganizing the Texinfo documentation and extracting its contents to more easily editable markdown or reStructuredText files. pandoc can be used to convert md and rst back to Texinfo until the next steps are performed (pandoc can read rst and md, and it can write all of rst/md/Texinfo).
    At this point, the bulk of the Texinfo documentation is already built from files residing in docs/.
  2. converting the QMP protocol documentation generator to emit reStructuredText instead of Texinfo.
  3. incorporating the contents of the docs/ folder into three manuals (user, developer, hardware).
  4. using sphinx to build beautiful documentation directly from markdown or reStructuredText.
  5. automating the publishing of this documentation on readthedocs.org.

Links:

Details:

  • Skill level: beginner-medium
  • Languages: Makefile, Python, Perl
  • Mentor: Paolo Bonzini <pbonzini@redhat.com> (bonzini on IRC)

qtest-os: a mini operating system written in Python

Summary: Write a Python library to interact with QEMU's qtest, and then as much as possible of a "mini-OS" written in Python

QEMU uses "qtest" as a mechanism for tests to interact with devices. qtest unit tests are currently written in C, using the GTest framework from glib and glue libraries called "libqtest" and "libqos". libqtest implements the qtest socket protocol, while libqos provides utility functions to deal with e.g. guest memory allocation and PCI devices. However, the functionality of libqos is limited, and using a high-level language like Python will make it easier to prototype and build more complex functionality in libqos.

This project will investigate using qtest from Python, including:

  • writing a Python library with the same functionality as libqtest
  • converting some of the existing tests from C to Python
  • extending qtest with a driver model ("qtest-os").

The last bullet splits a unit test in three parts: a description of QEMU's supported machine types, a set of drivers, and the unit test code proper. For example, given:

  • a SCSI unit test
  • a description of the machine type X saying that X a PCI bus
  • a driver for X's PCI host bridge
  • a driver for virtio-scsi

qtest would infer that the unit test can run by starting X with a virtio-scsi device.

Links:

Details:

  • Skill level: medium
  • Language: Python
  • Mentor: Paolo Bonzini <pbonzini@redhat.com> (bonzini on IRC)

VIRTIO 1.0 support in libqos

Summary: Add support for the latest VIRTIO paravirtualized device standard to the libqos test framework.

The VIRTIO 1.0 specification defines a family of paravirtualized devices, including virtio-net and virtio-blk, which are commonly used in QEMU guests. Paravirtualized devices are "hardware" interfaces designed specifically for emulation or virtualization.

QEMU has a low-level device test suite that uses a framework called libqos for common functionality like PCI and VIRTIO device access. These tests can exercise hardware registers to verify that emulated devices are implemented correctly by QEMU.

Currently libqos only supports legacy VIRTIO devices. The recent VIRTIO 1.0 standard introduces more flexible and performant memory layouts and hardware register interfaces.

The goal of this project is to add VIRTIO 1.0 support to libqos alongside the existing legacy VIRTIO support. This way both VIRTIO 1.0 and legacy code paths can be tested. If you complete this project early there are plenty of related tasks that you could tackle with the VIRTIO knowledge you will gain!

This project is suitable for candidates with an interest in device driver programming and hardware interfaces. You must be fluent in C programming.

Links:

Details:

  • Skill level: intermediate
  • Language: C
  • Mentor: Stefan Hajnoczi <stefanha@gmail.com> ("stefanha" on IRC)

Project idea template

=== TITLE ===
 
 '''Summary:''' Short description of the project
 
 Detailed description of the project.
 
 '''Links:'''
 * Wiki links to relevant material
 * External links to mailing lists or web sites
 
 '''Details:'''
 * Skill level: beginner or intermediate or advanced
 * Language: C
 * Mentor: Email address and IRC nick
 * Suggested by: Person who suggested the idea

Information for mentors

Mentors are responsible for keeping in touch with their candidate and assessing the candidate's progress.

The mentor typically gives advice, reviews the candidate's code, and has regular communication with the candidate to ensure progress is being made.

Being a mentor is a significant time commitment, plan for 5 hours per week. Make sure you can make this commitment because backing out during the summer will affect the candidate's experience.

The mentor chooses their candidate by reviewing candidate application forms, giving out bite-sized tasks so applicants can submit a patch upstream, and conducting IRC interviews with candidates. Depending on the number of candidates, this can be time-consuming in itself. Choosing the right candidate is critical so that both the mentor and the candidate can have a successful experience.