Internships/ProjectIdeas/QMP TUI

Interactive, asynchronous QEMU Machine Protocol (QMP) text user interface (TUI)

Summary: Write an interactive terminal program for issuing and receiving Qemu Monitor Protocol (QMP) commands from a running QEMU instance.

QMP is a JSON message-based protocol that serves as the primary method by which QEMU is controlled and managed by other applications. It is designed to be easy to send and parse from a variety of frameworks and languages, but it is not easy to type by hand. We have an existing python tool that some developers use called 'qmp-shell', but this tool has several fairly severe shortcomings.

qmp-shell is not asynchronous, so it cannot display responses from the server in realtime. Updating the tool to handle asynchronous input will require a fundamental rewrite of the tool to accommodate simultaneous writing of new commands by the human user while new input is received asynchronously from the server. If you are familiar with the console IRC chat program irssi, we are looking to create an interface that is similar. The program would have a message history that updates in realtime (like chat history in irssi), and a text editing bar to type new commands (like the text entry field in a chat room).

We have an existing synchronous QMP library, and an asyncio prototype has been developed to replace it. The focus of this project will be to use and polish that asyncio QMP library and write the actual TUI and interactive elements of the program itself.

Details:

• Skill level: Intermediate
• Language: Python 3.6
• Topic/Skill areas:
  • asyncio: We will be using Python's asyncio library. Experience with this library isn't required, but familiarity with async programming concepts will help: any of coroutines, cooperative scheduling, user threads, etc. If you've ever written a Discord.py bot, you've already used the asyncio library!
  • gradual typing: We will be using gradually typed Python 3.6, using mypy to statically validate those types. If you have not used types in Python before, it is not hard to learn as you go, and the mypy getting started guide is very approachable.
  • UI programming: We will likely be using urwid, a text console UI library for Python. If you have another toolkit/framework you are skilled with, we can likely use that instead. Some knowledge of UI programming concepts (generally class-based, using widgets and signals) will help you along.
  • Interactive console programs: Some knowledge of text-based interactive programs will help you know what good ideas to copy (or bad ones to avoid). If you've ever used links/lynx, irssi, emacs, vim, nano/pico, or even just bash, you'll have a good sense of TUI design basics.
• Mentor: John Snow <jsnow@redhat.com>
  • Pronouns: Any of the following; None (use 'jsnow'), he/him, or they/them at your preference. I don't use an honorific (no Mr., Mx., etc).
  • IRC nick: jsnow (OFTC). I am usually reachable here between 11AM EST and 7PM EST, Monday-Friday.
  • About: jsnow is the QEMU maintainer for various Python utilities, libraries and scripts used for testing and debugging in the QEMU codebase, has two cats, and really likes Pokemon.

Links:

• irssi: a good example of a text user interface with a live history/log and a textbox for inputting commands
• mitmproxy: A great example of a project that uses the urwid library to create a very effective TUI.
• urwid: Python TUI library used to implement mitmproxy's interface.
• asyncio: Python asynchronous library.
• aioconsole: An async python REPL for interactively writing async code in python. It might have good ideas to steal.
• urwid readline library: Implements readline-like hotkeys for urwid, which may be useful for writing a text entry box.
• Asynchronous QMP library: Work in progress; this is a prototype for a QMP library written for Python 3.6 using asyncio and mypy type hints.
• Synchronous QMP library: This is the existing QMP library used for various testing and debug utilities upstream in QEMU today. It is also strictly typed with mypy.
• qmp-shell: This is the existing interactive utility, and what this project aims to replace.

Recommended Research:

• Try using irssi to connect to irc.oftc.net and join the #qemu-gsoc channel. Say hello!
• Try installing mitmproxy and following along the mitmproxy tutorial. This will give you a good idea of the type of interface that inspired this project -- you only need to follow along until "27. You now know basics of mitmproxy's UI and how to control it."
• Read the mypy getting started guide for learning how gradual typing works in Python if you aren't already familiar.