Features/QMP/Asynchronous QMP-Shell

Asynchronous QMP-Shell

Introduction

The aim is to build an asynchronous Text User Interface(TUI) for issuing and receiving QEMU Monitor Protocol (QMP) commands from a running QEMU instance. Currently, this is done using some standard tools like telnet, socat, and a primitive interface from QEMU called qmp-shell. The qmp-shell is a primitive synchronous interface written in python, being synchronous interface it lacks support for asynchronous events. It requires the user to send an empty response to retrieve all the asynchronous events. This project aims to replace the qmp-shell with a more modern TUI interface that is asynchronous and provides improvised features compared to the existing interfaces.

Feature authors

• Mentor: John Snow
• GSoC Student: Niteesh Babu G S

FINAL REPORT

As of 23/08/2021 the following objectives have been achieved.

• Reviewed
  • Syntax highlights messages.
  • Syntax checker(Validate JSON syntax like missing quotes, brackets).
  • Log messages/events in TUI and to file.

• Work in progress
  • Auto-completion
  • Abstracted messages
  • Allowing overlays

Detailed description

QMP (QEMU Machine Protocol) is an API provided by QEMU to allow applications to operate a running QEMU instance. QMP is a JSON based protocol and features the following

• Lightweight, text-based, easy to parse data format
• Asynchronous messages support (ie. events)
• Capabilities Negotiation

QMP can be started on a number of interfaces. This makes it flexible for applications to connect to. The interfaces include UNIX sockets, TCP sockets, and STDIO. Some of the interfaces used by the developers to connect include socat, telnet and qmp-shell. The qmp-shell is a simple python interface that has a basic understanding of the protocol provides little more features compared to the standard tools like socat and telnet. Its features include simple command completion, readline features, and a simple syntax checker. Some of the shortcomings of this tool are as follows

• Synchronous: It cannot display the asynchronous event messages from the server in realtime.
• No syntax highlighting
• Lack of documentation: The syntax of the shell is not well documented.
• Lacks Notifications: The UI does not notify the status of the connection. So in case of a sudden connection loss, the user is not informed until a new command is issued.

The aim of this project is to build a versatile TUI interface that can provide all the lacking features of qmp-shell. This will replace the qmp-shell and will one of the major tools for issuing QMP commands. The initial goal is to build a TUI that can connect to QEMU and issue QMP commands, but the long-term goal is to augment this interface with more features related to testing and debugging.

The main features of this interface will be

• Asynchronous
• Syntax highlighting
• Notifications

The rest of the auxiliary features described below will be added in the future. All this is possible through the AQMP (Asynchronous QMP) library built by John Snow.

The initial prototype is expected to look something similar to this.

Libraries used for implementation

• urwid
• AQMP(Asynchronous QMP) library
• pygments

(More will be added as used)

Benefits to QEMU

• A versatile tool to issue QMP commands. And also maybe test and debug in the future.

Initial Prototype

The initial prototype is expected to look like the TUI shown in the above image. It should be capable of receiving asynchronous events, displaying the events (Should also allow scrolling), and issuing the commands. It should also be capable of syntax highlighting the JSON messages to make them easier to read. It should also contain a status bar that notifies it of any disconnections or other unexpected events from the server.

Implementation in Urwid

We will be using the Asyncio event loop in urwid, this will allow for getting inputs from the user and messages from the server without blocking the loop. The main widget of the TUI is the urwid Frame. The frame widget consists of three components, the header, footer, and body. The header will contain information like the current window name or the TUI name, this component is optional. The footer will contain the status bar that notifies events like disconnections or errors from the server-side. The body will host an urwid Listbox which will display all the messages sent/received. With these features, one should have a primitive TUI that is capable of connecting to, issuing, and displaying the messages to/from the QEMU instance.

GSoC Goals

The aim by end of GSoC is to improvise( or rewrite) the prototype to support more features including (ordered based on priority)

• Syntax highlights messages
• Syntax checker(Validate JSON syntax like missing quotes, brackets). JSON Schema validation TBD.
• Command completion
• Log messages

Long Term Goals

• Ability to filter messages
• Ability to show documentation for commands
• Different view modes for JSON messages (Condensed, Expanded, Raw)
• Replay commands
• Custom Key commands