Features/record-replay: Difference between revisions

From QEMU
No edit summary
 
(16 intermediate revisions by the same user not shown)
Line 11: Line 11:
be used for deterministic and reverse debugging of guest code through a gdb remote
be used for deterministic and reverse debugging of guest code through a gdb remote
interface.
interface.
One of the aims of deterministic/reverse debugging is eliminating Heisenbugs.
Stopping the program in the debugger
may cause timeout in data processing or data transfer.
The behavior of the connected device may change and the bug will disappear.
Each program run can expose different behavior of the program
without giving a chance to examine the bugs.
==Limitations==
Record/replay reuses icount to implement deterministic execution.
Therefore rr inherits icount limitations:
* Works only in single CPU TCG mode.
* Some platforms have incomplete icount implementation
Current record/replay implementation is incomplete and cannot be used with
* Passthrough USB devices
Patches for support of these devices will be added later.


==Using record/replay==
==Using record/replay==


Record/replay feature is tested for i386, x86_64, ARM, and MIPS platforms.
Record/replay feature is tested for i386, x86_64, ARM, and MIPS platforms.
Current version does not support replaying of block devices, serial ports,
passthrough USB, and network devices. Therefore record/replay may be used with the
following or similar command line options:
qemu-system-arm -M integratorcp -kernel zImage.integrator -initrd arm_root.img -net none


Execution recording may be enabled through icount command line option:
Execution recording may be enabled through icount command line option:
Line 27: Line 40:
To enable replaying icount option should look like this:
To enable replaying icount option should look like this:
-icount shift=7,rr=replay,rrfile=replay.bin
-icount shift=7,rr=replay,rrfile=replay.bin
To record and replay block operations the drive must be configured
as follows:
-drive file=disk.qcow,if=none,id=img-direct
-drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay
-device ide-hd,drive=img-blkreplay
blkreplay driver should be inserted between disk image and virtual driver
controller. Therefore all disk requests may be recorded and replayed.
Character devices connected to QEMU are recorded/replayed automatically.
Both of record and replay command lines should have equivalent number of
attached character devices.
Record and replay for network interactions is performed with the network filter.
Each backend must have its own instance of the replay filter as follows:
-netdev user,id=net1 -device rtl8139,netdev=net1
-object filter-replay,id=replay,netdev=net1
Record/replay for audio devices (-soundhw option) is enabled automatically.
===Supported inputs===
* Mouse input
* Keyboard input
* Host real time clock
* Character devices
* Network devices
* Audio input
===Snapshotting===
New VM snapshots may be created in replay mode. They can be used later
to recover the desired VM state. All VM states created in replay mode
are associated with the moment of time in the replay scenario.
After recovering the VM state replay will start from that position.
Default starting snapshot name may be specified with icount field
rrsnapshot as follows:
-icount shift=7,rr=record,rrfile=replay.bin,rrsnapshot=snapshot_name
This snapshot is created at start of recording and restored at start
of replaying. It also can be loaded while replaying to roll back
the execution.
==Features to add==
Full version of record/replay will include support of:
* Automatic VM snapshotting
* Reverse debugging through GDB


==How to get involved==
==How to get involved==
Line 35: Line 98:
You can mail Pavel Dovgalyuk to get information about patches
You can mail Pavel Dovgalyuk to get information about patches
that were not upstreamed yet.
that were not upstreamed yet.
==Links==
Papers with description of deterministic replay implementation:
* http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html
* http://dl.acm.org/citation.cfm?id=2786805.2803179
Prior QEMU version with block patches added:
https://github.com/Dovgalyuk/qemu/tree/rr-15

Latest revision as of 08:17, 2 March 2017

Overview

Record/replay feature is implementation of deterministic replay for system-level simulation (softmmu mode).

Record/replay functions are used for the reverse execution and deterministic replay of qemu execution. Determinitsic replay is used to record volatile system execution once and replay it for multiple times for the sake of analysis, debugging, logging, etc. This implementation of deterministic replay can be used for deterministic and reverse debugging of guest code through a gdb remote interface.

One of the aims of deterministic/reverse debugging is eliminating Heisenbugs. Stopping the program in the debugger may cause timeout in data processing or data transfer. The behavior of the connected device may change and the bug will disappear. Each program run can expose different behavior of the program without giving a chance to examine the bugs.

Limitations

Record/replay reuses icount to implement deterministic execution. Therefore rr inherits icount limitations:

  • Works only in single CPU TCG mode.
  • Some platforms have incomplete icount implementation

Current record/replay implementation is incomplete and cannot be used with

  • Passthrough USB devices

Patches for support of these devices will be added later.

Using record/replay

Record/replay feature is tested for i386, x86_64, ARM, and MIPS platforms.

Execution recording may be enabled through icount command line option: -icount shift=7,rr=record,rrfile=replay.bin

To enable replaying icount option should look like this: -icount shift=7,rr=replay,rrfile=replay.bin

To record and replay block operations the drive must be configured as follows:

-drive file=disk.qcow,if=none,id=img-direct
-drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay
-device ide-hd,drive=img-blkreplay

blkreplay driver should be inserted between disk image and virtual driver controller. Therefore all disk requests may be recorded and replayed.

Character devices connected to QEMU are recorded/replayed automatically. Both of record and replay command lines should have equivalent number of attached character devices.

Record and replay for network interactions is performed with the network filter. Each backend must have its own instance of the replay filter as follows:

-netdev user,id=net1 -device rtl8139,netdev=net1
-object filter-replay,id=replay,netdev=net1

Record/replay for audio devices (-soundhw option) is enabled automatically.

Supported inputs

  • Mouse input
  • Keyboard input
  • Host real time clock
  • Character devices
  • Network devices
  • Audio input

Snapshotting

New VM snapshots may be created in replay mode. They can be used later to recover the desired VM state. All VM states created in replay mode are associated with the moment of time in the replay scenario. After recovering the VM state replay will start from that position.

Default starting snapshot name may be specified with icount field rrsnapshot as follows:

-icount shift=7,rr=record,rrfile=replay.bin,rrsnapshot=snapshot_name

This snapshot is created at start of recording and restored at start of replaying. It also can be loaded while replaying to roll back the execution.

Features to add

Full version of record/replay will include support of:

  • Automatic VM snapshotting
  • Reverse debugging through GDB

How to get involved

First version of patches of record/replay feature is prepared by ISP RAS

You can mail Pavel Dovgalyuk to get information about patches that were not upstreamed yet.

Links

Papers with description of deterministic replay implementation:

Prior QEMU version with block patches added: https://github.com/Dovgalyuk/qemu/tree/rr-15