Internships/ProjectIdeas/QEMUPerformance

TCG Continuous Benchmarking

Summary: The nature of this project lies more in exploration, analysis and presentation than in coding. The performance of a software product will be examined to the greatest details. The software under examination will be QEMU emulator - across its modes, across its components, and across time.

QEMU may operate in so called user mode (an executable built for one processor (in QEMU parlance, target) is, by means of QEMU emulation, executed on the system with another processor (again, in QEMU slang, host)) and in system mode (the whole system of one kind (target) is emulated on the system of another kind (host)). These two modes will be examined separately:

TASK

PART I: (user mode)

• select around a dozen test programs (resembling components of SPEC benchmark, but all must be open source, and preferably having license compatible with QEMU); those test programs should be distributed like this: 4-5 FPU CPU-intensive, 4-5 non-FPU CPU intensive, 1-2 I/O intensive;
• measure execution time and other performance data of all selected test program across all targets on Intel and possibly other hosts, for the latest QEMU version:
  • try to improve performance if there is an obvious bottleneck;
  • develop tests that will be protection against performance regressions in future;
  • provide automated nightly tests for letting know QEMU developers if something changed performance-wise.
• measure execution time of all selected test programs for selected targets for all QEMU versions in last 5 years (there are appr. 15 such versions):
  • confirm performance improvements and/or detect performance degradation.
• summarize all results in a comprehensive form, using also graphics/data visualization.

PART II: (system mode)

• measure execution time and other performance data for boot/shutdown cycle for selected machines for the latest QEMU version:
  • try to improve performance if there is an obvious bottleneck.
• summarize all results in a comprehensive form.

DELIVERABLES

1) Each target maintainer for target will be given a list of top 25 functions in terms of spent host CPU time for each benchmark described in the previous section. Additional information and observations will be also provided, if the judgment is they are useful and relevant.

2) Each machine maintainer machine (that has successful boot/shutdown cycle) will be given a list of top 25 functions in terms of spent host time during boot/shutdown cycle. Additional information and observations may also be provided.

3) The community will be given all devised performance measurement methods in the form of easily reproducible step-by-step setup and execution procedures.

Deliverables should be gradually distributed over wider time interval of around two months.

Links:

• [1]
• [2]
• [3]

Details:

• Skill level: intermediate
• Languages:
  • C (for code analysis, performance improvements)
  • Python (for automatization)
  • potentially JavaScript (d3.js or similar library; for data visualization)
• Mentor: Aleksandar Markovic (aleksandar.markovic@rt-rk.com)
• Suggested by: Aleksandar Markovic