Features/CPUHotplug: Difference between revisions

From QEMU
Line 54: Line 54:


= device_add/device_del interface =
= device_add/device_del interface =
<sup>Legend:</sup> <sup><span style="color:green">green</span> - done, <span style="color:blue">blue</span> - in progress, <span style="color:red">red</span> - TBD</sup>
== Summary ==
== Summary ==




== Work in progress/TODOs ==
* <span style="color:blue">Conversion of features and other properties into static properties</span> provides following benefits:
** global properties for CPU, generalizing -cpu xxx,features_string template to a set of global properties 
** latest implementation doing only conversion to static properties tree: [https://github.com/imammedo/qemu/tree/x86-cpu-properties.WIP] [http://permalink.gmane.org/gmane.comp.emulators.qemu/196850| posted v7 series]
** conversion to global properties is postponed until CPU sub-classes 
* <span style="color:blue">CPU models as CPU subclasses</span>
** gives ability to create CPUs using CPU subclass name without any ad-hoc calls.
** there are several implementations in qemu-devel with following open questions:
*** <span style="color:red">if running in KVM mode, kvm_init() should be called before sub-classes class_init is called</span>
**** <span style="color:red">issue 1</span>: in KVM mode qemu provides '''host''' CPU model. <strike>This model depends on kvm being initialized before CPU could be created due to dependency on kvm_arch_get_supported_cpuid(). Due to lazy type initialization it usually doesn't cause problem because CPUs are created after kvm_init(). However if ''qemu'' is called with options ''-enable-kvm -cpu help'', it should display host model as available. With introduction of CPU sub-classes, '''host''''s cpu model type should be registered only when kvm is enabled, introducing dependency on kvm_init() being called first. The same issue applies to type introspection when it will be available.</strike> I think that we are reached consensus here that 'host' CPU type should be registered at kvm_arch_init() time.
**** <span style="color:red">issue 2</span>: <strike>when ''qemu'' is started with ''-enable-kvm'' and '''vendor''' feature is not overridden on command line, built-in '''vendor''' is replaced with host's vendor [see commit 8935499831312]. Again with lazy type initialization it doesn't cause problem because kvm_init() is called before CPU's type class_init() is called, so class_init() could overwrite built-in vendor with host's value. But if type introspection wouldn't require instantiated machine /i.e. be like ''-cpu help''/ it could cause problem that even with ''-enable-kvm'' it would return built-in vendor values instead of host's due to the lack of option dependencies which would say that ''-enable-kvm'' should be processed before '-show-types' or something like this over other interfaces.</strike>. 'vendor' issue evolved into a generic problem, there are several ideas how to proceed:
***** 1. Register all CPU sub-classes at QEMU start-up time and fix them up later if/when kvm_arch_init() called to apply KVM specific to them
****** keeps amount of CPU sub-classes == cpu_models
****** it requires enumeration of all CPU sub-classes to fixup default values. Therefore causing type class initialization for types which won't be used.
****** defaults could be deceiving/not valid if class introspection to happen before kvm_arch_init(), and there is no sure way to prevent misuse.
****** CPU sub-classes defaults are mutable depending on -enable-kvm option.
****** possible to get rid of x86_def_t type/array embedding it in class_[model]_init() functions
***** 2. Register *-tcg-* and *-kvm-* subclasses at QEMU start-up time, with TCG/KVM defaults hard-codded in respective class_[tcg|kvm]_init() functions.
****** CPU-subclasses defaults are not mutable depending on -enable-kvm option and doesn't require (now) kvm_init() being called first
****** doubles CPU sub-classes amount
****** exposes *-kvm-* CPU sub-classes to user and allows to create *-kvm-* based CPU in TCG mode and vice/verse.
****** hard to re-factor since users could start use class names instead of cpu_model names and won't allow to eliminate x86_def_t type/array.
***** 3. Register CPU sub-classes after kvm_init() but before machine init and set defaults in class_init() depending on if KVM is available/inited.
****** keeps amount of CPU sub-classes == cpu_models
****** user sees only one immutable set of classes. But set has different defaults depending on mode QEMU was started with (TCG/KVM).
****** possible to get rid of x86_def_t type/array embedding it in class_[model]_init() functions
****** requires global hook in vl.c between kvm_init() and machine_init()
****** CPU classes won't be available before this hook, so QMP, qom-get/set, list_cpus will be forced to be called after it to get access to CPU sub-classes
== Completed dependencies ==
<div style="color:green" class="mw-collapsible mw-collapsed">
* External CPU clean-ups. Move CPU internals inside CPU object
** move tcg init code CPU. commits d65e98, 84e3b60, eeec69d, 130a038
** move CPU reset from board level into CPU. commits 65dee3805, dd673288
** move APIC creation/initialization into CPU object. commit bdeec8021
* CPU as Device [commit: 961f839]
** necessary for converting "CPUID features" into static properties
** allows to use device_add command after CPU subclasses is implemented.
* QOM realize, device-only
** convert CPUs realizefn() to use DeviceRealize [commit: 2b6f294]
* CPUID features as properties
** provides an ability to set/get features using common FEAT_FOO=VAL property interfaces.
** Features related clean-ups and code reorganisation:
*** move feature flags fix-ups & checks to realize time
**** in OQM model any feature/property could be amended until realize time. Masking out unsupported kvm/tcg features too early could lead to invalid features to be exposed to guest
**** 9b15cd9 target-i386: Sanitize AMD's ext2_features at realize time
**** 4586f15 target-i386: Filter out unsupported features at realize time
**** 5ec01c2 Move kvm_check_features_against_host() check to realize time
*** separate features parsing from setting defaults
**** clean-up cpu_x86_parse_featurestr(), leaving in it only custom features parsing that should be set on CPU instance after all defaults are set. Later defaults initialization should be moved to CPU sub-classes and custom legacy features parser cpu_x86_parse_featurestr() could be converted to setting global properties, when CPU features are converted into static properties. [commits: 077c68c, fa2db3c, 8ba8a69, 99b88a1, 11acfdd, a91987c, 2c728df]
*** simplify vendor property
**** current 'vendor' feature implementation has complex semantic depending on if qemu is running in TCG or KVM mode and if vendor is overridden on command-line. ''if (kvm_enabled() == true)  vendor = host's vendor;  else vendor = built-in vendor; if (custom vendor) vendor = custom vendor''[commit: 99b88a1, 11acfdd]
</div>


[[Category:Obsolete feature pages]]
[[Category:Obsolete feature pages]]

Revision as of 10:41, 4 January 2017

Summary

There are 2 ways to hotplug CPU in QEMU:

  • dedicated legacy interface: cpu-add QMP command
  • generic device-add/device-del interface for hot-(un)plugging CPUs

Owner

  • Name: Igor Mammedov
  • Email: imammedo@redhat.com

cpu-add interface

Summary

{ 'command': 'cpu-add', 'data': {'id': 'int'} }

  • ID - a number in range [0..max-cpus)
  • Available since: 1.5
  • Supported targets: i386-softmmu, x86_64-softmmu, s399x (since 2.6)

Description

Command is an legacy solution for CPU hot-add and gives a simplified interface for it. It provides an opportunity to implement the feature for targets that currently can't implement CPU hot-add using device_add command due to their present design. Later targets that implement it could rewrite it to become a wrapper over device_add when it becomes usable for target.

Usage example

1. start QEMU with QMP socket available and with startup amount of CPUs less than maxcpus

./qemu-system-x86_64 -qmp unix:/tmp/qmp-sock,server,nowait -smp 1,maxcpus=4

2. Connect to QMP socket using qmp-shell command

./QMP/qmp-shell /tmp/qmp-sock

3. Add CPUs issuing cpu-add command in qmp-shell command prompt

cpu-add id=1

4. Optionally online newly added CPU inside guest

Linux kernel doesn't online hot-added CPUs automatically. Once CPU is hot-added it should be onlined using an appropriate udev script or manually by issuing a following command:

echo 1 > /sys/devices/system/cpu/cpu1/online

Sample udev script: Add the following line to /etc/udev/rules.d/99-hotPlugCPU.rules

SUBSYSTEM=="cpu",ACTION=="add",RUN+="/bin/sh -c '[ ! -e /sys$devpath/online ] || echo 1 > /sys$devpath/online'"


Current limitations

  1. migration target should be started with initial CPU count '-smp XX' that includes hot-added CPUs on migration source side.
  2. CPU shouldn't be hot-plugged during migration.
  3. adding CPUs should be done in successive order from lower to higher IDs in [0..max-cpus) range.
    It's possible to add arbitrary CPUs in random order, however that would cause migration to fail on its target side.

device_add/device_del interface

Summary