Features/CPUHotplug: Difference between revisions

From QEMU
Line 82: Line 82:
'''''query-hotpluggable-cpus
'''''query-hotpluggable-cpus
QMP command. User can query the capability using '''''query-machines
QMP command. User can query the capability using '''''query-machines
QMP command and check if property '''''hotpluggable-cpus
QMP command and check if property ''hotpluggable-cpus
is set to true. If machine reports that '''''hotpluggable-cpus
is set to true. If machine reports that ''hotpluggable-cpus
are supported, user can use '''''query-hotpluggable-cpus
are supported, user can use '''''query-hotpluggable-cpus
command to list possible to hotplug CPUs with list of necessary to hotplug properties.
command to list possible to hotplug CPUs with list of necessary to hotplug properties.
For example for following CLI:<br/>
For example for following CLI:<br/>
  qemu-system-x86_64 -smp 1,maxcpus=4 -qmp unix:/tmp/q,server,nowait
  qemu-system-x86_64 -smp 1,maxcpus=4 -qmp unix:/tmp/q,server,nowait
user would get following result:
user would get following
  qmp-shell -p /tmp/q
  qmp-shell -p /tmp/q
  Welcome to the QMP low-level shell!
  Welcome to the QMP low-level shell!
Line 117: Line 117:
  }
  }


 
list of possible CPUs for given at startup ''-cpu'' and ''-smp'' CLI options. Where
 
*''-cpu cpu_model'' is translated to corresponding CPU ''type'' name that should be used with '''''device_add''''' as ''driver'' property value
[[Category:Obsolete feature pages]]
*''-smp n,sockets=x,cores=y,threads=z,maxcpus=m'' is translated to set of possible to hotplug CPUs, where per CPU ''props'' list will provide a set of propery/value pairs necessary to hotplug given CPU instance with '''''device_add''''' command.

Revision as of 14:33, 4 January 2017

Summary

There are 2 ways to hotplug CPU in QEMU:

  • dedicated legacy interface: cpu-add QMP command
  • generic device-add/device-del interface for hot-(un)plugging CPUs

Owner

  • Name: Igor Mammedov
  • Email: imammedo@redhat.com

cpu-add interface

Summary

{ 'command': 'cpu-add', 'data': {'id': 'int'} }

  • ID - a number in range [0..max-cpus)
  • Available since: 1.5
  • Supported targets: i386-softmmu, x86_64-softmmu, s390x (since 2.6)

Description

Command is a legacy solution for CPU hot-add and gives a simplified interface for it. It provides an opportunity to implement the feature for targets that currently can't implement CPU hot-add using device_add command due to their present design. Later targets that implement it could rewrite it to become a wrapper over device_add when it becomes usable for target.

Usage example

1. start QEMU with QMP socket available and with startup amount of CPUs less than maxcpus

./qemu-system-x86_64 -qmp unix:/tmp/qmp-sock,server,nowait -smp 1,maxcpus=4

2. Connect to QMP socket using qmp-shell command

./QMP/qmp-shell /tmp/qmp-sock

3. Add CPUs issuing cpu-add command in qmp-shell command prompt

cpu-add id=1

4. Optionally online newly added CPU inside guest

Linux kernel doesn't online hot-added CPUs automatically. Once CPU is hot-added it should be onlined using an appropriate udev script or manually by issuing a following command:

echo 1 > /sys/devices/system/cpu/cpu1/online

Sample udev script: Add the following line to /etc/udev/rules.d/99-hotPlugCPU.rules

SUBSYSTEM=="cpu",ACTION=="add",RUN+="/bin/sh -c '[ ! -e /sys$devpath/online ] || echo 1 > /sys$devpath/online'"


Current limitations

  1. migration target should be started with initial CPU count '-smp XX' that includes hot-added CPUs on migration source side.
  2. CPU shouldn't be hot-plugged during migration.
  3. adding CPUs should be done in successive order from lower to higher IDs in [0..max-cpus) range.
    It's possible to add arbitrary CPUs in random order, however that would cause migration to fail on its target side.

device_add/device_del interface

Summary

Lists possible to hotplug CPUs:
{ 'command': 'query-hotpluggable-cpus', 'returns': ['HotpluggableCPU'] }

CPU hot-add:
{ 'command': 'device_add', 'data': {'driver': 'str', 'id': 'str', ... }}

  • mandatory properties for every CPU:
    • driver: cpu model type name
    • id: unique device name
  • target/configuration dependent properties
    • socket-id: socket number in range [0..max sockets)
    • core-id: core number in range [0..max cores)
    • thread-id: thread-id in range [..max threads)
    • node-id: NUMA node ID the CPU belongs to

CPU hot-remove:
{ 'command': 'device_del', 'data': {'id': 'str'} }

  • id: device name that has been used for CPU hotplug with device_add command or -device CLI option


Supported targets:

  • i386, x86_64 (since 2.7)
  • spapr (since 2.7)

Description

Target/machine type that supports generic device_add/device_del interface for hot-adding/removing CPUs must implement query-hotpluggable-cpus QMP command. User can query the capability using query-machines QMP command and check if property hotpluggable-cpus is set to true. If machine reports that hotpluggable-cpus are supported, user can use query-hotpluggable-cpus command to list possible to hotplug CPUs with list of necessary to hotplug properties. For example for following CLI:

qemu-system-x86_64 -smp 1,maxcpus=4 -qmp unix:/tmp/q,server,nowait

user would get following

qmp-shell -p /tmp/q
Welcome to the QMP low-level shell!
Connected to QEMU 2.8.50
(QEMU) query-hotpluggable-cpus
{
   "return": [
       {
           "type": "qemu64-x86_64-cpu", 
           "vcpus-count": 1, 
           "props": {
               "socket-id": 1, 
               "core-id": 0, 
               "thread-id": 0
           }
       }, 
       {
           "qom-path": "/machine/unattached/device[0]", 
           "type": "qemu64-x86_64-cpu", 
           "vcpus-count": 1, 
           "props": {
               "socket-id": 0, 
               "core-id": 0, 
               "thread-id": 0
           }
       }
   ]
}

list of possible CPUs for given at startup -cpu and -smp CLI options. Where

  • -cpu cpu_model is translated to corresponding CPU type name that should be used with device_add as driver property value
  • -smp n,sockets=x,cores=y,threads=z,maxcpus=m is translated to set of possible to hotplug CPUs, where per CPU props list will provide a set of propery/value pairs necessary to hotplug given CPU instance with device_add command.