nova-compute.log display incorrect information for CPUPinningUnknown.

Bug #1729784 reported by Charlotte Han
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Low
Charlotte Han
Pike
Fix Released
Low
Charlotte Han

Bug Description

Description
===========
When nova-compute occured CPUPinningUnknown exception, the information of nova-compute.log is wrong.

Steps to reproduce
==================
* I did create 6 instances with numa topology.
  [root@ctrl3 ~]# virsh list
 Id Name State
----------------------------------------------------
 1 instance-00000002 running
 2 instance-00000006 running
 3 instance-0000000a running
 4 instance-00000008 running
 5 instance-0000000c running
 6 instance-00000004 running

[root@ctrl3 ~]# virsh vcpupin 1
VCPU: CPU Affinity
----------------------------------
   0: 28

[root@ctrl3 ~]# virsh vcpupin 2
VCPU: CPU Affinity
----------------------------------
   0: 2

[root@ctrl3 ~]# virsh vcpupin 3
VCPU: CPU Affinity
----------------------------------
   0: 44

[root@ctrl3 ~]# virsh vcpupin 4
VCPU: CPU Affinity
----------------------------------
   0: 0

[root@ctrl3 ~]# virsh vcpupin 5
VCPU: CPU Affinity
----------------------------------
   0: 16

[root@ctrl3 ~]# virsh vcpupin 6
VCPU: CPU Affinity
----------------------------------
   0: 18

* then I did modify section vcpu_pin_set in nova.conf.
   [root@ctrl3 ~]# cat /etc/nova/nova.conf | grep vcpu_pin_set
    vcpu_pin_set = "0-47,^18"

* then I did restart openstack-nova-compute service
   Log display "Error updating resources for node ctrl3.: CPUPinningUnknown: CPU set to pin [18] must be a subset of known CPU set [28]".
   But this information is wrong, known CPU set is not only [28], it should be [0, 2, 4, 6, 8, 10, 12, 14, 16, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46]

Expected result
===============
CPUPinningUnknown: CPU set to pin [18] must be a subset of known CPU set [0, 2, 4, 6, 8, 10, 12, 14, 16, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46]

Actual result
=============
 CPUPinningUnknown: CPU set to pin [18] must be a subset of known CPU set [28]

Environment
===========
1. Exact version of OpenStack you are running. See the following
  Pike version

2. Which hypervisor did you use?
  Libvirt + KVM

Logs & Configs
==============
/lockutils.py:282
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager [req-8835b3d7-a453-4f91-abfa-5ccd0c54693b - - - - -] Error updating resources for node ctrl3.: CPUPinningUnknown: CPU set to pin [18] must be a subset of known CPU set [28]
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager Traceback (most recent call last):
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6647, in update_available_resource_for_node
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager rt.update_available_resource(context, nodename)
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 657, in update_available_resource
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager self._update_available_resource(context, resources)
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager return f(*args, **kwargs)
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 698, in _update_available_resource
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager self._update_usage_from_instances(context, instances, nodename)
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 1141, in _update_usage_from_instances
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager has_ocata_computes=has_ocata_computes)
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 1077, in _update_usage_from_instance
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager sign=sign)
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 875, in _update_usage
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager cn, usage, free)
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/hardware.py", line 1784, in get_host_numa_usage_from_instance
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager host_numa_topology, instance_numa_topology, free=free))
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/hardware.py", line 1640, in numa_usage_from_instances
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager newcell.pin_cpus(pinned_cpus)
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/objects/numa.py", line 82, in pin_cpus
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager cpuset=list(self.pinned_cpus))
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager CPUPinningUnknown: CPU set to pin [18] must be a subset of known CPU set [28]
2017-11-03 14:44:01.425 32568 ERROR nova.compute.manager

Charlotte Han (hanrong)
Changed in nova:
importance: Undecided → Low
assignee: nobody → Charlotte Han (hanrong)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/517592

Eli Qiao (taget-9)
Changed in nova:
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/523644

Charlotte Han (hanrong)
Changed in nova:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/517592
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ecf12a4dd132e4aa9c2e3563a41acc0ab6d13978
Submitter: Zuul
Branch: master

commit ecf12a4dd132e4aa9c2e3563a41acc0ab6d13978
Author: zte-hanrong <email address hidden>
Date: Thu Nov 2 16:09:26 2017 +0800

    Fix incorrect known vcpuset when CPUPinningUnknown raised

    When exception.CPUPinningUnknown or exception.CPUUnPinningUnknown
    is raised, the value of known vcpuset is NUMACell's pinned_cpus,
    it should be NUMACell's cpuset.

    Close-Bug: #1729784
    Change-Id: I1902dc40623af62fd83fac222099f664d7d19ed9

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/pike)

Reviewed: https://review.openstack.org/523644
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ae56863746cdd69e27f2e046a81f581541508168
Submitter: Zuul
Branch: stable/pike

commit ae56863746cdd69e27f2e046a81f581541508168
Author: zte-hanrong <email address hidden>
Date: Thu Nov 2 16:09:26 2017 +0800

    Fix incorrect known vcpuset when CPUPinningUnknown raised

    When exception.CPUPinningUnknown or exception.CPUUnPinningUnknown
    is raised, the value of known vcpuset is NUMACell's pinned_cpus,
    it should be NUMACell's cpuset.

    Close-Bug: #1729784
    Change-Id: I1902dc40623af62fd83fac222099f664d7d19ed9
    (cherry picked from commit ecf12a4dd132e4aa9c2e3563a41acc0ab6d13978)

tags: added: in-stable-pike
Charlotte Han (hanrong)
Changed in nova:
status: In Progress → Fix Committed
Changed in nova:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.