| 2018-11-21 18:57:25 |
Inbar Stolberg |
description |
Description
===========
server rebuild will fail in nova scheduler on NUMATopologyFilter if the computes do not have enough capacity (even though clearly the running server is already accounted into that calculation)
to resolve the issue a fix is required in NUMATopologyFilter to not perform the rebuild operation in the case that the request is due to rebuild.
the result of such a case will be that server rebuild will fail with error of "no valid host found"
(do not mix resize with rebuild functions...)
Steps to reproduce
==================
1. create a flavor that contain metadata that will point to a specific compute (use host aggregate with same key:value metadata
2. create a server on that compute (preferably using heat stack)
3. (try to) rebuild the server using stack update
4. issue reproduced
Expected result
===============
server in an active running state (if image was replaced in the rebuild command than with a reference to the new image in the server details.
Actual result
=============
server in error state with error of no valid host found.
Environment
===========
detected in Rocky release
KVM hypervisor
Ceph storage
Neutron networks
Logs & Configs
==============
in nova.conf:
enabled_filters=AggregateInstanceExtraSpecsFilter,RetryFilter,AvailabilityZoneFilter,NUMATopologyFilter,PciPassthroughFilter,RamFilter,ComputeFilter,ImagePropertiesFilter,CoreFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,DiskFilter,ComputeCapabilitiesFilter,AggregateRamFilter,SameHostFilter,DifferentHostFilter
logs: tbd |
Description
===========
server rebuild will fail in nova scheduler on NUMATopologyFilter if the computes do not have enough capacity (even though clearly the running server is already accounted into that calculation)
to resolve the issue a fix is required in NUMATopologyFilter to not perform the rebuild operation in the case that the request is due to rebuild.
the result of such a case will be that server rebuild will fail with error of "no valid host found"
(do not mix resize with rebuild functions...)
Steps to reproduce
==================
1. create a flavor that contain metadata that will point to a specific compute (use host aggregate with same key:value metadata
make sure flavor contain topology related metadata:
hw:cpu_cores='1', hw:cpu_policy='dedicated', hw:cpu_sockets='6', hw:cpu_thread_policy='prefer', hw:cpu_threads='1', hw:mem_page_size='large', location='area51'
2. create a server on that compute (preferably using heat stack)
3. (try to) rebuild the server using stack update
4. issue reproduced
Expected result
===============
server in an active running state (if image was replaced in the rebuild command than with a reference to the new image in the server details.
Actual result
=============
server in error state with error of no valid host found.
Environment
===========
detected in Rocky release
KVM hypervisor
Ceph storage
Neutron networks
Logs & Configs
==============
in nova.conf:
enabled_filters=AggregateInstanceExtraSpecsFilter,RetryFilter,AvailabilityZoneFilter,NUMATopologyFilter,PciPassthroughFilter,RamFilter,ComputeFilter,ImagePropertiesFilter,CoreFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,DiskFilter,ComputeCapabilitiesFilter,AggregateRamFilter,SameHostFilter,DifferentHostFilter
logs: tbd |
|
| 2018-11-26 21:14:10 |
Inbar Stolberg |
description |
Description
===========
server rebuild will fail in nova scheduler on NUMATopologyFilter if the computes do not have enough capacity (even though clearly the running server is already accounted into that calculation)
to resolve the issue a fix is required in NUMATopologyFilter to not perform the rebuild operation in the case that the request is due to rebuild.
the result of such a case will be that server rebuild will fail with error of "no valid host found"
(do not mix resize with rebuild functions...)
Steps to reproduce
==================
1. create a flavor that contain metadata that will point to a specific compute (use host aggregate with same key:value metadata
make sure flavor contain topology related metadata:
hw:cpu_cores='1', hw:cpu_policy='dedicated', hw:cpu_sockets='6', hw:cpu_thread_policy='prefer', hw:cpu_threads='1', hw:mem_page_size='large', location='area51'
2. create a server on that compute (preferably using heat stack)
3. (try to) rebuild the server using stack update
4. issue reproduced
Expected result
===============
server in an active running state (if image was replaced in the rebuild command than with a reference to the new image in the server details.
Actual result
=============
server in error state with error of no valid host found.
Environment
===========
detected in Rocky release
KVM hypervisor
Ceph storage
Neutron networks
Logs & Configs
==============
in nova.conf:
enabled_filters=AggregateInstanceExtraSpecsFilter,RetryFilter,AvailabilityZoneFilter,NUMATopologyFilter,PciPassthroughFilter,RamFilter,ComputeFilter,ImagePropertiesFilter,CoreFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,DiskFilter,ComputeCapabilitiesFilter,AggregateRamFilter,SameHostFilter,DifferentHostFilter
logs: tbd |
Description
===========
server rebuild will fail in nova scheduler on NUMATopologyFilter if the computes do not have enough capacity (even though clearly the running server is already accounted into that calculation)
to resolve the issue a fix is required in NUMATopologyFilter to not perform the rebuild operation in the case that the request is due to rebuild.
the result of such a case will be that server rebuild will fail with error of "no valid host found"
(do not mix resize with rebuild functions...)
Steps to reproduce
==================
1. create a flavor that contain metadata that will point to a specific compute (use host aggregate with same key:value metadata
make sure flavor contain topology related metadata:
hw:cpu_cores='1', hw:cpu_policy='dedicated', hw:cpu_sockets='6', hw:cpu_thread_policy='prefer', hw:cpu_threads='1', hw:mem_page_size='large', location='area51'
2. create a server on that compute (preferably using heat stack)
3. (try to) rebuild the server using stack update
4. issue reproduced
Expected result
===============
server in an active running state (if image was replaced in the rebuild command than with a reference to the new image in the server details.
Actual result
=============
server in error state with error of no valid host found.
Message
No valid host was found. There are not enough hosts available.
Code
500
Details
File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 966, in rebuild_instance return_alternates=False) File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 723, in _schedule_instances return_alternates=return_alternates) File "/usr/lib/python2.7/site-packages/nova/scheduler/utils.py", line 907, in wrapped return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 53, in select_destinations instance_uuids, return_objects, return_alternates) File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 37, in __run_method return getattr(self.instance, __name)(*args, **kwargs) File "/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 158, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 179, in call retry=self.retry) File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 133, in _send retry=retry) File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send call_monitor_timeout, retry=retry) File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 575, in _send raise result
Environment
===========
detected in Rocky release
KVM hypervisor
Ceph storage
Neutron networks
Logs & Configs
==============
in nova.conf:
enabled_filters=AggregateInstanceExtraSpecsFilter,RetryFilter,AvailabilityZoneFilter,NUMATopologyFilter,PciPassthroughFilter,RamFilter,ComputeFilter,ImagePropertiesFilter,CoreFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,DiskFilter,ComputeCapabilitiesFilter,AggregateRamFilter,SameHostFilter,DifferentHostFilter
logs: tbd |
|
