resize server fails silently

[Timofey Durakov]

Attempt to resize instance from bigger flavor to smaller fails, but no user notification in cli happened:

root@node-7:~# nova resize --poll a4 1

Server resizing... 100% complete

while nova-compute.log has trace:
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 137, in _dispatch_and_reply
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher incoming.message))
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 180, in _dispatch
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 126, in _do_dispatch
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher payload)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 298, in decorated_function
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher pass
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 284, in decorated_function
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 348, in decorated_function
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 272, in decorated_function
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher migration.instance_uuid, exc_info=True)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 259, in decorated_function
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 326, in decorated_function
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 314, in decorated_function
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3795, in resize_instance
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher self.instance_events.clear_events_for_instance(instance)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher self.gen.throw(type, value, traceback)
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 6123, in _error_out_instance_on_exception
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher raise error.inner_exception
2015-05-15 09:39:45.968 32005 TRACE oslo.messaging.rpc.dispatcher ResizeError: Resize error: Unable to resize disk down.

User should be notified about such fail.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/184495

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matt Riedemann (<email address hidden>) on branch: master
Review: https://review.openstack.org/195088
Reason: I'm considering this abandoned, if there is still intent to work on it I think we should have a blueprint like Andrew pointed out and plan that for Newton.

[Charlotte Han (hanrong)]

I reproduced this bug stable/mitaka version.

[stack@SBCJSlot5Rack2Centos7 root]$ openstack server resize --wait 0428ffdd-d8fe-4eb0-9cca-562d099f6868--flavor 1
Complete

2016-06-01 11:45:31.659 INFO nova.compute.manager [req-185643b2-94d6-4c0a-9b60-049b7db0a133 admin admin] [instance: 0428ffdd-d8fe-4eb0-9cca-562d099f6868] Setting instance back to ACTIVE after: Instance rollback performed due to: Resize error: Unable to resize disk down.

I think this disk down could be detected earlier in nova-conductor or nova-api.

[Timofey Durakov (tdurakov)]

I'm proposing to move this issue to Invalid state. Nova is designed to be as much non blocking as possible. So it accepts request and than check possibility to resize in background. While all this information is available over instance-actions. Also, it's worth to mention that xen could resize down so it's not possible to check this on API side.

[Charlotte Han (hanrong)]

Ok.