OpenStack Compute (nova)

Instance resize fails due to incorrect parameters order

Bug #1793177 reported by Vladyslav Drok
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
In Progress
Medium
Vladyslav Drok

Bug Description

This happens on pike on destination node (to be perfectly clear it is cold migration from ocata compute to pike compute):

2018-09-18 15:17:31.329 10975 INFO nova.compute.manager [req-70e235c5-8f73-4e27-b2a9-dd17cc89c72d c073ae48f68d498cbd719d11024c1054 e6d59eb5849e49ef81a9c7ab01ca68ad - - -] [instance: ffd00416-7eff-4a1f-8c14-17cb6bd40b17] Successfully reverted task state from resize_prep on failure for instance.
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server [req-70e235c5-8f73-4e27-b2a9-dd17cc89c72d c073ae48f68d498cbd719d11024c1054 e6d59eb5849e49ef81a9c7ab01ca68ad - - -] Exception during message handling: ValueError: A list is required in field reservations, not a bool
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 75, in wrapped
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server function_name, call_dict, binary)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server self.force_reraise()
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 66, in wrapped
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 188, in decorated_function
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server LOG.warning(msg, e, instance=instance)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server self.force_reraise()
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 157, in decorated_function
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/utils.py", line 686, in decorated_function
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 145, in decorated_function
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server migration.instance_uuid, exc_info=True)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server self.force_reraise()
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 122, in decorated_function
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 216, in decorated_function
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info())
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server self.force_reraise()
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 204, in decorated_function
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3885, in resize_instance
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server instance=instance)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/nova/objects/quotas.py", line 80, in from_reservations
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server quotas.reservations = reservations
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 72, in setter
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server field_value = field.coerce(self, name, value)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/fields.py", line 195, in coerce
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server return self._type.coerce(obj, attr, value)
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/fields.py", line 637, in coerce
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server {'attr': attr, 'type': type(value).__name__})
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server ValueError: A list is required in field reservations, not a bool
2018-09-18 15:17:31.342 10975 ERROR oslo_messaging.rpc.server

The problem here is simply incorrect order of positional parameters in the compute_rpcapi.resize_instance call

Vladyslav Drok (vdrok)
Changed in nova:
assignee: nobody → Vladyslav Drok (vdrok)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/603439

Revision history for this message
Matt Riedemann (mriedem) wrote :

What is the value of [upgrade_levels]/compute in nova.conf? It should be Ocata for compatibility while you're doing rolling upgrades (or 'auto' might work too).

Matt Riedemann (mriedem)
Changed in nova:
status: New → In Progress
Revision history for this message
Vladyslav Drok (vdrok) wrote :

Hey Matt, it was set to auto.

Matt Riedemann (mriedem)
Changed in nova:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/pike)

Reviewed: https://review.openstack.org/603439
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=76f937b50a50dd43b97d62ebab8459bc98cfe554
Submitter: Zuul
Branch: stable/pike

commit 76f937b50a50dd43b97d62ebab8459bc98cfe554
Author: Vladyslav Drok <email address hidden>
Date: Tue Sep 18 19:32:16 2018 +0300

    [pike-only] Fix resize_instance rpcapi call

    After reservations parameter was removed from the call in compute
    manager (change I9269ffa2b80e48db96c622d0dc0817738854f602), it began
    passing clean_shutdown into reservations instead.
    This is pike only change as this code was refactored in queens
    (commit eae37a27caa5ca8b0ca50187928bde81f28a24e1) and reservations
    parameter was removed.

    Closes-Bug: 1793177
    Change-Id: Ia59ec50bda18246b2747aefc6cc621ad54b698b0

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 16.1.8

This issue was fixed in the openstack/nova 16.1.8 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.