nova compute fail to remove instance with port if network is broken

Bug #1271958 reported by George Shuklin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
New
Undecided
Unassigned

Bug Description

If user was capable to create broken network configuration, instance become undeletable. Reason why user can create broken networking is under investigation (current hypothesis: if network (neutron) created in one tennant and instance in other, and user is admin in both tenants, it cause broken configuration).

But such instance deletetion cause trace on nova-compute:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
    **args)
  File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
    result = getattr(proxyobj, method)(ctxt, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 353, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 90, in wrapped
    payload)
  File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 73, in wrapped
    return f(self, context, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 243, in decorated_function
    pass
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 229, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 294, in decorated_function
    function(self, context, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 271, in decorated_function
    e, sys.exc_info())
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 258, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1792, in terminate_instance
    do_terminate_instance(instance, bdms)
  File "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line 246, in inner
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1784, in do_terminate_instance
    reservations=reservations)
  File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 105, in inner
    rv = f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1757, in _delete_instance
    user_id=user_id)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1729, in _delete_instance
    self._shutdown_instance(context, db_inst, bdms)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1639, in _shutdown_instance
    network_info = self._get_instance_nw_info(context, instance)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 876, in _get_instance_nw_info
    instance)
  File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 455, in get_instance_nw_info
    result = self._get_instance_nw_info(context, instance, networks)
  File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 463, in _get_instance_nw_info
    nw_info = self._build_network_info_model(context, instance, networks)
  File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 1009, in _build_network_info_model
    subnets)
  File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 962, in _nw_info_build_network
    label=network_name,
UnboundLocalError: local variable 'network_name' referenced before assignment

The reason is following code :

    def _nw_info_build_network(self, port, networks, subnets):
        # NOTE(danms): This loop can't fail to find a network since we
        # filtered ports to only the ones matching networks in our parent
        for net in networks:
            if port['network_id'] == net['id']:
                network_name = net['name']
                break

(if no net found network_name become undefined).

Following patch should allow instance deletion in case of networking problems:

diff --git a/nova/network/neutronv2/api.py b/nova/network/neutronv2/api.py
index a41924d..8a44f99 100644
--- a/nova/network/neutronv2/api.py
+++ b/nova/network/neutronv2/api.py
@@ -939,6 +939,8 @@ class API(base.Base):
             if port['network_id'] == net['id']:
                 network_name = net['name']
                 break
+ else:
+ network_name = ""

         bridge = None
         ovs_interfaceid = None

Revision history for this message
George Shuklin (george-shuklin) wrote :
description: updated
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.