OpenStack Compute (nova)

attaching all devstack quantum networks to a nova server results in un-deletable server

Bug #1236116 reported by Steven Dake on 2013-10-07

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Invalid	Undecided	Steven Dake
	OpenStack Heat	Invalid	Undecided	Steven Dake

Bug Description

Attaching multiple networks results in a backtrace and undeleteable instance when used with Neutron.

Run reproducer as follows:
[sdake@bigiron ~]$ less reproducer
#!/bin/bash

glance image-create --name=cirros-0.3.0-x86_64 --disk-format=qcow2 --container-f
ormat=bare < cirros.img
id1=`neutron net-list -c id -f csv --quote none | grep -v id | tail -1 | tr -d '
\r'`
id2=`neutron net-list -c id -f csv --quote none | grep -v id | head -1 | tr -d '
\r'`
nova boot --flavor m1.tiny --image cirros-0.3.0-x86_64 --security_group default
--nic net-id=$id1 --nic net-id=$id2 cirros

Run nova list waiting for server to become active. Once server is active, delete the server via the nova delete <id> operation. Server will enter an undeleteable saying it is "ACTIVE". It is important that both networks are connected when the delete operation is run, as for some reason one of the networks gets disconnected by some component (not sure which).

Further delete operations are either unsuccessful or block further ability to create instances with instances finishing in the ERROR state after creation.

n-cpu backtraces with:
2013-10-06 18:03:11.269 ERROR nova.openstack.common.rpc.amqp [req-4f7cf630-d1eb-4fcd-af22-c11fa77fd3dd admin admin] Exception during message handling
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp **args)
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 353, in decorated_function
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/exception.py", line 90, in wrapped
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp payload)
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/exception.py", line 73, in wrapped
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp return f(self, context, *args, **kw)
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 243, in decorated_function
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp pass
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 229, in decorated_function
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 294, in decorated_function
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp function(self, context, *args, **kwargs)
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 271, in decorated_function
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp e, sys.exc_info())
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 258, in decorated_function
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-10-06 18:03:11.269 TRACE nova.openstack.common.rpc.amqp File 2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 294, in decorated_function
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp function(self, context, *args, **kwargs)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 271, in decorated_function
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp e, sys.exc_info())
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 258, in decorated_function
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 1793, in terminate_instance
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp do_terminate_instance(instance, bdms)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 246, in inner
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp return f(*args, **kwargs)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 1785, in do_terminate_instance
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp reservations=reservations)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/hooks.py", line 105, in inner
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp rv = f(*args, **kwargs)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 1758, in _delete_instance
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp user_id=user_id)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 1730, in _delete_instance
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp self._shutdown_instance(context, db_inst, bdms)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 1640, in _shutdown_instance
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp network_info = self._get_instance_nw_info(context, instance)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/compute/manager.py", line 877, in _get_instance_nw_info
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp instance)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/network/neutronv2/api.py", line 455, in get_instance_nw_info
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp result = self._get_instance_nw_info(context, instance, networks)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/network/neutronv2/api.py", line 463, in _get_instance_nw_info
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp nw_info = self._build_network_info_model(context, instance, networks)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/network/neutronv2/api.py", line 1009, in _build_network_info_model
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp subnets)
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/nova/nova/network/neutronv2/api.py", line 962, in _nw_info_build_network
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp label=network_name,
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp UnboundLocalError: local variable 'network_name' referenced before assignment
2013-10-06 18:13:02.653 TRACE nova.openstack.common.rpc.amqp

Steven Dake (sdake) on 2013-10-07

Changed in nova:
assignee:	nobody → Steven Dake (sdake)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2013-10-07: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/49976

Changed in nova:
status:	New → In Progress

Steven Dake (sdake) on 2013-10-07

Changed in heat:
status:	New → In Progress
assignee:	nobody → Steven Dake (sdake)

Davanum Srinivas (DIMS) (dims-v) on 2013-10-07

tags:

added: havana-rc-potential

Thierry Carrez (ttx) on 2013-10-14

tags:

removed: havana-rc-potential

Revision history for this message

Joe Gordon (jogo) wrote on 2014-07-10:

Is this still valid, the patch was abandoned.

Changed in nova:
status:	In Progress → Incomplete

Revision history for this message

Sean Dague (sdague) wrote on 2014-09-17:

Old incomplete bug. Please reopen if still an issue.

Changed in nova:
status:	Incomplete → Invalid

Steve Baker (steve-stevebaker) on 2015-08-26

Changed in heat:
status:	In Progress → Invalid

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.