Removing network bridge cause ERROR state of instance during deletion

Bug #1428182 reported by Marian Horban
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Low
Marian Horban

Bug Description

Instance moves to ERROR state after deleting this instance
Configuration:
  1. network_manager = nova.network.manager.VlanManager
  2. teardown_unused_network_gateway = true
Steps to reproduce:
  1. launch instance
  2. remove instance
Expected result:
instance is removed without error
Actual result:
instance is not removed, instance' state become ERROR:
root@node-1:~# nova list
+--------------------------------------+-----------------+--------+------------+-------------+----------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-----------------+--------+------------+-------------+----------------------+
| fd581745-8ecd-4fa2-af80-82493d083b97 | test_del_bridge | ERROR | deleting | Running | novanetwork=10.0.0.3 |
+--------------------------------------+-----------------+--------+------------+-------------+----------------------+
root@node-1:~# nova show fd581745-8ecd-4fa2-af80-82493d083b97
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | node-2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | node-2 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000002 |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | deleting |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | 2015-03-04T15:14:44.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2015-03-04T15:14:40Z |
| fault | {"message": "Remote error: ProcessExecutionError Unexpected error while running command. |
| | Command: sudo nova-rootwrap /etc/nova/rootwrap.conf dhcp_release br103 10.0.0.3 fa:16:3e:d2:1c:9a |
| | Exit code: 1 |
| | Stdout: '' |
| | Stderr: 'cannot setup interface: No such device\ |
| | ' |
| | [u'Tra", "code": 500, "details": " File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 296, in decorated_function |
| | return function(self, context, *args, **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2262, in terminate_instance |
| | do_terminate_instance(instance, bdms) |
| | File \"/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py\", line 249, in inner |
| | return f(*args, **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2260, in do_terminate_instance |
| | self._set_instance_error_state(context, instance['uuid']) |
| | File \"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py\", line 68, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2250, in do_terminate_instance |
| | reservations=reservations) |
| | File \"/usr/lib/python2.7/dist-packages/nova/hooks.py\", line 103, in inner |
| | rv = f(*args, **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2220, in _delete_instance |
| | user_id=user_id) |
| | File \"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py\", line 68, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2190, in _delete_instance |
| | self._shutdown_instance(context, db_inst, bdms) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2127, in _shutdown_instance |
| | self._try_deallocate_network(context, instance, requested_networks) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2086, in _try_deallocate_network |
| | self._set_instance_error_state(context, instance['uuid']) |
| | File \"/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py\", line 68, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 2081, in _try_deallocate_network |
| | self._deallocate_network(context, instance, requested_networks) |
| | File \"/usr/lib/python2.7/dist-packages/nova/compute/manager.py\", line 1775, in _deallocate_network |
| | context, instance, requested_networks=requested_networks) |
| | File \"/usr/lib/python2.7/dist-packages/nova/network/api.py\", line 94, in wrapped |
| | return func(self, context, *args, **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/nova/network/api.py\", line 318, in deallocate_for_instance |
| | requested_networks=requested_networks) |
| | File \"/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py\", line 190, in deallocate_for_instance |
| | return cctxt.call(ctxt, 'deallocate_for_instance', **kwargs) |
| | File \"/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py\", line 150, in call |
| | wait_for_reply=True, timeout=timeout) |
| | File \"/usr/lib/python2.7/dist-packages/oslo/messaging/transport.py\", line 90, in _send |
| | timeout=timeout) |
| | File \"/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py\", line 409, in send |
| | return self._send(target, ctxt, message, wait_for_reply, timeout) |
| | File \"/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py\", line 402, in _send |
| | raise result |
| | ", "created": "2015-03-04T15:14:48Z"} |
| flavor | m1.tiny (1) |
| hostId | 8a17e6dfb8afa7a82fa5964edc54bb5355cc46fb73e000bc2b310623 |
| id | fd581745-8ecd-4fa2-af80-82493d083b97 |
| image | TestVM (a671fa04-d19c-49d1-888d-9a43412e13ac) |
| key_name | KEY_PAIR |
| metadata | {} |
| name | test_del_bridge |
| novanetwork network | 10.0.0.3 |
| os-extended-volumes:volumes_attached | [] |
| security_groups | default |
| status | ERROR |
| tenant_id | dc669bf633ae4bd7b995c885e0428fec |
| updated | 2015-03-04T15:14:48Z |
| user_id | bd2a2c6487484a24937c39740a76e501 |
+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+

Marian Horban (mhorban)
Changed in nova:
status: New → In Progress
assignee: nobody → Marian Horban (mhorban)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/161289

Changed in nova:
importance: Undecided → Low
Revision history for this message
Jay Pipes (jaypipes) wrote :

There is a second bug here, that the fault field contains a traceback. Users should never see Python tracebacks in the fault field in `nova show <UUID>`...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/161289
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ba08070540431828fb3c741b499b1d3db7f873ed
Submitter: Jenkins
Branch: master

commit ba08070540431828fb3c741b499b1d3db7f873ed
Author: Marian Horban <email address hidden>
Date: Wed Mar 4 10:58:19 2015 -0500

    Releasing DHCP in nova-network fixed

    Added checking if device exists before executing
    `dhcp_release dev` command.

    Change-Id: I1fcd6f817121fb63608e166043ae37f185f2d4b6
    Closes-Bug: #1428182

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → kilo-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: kilo-rc1 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.