reschedule failed because port still in use

Bug #1308405 reported by Liusheng
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Invalid
High
Liusheng

Bug Description

When booting an instance with a port specified, if the instance spawn faild for libvit error, the instance will be rescheduled, and will raise a "PortInUse" exception.

To reproduce, we can add "raise Exception" after spawn in _build_instance() and restart nova-compute.

more details plz see:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/site-packages/nova/compute/manager.py", line 1043, in _build_instance
    set_access_ip=set_access_ip)
  File "/usr/lib64/python2.6/site-packages/nova/compute/manager.py", line 1426, in _spawn
    LOG.exception(_('Instance failed to spawn'), instance=instance)
  File "/usr/lib64/python2.6/site-packages/nova/compute/manager.py", line 1423, in _spawn
    block_device_info)
  File "/usr/lib64/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2083, in spawn
    admin_pass=admin_password)
  File "/usr/lib64/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2480, in _create_image
    net = netutils.get_injected_network_template(network_info)
  File "/usr/lib64/python2.6/site-packages/nova/virt/netutils.py", line 74, in get_injected_network_
    if not (network_info and template):
  File "/usr/lib64/python2.6/site-packages/nova/network/model.py", line 379, in __len__
    return self._sync_wrapper(fn, *args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/nova/network/model.py", line 366, in _sync_wrapper
    self.wait()
  File "/usr/lib64/python2.6/site-packages/nova/network/model.py", line 398, in wait
    self[:] = self._gt.wait()
  File "/usr/lib64/python2.6/site-packages/eventlet/greenthread.py", line 168, in wait
    return self._exit_event.wait()
  File "/usr/lib64/python2.6/site-packages/eventlet/event.py", line 120, in wait
    current.throw(*self._exc)
  File "/usr/lib64/python2.6/site-packages/eventlet/greenthread.py", line 194, in main
    result = function(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/nova/compute/manager.py", line 1244, in _allocate_network
    dhcp_options=dhcp_options)
  File "/usr/lib64/python2.6/site-packages/nova/network/neutronv2/api.py", line 243, in allocate_for
    raise exception.PortInUse(port_id=port_id)
PortInUse: Port faf3aa64-11f8-4fc7-81bc-084098014f4a is still in use.

Liusheng (liusheng)
Changed in nova:
assignee: nobody → Liusheng (liusheng)
Liusheng (liusheng)
description: updated
description: updated
Revision history for this message
Openstack Gerrit (openstack-gerrit) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/87915

Changed in nova:
status: New → In Progress
Revision history for this message
Openstack Gerrit (openstack-gerrit) wrote : Fix proposed to nova (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/89505

Aaron Rosen (arosen)
tags: added: network
tags: added: havana-backport-potential icehouse-backport-potential
Changed in nova:
importance: Undecided → High
Revision history for this message
Chris Behrens (cbehrens) wrote :

actually, this bug may no longer be valid... because I think the build process doesn't deallocate and attempt to reallocate on reschedules anymore (unless the driver is baremetal/ironic)... Can you check the latest code from master to see if this is still a problem?

Revision history for this message
Chris Behrens (cbehrens) wrote :

Ignore that last comment.

Revision history for this message
Chris Behrens (cbehrens) wrote :

So I think the real problem here is that the device_id was never unassigned before the instance rescheduled. I suspect this review that landed 5 days ago happens to fix this bug also:

https://review.openstack.org/#/c/99400/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by liusheng (<email address hidden>) on branch: master
Review: https://review.openstack.org/87915
Reason: abandon, the https://review.openstack.org/#/c/99400/ has fixed this bug :(

Revision history for this message
Liusheng (liusheng) wrote :

Chris Behrens: thanks, it is. this bug has been fixed in https://review.openstack.org/#/c/99400/.

Changed in nova:
status: In Progress → Invalid
Revision history for this message
Chris Behrens (cbehrens) wrote :

Confirmed that review above does fix this problem too. Marking this as duplicate as 1321381 and closing.

Revision history for this message
Chris Behrens (cbehrens) wrote :

nevermind... was changed to invalid already.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.