Quantum Manager VM launch failure

[dan wendlandt]

If the first VM to be spun with DHCP-enabled (default on) is quickly followed by a second VM (e.g., do to an automated script launching VMs), you will reliably hit a race condition that causes the second VM to fail to successfully launch.

This occurs with Quantum Manager, and the race condition is related to the check that determines if a quantum port for the DHCP server has been created yet. The solution is a simple one-liner: to add @synchronized for the enable_dhcp method so that it cannot be run simultaneously for two VMs.

diff --git a/nova/network/quantum/manager.py b/nova/network/quantum/manager.py
index b433652..4fc6f37 100644
--- a/nova/network/quantum/manager.py
+++ b/nova/network/quantum/manager.py
@@ -364,6 +364,7 @@ class QuantumManager(manager.FloatingIP, manager.FlatManager):
                                          instance['uuid'],
                                          instance_type_id, host)

+ @utils.synchronized('quantum-enable-dhcp')
     def enable_dhcp(self, context, quantum_net_id, network_ref, vif_rec,
             project_id):
         LOG.info("Using DHCP for network: %s" % network_ref['label'])

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/3447
Committed: http://github.com/openstack/nova/commit/6142230ccf2555650dbb902a5c342a342e9b2582
Submitter: Jenkins
Branch: master

commit 6142230ccf2555650dbb902a5c342a342e9b2582
Author: Dan Wendlandt <email address hidden>
Date: Wed Jan 25 21:36:22 2012 -0800

    bug 921931: fix Quantum Manager VM launch race condition

    Change-Id: I1f3a0f001477caa5f595c93c49f6c9e03a50149a

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/3452

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/3452
Committed: http://github.com/openstack/nova/commit/e77e28705c6b409a18ebefbbaed9474c07f66be0
Submitter: Jenkins
Branch: milestone-proposed

commit e77e28705c6b409a18ebefbbaed9474c07f66be0
Author: Dan Wendlandt <email address hidden>
Date: Wed Jan 25 21:36:22 2012 -0800

    bug 921931: fix Quantum Manager VM launch race condition

    Change-Id: I1f3a0f001477caa5f595c93c49f6c9e03a50149a