few VMs fail to get ip address in devstack

[Sarada]

I am running Juno int code on multi node devstack environment. I am trying to boot 10 VMs in each different network. If so, few VMs, will fail to boot. If i delete & recreate a VM in the same network, it will fail to get the ip address.

Here are the steps followed to hit this issue

1. Create a network
2. Create a subnet
3. Create a distributed router. Add the subnet to this DVR
4. Boot a VM.
5. Follow the steps 1-4 for 9 more VMs.
6. In this case, few VMs will not get the ip address.
7. I see that dhcp request is not reaching the NN. Please note that few more VMs hosted on the same CN are getting the ip address.

On both Openstack Controller & Compute node logs does not show any Errors.

[Sarada (sarada-a)]

Detailed Steps to recreate the problem

1. Create a network
2. Create a subnet
3. Create a distributed router. Add the subnet to this DVR
4. Boot a VM.
5. Follow the steps 1-4 for 9 more VMs.
6. When i boot the VMs in bulk, few VMs will fail to boot. It will be in Build state itself.
7. If i delete & try to recreate another VM in the same networks, it will fail to get the ip address

[Sarada (sarada-a)]

I verified by adding Legacy Routers in step3, all the VMs will boot at the first place & get the ip address successfully. Hoping DVR is causing some issue here.

[Carl Baldwin (carl-baldwin)]

Marking this High for now. We need to attempt to reproduce this to confirm the problem. Assuming we can confirm, we need to figure out where the problem is.

[Stephen Ma (stephen-ma)]

I created two node setup. One compute node plus one controller node. I created 9 VMs using the steps 1-5 outlined in the description. Checking the VMs' console logs after boot-up, the eth0 of two of the VMs never received their fixed IP addresses. Reboots of these VMs didn't correct the problem.

[Armando Migliaccio (armando-migliaccio)]

https://review.openstack.org/#/c/112229

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/112229
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=dc658273e7cfe72e50dad40203db8bb0d9bb2188
Submitter: Jenkins
Branch: master

commit dc658273e7cfe72e50dad40203db8bb0d9bb2188
Author: Vivekanandan Narasimhan <email address hidden>
Date: Tue Aug 5 11:36:35 2014 -0700

    Fix to enable L2pop to serve DVR

    This change fixes the information used by the L2pop
    driver to populate l2pop rules that enables DVR to
    route packets across compute servers that have
    tenant VMs that belong to different networks.
    It also fixes the case where VMs were not able to
    obtain IP Addresses when such VMs are on DVR
    hosted subnets.

    Change-Id: Ib630e57c186da60eb15f9ffa6b1b0bfa74f48caa
    Closes-Bug: #1350485
    Closes-Bug: #1352857