Linux bridge and devstack does not work

[Gary Kotton]

(Sumit's analysis: This is probably not a devstack issue. The problem is with the LB gateway driver wherein the IP address is trying to be set on a bridge device which already has an IP address. The check for an existing bridge is being performed, but the IP address is being set outside that check. Ideally, this code should not have been invoked if the gateway was already set. Something seems to have changed in the QuantumManager as result of which this code is being invoked again. At any rate, I will fix the LB gateway driver, and we will not see this)

For example when devstack is up and running prior to deploying an instance we have:

brq744ec2f4-c0 Link encap:Ethernet HWaddr fa:16:3e:03:a6:55
          inet addr:10.0.0.1 Bcast:0.0.0.0 Mask:255.255.255.0
          UP BROADCAST MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

gw-744ec2f4-c0 Link encap:Ethernet HWaddr fa:16:3e:03:a6:55
          inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
          UP BROADCAST MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

When an instance is deployed the following happens:

2012-05-15 01:59:18 DEBUG nova.utils [req-4d50ed10-46e1-406c-9074-dc45da860365 df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] Running cmd (subprocess): sudo /usr/local/bin/nova-rootwrap ip address add 10.0.0.1/24 dev brq744ec2f4-c0 from (pid=4234) execute /opt/stack/nova/nova/utils.py:178
2012-05-15 01:59:18 DEBUG nova.utils [req-4d50ed10-46e1-406c-9074-dc45da860365 df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] Result was 254 from (pid=4234) execute /opt/stack/nova/nova/utils.py:194
2012-05-15 01:59:18 ERROR nova.rpc.amqp [req-4d50ed10-46e1-406c-9074-dc45da860365 df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] Exception during message handling
2012-05-15 01:59:18 TRACE nova.rpc.amqp Traceback (most recent call last):
2012-05-15 01:59:18 TRACE nova.rpc.amqp File "/opt/stack/nova/nova/rpc/amqp.py", line 263, in _process_data
2012-05-15 01:59:18 TRACE nova.rpc.amqp rval = node_func(context=ctxt, **node_args)
2012-05-15 01:59:18 TRACE nova.rpc.amqp File "/opt/stack/nova/nova/network/quantum/manager.py", line 390, in allocate_for_instance
2012-05-15 01:59:18 TRACE nova.rpc.amqp network, vif_rec, network['net_tenant_id'])
2012-05-15 01:59:18 TRACE nova.rpc.amqp File "/opt/stack/nova/nova/utils.py", line 880, in inner
2012-05-15 01:59:18 TRACE nova.rpc.amqp retval = f(*args, **kwargs)
2012-05-15 01:59:18 TRACE nova.rpc.amqp File "/opt/stack/nova/nova/network/quantum/manager.py", line 501, in enable_dhcp
2012-05-15 01:59:18 TRACE nova.rpc.amqp self.l3driver.initialize_gateway(network_ref)
2012-05-15 01:59:18 TRACE nova.rpc.amqp File "/opt/stack/nova/nova/network/l3.py", line 98, in initialize_gateway
2012-05-15 01:59:18 TRACE nova.rpc.amqp gateway=(network_ref['gateway'] is not None))
2012-05-15 01:59:18 TRACE nova.rpc.amqp File "/opt/stack/nova/nova/network/linux_net.py", line 900, in plug
2012-05-15 01:59:18 TRACE nova.rpc.amqp return _get_interface_driver().plug(network, mac_address, gateway)
2012-05-15 01:59:18 TRACE nova.rpc.amqp File "/opt/stack/nova/nova/network/linux_net.py", line 1160, in plug
2012-05-15 01:59:18 TRACE nova.rpc.amqp run_as_root=True)
2012-05-15 01:59:18 TRACE nova.rpc.amqp File "/opt/stack/nova/nova/utils.py", line 201, in execute
2012-05-15 01:59:18 TRACE nova.rpc.amqp cmd=' '.join(cmd))
2012-05-15 01:59:18 TRACE nova.rpc.amqp ProcessExecutionError: Unexpected error while running command.
2012-05-15 01:59:18 TRACE nova.rpc.amqp Command: sudo /usr/local/bin/nova-rootwrap ip address add 10.0.0.1/24 dev brq744ec2f4-c0
2012-05-15 01:59:18 TRACE nova.rpc.amqp Exit code: 254
2012-05-15 01:59:18 TRACE nova.rpc.amqp Stdout: ''
2012-05-15 01:59:18 TRACE nova.rpc.amqp Stderr: 'RTNETLINK answers: File exists\n'
2012-05-15 01:59:18 TRACE nova.rpc.amqp
2012-05-15 01:59:18 ERROR nova.rpc.common [req-4d50ed10-46e1-406c-9074-dc45da860365 df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] Returning exception Unexpected error while running command.
Command: sudo /usr/local/bin/nova-rootwrap ip address add 10.0.0.1/24 dev brq744ec2f4-c0
Exit code: 254
Stdout: ''
Stderr: 'RTNETLINK answers: File exists\n' to caller
2012-05-15 01:59:18 ERROR nova.rpc.common [req-4d50ed10-46e1-406c-9074-dc45da860365 df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/rpc/amqp.py", line 263, in _process_data\n rval = node_func(context=ctxt, **node_args)\n', ' File "/opt/stack/nova/nova/network/quantum/manager.py", line 390, in allocate_for_instance\n network, vif_rec, network[\'net_tenant_id\'])\n', ' File "/opt/stack/nova/nova/utils.py", line 880, in inner\n retval = f(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/quantum/manager.py", line 501, in enable_dhcp\n self.l3driver.initialize_gateway(network_ref)\n', ' File "/opt/stack/nova/nova/network/l3.py", line 98, in initialize_gateway\n gateway=(network_ref[\'gateway\'] is not None))\n', ' File "/opt/stack/nova/nova/network/linux_net.py", line 900, in plug\n return _get_interface_driver().plug(network, mac_address, gateway)\n', ' File "/opt/stack/nova/nova/network/linux_net.py", line 1160, in plug\n run_as_root=True)\n', ' File "/opt/stack/nova/nova/utils.py", line 201, in execute\n cmd=\' \'.join(cmd))\n', "ProcessExecutionError: Unexpected error while running command.\nCommand: sudo /usr/local/bin/nova-rootwrap ip address add 10.0.0.1/24 dev brq744ec2f4-c0\nExit code: 254\nStdout: ''\nStderr: 'RTNETLINK answers: File exists\\n'\n"]