nova-network associates wrong network

[Yves Vogl]

OpenStack Grizzly on Ubuntu 12.04, Multi host setup.

I'm creating 3 different networks:

# nova-manage network create --label compute-1 --fixed_range_v4 192.168.1.0/27 --num_networks 1 --network_size 32
# nova-manage network create --label compute-2 --fixed_range_v4 192.168.2.0/27 --num_networks 1 --network_size 32
# nova-manage network create --label compute-3 --fixed_range_v4 192.168.3.0/27 --num_networks 1 --network_size 32

Not every compute node may use every network because they are distributed over different data centers. So I need to associate the hosts to the networks:

# nova-manage network modify --host=compute-1 192.168.1.0/27
# nova-manage network modify --host=compute-2 192.168.2.0/27
# nova-manage network modify --host=compute-3 192.168.3.0/27

1. Issue

Now, when I call the following command:

  `nova boot --image <uuid> --flavor m1.medium --availability-zone eu-central-1 test`

The created VM gets 3 NICs due to the multinic feature. But in fact, only one network is suitable because the other 2 networks aren't even associated with the hosts.

Workaround:

I need to specify the host and the network.

  `nova boot --image --nic net-id=<uuid-of-net-on-compute-1> <uuid> --flavor m1.medium --availability-zone eu-central-1:compute-1 test`

Unfortunately this bypasses some useful scheduling balancer algorithms.

2. Issue

After creating the networks, I can see appropriate dnsmasq processes. On compute-1 I see:

/usr/sbin/dnsmasq (...) -listen-address=192.168.1.1 --except-interface=lo --dhcp-range=set:compute-1,192.168.1.1,static,255.255.255.224,120s (...)

On compute-2 I see:

/usr/sbin/dnsmasq (...) --listen-address=192.168.2.1 --except-interface=lo --dhcp-range=set:compute-2,192.168.2.1,static,255.255.255.224,120s (...)

But when I kill those dnsmasq processes and restart nova-network, then compute-2 starts dnsmasq with the configuration for compute-1.

[Yves Vogl (vogl)]

I explored that this only happens if there are active VMs. It seems that the affected VMs „loose“ their compute-node association.

[Yves Vogl (vogl)]

https://github.com/openstack/nova/blob/9e9b3328af92cb9697f1aa4be22279b09f0fa14b/nova/network/manager.py#L455

I think this should only get the networks for the host… not just all networks.
I try to validate and fix this.

[Yves Vogl (vogl)]

I can validate that this line causes the issue.

The following should work – but self.host is undefined in this context. So I need to figure out where to get the valid host instance from.

`networks = self.db.network_get_all_by_host(context, self.host)`

[Yves Vogl (vogl)]

Obviously self.host was the right way ;-)

I fixed this and sent a pull request:

https://github.com/openstack/nova/pull/27

[Mark McLoughlin (markmc)]

Yves - it would be great if you could submit your patch through gerrit. Thanks!

[Yves Vogl (vogl)]

Thanks, I'll do so.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/43668

[Joe Gordon (jogo)]

patch was abandoned, moving back to new to be re-triaged

[Sean Dague (sdague)]

The code in this area has substantially changed. Is this still an issue?

[Yves Vogl (vogl)]

I had no time to check this out any further because I left the company who was affected by this bug.

Maybe Alexander Papaspyrou (https://launchpad.net/~lxndrp) can help you. I'll contact him.

[Yves Vogl (vogl)]

My former colleague confirms that this bug still exists.

[loosi (looserpain)]

In confirm that this bug is still present, too.

Im currently using Icehouse on Ubuntu/OpenSuse Nodes with nova-network and associated and non-associated networks seems to be completely ignored when not using the "nic" param for nova-boot. I do believe that this is a basic feature that isn't working as expected.

I cannot confirm second issue, since im not using any dnsmasq features.

offtopic
@vogl : good explanation on hetzner forum how some time ago. Setup with icehouse for hetzner is even more easy these days and works like a charm.

[Steve Wyamch (steve-wyamch)]

I also have this issue, icehouse, ubuntu 14.04.
This bug still exists.

[Markus Zoeller (markus_z) (mzoeller)]

This is an automated cleanup. This bug report has been closed because it
is older than 18 months and there is no open code change to fix this.
After this time it is unlikely that the circumstances which lead to
the observed issue can be reproduced.

If you can reproduce the bug, please:
* reopen the bug report (set to status "New")
* AND add the detailed steps to reproduce the issue (if applicable)
* AND leave a comment "CONFIRMED FOR: <RELEASE_NAME>"
  Only still supported release names are valid (LIBERTY, MITAKA, OCATA, NEWTON).
  Valid example: CONFIRMED FOR: LIBERTY