neutron dhcp agent failed to spawn dnsmasq because tap interface had address configured by RA from external router

[Jerry Zhao]

i was configuring an ipv6 subnet in slaac address mode and N/S ipv6 ra mode, because external router provides router advertisement.
there was also an ipv4 subnet in the network.
after i created the ipv6 subnet, i found that dnsmasq was down because when neutron-rootwrap configured ipv6 address to the tap interface which dnsmasq listened to, the ipv6 address had already been auto configured by the router advertisement from external router, hence neutron-rootwrap exited with RTNETLINK answers: File exists

steps to reproduce the bug:
have an external router or radvd sending RA
add an ipv4 subnet
wait for a while until router advertisement reaches the tap interface dnsmasq binds to and gets an ipv6 address.
add an ipv6 subnet with ipv6 address mode: slaac ; ipv6 ra mode: not specified.
dnsmasq will fail to restart.

[Jerry Zhao (zhaoxinyu)]

ip netns exec qdhcp-609ce0e0-671e-47ad-b285-4f0afddf4457 tcpdump -i tap3195cce6-10 -vv ip6 -lnSxs 0
tcpdump: listening on tap3195cce6-10, link-type EN10MB (Ethernet), capture size 65535 bytes
08:11:58.592621 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 64) fe80::da49:bff:fe8c:36b9 > ff02::1: [icmp6 sum ok] ICMP6, router advertisement, length 64
 hop limit 64, Flags [none], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
   source link-address option (1), length 8 (1): d8:49:0b:8c:36:b9
     0x0000: d849 0b8c 36b9
   mtu option (5), length 8 (1): 1500
     0x0000: 0000 0000 05dc
   prefix info option (3), length 32 (4): 2001:470:bb86:1::/64, Flags [onlink, auto], valid time 2592000s, pref. time 604800s
     0x0000: 40c0 0027 8d00 0009 3a80 0000 0000 2001
     0x0010: 0470 bb86 0001 0000 0000 0000 0000
 0x0000: 6000 0000 0040 3aff fe80 0000 0000 0000
 0x0010: da49 0bff fe8c 36b9 ff02 0000 0000 0000
 0x0020: 0000 0000 0000 0001 8600 078e 4000 0708
 0x0030: 0000 0000 0000 0000 0101 d849 0b8c 36b9
 0x0040: 0501 0000 0000 05dc 0304 40c0 0027 8d00
 0x0050: 0009 3a80 0000 0000 2001 0470 bb86 0001
 0x0060: 0000 0000 0000 0000

ip netns exec qdhcp-609ce0e0-671e-47ad-b285-4f0afddf4457 ifconfig
lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:65536 Metric:1
          RX packets:27 errors:0 dropped:0 overruns:0 frame:0
          TX packets:27 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4672 (4.6 KB) TX bytes:4672 (4.6 KB)

tap3195cce6-10 Link encap:Ethernet HWaddr fa:16:3e:95:83:d0
          inet addr:162.3.122.5 Bcast:162.3.122.255 Mask:255.255.255.0
          inet6 addr: fe80::f816:3eff:fe95:83d0/64 Scope:Link
          inet6 addr: 2001:470:bb86:1:f816:3eff:fe95:83d0/64 Scope:Global
          UP BROADCAST RUNNING MTU:1500 Metric:1
          RX packets:18861 errors:0 dropped:0 overruns:0 frame:0
          TX packets:355 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2302847 (2.3 MB) TX bytes:31988 (31.9 KB)

neutron subnet-create --ip-version=6 --name=ipv6 --ipv6-address-mode=slaac provider-net 2001:470:bb86:0001::/64 --allocation-pool start=2001:470:bb86:0001::20,end=2001:470:bb86:0001::fffe --gateway 2001:470:bb86:0001::1
Created a new subnet:
+-------------------+------------------------------------------------------------------------+
| Field | Value |
+-------------------+------------------------------------------------------------------------+
| allocation_pools | {"start": "2001:470:bb86:0001::20", "end": "2001:470:bb86:0001::fffe"} |
| cidr | 2001:470:bb86:1::/64 |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 2001:470:bb86:0001::1 |...

[Xu Han Peng (xuhanp)]

I gave it a try today and in my environment, dnsmasq can successfully start:

nobody 28889 1 0 00:41 ? 00:00:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tapd8a07096-a2 --except-interface=lo --pid-file=/var/lib/neutron/dhcp/3d39fc2c-bd21-4ce2-8574-c0285627f6d2/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/3d39fc2c-bd21-4ce2-8574-c0285627f6d2/host --addn-hosts=/var/lib/neutron/dhcp/3d39fc2c-bd21-4ce2-8574-c0285627f6d2/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/3d39fc2c-bd21-4ce2-8574-c0285627f6d2/opts --leasefile-ro --dhcp-range=set:tag1,10.1.0.0,static,86400s --dhcp-lease-max=256 --conf-file=/etc/neutron/dnsmasq.conf --domain=openstacklocal

Since I am also using external router to send out RA, my dhcp tap device has both SLAAC IPv6 address and IPv4 address:

21: tapd8a07096-a2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:ff:74:d6 brd ff:ff:ff:ff:ff:ff
    inet 10.1.0.3/24 brd 10.1.0.255 scope global tapd8a07096-a2
       valid_lft forever preferred_lft forever
    inet6 2001:db8:1:0:f816:3eff:feff:74d6/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feff:74d6/64 scope link
       valid_lft forever preferred_lft forever

2001:db8:1:0:f816:3eff:feff:74d6 here is the SLAAC address using the prefix in external RA.

Jerry, is my test same with yours?

[Xu Han Peng (xuhanp)]

I recreated this problem in my env today. The trick is to let DHCP tap device get SLAAC first (by external RA), then add the ipv6 subnet to network.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/149503

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/149503
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5595956b88d9b7bb8f1f293be83c44e766be2917
Submitter: Jenkins
Branch: master

commit 5595956b88d9b7bb8f1f293be83c44e766be2917
Author: Xu Han Peng <email address hidden>
Date: Fri Jan 23 13:29:47 2015 +0800

    Not assign dynamic IPv6 address on dhcp interface

    When SLAAC IPv6 subnet using external router advertisement
    (ipv6_ra_mode=None, ipv6_address_mode=slaac) is added to a
    dual-stack network, the device of dnsmasq or router gateway may
    already be assigned with dynamic SLAAC IPv6 address. Current
    code only checks for permanent address on a device when
    initilizing the l3 address of that device. This results
    assigning duplicated address to dhcp tap device, which leads
    to dhcp agent failure.

    This fix changes dhcp code to not init dynamic IPv6 address
    on dhcp device so dynamic address can be auto-configured by
    either internal or external RA device.

    Change-Id: I72b85eb93d1849299d523eedf563ab201b4a0352
    Closes-Bug:1409634