Comment 1 for bug 1811639

Revision history for this message
Bence Romsics (bence-romsics) wrote :

To my understanding this bug was first observed in a downstream modified (Pike-based) neutron. The following is my best reproduction on upstream master:

# create a network with a v6 subnet in dhcp stateless mode
openstack network create net1
openstack subnet create subnet1 --network net1 --subnet-range 10.0.4.0/24
openstack subnet create subnet1 --network net1 --ip-version 6 --ipv6-ra-mode dhcpv6-stateless --ipv6-address-mode dhcpv6-stateless --subnet-range fda6:dd28:1656::/64

I need a vm in which I could control dhclient, therefore I create an image with root password pre-set and I access it via 'libvirt console' from the compute host after openstack booted it.

openstack image create u1804 --container-format bare --disk-format qcow2 --public --file ~/u1804-with-root-password.img
openstack server create vm0 --flavor ds512M --image u1804 --nic net-id=net1 --wait

# on the compute host
source ~/src/os/openstack/devstack/openrc admin admin
sudo virsh console $( openstack server show vm0 -f value -c OS-EXT-SRV-ATTR:instance_name )
# login with pre-set root password
# trigger v6 dhclient as needed
dhclient -v -6 ens2

The client cannot acquire an address, but it seems we have multiple issues:

root@vm0:~# dhclient -v -6 ens2
Internet Systems Consortium DHCP Client 4.3.5
Copyright 2004-2016 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on Socket/ens2
Sending on Socket/ens2
PRC: Soliciting for leases (INIT).
XMT: Forming Solicit, 0 ms elapsed.
XMT: X-- IA_NA 3e:e4:10:e4
XMT: | X-- Request renew in +3600
XMT: | X-- Request rebind in +5400
XMT: Solicit on ens2, interval 1010ms.
RCV: Advertise message on ens2 from fe80::f816:3eff:fe4a:e696.
RCV: X-- Server ID: 00:01:00:01:23:cf:43:eb:fa:16:3e:4a:e6:96
PRC: Lease failed to satisfy.

The first issue is that (likely a configuration error due to my insufficient IPv6 knowledge) the dhcpv6 server thinks there are no v6 addresses available to advertise:

# captured while dhclient was running
$ sudo tcpdump -vvv -n -i $( sudo virsh dumpxml $( openstack server show vm0 -f value -c OS-EXT-SRV-ATTR:instance_name ) | egrep tap | cut -d\' -f2 )
tcpdump: listening on tapdead586f-16, link-type EN10MB (Ethernet), capture size 262144 bytes
14:02:39.194410 IP6 (flowlabel 0x449a9, hlim 1, next-header UDP (17) payload length: 64) fe80::f816:3eff:fee4:10e4.546 > ff02::1:2.547: [bad udp cksum 0x44b7 -> 0x2b6b!] dhcp6 solicit (xid=4961ec (client-ID hwaddr/time type 1 time 600786647 fa163ee410e4) (option-request DNS-server DNS-search-list Client-FQDN SNTP-servers) (elapsed-time 0) (IA_NA IAID:1055133924 T1:3600 T2:5400))
14:02:39.195553 IP6 (class 0xc0, flowlabel 0x27db9, hlim 64, next-header UDP (17) payload length: 76) fe80::f816:3eff:fe4a:e696.547 > fe80::f816:3eff:fee4:10e4.546: [udp sum ok] dhcp6 advertise (xid=4961ec (client-ID hwaddr/time type 1 time 600786647 fa163ee410e4) (server-ID hwaddr/time type 1 time 600785899 fa163e4ae696) (status-code NoAddrsAvail))
14:02:40.325718 IP6 (flowlabel 0x449a9, hlim 1, next-header UDP (17) payload length: 64) fe80::f816:3eff:fee4:10e4.546 > ff02::1:2.547: [bad udp cksum 0x44b7 -> 0x2afb!] dhcp6 solicit (xid=4961ec (client-ID hwaddr/time type 1 time 600786647 fa163ee410e4) (option-request DNS-server DNS-search-list Client-FQDN SNTP-servers) (elapsed-time 112) (IA_NA IAID:1055133924 T1:3600 T2:5400))
14:02:40.326058 IP6 (class 0xc0, flowlabel 0x27db9, hlim 64, next-header UDP (17) payload length: 76) fe80::f816:3eff:fe4a:e696.547 > fe80::f816:3eff:fee4:10e4.546: [bad udp cksum 0x6036 -> 0xef08!] dhcp6 advertise (xid=4961ec (client-ID hwaddr/time type 1 time 600786647 fa163ee410e4) (server-ID hwaddr/time type 1 time 600785899 fa163e4ae696) (status-code NoAddrsAvail))
14:02:42.438350 IP6 (flowlabel 0x449a9, hlim 1, next-header UDP (17) payload length: 64) fe80::f816:3eff:fee4:10e4.546 > ff02::1:2.547: [bad udp cksum 0x44b7 -> 0x2a27!] dhcp6 solicit (xid=4961ec (client-ID hwaddr/time type 1 time 600786647 fa163ee410e4) (option-request DNS-server DNS-search-list Client-FQDN SNTP-servers) (elapsed-time 324) (IA_NA IAID:1055133924 T1:3600 T2:5400))
14:02:42.438575 IP6 (class 0xc0, flowlabel 0x27db9, hlim 64, next-header UDP (17) payload length: 76) fe80::f816:3eff:fe4a:e696.547 > fe80::f816:3eff:fee4:10e4.546: [bad udp cksum 0x6036 -> 0xef08!] dhcp6 advertise (xid=4961ec (client-ID hwaddr/time type 1 time 600786647 fa163ee410e4) (server-ID hwaddr/time type 1 time 600785899 fa163e4ae696) (status-code NoAddrsAvail))

# also observe the same from dnsmasq logs
$ sudo journalctl -f | egrep dnsmasq
jan 14 14:04:04 devstack0 dnsmasq-dhcp[7053]: DHCPSOLICIT(tapbce96e36-f6) 00:01:00:01:23:cf:46:d7:fa:16:3e:e4:10:e4
jan 14 14:04:04 devstack0 dnsmasq-dhcp[7053]: DHCPADVERTISE(tapbce96e36-f6) 00:01:00:01:23:cf:46:d7:fa:16:3e:e4:10:e4 no addresses available
jan 14 14:04:05 devstack0 dnsmasq-dhcp[7053]: DHCPSOLICIT(tapbce96e36-f6) 00:01:00:01:23:cf:46:d7:fa:16:3e:e4:10:e4
jan 14 14:04:05 devstack0 dnsmasq-dhcp[7053]: DHCPADVERTISE(tapbce96e36-f6) 00:01:00:01:23:cf:46:d7:fa:16:3e:e4:10:e4 no addresses available

While something is still clearly misconfigured here (as proved by "no addresses available") please observe that most dhcp pakcets have incorrect udp checksums. Both from client to server and from server to client. On the other hand it is very interesting that the first advertise packet has the correct checksum. I don't have an explanation at the moment why.

# side note: the fix of bug #1244589 (the old v4 variant) is present
$ sudo ip netns exec qdhcp-$( openstack network show net1 -f value -c id ) iptables -t mangle -nvL | egrep -i checksum
    8 2916 CHECKSUM udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:68 CHECKSUM fill
# but of course there's no rule installed for v6
$ sudo ip netns exec qdhcp-$( openstack network show net1 -f value -c id ) ip6tables -t mangle -nvL | egrep -i checksum
[nothing]