instance cannot get ip automatically under FlatDHCP mode

Bug #1072014 reported by stanzgy
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Invalid
Undecided
Unassigned

Bug Description

I installed the folsom-2012.2 nova/glance/keystone in a singe node and all things seemed right
except for that the instance can't get their private fixed IP via DHCP protocol under FlatDHCP network mode.

After I used vnc to get access to vm instance and use command `ip addr add %FIXED-IP%/%NETMASK% dev eth0` to configure
the ip address for instance manually, the network of the instance became to work.
(the image of the instances is debian-6.0.4-amd64-standard)

Then I tried to use tcpdump to capture the udp packets on br100(which nova-network is using), found that the dnsmasq(which acted as dhcp server)
didn't respond to the DHCPDISCOVER requests send by dhcp client inside the instance.

I also found similar problem reported by others, but in this case my server's kernel version is 3.2

DHCP broken for Openstack Nova instances since kernel v3.3
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1035172

Bellow are debug infos, can anyone give some help? thx

kernel version

    hzzhanggy % uname -a
    Linux DEV6 3.2.0-3-amd64 #1 SMP Mon Jul 23 02:45:17 UTC 2012 x86_64 GNU/Linux

dnsmasq version

    hzzhanggy % sudo aptitude show dnsmasq
    Package: dnsmasq
    State: installed
    Automatically installed: no
    Version: 2.63-4
    Priority: optional
    Section: net
    Maintainer: Simon Kelley <email address hidden>
    Architecture: all
    Uncompressed Size: 39.9 k
    Depends: netbase, dnsmasq-base (>= 2.63-4)
    Suggests: resolvconf
    Conflicts: resolvconf (< 1.15)
    Description: Small caching DNS proxy and DHCP/TFTP server
     Dnsmasq is a lightweight, easy to configure, DNS forwarder and DHCP server. It is designed to provide DNS and optionally, DHCP, to a small network. It can serve the names of local machines which are
     not in the global DNS. The DHCP server integrates with the DNS server and allows machines with DHCP-allocated addresses to appear in the DNS with names configured either in each host or in a central
     configuration file. Dnsmasq supports static and dynamic DHCP leases and BOOTP/TFTP for network booting of diskless machines.

network interface

    hzzhanggy % ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
        inet 169.254.169.254/32 scope link lo
        inet6 ::1/128 scope host
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
        link/ether 5c:f3:fc:98:97:d8 brd ff:ff:ff:ff:ff:ff
        inet xxx.xxx.xxx.6/24 brd xxx.xxx.xxx.255 scope global eth0
        inet6 fe80::5ef3:fcff:fe98:97d8/64 scope link
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br100 state UP qlen 1000
        link/ether 5c:f3:fc:98:97:da brd ff:ff:ff:ff:ff:ff
        inet6 fe80::5ef3:fcff:fe98:97da/64 scope link
           valid_lft forever preferred_lft forever
    4: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
        link/ether 5e:f3:fc:9c:97:db brd ff:ff:ff:ff:ff:ff
    78: br101: <BROADCAST,MULTICAST,PROMISC> mtu 1500 qdisc noqueue state DOWN
        link/ether 2a:bc:e5:2f:4b:4c brd ff:ff:ff:ff:ff:ff
        inet 10.120.33.1/25 brd 10.120.33.127 scope global br101
    81: br100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
        link/ether 5c:f3:fc:98:97:da brd ff:ff:ff:ff:ff:ff
        inet 10.120.33.1/25 brd 10.120.33.127 scope global br100
        inet6 fe80::5ef3:fcff:fe98:97da/64 scope link
           valid_lft forever preferred_lft forever
    102: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br100 state UNKNOWN qlen 500
        link/ether fe:16:3e:06:c4:71 brd ff:ff:ff:ff:ff:ff
        inet6 fe80::fc16:3eff:fe06:c471/64 scope link
           valid_lft forever preferred_lft forever
    103: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br100 state UNKNOWN qlen 500
        link/ether fe:16:3e:08:1d:0d brd ff:ff:ff:ff:ff:ff
        inet6 fe80::fc16:3eff:fe08:1d0d/64 scope link
           valid_lft forever preferred_lft forever

dnsmasq process

    hzzhanggy % ps aux|grep dnsmasq
    nobody 24867 0.0 0.0 21360 952 ? SN 17:49 0:00 /usr/sbin/dnsmasq --strict-order --bind-interfaces --conf-file= --domain=novalocal --pid-file=/home/hzzhanggy/lib/nova/networks/nova-br100.pid --listen-address=10.120.33.1 --except-interface=lo --dhcp-range=set:'private',10.120.33.2,static,120s --dhcp-lease-max=128 --dhcp-hostsfile=/home/hzzhanggy/lib/nova/networks/nova-br100.conf --dhcp-script=/usr/local/bin/nova-dhcpbridge --leasefile-ro
    root 24868 0.0 0.0 21332 420 ? SN 17:49 0:00 /usr/sbin/dnsmasq --strict-order --bind-interfaces --conf-file= --domain=novalocal --pid-file=/home/hzzhanggy/lib/nova/networks/nova-br100.pid --listen-address=10.120.33.1 --except-interface=lo --dhcp-range=set:'private',10.120.33.2,static,120s --dhcp-lease-max=128 --dhcp-hostsfile=/home/hzzhanggy/lib/nova/networks/nova-br100.conf --dhcp-script=/usr/local/bin/nova-dhcpbridge --leasefile-ro

the dnamasq listening port

    hzzhanggy % sudo netstat -anpu|grep dns
    udp 0 0 10.120.33.1:53 0.0.0.0:* 24867/dnsmasq
    udp 0 0 0.0.0.0:67 0.0.0.0:* 24867/dnsmasq

the content of dhcp-hostsfile

    hzzhanggy % cat ~/lib/nova/networks/nova-br100.conf
    fa:16:3e:08:1d:0d,server-001.novalocal,10.120.33.44
    fa:16:3e:06:c4:71,server-002.novalocal,10.120.33.45

run dhclient in instance with mac "fa:16:3e:08:1d:0d", while running tcpdump to capture udp packets on br100 on openstack server,
just DHCPDISCOVER packets captured.

    root # dhclient eth0

    hzzhanggy % sudo tcpdump -v -i br100 udp

    22:01:03.347651 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
        0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:08:1d:0d (oui Unknown), length 300, xid 0xd5a2dd48, Flags [none]
              Client-Ethernet-Address fa:16:3e:08:1d:0d (oui Unknown)
              Vendor-rfc1048 Extensions
                Magic Cookie 0x63825363
                DHCP-Message Option 53, length 1: Discover
                Parameter-Request Option 55, length 13:
                  Subnet-Mask, BR, Time-Zone, Default-Gateway
                  Domain-Name, Domain-Name-Server, Option 119, Hostname
                  Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
                  NTP
    22:01:05.351754 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
        0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:08:1d:0d (oui Unknown), length 300, xid 0xa59d1867, secs 8, Flags [none]
              Client-Ethernet-Address fa:16:3e:08:1d:0d (oui Unknown)
              Vendor-rfc1048 Extensions
                Magic Cookie 0x63825363
                DHCP-Message Option 53, length 1: Discover
                Parameter-Request Option 55, length 13:
                  Subnet-Mask, BR, Time-Zone, Default-Gateway
                  Domain-Name, Domain-Name-Server, Option 119, Hostname
                  Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
                  NTP
    22:01:05.443905 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
        0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:08:1d:0d (oui Unknown), length 300, xid 0x56275753, Flags [none]
              Client-Ethernet-Address fa:16:3e:08:1d:0d (oui Unknown)
              Vendor-rfc1048 Extensions
                Magic Cookie 0x63825363
                DHCP-Message Option 53, length 1: Discover
                Parameter-Request Option 55, length 13:
                  Subnet-Mask, BR, Time-Zone, Default-Gateway
                  Domain-Name, Domain-Name-Server, Option 119, Hostname
                  Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
                  NTP
    22:01:06.346870 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
        0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:08:1d:0d (oui Unknown), length 300, xid 0xd5a2dd48, secs 3, Flags [none]
              Client-Ethernet-Address fa:16:3e:08:1d:0d (oui Unknown)
              Vendor-rfc1048 Extensions
                Magic Cookie 0x63825363
                DHCP-Message Option 53, length 1: Discover
                Parameter-Request Option 55, length 13:
                  Subnet-Mask, BR, Time-Zone, Default-Gateway
                  Domain-Name, Domain-Name-Server, Option 119, Hostname
                  Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
                  NTP

trace iptables, the dhcp request matched an ACCEPT rule

    hzzhanggy % dmesg|grep "fa:16:3e:08:1d:0d"|tail -n 10

    [1481865.887860] TRACE: filter:nova-network-local:return:1 IN=br100 OUT=br100 PHYSIN=vnet0 PHYSOUT=eth1 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308
    [1481865.887874] TRACE: filter:nova-filter-top:rule:2 IN=br100 OUT=br100 PHYSIN=vnet0 PHYSOUT=eth1 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308
    [1481865.887888] TRACE: filter:nova-compute-local:return:3 IN=br100 OUT=br100 PHYSIN=vnet0 PHYSOUT=eth1 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308
    [1481865.887903] TRACE: filter:nova-filter-top:rule:3 IN=br100 OUT=br100 PHYSIN=vnet0 PHYSOUT=eth1 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308
    [1481865.887916] TRACE: filter:nova-api-local:return:1 IN=br100 OUT=br100 PHYSIN=vnet0 PHYSOUT=eth1 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308
    [1481865.887931] TRACE: filter:nova-filter-top:return:4 IN=br100 OUT=br100 PHYSIN=vnet0 PHYSOUT=eth1 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308
    [1481865.887943] TRACE: filter:FORWARD:rule:2 IN=br100 OUT=br100 PHYSIN=vnet0 PHYSOUT=eth1 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308
    [1481865.887958] TRACE: filter:nova-network-FORWARD:rule:1 IN=br100 OUT=br100 PHYSIN=vnet0 PHYSOUT=eth1 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308
    [1481865.887980] TRACE: filter:INPUT:rule:1 IN=br100 OUT= PHYSIN=vnet0 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308
    [1481865.887995] TRACE: filter:nova-network-INPUT:rule:1 IN=br100 OUT= PHYSIN=vnet0 MAC=ff:ff:ff:ff:ff:ff:fa:16:3e:08:1d:0d:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=328 TOS=0x10 PREC=0x00 TTL=128 ID=0 PROTO=UDP SPT=68 DPT=67 LEN=308

    hzzhanggy % sudo iptables-save|grep nova-network-INPUT

    :nova-network-INPUT - [0:0]
    -A INPUT -j nova-network-INPUT
    -A nova-network-INPUT -i br100 -p udp -m udp --dport 67 -j ACCEPT
    -A nova-network-INPUT -i br100 -p tcp -m tcp --dport 67 -j ACCEPT
    -A nova-network-INPUT -i br100 -p udp -m udp --dport 53 -j ACCEPT
    -A nova-network-INPUT -i br100 -p tcp -m tcp --dport 53 -j ACCEPT

strace dnsmasq, found that the dnsmasq wait at the "select" operation, seemded not data reveiced

    hzzhanggy % sudo strace -p 24867

    time(NULL) = 1351256984
    fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
    fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
    recvmsg(4, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000400}, msg_iov(1)=[{"\200\0\0\0\31\0\0\0\0\0\0\0\0\0\0\0\n\200\0\0\377\0\0\1\0\2\0\0\10\0\17\0"..., 384}], msg_controllen=0, msg_flags=0}, MSG_PEEK|MSG_TRUNC) = 128
    recvmsg(4, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000400}, msg_iov(1)=[{"\200\0\0\0\31\0\0\0\0\0\0\0\0\0\0\0\n\200\0\0\377\0\0\1\0\2\0\0\10\0\17\0"..., 384}], msg_controllen=0, msg_flags=0}, 0) = 128
    fcntl(4, F_SETFL, O_RDWR) = 0
    select(8, [3 4 5 6 7], [], [], NULL

lsof result of dnsmasq

    hzzhanggy % sudo lsof -p 24867

    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    dnsmasq 24867 nobody cwd DIR 8,1 4096 2 /
    dnsmasq 24867 nobody rtd DIR 8,1 4096 2 /
    dnsmasq 24867 nobody txt REG 8,1 253592 81988 /usr/sbin/dnsmasq
    dnsmasq 24867 nobody mem REG 8,1 14768 123450 /lib/x86_64-linux-gnu/libdl-2.13.so
    dnsmasq 24867 nobody mem REG 8,1 31744 123456 /lib/x86_64-linux-gnu/librt-2.13.so
    dnsmasq 24867 nobody mem REG 8,1 131107 123454 /lib/x86_64-linux-gnu/libpthread-2.13.so
    dnsmasq 24867 nobody mem REG 8,1 1583120 123435 /lib/x86_64-linux-gnu/libc-2.13.so
    dnsmasq 24867 nobody mem REG 8,1 211376 123191 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.8
    dnsmasq 24867 nobody mem REG 8,1 24712 82068 /usr/lib/libnfnetlink.so.0.2.0
    dnsmasq 24867 nobody mem REG 8,1 96520 123284 /usr/lib/x86_64-linux-gnu/libnetfilter_conntrack.so.3.3.0
    dnsmasq 24867 nobody mem REG 8,1 286488 123894 /lib/x86_64-linux-gnu/libdbus-1.so.3.7.2
    dnsmasq 24867 nobody mem REG 8,1 136936 123458 /lib/x86_64-linux-gnu/ld-2.13.so
    dnsmasq 24867 nobody mem REG 8,6 217016 851971 /var/cache/nscd/group
    dnsmasq 24867 nobody mem REG 8,6 217016 851970 /var/cache/nscd/passwd
    dnsmasq 24867 nobody mem REG 8,1 1534672 92791 /usr/lib/locale/locale-archive
    dnsmasq 24867 nobody mem REG 8,1 26066 123643 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
    dnsmasq 24867 nobody 0u CHR 1,3 0t0 1028 /dev/null
    dnsmasq 24867 nobody 1u CHR 1,3 0t0 1028 /dev/null
    dnsmasq 24867 nobody 2u CHR 1,3 0t0 1028 /dev/null
    dnsmasq 24867 nobody 3u IPv4 387210551 0t0 UDP *:bootps
    dnsmasq 24867 nobody 4u netlink 0t0 387210552 ROUTE
    dnsmasq 24867 nobody 5u IPv4 387210563 0t0 UDP 10.120.33.1:domain
    dnsmasq 24867 nobody 6u IPv4 387210564 0t0 TCP 10.120.33.1:domain (LISTEN)
    dnsmasq 24867 nobody 7r FIFO 0,8 0t0 387208289 pipe
    dnsmasq 24867 nobody 8w FIFO 0,8 0t0 387208289 pipe
    dnsmasq 24867 nobody 9u unix 0xffff880665f44c40 0t0 387210157 socket
    dnsmasq 24867 nobody 12w FIFO 0,8 0t0 387210158 pipe

but when I kill the dnsmasq process and use nc to listen on udp 67 port, it definately could received something.

    hzzhanggy % _ nc -l -u -p 67|xxd
    0000000: 0101 0600 c12f 2d0f 0014 0000 0000 0000 ...../-.........
    0000010: 0000 0000 0000 0000 0000 0000 fa16 3e75 ..............>u
    0000020: 38c3 0000 0000 0000 0000 0000 0000 0000 8...............
    0000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0000060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    00000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    00000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    00000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    00000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
    00000e0: 0000 0000 0000 0000 0000 0000 6382 5363 ............c.Sc
    00000f0: 3501 0137 0d01 1c02 030f 0677 0c2c 2f1a 5..7.......w.,/.
    0000100: 792a ff00 0000 0000 0000 0000 0000 0000 y*..............
    0000110: 0000 0000 0000 0000 0000 0000 0000 0000 ................

Revision history for this message
stanzgy (stanzgy) wrote :
Revision history for this message
Dan Smith (danms) wrote :

Any chance your guest images are configured to change the mac address of the primary ethernet adapter via udev or similar? What image are you trying and where did you get it? Does CirrOS work?

Changed in nova:
status: New → Triaged
Revision history for this message
stanzgy (stanzgy) wrote :
Download full text (4.4 KiB)

hi, i'm using debian-6.0.4-amd64-stand w/o any modifications as my guest's image.
there is no chance that guest vm will change its mac address.

it's has been a long time since i reported this bug.
the system's environment on this server has changed significantly and couldn't reproduce the bug on it.

but i succeeded to reproduce this bug on my personal server w/ both debian-6.0.4 and cirros guest images.

stanzgy % uname -a
Linux stanzgy-netease 3.6.8-1-ARCH #1 SMP PREEMPT Mon Nov 26 22:10:40 CET 2012 x86_64 GNU/Linux

stanzgy % qemu-kvm --version
QEMU emulator version 1.2.0 (qemu-kvm-1.2.0), Copyright (c) 2003-2008 Fabrice Bellard

stanzgy % libvirtd --version
libvirtd (libvirt) 1.0.0

stanzgy % dnsmasq --version
Dnsmasq version 2.63rc6 Copyright (c) 2000-2012 Simon Kelley
Compile time options: IPv6 GNU-getopt DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack

(in guest vm) root # dhclient -v -4 -1 eth0
Internet Systems Consortium DHCP Client 4.1.1-P1
Copyright 2004-2010 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/eth0/fa:16:3e:5a:cf:47
Sending on LPF/eth0/fa:16:3e:5a:cf:47
Sending on Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 9
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 9
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 20
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 15
5 bad udp checksums in 5 packets
No DHCPOFFERS received.
Unable to obtain a lease on first try. Exiting.

stanzgy % sudo tcpdump -ni br100 udp port 67 or udp port 68
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br100, link-type EN10MB (Ethernet), capture size 65535 bytes
17:22:53.501451 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:5a:cf:47, length 300
17:22:53.501633 IP 10.0.0.1.67 > 10.0.1.13.68: BOOTP/DHCP, Reply, length 309
17:23:01.507726 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:5a:cf:47, length 300
17:23:01.507874 IP 10.0.0.1.67 > 10.0.1.13.68: BOOTP/DHCP, Reply, length 309
17:23:10.508736 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:5a:cf:47, length 300
17:23:10.508909 IP 10.0.0.1.67 > 10.0.1.13.68: BOOTP/DHCP, Reply, length 309
17:23:19.508799 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:5a:cf:47, length 300
17:23:19.508963 IP 10.0.0.1.67 > 10.0.1.13.68: BOOTP/DHCP, Reply, length 309
17:23:39.519797 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:5a:cf:47, length 300
17:23:39.519947 IP 10.0.0.1.67 > 10.0.1.13.68: BOOTP/DHCP, Reply, length 309

10 packets captured
10 packets received by filter
0 packets dropped by kernel

it looked like sth got broken with udp checksum. after a search i found explanation about this bug here:
https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Host_Configuration_and_Guest_Installation_Guide/ch11s02.html

the temporarily workaround is to manually add an iptable rule
stanzgy % sudo iptables -t mangle -A POSTROUT...

Read more...

Revision history for this message
stanzgy (stanzgy) wrote :

i don't know whether the reason that dhcp didn't work on these two server is same, but currently it's sure that the nova-network dhcp service won't work under the situation described above.

Revision history for this message
Tracy Jones (tjones-i) wrote :

This has not been touched in over 2 years. Im marking it incomplete for now. If someone wants to repo on juno and it is still an issue please attach logs and reopen.

Changed in nova:
status: Triaged → Incomplete
Sean Dague (sdague)
Changed in nova:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.