Hyper-V NIC cannot pass IPv6 UDP packets by default until protocol offload is disabled

[Frédéric Druilhet]

Hi,

On a client server, I can't get a response from a name server in ipv6+udp. On the same server, it works fine in tcp.
If I log on my name server, I can get a response from himself but, as a client, it can't get a response from another name server.

here are the steps to reproduce :
- Create a new server using Ubuntu server 16.04. Set it to use ipv4 and use your preferred name server. Select absolutely nothing as functionality.
- check some nslookup and apt-get stuff to verify network is OK
- add ipv6 to the interface and refer to an IPV6 nameserver. I have 3 and thats why I noticed the problem >> All 3 ipv6 nameservers are filling the 3 places in resolv.conf.
- if you don't have 3 ipv6 nameservers, just add one and comment the line with the ipv4 name server.
- verify your resolv.conf. Should only content ipv6 reference.
- Now, you can't nslookup anymore...
- add some tests : "dig @ipv4server google.com" works but "dig @ipv6server google.com" don't

Here a more tests :

Nameserver1 : Nameserver1.MyDomain : XXX.YYY.ZZZ.250 / xxxx:yyyy:zzzz::250
Client StandAloneServer.MyDomain : XXX.YYY.ZZZ.209 / xxxx:yyyy:zzzz::209

With IPV4

# dig @XXX.YYY.ZZZ.226 www.google.fr

/var/log/named/queries.log on Nameserver1:
20-May-2016 12:25:02.834 queries: info: client XXX.YYY.ZZZ.209#35116 (www.google.fr): query: www.google.fr IN A +E (XXX.YYY.ZZZ.226)

Tcp Dump on Nameserver1:
12:25:02.834092 IP StandAloneServer.MyDomain.35116 > Nameserver1.MyDomain.domain: 62949+ [1au] A? www.google.fr. (42)
12:25:02.834395 IP Nameserver1.MyDomain.domain > StandAloneServer.MyDomain.35116: 62949 1/4/5 A 216.58.210.195 (204)

Tcp Dump on Client
12:25:02.668200 IP StandAloneServer.MyDomain.35116 > Nameserver1.MyDomain.domain: 62949+ [1au] A? www.google.fr. (42)
12:25:02.669573 IP Nameserver1.MyDomain.domain > StandAloneServer.MyDomain.35116: 62949 1/4/5 A 216.58.210.195 (204)

RESULT:

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @XXX.YYY.ZZZ.226 www.google.fr
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49591
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 5
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.google.fr. IN A
;; ANSWER SECTION:
www.google.fr. 300 IN A 216.58.210.195
;; AUTHORITY SECTION:
google.fr. 171625 IN NS ns1.google.com.
[TRUNCATED]

WITH IPV6
# dig @xxxx:yyyy:zzzz::226 www.google.fr

NOTE : I only asked once, it created 3 queries with 5 secs d’intervalle environ

/var/log/named/queries.log on Nameserver1:
20-May-2016 12:32:34.902 queries: info: client xxxx:yyyy:zzzz::209#35362 (www.google.fr): query: www.google.fr IN A +E (xxxx:yyyy:zzzz::226)
20-May-2016 12:32:39.902 queries: info: client xxxx:yyyy:zzzz::209#35362 (www.google.fr): query: www.google.fr IN A +E (xxxx:yyyy:zzzz::226)
20-May-2016 12:32:44.902 queries: info: client xxxx:yyyy:zzzz::209#35362 (www.google.fr): query: www.google.fr IN A +E (xxxx:yyyy:zzzz::226)

Tcp Dump on Nameserver1:
12:32:34.902598 IP6 StandAloneServer.MyDomain.35362 > Nameserver1.MyDomain.domain: 23393+ [1au] A? www.google.fr. (42)
12:32:34.902994 IP6 Nameserver1.MyDomain.domain > StandAloneServer.MyDomain.35362: 23393 1/4/5 A 216.58.210.195 (204)
…
12:32:39.902644 IP6 StandAloneServer.MyDomain.35362 > Nameserver1.MyDomain.domain: 23393+ [1au] A? www.google.fr. (42)
12:32:39.902975 IP6 Nameserver1.MyDomain.domain > StandAloneServer.MyDomain.35362: 23393 1/4/5 A 216.58.210.195 (204)
…
12:32:39.902644 IP6 StandAloneServer.MyDomain.35362 > Nameserver1.MyDomain.domain: 23393+ [1au] A? www.google.fr. (42)
12:32:39.902975 IP6 Nameserver1.MyDomain.domain > StandAloneServer.MyDomain.35362: 23393 1/4/5 A 216.58.210.195 (204)

Tcp Dump on Client12:32:34.725312 IP6 StandAloneServer.MyDomain.35362 > Nameserver1.MyDomain.domain: 23393+ [1au] A? www.google.fr. (42)
12:32:34.726036 IP6 Nameserver1.MyDomain.domain > StandAloneServer.MyDomain.35362: 23393 1/4/5 A 216.58.210.195 (204)
…
12:32:39.725199 IP6 StandAloneServer.MyDomain.35362 > Nameserver1.MyDomain.domain: 23393+ [1au] A? www.google.fr. (42)
12:32:39.726045 IP6 Nameserver1.MyDomain.domain > StandAloneServer.MyDomain.35362: 23393 1/4/5 A 216.58.210.195 (204)
…
12:32:44.725288 IP6 StandAloneServer.MyDomain.35362 > Nameserver1.MyDomain.domain: 23393+ [1au] A? www.google.fr. (42)
12:32:44.725944 IP6 Nameserver1.MyDomain.domain > StandAloneServer.MyDomain.35362: 23393 1/4/5 A 216.58.210.195 (204)

RESULT

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @xxxx:yyyy:zzzz::226 www.google.fr
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

but, if I do the same query with the tcp flag, all is OK :

# dig +tcp @xxxx:yyyy:zzzz::226 www.google.fr

; <<>> DiG 9.10.3-P4-Ubuntu <<>> +tcp xxxx:yyyy:zzzz::226 www.google.fr
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41105
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 5

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.google.fr. IN A

;; ANSWER SECTION:
www.google.fr. 300 IN A 216.58.210.195

[Frédéric Druilhet (frederic-druilhet)]

One colleague, linux expert, pointed me to this : #1527902

he did some strace and find :
60735 gettimeofday({1463755505, 995830}, NULL) = 0
60735 sendmsg(20, {msg_name(28)={sa_family=AF_INET6, sin6_port=htons(53), inet_pton(AF_INET6, "2001:660:660c:120::226", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, msg_iov(1)=[{"\236\207\1 \
0\1\0\0\0\0\0\1\3www\6google\2fr\0\0\1\0\1\0"..., 42}], msg_controllen=0, msg_flags=0}, 0) = 42
60735 futex(0x7f7ab325d0a4, FUTEX_WAIT_PRIVATE, 3, NULL <unfinished ...>
60737 <... epoll_wait resumed> [{EPOLLIN, {u32=3, u64=3}}], 64, -1) = 1
60737 read(3, "\24\0\0\0\375\377\377\377", 8) = 8
60737 epoll_ctl(5, EPOLL_CTL_ADD, 20, {EPOLLIN, {u32=20, u64=20}}) = 0
60737 read(3, 0x7f7aac706e40, 8) = -1 EAGAIN (Resource temporarily unavailable)
60737 epoll_wait(5, <unfinished ...>
60736 <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)

[Frédéric Druilhet (frederic-druilhet)]

Sorry, i'm tired, here is the link:
https://bugs.launchpad.net/ubuntu/+source/linux-lts-trusty/+bug/1527902

[Frédéric Druilhet (frederic-druilhet)]

Just to add a point about how I noticed the bug. It as one important information.
I have 3 nameservers, all ipv6/4. No particular problem.
Problem occurred when I added a 4th Ubuntu server. This one installs fine as I set ip IPv4 first with the 3 nameservers ipv4 address. But, when I added the ipv6 interface, and all 3 nameservers with their ipv6 address, it failed. I spent a lot of time checking network, Ipv6, reinstalling Ubuntu till I end up notifying that as a real bug.

In fact, adding 3 ipv6 nameservers address changes the resolv.conf and the 3 nameservers stack is filled with ipv6 adresses. So, it leads to a non responding resolution.
I made some tests on my nameservers and noticed that they are in the same setup but, and thats the important discovery, they get a response with ipv6 when then request themselves. Server1 doesn't get a response from server2 et 3, but it does with server1. That's one I didn't noticed the bug because each nameserver has it's own address in the resolv.conf so requests end with responding

To Bypass the bug, each interface has only ipv4 nameserver addresses except on each nameserver where I left it's own ipv6 address in the interface setup

[Frédéric Druilhet (frederic-druilhet)]

The same with ntp:

@client:~# ntpdate aaa.bbb.ccc.188
24 May 12:56:09 ntpdate[26846]: adjust time server aaa.bbb.ccc.188 offset 0.083226 sec

@client:~# ntpdate AAAA:BBBB::188
24 May 12:56:38 ntpdate[27174]: no server suitable for synchronization found

[Robie Basak (racb)]

Thank you for taking the time to report this bug and helping to make Ubuntu better.

I'm sorry, I don't understand your report.

You've reported the bug against bind9, so are you saying that the problem is with bind9 serving as a forwarder? But in that case, I don't understand why you're instructing me to change resolv.conf in your reproduction steps, since bind doesn't use that.

You're also instructing me to use dig @XXX for a failure case, which also doesn't use resolv.conf but uses the nameserver specified directly.

I have just tested and confirmed that dig works correctly with IPv6 on Xenial against one of my ISPs IPv6 resolvers. I tried changing resolv.conf temporarily to point to an IPv6 nameserver only, and it still works:

$ dig www.google.com @2001:8b0::2020

; <<>> DiG 9.10.3-P4-Ubuntu <<>> www.google.com @2001:8b0::2020
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57112
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.google.com. IN A

;; ANSWER SECTION:
www.google.com. 202 IN A 216.58.198.100

;; Query time: 20 msec
;; SERVER: 2001:8b0::2020#53(2001:8b0::2020)
;; WHEN: Wed May 25 14:59:56 BST 2016
;; MSG SIZE rcvd: 59

Are you sure you don't have a simple firewall issue?

Since I cannot understand your report, I regret that this will not make any progress without further information.

Please could you provide a simple set of steps to reproduce that uses a single minimal set of DNS tooling, rather than all the tools? For example, if it is a problem with resolv.conf and nss, then your instructions should involve changing resolv.conf and using "getent hosts" or similar only. Using "dig @..." makes no sense since this bypasses resolv.conf and nss. Concurrent commentary using tcpdump said are appreciated and useful; my point is that the failure should be able reproduce using minimal tooling.

Once done, please change the bug status back to New.

[Frédéric Druilhet (frederic-druilhet)]

Hi, thanks for reading.
You are right, this is not a bind issue. It appears with ntp also.
I spoke about resolv.conf because it masked me the problem: it contains my 3 nameservers
But As I mentioned, when one server tries resolving using itself, if works. That's why I noticed lately the problem when adding a non-nameserver ubuntu.

Of course, I made my best to find any network issues, there a no firewall and all takes place in a HyperV network.

So, I asked a more general and simple question here : https://answers.launchpad.net/ubuntu/+question/294135

When I switch to ipv4 (resolving, ntp) with the same host, all is fine.
TO help reproduce, I'll add 2 more servers, strictly ipv6, one in Ubuntu 16.04 and one in 15.10 and check.

I'll tell later
Fred

[Frédéric Druilhet (frederic-druilhet)]

PS : If you can reaffect this bug report to the right package, thanks for that. I haven't found how to get more generic ...

[Frédéric Druilhet (frederic-druilhet)]

Hi back.
I (really) quickly set up 2 virtual machines on my HyperV cluster. Each With ubuntu OOB iso image, one in 15.10, one in 16.04. No extra setup, only ipv6 adresses.
I attached the result.
I'll post some tcpdump in a few minutes

[Frédéric Druilhet (frederic-druilhet)]

========== TcpDump =========

---- from the client ----
root@14Ext-NS:~# tcpdump host 14Ext-Code.dr14.cnrs.fr -v
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:27:46.837544 IP6 (flowlabel 0x9a4b5, hlim 64, next-header UDP (17) payload length: 50) 14Ext-Code.dr14.cnrs.fr.46138 > 14Ext-NS.dr14.cnrs.fr.domain: [bad udp cksum 0xe623 -> 0x2df7!] 50905+ [1au] A? www.google.fr. (42)
18:27:49.195988 IP6 (flowlabel 0x4b7e2, hlim 64, next-header UDP (17) payload length: 212) 14Ext-NS.dr14.cnrs.fr.domain > 14Ext-Code.dr14.cnrs.fr.46138: [bad udp cksum 0x203e -> 0xa01b!] 50905 1/4/5 www.google.fr. A 216.58.210.227 (204)
18:27:49.196277 IP6 (flowlabel 0x4b7e2, hlim 64, next-header UDP (17) payload length: 212) 14Ext-NS.dr14.cnrs.fr.domain > 14Ext-Code.dr14.cnrs.fr.46138: [bad udp cksum 0x203e -> 0x301c!] 50905 1/4/5 www.google.fr. A 216.58.210.227 (204)
18:27:51.837791 IP6 (flowlabel 0x9a4b5, hlim 64, next-header UDP (17) payload length: 50) 14Ext-Code.dr14.cnrs.fr.46138 > 14Ext-NS.dr14.cnrs.fr.domain: [bad udp cksum 0xe623 -> 0x2df7!] 50905+ [1au] A? www.google.fr. (42)
18:27:51.838131 IP6 (flowlabel 0x4b7e2, hlim 64, next-header UDP (17) payload length: 212) 14Ext-NS.dr14.cnrs.fr.domain > 14Ext-Code.dr14.cnrs.fr.46138: [bad udp cksum 0x203e -> 0x421c!] 50905 1/4/5 www.google.fr. A 216.58.210.227 (204)
18:27:54.208962 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::215:fdff:fefd:fd10 > 14Ext-Code.dr14.cnrs.fr: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 14Ext-Code.dr14.cnrs.fr
   source link-address option (1), length 8 (1): 00:15:fd:fd:fd:10
18:27:54.209228 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 24) 14Ext-Code.dr14.cnrs.fr > fe80::215:fdff:fefd:fd10: [icmp6 sum ok] ICMP6, neighbor advertisement, length 24, tgt is 14Ext-Code.dr14.cnrs.fr, Flags [solicited]
^C
7 packets captured

------ from the nameserver --------
root@14Ext-Code:~# tcpdump host 14Ext-NS.dr14.cnrs.fr -v
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:27:41.607983 IP6 (flowlabel 0x9a4b5, hlim 64, next-header UDP (17) payload length: 50) 14Ext-Code.dr14.cnrs.fr.46138 > 14Ext-NS.dr14.cnrs.fr.domain: [bad udp cksum 0x1f9c -> 0x2df7!] 50905+ [1au] A? www.google.fr. (42)
18:27:46.607895 IP6 (flowlabel 0x9a4b5, hlim 64, next-header UDP (17) payload length: 50) 14Ext-Code.dr14.cnrs.fr.46138 > 14Ext-NS.dr14.cnrs.fr.domain: [bad udp cksum 0x1f9c -> 0x2df7!] 50905+ [1au] A? www.google.fr. (42)
18:27:46.620713 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) fe80::215:5dff:fe01:140e > 14Ext-NS.dr14.cnrs.fr: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 14Ext-NS.dr14.cnrs.fr
   source link-address option (1), length 8 (1): 00:15:5d:01:14:0e
18:27:46.621042 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 24) 14Ext-NS.dr14.cnrs.fr > fe80::215:5dff:fe01:140e: [icmp6 sum ok] ICMP6, neighbor advertisement, length 24, tgt is 14Ext-NS.dr14.cnrs.fr, Flags [solicited]
18:27:48.966683 IP6 (flowlabel 0x4b7e2, hlim 64, next-header UDP (17) payload length: 212) 14Ext-NS.dr14.cnrs.fr.domain > 14Ext-Code.dr14.cnrs.fr.46138: [bad udp cksum 0x5848 -> 0...

[Robie Basak (racb)]

Please provide a single minimal set of steps to reproduce. Requiring two 16.04 VMs connected to each other would be ideal, with one server running bind and one server running dig that queries the server running bind over IPv6. Assuming that the two machines can communicate with each other over IPv6, tell me exactly what commands to type on each machine to install and set up bind and query using dig.

Before you post the instructions, please check that they reproduce the problem in your environment and confirm that you did this in your comment.

Also, it might be worth trying to reproduce without Hyper-V? Bad UDP checksums may be normal (it's a common optimisation) but may also be the problem. If it does affect Hyper-V only it still may be a bug in Ubuntu (for example in the kernel that handles the Hyper-V virtual NICs) but it would be useful to know, because I don't have Hyper-V here so will not be able to reproduce a Hyper-V-specific problem.

[Frédéric Druilhet (frederic-druilhet)]

Hi Robie,
Thanks for your reply. I'll do my best to be as clear as possible

First, I must say that I found how to resolve the problem in this thread: http://ubuntuforums.org/showthread.php?t=1940190
That way, you'll have a better on where to search. Before answering your request, remember that I also have 2 windows 2012R2 servers on the same environment (HyperV, ipv4/v6 subnet, vlan) and they do not need to have rx/tx checksum disabled to query/answer each other or Ubuntu servers. SO problems appear to be un ubuntu network drivers.

So, your request:

Setup: I have an HyperV failover cluster made of 5 2012R2 servers. On each server (Dell with Broadcom cards), 2 network cards are teamed using 2012-R2 functionality for LAN access.

Virtual machines: 5 Ubuntu 16.06 servers. Some upgraded from 13.X setup, some completely new. Each has dual stack ipv4/v6. Among them, 3 are nameservers running latest bin. From previous reply, I added 2 new machines from scratch: Ubuntu 15.10 and 16.04. I also have 3 virtual windows servers 2012R2 on this subnet (2 are domain controllers so also name servers).
All Ubuntu are at the latest version, no more updates proposed/ Here is lshw setup for network
  *-network
       description: Ethernet interface
       physical id: 1
       logical name: eth0
       serial: 00:15:fd:fd:fd:10
       capabilities: ethernet physical
       configuration: broadcast=yes driver=hv_netvsc firmware=N/A ip=xxx.xx.128.235 link=yes multicast=yes

state (intend: with rx/tx enabled on ubuntu machines):
- Any server can dig with any bind server on ipv4 udp
- Any server can dig with any bind server on ipv4 tcp
- NO server can dig with any bind server on ipv6 udp (No server: understand bind or not bind ones)
- Any server can dig with any bind server on ipv6 tcp
- each bind server can query himself on ipv6 udp using ::1 or its real ipv6 address

Tests :
- changing any HyperV network card setup does change nothing (there are network protection, router or DHCP protection). I revert to basic setup
- live migrating the VMs I'm using to a single HyperV server to get rid of any network use was useless
- Disabling rt/tx on HyperV Teamed NIC driver was useless
- Disabling rt/tx on HyperV real network card broadcom driver was useless
- Disabling rt/tx on my virtual Windows servers was useless : they can query/get queried without problem with leads me to think it's really an Ubuntu problem with hv_netvsc driver.
- Disabling "light network protection" in HyperV on the virtual switch is useless)
- No physical machine was on this network

Resolution:
- install ethtool (one machine needed it)
- run :$ ethtool --offload eth0 rx off tx off

Now, I'll do what you ask:
- create 2 VMs (Generation 1) with basically default options. On the same HV server. I didn't add the VM to the failover cluster to prevent migration. One named UbuntuServer, the other Ubuntu Client
- install ubuntu 16.04 server (I default to english/unitedstates (i'm french on others)). No DHCP, network manually configured in IPV6 directly. Server ends with ::2000, client ends with ::1000. On the nameserver question, I've added the server ::2000 ipv6 on both. Of course, no updates. Simp...

[Frédéric Druilhet (frederic-druilhet)]

My previous message without trace wrapping

[Robie Basak (racb)]

Frédéric,

Thank you. This is clear to me now. As it's a driver issue, I'm reassigning to the kernel package. The kernel team will probably need more information about your (virtual) hardware and will want you to test the latest upstream kernel version or similar.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1584042

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Frédéric Druilhet (frederic-druilhet)]

Hi ! I'sorry but I can't apport-collect;
Due to ipv4 adress lack and network isolation, my servers are setup ipv6 only and launchpad.net only resolves in ipv4

[Robie Basak (racb)]

Please at least report your kernel version.

[Frédéric Druilhet (frederic-druilhet)]

root@Client:/home/ssi# uname -a
Linux Client 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

root@Client:/home/ssi# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04 LTS
Release: 16.04
Codename: denial

root@Client:/home/ssi# lsblk -o NAME,SIZE
NAME SIZE
fd0 4K
sda 15G
├─sda1 487M
├─sda2 1K
└─sda5 14.5G
  ├─Client--vg-root 13.5G
  └─Client--vg-swap_1 1G
sr0 1024M

root@Client:/home/ssi# lspci
00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (AGP disabled) (rev 03)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 01)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 02)
00:08.0 VGA compatible controller: Microsoft Corporation Hyper-V virtual VGA

root@Client:/home/ssi# lshw -sanitize
computer
    description: Computer
    width: 64 bits
    capabilities: smbios-2.3 vsyscall32
  *-core
       description: Motherboard
       physical id: 0
     *-memory
          description: System memory
          physical id: 0
          size: 983MiB
     *-cpu
          product: Intel(R) Xeon(R) CPU E5507 @ 2.27GHz
          vendor: Intel Corp.
          physical id: 1
          bus info: cpu@0
          width: 64 bits
          capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx x86-64 constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 popcnt hypervisor lahf_lm
     *-pci
          description: Host bridge
          product: 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (AGP disabled)
          vendor: Intel Corporation
          physical id: 100
          bus info: pci@0000:00:00.0
          version: 03
          width: 32 bits
          clock: 33MHz
        *-isa
             description: ISA bridge
             product: 82371AB/EB/MB PIIX4 ISA
             vendor: Intel Corporation
             physical id: 7
             bus info: pci@0000:00:07.0
             version: 01
             width: 32 bits
             clock: 33MHz
             capabilities: isa bus_master
             configuration: latency=0
        *-ide
             description: IDE interface
             product: 82371AB/EB/MB PIIX4 IDE
             vendor: Intel Corporation
             physical id: 7.1
             bus info: pci@0000:00:07.1
             version: 01
             width: 32 bits
             clock: 33MHz
             capabilities: ide bus_master
             configuration: driver=ata_piix latency=0
             resources: irq:0 ioport:1f0(size=8) ioport:3f6 ioport:170(size=8) ioport:376 ioport:ffa0(size=16)
        *-bridge UNCLAIMED
             description: Bridge
             product: 82371AB/EB/MB PIIX4 ACPI
             vendor: Intel Corporation
             physical id: 7.3
             bus info: pci@0000:00:07.3
             version: 02
             width: 32 bits
             clock: 33MHz
       ...

[Joseph Salisbury (jsalisbury)]

Did this issue start happening after an update/upgrade? Was there a prior kernel version where you were not having this particular problem?

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.6 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.6-yakkety

[Frédéric Druilhet (frederic-druilhet)]

This issue could exist since a long time. Having only 3 servers, each nameserver, did hide the problem (as each could resolve)
In my numerous tests, I made a virtual machine using an old Ubuntu server 15.04 and the problem was present. But,in the meanwhile, my bind server was having troubles (rx/tx activated). So I don't know if its due to 15.04 or the production bind server

I'll try to test on the upstream build as soon as possible. I'll tell you

[Frédéric Druilhet (frederic-druilhet)]

On "client" and "server":
Setting up linux-headers-4.6.0-040600 (4.6.0-040600.201605151930) ...
Setting up linux-headers-4.6.0-040600-generic (4.6.0-040600.201605151930) ...
Setting up linux-image-4.6.0-040600-generic (4.6.0-040600.201605151930) ...

root@Server:/# uname -a
Linux Server 4.6.0-040600-generic #201605151930 SMP Sun May 15 23:32:59 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

root@Client:/# uname -a
Linux Client 4.6.0-040600-generic #201605151930 SMP Sun May 15 23:32:59 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

root@Client:/# ethtool --show-offload eth0
Features for eth0:
rx-checksumming: on
tx-checksumming: on
 tx-checksum-ipv4: off [fixed]
 tx-checksum-ip-generic: on
 tx-checksum-ipv6: off [fixed]
 tx-checksum-fcoe-crc: off [fixed]
 tx-checksum-sctp: off [fixed]

root@Client:/# ethtool --offload eth0 rx off tx off
Actual changes:
rx-checksumming: off
tx-checksumming: off

root@Server:/# ethtool --offload eth0 rx off tx off
Actual changes:
rx-checksumming: off
tx-checksumming: off

oot@Client:/# dig @2001:660:660c:120::2000 testubuntu NS

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @2001:660:660c:120::2000 testubuntu NS
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2664
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;testubuntu. IN NS

;; ANSWER SECTION:
testubuntu. 172800 IN NS server.testubuntu.

--------------------------------------
Bug still present upstream !!
Confirmed
tag: kernel-bug-exists-upstream

[Frédéric Druilhet (frederic-druilhet)]

Sorry, on my previous post, I forgot to copy paste this fail before setting offload off:
Also, I didn't find where to add the tag kernel-bug-exists-upstream ??

root@Client:/# dig @2001:660:660c:120::2000 testubuntu NS

; <<>> DiG 9.10.3-P4-Ubuntu <<>> @2001:660:660c:120::2000 testubuntu NS
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

[Robie Basak (racb)]

It's at the bottom of the bug description.

[Frédéric Druilhet (frederic-druilhet)]

Spotted, sorry :)

[Guillaume Mazoyer (respawneral)]

I can confirm this issue.
I'm currently experiencing it with newly installed Ubuntu Server 16.04.
I also can confirm that this issue was not present in 14.04.

[Max Chen (maxchen)]

I also confirm this issue. All IPv6 udp are blocked (guest vm by Hyper-V)

using frederic-druilhet's comment 11 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1584042/comments/11

    ethtool --offload eth0 rx off tx off

the IPv6 udp works.

[Max Chen (maxchen)]

I've just read this: https://support.microsoft.com/en-hk/kb/2701206 , is it possible that MS should do the fixing? not the side on Linux?

Besides, I install Ubuntu 16.04.1 on OpenVZ and VirtualBox, they are working happily, only the guest VM in Hyper-V has this issue.

[Frédéric Druilhet (frederic-druilhet)]

This is probably a bug in the Linux HyperV Card driver.
Maybe it's MS job to correct this. I don't know who has responsibility on this package.
But, I confirm it is not a bug in the guest hyperV system or windows. I have ipv6 windows servers (multiple versions) and they all work correctly with Ipv6. Even my HyperV Servers are on ipv6 and work fine.
On the other hand, if the bug is on Linux, but more widely that only on Hyperv Card driver, we would have heard about it way faster ;)
At last, changing a settings with ethtool resolves the problem is pointing the investigation starting point I guess.

[Max Chen (maxchen)]

en, even following https://technet.microsoft.com/en-us/windows-server-docs/compute/hyper-v/supported-ubuntu-virtual-machines-on-hyper-v , and install the virtual kernel:

`sudo ethtool eth0` still not able to get the detail info of NIC

Settings for eth0:
 Supported ports: [ ]
 Supported link modes: Not reported
 Supported pause frame use: No
 Supports auto-negotiation: No
 Advertised link modes: Not reported
 Advertised pause frame use: No
 Advertised auto-negotiation: No
 Speed: Unknown!
 Duplex: Unknown! (255)
 Port: Other
 PHYAD: 0
 Transceiver: internal
 Auto-negotiation: off
 Link detected: yes

Besides, 'enp0s3' is shown instead of 'eth0', if the vm is guest of VirtualBox.

[Max Chen (maxchen)]

Just now, I was writing memo of the Ubuntu VM in Hyper-V, and read again https://technet.microsoft.com/en-us/windows-server-docs/compute/hyper-v/supported-ubuntu-virtual-machines-on-hyper-v , this time. I found that "TCP Segmentation and Checksum Offloads" are available on Windows Server operating system version " 2016, 2012 R2, 2012, 2008 R2". They do not mention UDP's.