[regression] all network apps / browsers suffer from multi-second delays by default due to IPv6 DNS lookups

[camper365]

In Karmic, DNS lookups take a very long time with some routers, because glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no (non-loopback) IPv6 interfaces configured. Routers which do not repond to this cause the lookup to take 20 seconds (until the IPv6 query times out).

*** PLEASE DO NOT COMMENT ON THIS BUG unless you have something constructive to say. Everything that can be said has already been said, and if you comment, you are just adding noise. Please let those that actually know what they are doing concentrate on fixing this bug from now on. ***

If disabling IPv6 or using good DNS servers like openDNS fixes the problem, you are not dealing with this bug. Please refrain from complaining here in that case

[mrmx1 (mrmx1-redhat-bugs)]

+++ This bug was initially created as a clone of Bug #459756 +++

Description of problem:

Just installed Fedora 11 from x86_64 ISO image, and I encountered the same problem as with Fedora 10 when I first installed it: "DNS resolver not reliable", which was earlier reported as Bug #459756.

What I see:
yum does not connect to external repositories.
ping does resolve external names, and works OK.
Firefox does resolve Internet names only if "network.dns.disableIPv6" is set to TRUE in about:config
Evolution does not connect to my mailboxes, I believe due to DNS failure.

My Network Configuration Ethernet Device is configured (in the GUI) with "Enable IPv6 configuration for this interface" unchecked.

My uname -a reports:
Linux localhost.localdomain 2.6.29.4-167.fc11.x86_64 #1 SMP Wed May 27 17:27:08 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

My glibc is:
glibc-2.10.1-2

In my case this is reproducible always (firefox, yum), as it was with Fedora 10.
glibc-2.9-3 resolved the problem in Fedora 10 fine for me, with my DNS.
Maybe this is reproducible for me because my old DNS has no IPv6 at all, I guess.

[jonathansteffan (jonathansteffan-redhat-bugs)]

I will also confirm this issue. We are using load balanced instances of bind-9.3.4-6.0.2.P1.el5_2 for our DNS services. Default is to hand out that DNS service as a resolver target. If resolv.conf is manually changed to remove the DNS VIP and put a resolver that does not go across our firewalls, everything works as expected.

[drepper (drepper-redhat-bugs)]

First: why haven't any of you testing the test releases?

Second: where is the data? Logs of the DNS traffic etc.

There is a work-around in place to catch broken installations which should catch this. It will definitely delay the operation but this is your own fault and it's documented in the release notes.

[mrmx1 (mrmx1-redhat-bugs)]

I have to apologize because, after such a long time, I realized that at least in my case it isn't a problem with Fedora - I managed to get an updated firmware for my DSL box (which includes my DNS) from somewhere in Australia and as a kind of magic everything seems to be working fine, now...

Most misleading was that all previous releases of Fedora and Red Hat Linux worked just fine since ever.

[bugsrep (bugsrep-redhat-bugs)]

  I have Fedora 10, Fedora 11 and Windows XP on my computer. I have just installed Fedora 11. In F10 and WinXP domain names are resolved quickly while in F11 it can take up to 10-15 seconds in both Firexox 3.5 beta 4 and Konqueror.

  After finding this bug report I changed network.dns.disableIPv6 from false to true. As soon as I did it domain names begin to resolve immediately. If set back to false, name resolution is very slow again.

  I do not think that this is a real cure, probably just hiding the problem.

[drepper (drepper-redhat-bugs)]

(In reply to comment #4)
> I have Fedora 10, Fedora 11 and Windows XP on my computer. I have just
> installed Fedora 11. In F10 and WinXP domain names are resolved quickly while
> in F11 it can take up to 10-15 seconds in both Firexox 3.5 beta 4 and
> Konqueror.

There was one little additional problem which creates this behavior. Every lookup is slow in the F11 GA release's glibc. I've fixed that since. Now, as it should be, only the fist lookup in a process is slow.

And yes, that will be the final solution. It is not acceptable that people with broken setups prevent progress for all the rest. You can set the appropriate option in resolv.conf and always get fast lookups in your broken environment but the default will be as it is now.

[jonathansteffan (jonathansteffan-redhat-bugs)]

We have identified the network device that is causing issues with the new glibc dns resolution behaviour. We have a Juniper SSG-320M that does stateful inspection of the DNS UDP traffic. The issue is that the new behaviour sends two packets with the same signature. "Signature" as in it has the same source and destination for host/port and this causes only one packet to make it back. In many/most cases, that is the AAAA packet and the A has to be requested multiple times before a response properly comes back. We are going to contact the vendor, but I expect others would run into similar issues.

Is it feasible to have both requests go out at the same time but have different source ports, or is that going to be just as slow as requesting A, then AAAA?

[kernel (kernel-redhat-bugs)]

Regarding comment #6, that's similar to what I reported on the original bug. copied from https://bugzilla.redhat.com/show_bug.cgi?id=459756#c108:

Just a further datapoint on this, since I too spent a few days scratching my
head on it. It looks like what changed in F10 is that both the AAAA and A
requests are sent using the SAME SOURCE PORT, while pre-F10 used different
source ports for the two requests.

For me, that change spelled trouble in the form of a race for my loadbalancer.
I saw this:

1) receive A request, creating session table entry with NAT'd reply IP
2) receive AAAA request on port x, reusing session table entry from #1
3) respond to AAAA request on port x and remove session table entry
4) loadbalancer receives response from DNS server for A request, but since
session table entry (with VIP response IP) is gone, it simply forwards the
traffic, so client receives a reply from a different IP (the IP of the server
itself, NOT the vip) and ignores it

So for me, the simple solution to this is to go back to the old behaviour of
having the A and AAAA requests use unique source ports. Wouldn't that be more
secure anyway? Seems like a step backward to reuse the port.

[hugh (hugh-redhat-bugs-1)]

re comments #6 and #7: are these not bugs in your Juniper SSG-320M and load balancer? Should you not get them fixed?

I discussed this in https://bugzilla.redhat.com/show_bug.cgi?id=459756#c116

Ulrich: here's what I understand your fix to do: if the first AAAA query doesn't get answered, never ask for AAAA records again. But since UDP is unreliable, this seems actually wrong. Wrong for everyone, not just those with broken networks.

To avoid damage from these broken devices, glibc would have to use a distinct port for each query in flight. That would take some book-keeping in glibc, I would think (unless this case is the only way more than one query could be in flight).

[kernel (kernel-redhat-bugs)]

But the question remains, WHY did the behavior change? Originally, glibc DID use unique ports for the AAAA and A queries. From a "predictability" perspective, that is a more secure approach, no? Similar to how ISNs are now randomized in TCP.

It seems many people's problems would be solved by going back to the (arguably more secure) method of using distinct ports for the A and AAAA queries.

[hugh (hugh-redhat-bugs-1)]

I don't know why the behaviour changed.

To randomize the port, glibc would have to ask the OS to allocate a port for each DNS query (and, actually, two in the case we are talking about because it is actually two queries). And free the port afterwards. Port allocation is done by the kernel, on a per-interface basis. So this would multiply the number of system calls (modestly). I don't know enough about how expensive these system calls are (they should be cheap).

The way to secure DNS is through DNSSec. I've been interested in that for a decade. Looks like it might happen soon. I cannot understand why, for all the crap that 911 justified, it didn't spur the deployment DNSSec.

[jonathansteffan (jonathansteffan-redhat-bugs)]

(In reply to comment #7, #8)

re: #8, Sorta. The addition of UDP "sessions" is not always/ever a great idea. Individual vendor implementations might have different methods that work in some/most cases and others that don't work at all. However, stateful packet inspection is something that everyone has to deal with and I would be surprised if our particular firewall vendor is not the only firewall in the world that will run into this issue. As a note, we also have this issue with our Juniper ISGs.

re: #7, We also have an issue using our Foundry ServerIron 450s as load balancers for our DNS traffic. We have "worked around" our issues with a few of our firewalls by adding rules to allow the sessions but have yet to solve how to properly implement a working LB situation for DNS clients that behave the way that glibc is right now.

If it's not that expensive to open up two ports to send the packets out at the same time not waiting for one response or the other, it might be best.

[p.mayers (p.mayers-redhat-bugs)]

For reference, we also see this with our Cisco ACE20s, though Cisco are investigating whether the behaviour is intentional or not. I hold no opinion on the wisdom or not of using the same source port.

I am interested (purely from a curiosity PoV) why glibc doesn't send a single query packet with A and AAAA queries. Presumably there are some broken DNS servers that eat the whole packet?

Ulrich - re #5 which specific Fedora update resolved the slowness to be once per-process? Then I can ask our Unix team to pre-apply it to the build.

FYI, we are having timeouts at login - the number of DNS lookups involved in resolving the LDAP server, kerberos TCP & UDP SRV & A records exceeds 60 seconds!

[drepper (drepper-redhat-bugs)]

A couple of points:

- almost no server handles two or more requests in one package. Very unfortunate
  but true.

- there always was the option to keep the socket open but hardly any program
  took advantage of this (setting the RES_STAYOPEN flag)

- any solution will punish those broken setups. I'll implement a second fallback.
  If the environment really prevents reuse of the same source port then we'll
  automatically use it only if separate requests on the same port fails.

I really hope that people will file this misbehavior of the routers as bugs. It has always been valid of the DNS library.

[p.mayers (p.mayers-redhat-bugs)]

Ah, I suspected servers would choke on A & AAAA in the same packet.

FYI, in this case it's actually a load-balancer. Cisco currently believe the problem is actually timing related - effectively, glibc sends both DNS requests so quickly that the load-balancer is still setting up the session when the 2nd packet arrives, and it's a race-condition of sorts.

They seem receptive to addressing it. I suspect many other NATs and load-balancers might not be completely broken per-se, but racey when two requests come in so "quickly". Obviously this a slightly different issue than broken DNS servers which don't answer at all.

For those who are interested, I'll attach a python/twisted script that reproduces glibc behaviour and can be run on-demand. This is useful when opening bug with SLB/NAT vendors.

[p.mayers (p.mayers-redhat-bugs)]

Created attachment 349114
python/twisted script to send two DNS queries "fast" i.e. back-to-back like glibc 2.10

This script can be used to test the issue. It behaves (I think) similarly to glibc, in that it very quickly sends an A and AAAA query to the given DNS server from the same source port.

[jonathansteffan (jonathansteffan-redhat-bugs)]

We have worked around the issue by adding custom policies on our firewalls. Additionally, we discovered the failure of the subsequent DNS request to be resolved when adding the commands listed below to the Foundry ServerIron450 on the Virtual Server configuration:

Port dns udp-normal-age
Udp-age 2

Basically.. rather then closing the session after the first packet makes it back through, the LB will now consider that session valid for a longer period of time and DNS is working as expected.

[p.mayers (p.mayers-redhat-bugs)]

Cisco have confirmed that this is a timing related bug, fixed in ACE software 1.5. The Cisco bug number is CSCsw52831.

I have also found another interesting behaviour, which I'll document here for reference; we have a 2nd DNS IP that passes through our ACE (but is not handled by the ACE). This was also suffering problems, but my test script was not.

The difference appears to be in the use of connected versus unconnected UDP sockets. Specifically, unconnected UDP sockets seem (under Linux) to always have an IP ID==0, and pass through the ACE fine. Connected UDP sockets seem to have incrementing IP IDs, and seem to get treated in a session-aware manner, and subject to the same timing bug.

glibc seems to use connected sockets, and thus hits the bug.

I hope this info is of interest. If someone knows which version of the F11 glibc RPM contains the "only 1st lookup is slow" fix, that would be useful.

[ayourtch (ayourtch-redhat-bugs)]

Phil, I verified and my C test program with which I reproduced the problem yesterday does send the IP ID of 0 (with the DF bit set), so I'd still maintain it is just timing-related for the ACE. :-)

[amcnabb (amcnabb-redhat-bugs)]

Somebody pointed out that there may be security reasons to use two separate ports. Wouldn't it be best to just always use two separate ports? I don't see any drawback to having separate ports.

[ayourtch (ayourtch-redhat-bugs)]

With my protocol purist hat on, I agree with Ulrich that it is a pure bug in the middleboxes, whatever they are - but the practicalities make this tougher to get in if the same queries are sent in the bangbang manner over the same four-tuple. OTOH, the separate ports suck by doubling the state on the "dumb NAPT" boxes in the middle. But other than that (and the added PITA of handling two sockets) - Ulrich/Jakub, are there any obstacles why this is bad, besides the extra code to deal with 2 sockets on the clientside ?

With the practical hat on, having the queries originate from different port should be very practical. Of course, there are "dumb NAPTs", but there the issue is mostly the CPU spent, while for the stateful boxes those would be some significant changes in the code, if they were not coded with the assumption that there can be parallel outstanding requests over the same 4-tuple.

Of course I'd be biased to have this fixed in the middleboxes, but the trouble is that some customers will be in the different administrative domain than the middleboxes - so it's gonna be very challenging for them.

[drepper (drepper-redhat-bugs)]

Opening two sockets is slow and eats up precious resources (yes, I consider 64k sockets a precious resource). I already said I'll implement this as a fallback solution in case everything else fails and the fallback mode will be automatically reached (after various timeouts). That'll likely happen next week. But it won't be the default. If workarounds to broken behavior is the default the underlying bugs will never be fixed. If people are inconvenienced because of the bugs but it still works they hopefully continue to harass the responsible parties.

The good news is that at least some of the vendors are responsive. Cisco apparently already fixed some of their code. Let's hope this continues to be the case for others as well.

[drepper (drepper-redhat-bugs)]

I've pushed to the upstream repository a change which implements the second fallback mode. It is automatically used (after appropriate timeouts) or it can be requested with the new resolver option single-request-reopen.

Andreas will hopefully build a scratch glibc sometime soon (perhaps for F12, but that version could be used on F11, too). When this happens please test it and don't wait until after F12 is released, as it happened this time once again.

[schwab (schwab-redhat-bugs)]

An F12 build is available here: <http://koji.fedoraproject.org/koji/taskinfo?taskID=1436854>.

[jclere (jclere-redhat-bugs)]

It seems to be the same F10 had after the release:
ping ok.
wget, FF, wget etc not ok.
On F10 it was https://bugzilla.redhat.com/show_bug.cgi?id=459756
The first fix was:
+++
I have fixed the problem by installing glibc-2.9-3 from koji
http://koji.fedoraproject.org/koji/buildinfo?buildID=73861
+++

[arxs (arxs-redhat-bugs)]

*** Bug 504951 has been marked as a duplicate of this bug. ***

[software (software-redhat-bugs)]

Created attachment 353850
wireshark capture of dns failure

I had this problem with F10, and now again with F11.
It prevents me from connecting to yum, firefox, or
ssh, which makes fixing it difficult, although
dig and ping work just fine.

I have captured the network traffic of a failed ssh
session which is attached. The Fedora 11 client sends
out two queries (both A and AAAA) for the same name
at the same time from the same port. The queries have
different transaction ID numbers so they both get a
response, but unfortunately the socket appears to be
closed after the first response, because the second
response elicits an ICMP destination unreachable from
the client. Unfortunately my DNS server is really
fast for the AAAA failures, so all my IPV4 connections
fail. It's a Motorola DSL modem from AT&T.

If the two transactions need to use the same socket,
libc needs to leave the socket open to get both
responses. The impact is huge.

[redhat-bugzilla (redhat-bugzilla-redhat-bugs)]

Frederick, blame the vendor/administrator of your DNS server for this,
please. Try the RPM packages from comment #23 and return some feedback
here. These packages can be used at Fedora 11 (or at least should be).

[schwab (schwab-redhat-bugs)]

There is also an F11 build here: <http://koji.fedoraproject.org/koji/taskinfo?taskID=1475122>.

[arxs (arxs-redhat-bugs)]

*** Bug 509166 has been marked as a duplicate of this bug. ***

[software (software-redhat-bugs)]

Created attachment 354162
wireshark capture of failed DNS session

(In reply to comment #27)

Sorry for the slow response but I tested the new glibc-2.10.1-3
i686/i586. Unfortunately it has not fixed my problem. The ICMP
messages have gone away, but the DNS server is responding in the
same order as the requests which may explain it. A capture
is attached...

  [fdd@jibsheet ~]$ ssh -v fdd.com
  OpenSSH_5.2p1, OpenSSL 0.9.8k-fips 25 Mar 2009
  debug1: Reading configuration data /etc/ssh/ssh_config
  debug1: Applying options for *
  ssh: Could not resolve hostname fdd.com: Temporary failure in name resolution

Strangely, the new glibc also breaks gnome-terminal. The window
comes up with menus but no text in terminal area. There is a
pop-up error of, "There was an error creating the child process for
this terminal." xterm works fine, so it I tried running
gnome-terminal on the command line, but it prints
nothing more informative to stdout. I tried strac'ing
gnome-terminal and it never forks. Going back and forth
between glibc versions shows the breakage is clearly
dependent on the new glibc being installed. I don't need
to reboot, but I do need to kill all gnome-terminals.
Apparently it is bug 509632, and the remount command of
bug 509632 comment 9 fixes it.

[drepper (drepper-redhat-bugs)]

(In reply to comment #30)
> Unfortunately it has not fixed my problem. The ICMP
> messages have gone away, but the DNS server is responding in the
> same order as the requests which may explain it. A capture
> is attached...

That data makes no sense. Or more correctly: it makes no sense that you're having problems.

The log shows that the server correctly answers both requests. I see the same pattern here (as expected) and everything works fine.

Where is the data captured? On the client machine (172.22.2.7 in your case)? Is there a firewall running?

Also, please use getent for the tests:

   getent ahosts fdd.com

That's more predictable, all the code involved comes from glibc.

Furthermore, I suee fdd.com is your domain. It doesn't happen for other domains?

[software (software-redhat-bugs)]

Created attachment 354195
wireshark capture

I agree it's hard to understand. It is not fixed by
"sudo /etc/init.d/iptables stop". The capture was made
on the client machine 172.22.2.7. It happens for all domains,
unless I add them to the /etc/hosts file. I can ping and
dig them just fine. I'm attaching an unfiltered capture
of this session...

  [root@jibsheet ~]# rpm -q glibc
  glibc-2.10.1-3.i686
  [root@jibsheet ~]# iptables-save
  # Generated by iptables-save v1.4.3.1 on Fri Jul 17 13:48:59 2009
  *filter
  :INPUT ACCEPT [534:96543]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [136:21713]
  COMMIT
  # Completed on Fri Jul 17 13:48:59 2009
  [root@jibsheet ~]# ping -c 1 google.com
  PING google.com (74.125.45.100) 56(84) bytes of data.
  64 bytes from yx-in-f100.google.com (74.125.45.100): icmp_seq=1 ttl=54 time=39.4 ms

  --- google.com ping statistics ---
  1 packets transmitted, 1 received, 0% packet loss, time 51ms
  rtt min/avg/max/mdev = 39.430/39.430/39.430/0.000 ms
  [root@jibsheet ~]# ssh -v google.com
  OpenSSH_5.2p1, OpenSSL 0.9.8k-fips 25 Mar 2009
  debug1: Reading configuration data /etc/ssh/ssh_config
  debug1: Applying options for *
  ssh: Could not resolve hostname google.com: Temporary failure in name resolution
  [root@jibsheet ~]# getent ahosts fdd.com
  [root@jibsheet ~]# getent ahosts google.com

The machine is newly installed F11, with updates applied by yum
after adding hostnames to /etc/hosts. I've added vlc
from rpmfusion, and now your glibc, but basically nothing
else.

Trying again with glibc-2.10.1-2 produces the same
results as -3, i.e. no ICMP. I'm sorry to be such trouble.

[software (software-redhat-bugs)]

Created attachment 354226
wireshark of DNS traffic that works

(In reply to comment #32)

The computer works fine elsewhere, so the difference must
be in the DNS AAAA response. The only difference I can
see is that the working response includes a SOA record
that the failing one doesn't. The successful capture
is attached.

[cdahlin (cdahlin-redhat-bugs)]

I'm getting this same issue, and I also have an AT&T Motorola modem. Considering buying a router in hopes that it will act as a DNS relay and protect me from whatever quirk is causing this (I'm having serious internet withdrawal :). Going into the modem interface and asking it to resolve a name directly seems to work.

[Micah Gersten (micahg)]

Thank you for reporting this to Ubuntu. Could you please see if you have the same trouble with other browsers such as epiphany-webkit or midori?

[camper365 (camper365)]

Yes it does apply to other browsers.

[Micah Gersten (micahg)]

This would appear to be more than a Firefox problem since other browsers are involved. I'm removing the Firefox 3.5 package from the bug and asking for reassignment to the appropriate package.

[steven.chapel (steven.chapel-redhat-bugs)]

I had this issue with x86-64 Fedora 11, and I'm having the same issue with x86-64 Fedora 12 alpha. Let me know what I can do to help.

[drepper (drepper-redhat-bugs)]

(In reply to comment #35)
> I had this issue with x86-64 Fedora 11, and I'm having the same issue with
> x86-64 Fedora 12 alpha. Let me know what I can do to help.

Look through the comments. What broken hardware do you use? What are the requests that are sent etc etc.

[steven.chapel (steven.chapel-redhat-bugs)]

I have a HomePortal 1000HW wireless DSL modem from 2wire. The problem occurs whether I connect to the router wirelessly or with an Ethernet cable. My ISP is AT&T and I use their domain name servers.

I don't know how to see the requests that are sent. I'm not an expert at TCP/IP networking. Are the comments saying that this is essentially a problem with the DNS server at my ISP? I'm able to use Firefox if I go to about:config and disable IPv6. Will disabling IPv6 in Fedora work around the problem?

[steven.chapel (steven.chapel-redhat-bugs)]

I added the line:
install ipv6 /bin/true
to the top of the file /etc/modprobe.d/dist.conf to disable IPv6, and the problems have gone away.

[clodoaldo.pinto.neto (clodoaldo.pinto.neto-redhat-bugs)]

I have two machines, one F10 and the other F11, behind the same ADSL router, dlink DSL 500B. F11 has no problems.

The DNS servers are set in resolv.conf to those of opendns.com

In F10 Firefox only works if network.dns.disableIPv6 is set to true. The weather applet can't connect. Yum has to try many mirrors before it finds one that works. folding@home can't connect at all.

Added these lines to modprobe.conf and rebooted:
alias net-pf-10 off
alias ipv6 off

After that Firefox works with network.dns.disableIPv6 set to false. The weather applet connects. Still Yum has try many mirrors and folding@home can't connect.

Is this the same issue reported in this bug?

# cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=d2.localdomain

# cat /etc/networks
default 0.0.0.0
loopback 127.0.0.0
link-local 169.254.0.0

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
# Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit
Ethernet controller
DEVICE=eth0
BOOTPROTO=none
DNS1=208.67.220.220
DNS2=208.67.222.222
DNS3=10.1.1.1
GATEWAY=10.1.1.1
HWADDR=00:21:97:00:79:21
IPADDR=10.1.1.110
NETMASK=255.0.0.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
IPV6INIT=yes
NM_CONTROLLED=yes
PEERDNS=yes

Also tried whith IPV6INIT=no.

[clodoaldo.pinto.neto (clodoaldo.pinto.neto-redhat-bugs)]

I reverted to the default configuration, IPv6 enabled.

While capturing the traffic in wireshark there is no IPv6 traffic reaching the eth0 interface. When I disable IPv6 in Firefox I can see the IPv4 traffic in wireshark. So I guess this has nothing to do with the issue reported in this bug. I will post in the Fedora list and later, if necessary, open another bug.

Steve #38, does Yum work correctly with IPv6 disabled, I mean, it finds a suitable server in the first tries?

[davem (davem-redhat-bugs)]

The bug has nothing to do with whether IPV6 traffic is generated or not.

If you have ipv6 enabled, GLIBC will try AAAA DNS requests, and the
behavior of how this is done in conjunction with normal A record
DNS requests is what causes problems with some equipment.

But these DNS queries happen over ipv4 in your configuration.
So if you want to "see it happen" you need to trace ipv4 traffic,
looking for DNS queries.

[steven.chapel (steven.chapel-redhat-bugs)]

(In reply to comment #40)
>
> Steve #38, does Yum work correctly with IPv6 disabled, I mean, it finds a
> suitable server in the first tries?

With Fedora 12 alpha, I cannot get any software update to work at all until I apply my workaround. After my workaround, I still have some problems (bug #516957) but at least software update works to some extent.

[steven.chapel (steven.chapel-redhat-bugs)]

I was also able to work around the problem by going into IPv4 Settings in the Network Connections Preferences and selecting Automatic (DHCP) addresses only and typing in 208.67.222.222, 208.67.220.220 for the DNS servers to use OpenDNS, then rebooting. I still have the same problems with software update as I did with IPv6 disabled.

[Martin Olsson (mnemo)]

I've been struggling with this bug as well, for me it started with updates I installed on 3rd sept even though I had no problems like this in karmic earlier (at this point I installed updates from about two weeks back though). It affects all network apps (not just browsers). I originally filed a ticket with my ISP because I thought it was their DNS servers that were slow.

[Martin Olsson (mnemo)]

What I was seeing what 20-40 seconds page loads for certain webpages, when I set network.dns.disableIPv6 to true most pages loads with 1-3 seconds.

[Jeroen Massar (massar)]

This is a problem with the DNS resolver.

This problem will occur for any DNS request which the DNS resolver does not support.
The proper solution is to fix the DNS resolver.

What happens:
 - Program is IPv6 enabled.
 - When it looks up a hostname, getaddrinfo() asks first for a AAAA record
 - the DNS resolver sees the request for the AAAA record, goes "uhmmm I dunno what it is, lets throw it away"
 - DNS client (getaddrinfo() in libc) waits for a response..... has to time out as there is no response. (THIS IS THE DELAY)
 - No records received yet, thus getaddrinfo() goes for a the A record request. This works.
 - Program gets the A records and uses those.

This does NOT only affect IPv6 (AAAA) records, it also affects any other DNS record that the resolver does not support.
Generally these resolvers are embedded into the "NAT boxes" that consumers have.

Working solution, as we are on Linux anyway: don't use the DNS resolver in the NAT box, but install eg pdns-recursor and use that.

Of course that does not fix the broken box, which might be the NAT box, or the resolvers at the ISP.
Some other people start using OpenDNS because those "work" (But that is not really true either: https://lists.dns-oarc.net/pipermail/dns-operations/2009-July/004217.html)

Note that the DNS queries go over IPv4 (transport), there is no IPv6 _connectivity_ involved here.

[matt (matt-redhat-bugs-1)]

I have repeated this behavior on Fedora 11. While yum updates are succeeding with some timeouts, the larger problem is that other applications do fail. I've further had no success with the workarounds (other than the explicit fix for Firefox).

I have tried:

a. installation of glibc-2.10.90-22, and using the option single-request-reopen in resolv.conf

b. disabling ipv6

I would appreciate any other workaround ideas, ideally not, it's the fault of other equipment.

[clodoaldo.pinto.neto (clodoaldo.pinto.neto-redhat-bugs)]

@44(In reply to comment #44)

I opened another bug as this one is not the one I'm experiencing:

https://bugzilla.redhat.com/show_bug.cgi?id=520304

[Markus Thielmann (thielmann)]

I'm not convinced, that this is a resolver bug. I'm running an IPv6 enabled system (aiccu tunnel with sixxs.net), so all IPv6 requests are answered by an IPv6 enabled DNS server. I'm still experiencing the same problems. Additional to that, this bug was introduced by Karmic and didn't happen before.

[matzilla (matzilla-redhat-bugs)]

Created attachment 362819
Capture of dns trafic while wget

Capture of dns trafic while wget lwn.net with fedora11+normal updates
provider is orange/france telecom. dns server looks like it's the ip of the livebox (there is several millions of users behind a livebox I think)

wget a little more than 10s
real 0m10.922s
user 0m0.005s
sys 0m0.013s

dns capture gives :
Fedora send two request , one A and one AAAA
answer for A is given at once
after 5s, retry of A (why ? we already have received the result ...)
answer with A immediately
then retry of AAAA, this time after the A answer come

after 10s, there is no dns trafic but wget decides to get the page (it was resolving host before)

after15s and 20s, the dns server timeout the aaaa answer

this looks timing related and dependant on the behaviour of the dns server as the same pc has a different behaviour when connecting to another provider.
as the dns answer immediately with a good ip,why is the linux waiting for so long ?

so my questions :
is the linux correctly getting the first A answer or is it rejecting it (believing it to the be an anwser not matching for the aaaa) ?
what are the timing used ?

[matzilla (matzilla-redhat-bugs)]

I'm using glibc 2.10.1-5
disabling ipv6 in about config firefox also workaround the pb.(didn't try globally as this should be working by default)

[Bernard Bou (bbou)]

The 5 second lag occurs with the Livebox (used by Orange, 12 million broadband internet customers in Europe). Better fix this unless you want a number of users to tweak their config files to either disable ipv6 or disable box-based dns server, not something anybody enjoys doing.

[max123 (maxrest)]

I also suffer from this problem, it _is_ the DNS-resolver, like Jeroen analysed - there should really be a fix for Karmic RS..

[Jeroen Massar (massar)]

@ Markus's #8 comment: as I mentioned "Note that the DNS queries go over IPv4 (transport), there is no IPv6 _connectivity_ involved here.".

You also state 'so all IPv6 requests are answered by an IPv6 enabled DNS server."; well, unless you configured IPv6 DNS resolver addresses in your /etc/resolv.conf then queries will still go over IPv4 (transport), even though they are AAAA queries. AICCU only provides IPv6 connectivity (transport) it does not configure DNS resolvers though.

@ Bernard's #9 comment: most likely your livebox contains one of these broken DNS resolvers. Happens a lot that CPEs have this issue. Try the below to check this out. Configuring resolv.conf with OpenDNS or other working DNS servers (eg the ones of your ISP directly, instead of the livebox) might solve your problem. Do also please realize that this problem ALSO occurs on other platforms than Linux, eg Windows, which is what the majority of people are using; what to use is a choice of the user afterall....

To verify this, do a:
for i in `cat /etc/resolv.conf | grep ^nameserver | cut -f2 -d' '`; do dig @$i www.microsoft.com AAAA; done

This should return quite quickly, even though no AAAA records for www.microsoft.com exist yet. Now, if you have a broken resolver somewhere along the way, these requests won't return quickly (unless they are locally or on-path cached as negative).

[Stephen Hall (stephen-richard-hall)]

I have had the same problem, affecting all network activities. Particularly a problem when performing upgrades via aptitude. Problem completely solved by specifying Opendns as my DNS servers. I do not have this problem when running Jaunty, XP or Vista.

[David Solbach (d-vidsolbach)]

Just updated to karmic and experienced the same problem (1-3 second delays on dns lookup).
Switching off IPv6 dns support in firefox "fixes" the problem.

Do do that open up Mozilla or Firefox and type in 'about:config' in the address bar
Scroll down to "network.dns.disableIPv6", it's defaulted to a value of false, change it to true.

hope that helps.

[camper365 (camper365)]

That's a "fix" in Firefox, but the problem still exists in every other network app (evolution, aptitude, etc.)
So web browsing (which is what most users are doing anyway) is normal speed but everything else is behaving like you have dial-up.
After the final release, if the bug isn't fixed in every app people might start complaining to their isp or even drop ubuntu (or not upgrade to Karmic)

[Pconfig (thomas9999)]

I also notice the same problem in kubuntu. I remember this happened before on my upgrade to intrepid.

[Brian Pitts (bpitts)]

This is still present in the today's build. I don't understand why this isn't prioritized as release critical, since it makes web browsing and other network-related tasks unbearably slow.

[max123 (maxrest)]

Right, I also stress the incredible delay in thunderbird, network apps on the shell, update manager and everything else network related due to ipv6 lookups without having an ipv6 ip again!

At the university, where I get ipv6 as well, everything works as usual but at home with ipv4 its hardly usable..

Please investigate into this bug, I provide myself for testing things on this topic..

regards, max

[Pconfig (thomas9999)]

Temporary workaround can be found here:

http://ubuntuforums.org/archive/index.php/t-1281820.html

This proves that it has something to do with DNS resolving.

[Jeroen Massar (massar)]

For everybody not reading the other comments, #7 actually explains what goes on....

Yes, indeed, probably the best solution is to use just install a local DNS resolver (pdns-resolver), which hits the roots/gtld's etc itself. This is not very friendly to the general Internet, but heck, with the largest DNS server doing short TTLs and based on geography it might not matter too much.

Thus kids, "apt-get install pdns-recursor" and edit your /etc/resolv.conf to point to 127.0.0.1 when you get hit by this issue.

[Pconfig (thomas9999)]

I really think the opendns workaround is better at the time. But both solutions aren't good enough. You can't tell your grandmother to edit some config files because her internet is slow

[camper365 (camper365)]

I agree that this bug should be considered release critical, if not just applying the workaround. What could be added to network-manager is a feature for when you connect it tries to obtain an ipv6 dns and if it succeeds, it uses the network dns resolver or if it fails then it uses pdns-resolver (I just don't think that would work for this release, maybe in Lucid)

[Zack Evans (zevans23)]

I have had a privoxy go-slow - several seconds on every lookup - since installing Karmic beta. Hadn't really noticed a problem in any other app but web browsing did sometimes feel sluggish.

In a brainwave just now I have tried disabling ipv6 (using grub method) and now privoxy is working beautifully. I have also noticed that web browsing feels snappier generally, so I think this was slowing -all- of my apps down by a large enough fraction for me to feel the difference now.

Just to reiterate: it's repeatable for me with privoxy. IPV6 on - privoxy massive latency. IPV6 off - privoxy works fine.

I have a Draytek so blaming the router isn't practical - these have a MASSIVE installed base. Whether it's strictly the router's fault or not, it would not be ubuntu of Ubuntu to get all academically correct about it, we need some sort of workaround that can be achieved by clicking buttons.

To be honest, only the advanced users would want IPv6 anyway, so why not have it off by default and make it very easy to switch on?

[Zack Evans (zevans23)]

Should also say quite happy to test any other proposed workaround.

[camper365 (camper365)]

I have found that when I ping a site (for example, www.google.com) and I ping the url (www.google.com) it takes a while, but if I ping the IP address (63.251.179.13) then the lag is gone

[Ragnarel (ragnarel)]

¿Why I haven't delays with wireless connection and yes with wired?

[Jeroen Massar (massar)]

@ Pconfig / #20

> You can't tell your grandmother to edit some config files because her internet is slow

Does your grandmother use Ubuntu then? If so, then just help her out in fixing the issue :)

@ Zack Evans / #23

> I have a Draytek so blaming the router isn't practical - these have a MASSIVE installed base

This problem also is in effect when the user has Windows and IPv6 enabled on that. The problem lies in the DNS resolver (which might not be the NAT box (what you call "router") but might be even your ISP, and thus you can avoid the problem by not using the DNS resolver in the NAT box. You might of course also try to upgrade your router, maybe they fixed the problem (you upgrade your Ubuntu and other things too, because they have issues, thus try that)

> To be honest, only the advanced users would want IPv6 anyway, so why not have it off by default and make it very easy to switch on?

Because in a few years or so you will have to enable IPv6 as there won't be any new hosts with IPv4 addresses. As such, better bite the apple today and fix those IPv6 issues, then wait till you really need it.

@ camper365 / #24

yes, that is correct, as when you ping www.google.com it has to lookup the hostname in DNS, while if you ping the address, it doesn't. DNS resolving (thus figuring out which address belongs to the requested hostname) is where the problem lies. See the hints about OpenDNS or pdns-recursor to solve it.

@ Ragnarel / #25

as per comment #11 try a:
  for i in `cat /etc/resolv.conf | grep ^nameserver | cut -f2 -d' '`; do dig @$i www.microsoft.com AAAA; done
when connected to wireless and when not connected to wireless. Or just for that matter, check if you are using the same nameservers when connected to wireless and wired, if they are different then you already got a small part of the answer.

[Martin Olsson (mnemo)]

Since we're running out of time, maybe we can just ship "network.dns.disableIPv6==true" as the Firefox default? I'd love a real fix for this bug but the RC is coming up very very soon now.

[Markus Thielmann (thielmann)]

Is it possible, that some patch changed the usage order of the nameserver from /etc/resolv.conf?

My router does deliver a "dead" nameserver via DHCP [1], which was never a problem since Ubuntu used to question the first (local) nameserver. The local nameserver resolves any given request without a problem [2]. If I remove the dead nameserver from resolv.conf, I no longer have any problems resolving DNS queries.

So it *might* be a solution to just change the usage order of the DNS servers to solve this "bug". Please notice, that a lot of users never experienced this problem before Karmic, so it might be hard to blame their hardware for this, even if it might be technically true... :-)

[1] It's a SE515, which delivers 217.237.151.97, despite any configuration.
[2] dig @192.168.1.1 www.microsoft.com AAAA without any noticeable delay

[Micah Gersten (micahg)]

There's at least enough information here to confirm the issue. I'll see if I can get someone to look at it.

[DodgeV83 (spamfrelow)]

This 100% fixed my problem!

1. In /etc/dhcp3/dhclient.conf add the following line:

prepend domain-name-servers 208.67.222.222,208.67.220.220;

2. In /etc/nsswitch.conf edit this line

hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4

to this

hosts: files dns

I'm not sure which one of these did it to be honest, but it's fixed!

[Darren Worrall (dazworrall)]

My results are a little different. At the moment I'm using a draytek router, and am indeed suffering slow resolution in all my apps. Running the snippet above:

for i in `cat /etc/resolv.conf | grep ^nameserver | cut -f2 -d' '`; do dig @$i www.microsoft.com AAAA; done

Is very quick though. I consistently have slow resolution when running updates, but the same command against archive.ubuntu.com is also very quick.

The router has something to do with it I'm sure, my router at home doesn't give me any trouble at all, but querying so directly like this is reproducibly fast, while querying indirectly through update-manager, is reproducibly slow.

[csulok (shikakaa)]

For the what ultimately AND universally fixed/worked around the problem was the following:

edit /etc/sysctl.conf and add the following to the bottom:

#Disable IPv6
net.ipv6.conf.all.disable_ipv6=1

[Jeroen Massar (massar)]

@ csulok / #32

What that does is avoid fixing the problem. You disable IPv6, and thus glibc plays smart and does not resolve AAAA records anymore.

Your DNS resolver though is still broken. You might not notice it now, but if for instance per next year DNSSEC gets turned on you will run into it again.... (and you will probably just disable DNSSEC....)

[Ragnarel (ragnarel)]

csulok, your trick didn't solve my problem.

2009/10/21 Jeroen Massar <email address hidden>:
> @ csulok / #32
>
> What that does is avoid fixing the problem. You disable IPv6, and thus
> glibc plays smart and does not resolve AAAA records anymore.
>
> Your DNS resolver though is still broken. You might not notice it now,
> but if for instance per next year DNSSEC gets turned on you will run
> into it again.... (and you will probably just disable DNSSEC....)
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>

[Antonio Roberts (hellocatfood)]

I'm experiencing these problems on my Dell Studio 1555 laptop with Karmic Beta. I hope it gets fixed soon!

[Nech (gerard-guadall)]

I have two targets
07:02.0 Network controller: Broadcom Corporation BCM4318 [AirForce One 54g] 802.11g Wireless LAN Controller (rev 02)
00:19.0 Ethernet controller: Intel Corporation 82566DC Gigabit Network Connection (rev 02)

I work better using wifi than using wired connection.

[Jeroen Massar (massar)]

@ Nech / #36

> I work better using wifi than using wired connection.

So, like I ask everybody else, check to see if there is a huge latency time difference when doing:

for i in `cat /etc/resolv.conf | grep ^nameserver | cut -f2 -d' '`; do dig @$i www.microsoft.com AAAA; done

Over the wired or wireless; quiker maybe is to check if you get a different set of DNS servers when connected over wired or wireless (just check if /etc/resolv.conf changes).

[Nech (gerard-guadall)]

I think the problem is not DNS. Actually, when I visit different websites in a short period of time, then everything get saturated. Some websites not load, and other take up to 2 or 3 minutes to do so. I tried it using Google Chromium also, and the result was the same. Is a new instalation the karmic, and the upgrade just happened

Results of wireless
-----------------------
; <<>> DiG 9.6.1-P1 <<>> @80.58.0.33 www.microsoft.com AAAA
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49350
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.microsoft.com. IN AAAA

;; ANSWER SECTION:
www.microsoft.com. 2921 IN CNAME toggle.www.ms.akadns.net.
toggle.www.ms.akadns.net. 247 IN CNAME g.www.ms.akadns.net.
g.www.ms.akadns.net. 265 IN CNAME lb1.www.ms.akadns.net.

;; AUTHORITY SECTION:
akadns.net. 90 IN SOA internal.akadns.net. hostmaster.akamai.com. 1256289512 90000 90000 90000 180

;; Query time: 104 msec
;; SERVER: 80.58.0.33#53(80.58.0.33)
;; WHEN: Fri Oct 23 11:20:03 2009
;; MSG SIZE rcvd: 170

wired
-------
; <<>> DiG 9.6.1-P1 <<>> @80.58.0.33 www.microsoft.com AAAA
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1196
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.microsoft.com. IN AAAA

;; ANSWER SECTION:
www.microsoft.com. 2805 IN CNAME toggle.www.ms.akadns.net.
toggle.www.ms.akadns.net. 140 IN CNAME g.www.ms.akadns.net.
g.www.ms.akadns.net. 144 IN CNAME lb1.www.ms.akadns.net.

;; AUTHORITY SECTION:
akadns.net. 91 IN SOA internal.akadns.net. hostmaster.akamai.com. 1256289631 90000 90000 90000 180

;; Query time: 102 msec
;; SERVER: 80.58.0.33#53(80.58.0.33)
;; WHEN: Fri Oct 23 11:22:00 2009
;; MSG SIZE rcvd: 170

[Jeroen Massar (massar)]

@ Nech / #38

As you have the same DNS server for both wired and wireless, most very likely _your problem_ is not a DNS issue* like what the others show here.

* = unless an upstream of your DNS server has the "drop unknown DNS records" problem and your resolver caches the negative answer correctly, which will cause any subsequent query, like the ones above, to be quick again.

To solve your problem, I guess you'll have to take a peek with Wireshark...

[Zack Evans (zevans23)]

My problem goes away if I disable IPv6. If I boot with IPv6 though, so I have the problem, DNS lookups from the command line happen quickly.

for i in `cat /etc/resolv.conf | grep ^nameserver | cut -f2 -d' '`; do dig @$i www.microsoft.com AAAA; done

is practically instant. (I have also tried it with some other hostnames to check it is not cacheing hiding the problem.)

I should add I did not have this problem in Jaunty, and no equipment has changed, only the upgrade to Karmic.

So, as I type, with IPv6 enabled, Privoxy is grinding, everything else seems OK.

If I reboot with IPv6 off, Privoxy and everything else will be OK. DNS AAAA lookups seem OK whether enabled or disabled.

So is there some other subtle interaction between Privoxy and IPv6?

[Jeroen Massar (massar)]

@ Zack Evans #40

I quickly looked at the Privoxy 3.0.12 IPv6-patch included in Debian (and probably the same thing in Ubuntu) and it seems that it is quite incomplete and has quite a number of broken assumptions. One of the primary brokeness is actually documented in the patch with: /* TODO: Allow multihomed hostnames */ indeed, that also means that if a host has multiple A and/or AAAA records, it will only try to use the first one, which might actually be an IPv6 address (which is generally preferred). As an example www.google.com has generally 4 IPv4 IP addresses (A records) and as that Privoxy patch only supports 1 address per hostname, it will only use one of those. If that single address is then IPv6, it will try IPv6.

If your IPv6 connectivity is then broken (which might also be the problem for other hosts) then of course it will take quite some time for that to timeout....

I would say: file a bug report against privoxy as clearly the patch that is included can various websites which have multiple addresses to only have the use of one address. and of course only IPv6 or IPv4 is then supported by it... in other words: BROKEN.

With the above in mind, people who have problems with "IPv6" should perform two tests:
 - ip ro sho |grep default
   ^---- if you have a default route somewhere then of course it will be used, thus check where it goes and if that makes sense for you.
 - for i in `cat /etc/resolv.conf | grep ^nameserver | cut -f2 -d' '`; do dig @$i www.microsoft.com AAAA; done
  ^-- if this has huge latency then your DNS resolver (or one above it) is broken. Do test this also with different hostnames then just www.microsoft.com, try especially hostnames that you didn't check in a while as they might be cached somewhere.

[PhilGil (pgilston)]

I've also been hit by this bug on a new Karmic install. Had no problems with Jaunty on the same hardware.

Tried csulok's work-around from post #32, which didn't work. Disabling IPV6 in Firefox does work (but only for Firefox, of course). Changing my IPV4 settings to use Open DNS works universally, but seems like a rather unpleasant kludge to use as a permanent fix.

[dhenry (tfc-duke)]

@PhilGil in #42: disabling IPV6 did not fix the problem for you? sounds strange. Did you run "sysctl -p /etc/sysctl.conf" after having set net.ipv6.conf.all.disable_ipv6 in /etc/sysctl.conf?

[dhenry (tfc-duke)]

Well, after a bit more testing, the fix in #32 only partially works. For example, it works for browsers but not for apt-get, wget or lftp.

[James M. Cooper (jmichc)]

It would certainly be good if this were somehow addressed before stable release.

Is there any chance of that at all at this point?

[Luciferion (mikolaj-q)]

That's good question James!

[aliases (nickolukianov)]

Hi there i had those problems too. It seems to work after i disables all iptables modules.
I think this bug caused by wrong-not good iptables policy .

As I'm a regular desktop user, and i sit behind router I find it ok to remove iptables.

rmmod iptable_filter
rmmod x_tables
rmmod ip_tables

worked for me :)

[camper365 (camper365)]

I am surprised that this isn't release critical. Probably the biggest reason people get on the computer is to use the internet and if people associate Ubuntu with slow internet, no matter how fast you make it boot, they won't want to use it. Even if it is just a workaround that is applied to the final release, it still should be done.

[Andy Langdon (andy-langdon)]

I have not used launchpad before and am not sure of the procedures. This bug has been nominated as a confirmed and high bug in the Network Manager but has not been confirmed or assigned an urgency under Linux (Ubuntu). This means it does not appear as a high urgency problem when you search the bugs for Karmic which I would have expected it to. Could it be that this problem is getting overlooked?

[Szabolcs (szhorvat)]

I am affected by this problem too, and I can't help but wonder why this bug isn't blocking release. It makes Ubuntu Karmic practically unusable for me (and most likely everyone else who is affected).

[asdf (joseph-boese)]

Post #30 directions seems to be the common advice given to fix the issue and fixed it for me (thanks Dodge V83). Still I agree this a major problem and Notwork Manager is the biggest pile of crap software that comes with gnome and I have no idea why they don't toss the shiiteware out like they should. A much better alternative I hear is wicd. To install type

sudo apt-get install wicd

[asdf (joseph-boese)]

Ack don't use wicd if you have a netbook and use the ath5k driver (still very beta nice way to put it) as suspend and resume hose up the ath5k driver (pretty sure Network Manager handles this by doing an rmmod and modprobe). These network issues are precisely why Windows is owning Linux on the netbooks. I got mine running fine but not everyone has a CS degree. Most people just want it to work out of the box. Oh well maybe next version.

[camper365 (camper365)]

Next version? This should have delayed the release at least long enough for ipv6 not to be seeded on the LiveCD. People may consider the slow internet on the livecd to be caused by the fact that you're running from a cd, but when they install it they expect it to be full speed, which should be faster than Windows. I know that ipv6 needs to replace ipv4 soon, but if trying to do it slows down the internet, and Windows isn't doing the same thing, then it is considered an Ubuntu problem and not an isp or DNS resolver problem (whether it is or not). A lot of people are willing to take any fault with Linux to an extreme and say that is why you shouldn't use it (even if it isn't true). This will make people like that go nuts because they have found a problem with Ubuntu that isn't in Windows, completely disregarding all of the problems in Windows that aren't in Ubuntu. I have not seen Windows 7, but I have heard that it is a lot to compete against and if Ubuntu has slow internet, people won't even bother looking at it. You can't say that a fix is to go this bug report and look at comment #30. If there is a workaround, it should be applied before it is installed.

[asdf (joseph-boese)]

I agree it is a pretty serious issue for many and by no way mean to imply windows is anything but a well polished turd. All in all the upgrade to 9.10 on my netbook (Samsung NC10) was largely seemless (a few minor issues including this one that took maybe an hour to resolve but again I am a tech geek that loves the command line) and is much snappier than any windows as well as even 9.04. But for Joe Q Public this issue would be a showstopper. I agree this needs to be resolved asap as networking is always one of those issues that scare non techies (and with crappy drivers like ath5k and imho not so good Network Manager they should be scared). We are on the same page.

[Cruxic (cruxic)]

I too am surprised this did not block the release. Can we appeal to a higher power? I'm not too familiar with the development process of Ubuntu but surely there's some IRC channel or mailing list where we can get the attention of somebody with "connections".

[A. Tombol (atombol)]

i'm adding this comment, because it might help some who think that they problem with the connection is related to this ipv6 bug (i thought the same)

on my laptop, the ethernet card picked up the connection to the local router, but browsing didn't work. (but it opened google pages as is realized later)
wifi connection was fine, with browsing and everything
i tried every workaround in this thread, but none of them worked

after a half day of struggle i ended up searching for driver bugs, and ran into this thread
http://ubuntuforums.org/showthread.php?p=2545109

and voilà, setting MTU from automatic to 1400 in Network Manager, eth0 settings, did the trick!

i hope it helps at least some of you

[magi (magnus-hagdorn-marsupium)]

I seem to be suffering from the same problem. On my desktop with wired ethernet connection I seem to have a problem with DNS:
- I get a timeout when I ping a host, e.g. ping google.com
- I cannot resolve the address when I do host google.com
- I can resolve the address when I do host google.com ip_of_my_dns
- the info field of the network-manager reports the correct dns servers
- it works as expected after I have restarted network-manager

I have no problems on my laptop which is connected wirelessly to the same router and thus is using the same settings.

[magi (magnus-hagdorn-marsupium)]

ok, it looks like I am suffering from a different problem: reported Bug #465757

[Markus Thielmann (thielmann)]

If you're hit by this bug just in karmic, while it does work in Jaunty, could you please try this:

Open /etc/resolv.conf and reorder the nameserver lines (first to last, etc.). Do not restart, do not reconnect, just have a look if this solves your problem.

[Tim Coffey (coffey67)]

This bug affects me too but I found a simple work around. Edit your settings in network manager and put your routers IP address as the last one in the DNS servers box. Find your providers primary and secondary DNS server IP addresses and ensure they are listed first. Once you make the change you need to disable networking and then re enable it for the change to take effect. Browsing is speedy now! I hope this helps everyone get going for now but I also hope the bug gets fixed because there is no reason your router should not be able to be listed first.

[Suparshwa Shah (suparshwa-great)]

Hi, Tim Coffey

I dont know how but this seems to solve my problem.

[Fer Servadu (ferservadu)]

I can confirm that Markus Thielmann's suggestion (#59) solved the problem for me. Thank you.

[Jeroen Massar (massar)]

Of course it fixes the problem, this as the problem lies in the DNS resolver that is inside the DSL/cable/etc modem.

Thus by entering/using OTHER dns servers, preferably the ones from your ISP, otherwise the ones from the OpenDNS or similar providers, you bypass the DNS resolver in the modem, and thus the device that would otherwise drop DNS queries for AAAA records.

[Tim Coffey (coffey67)]

#63 Jeroen Massar,

I do not think the problem is with the DNS resolver in the modem because other PC's do not have this problem and this PC did not have the problem until an upgrade to Karmic. I think the problem is how Karmic interacts with the modem.

[Jeroen Massar (massar)]

#64 Tim Coffey,

Just try the following: dig @<dns server that is broken> www.bingabongobango.com AAAA

That will take quite a long time to return.

The problem is that when you install Karmic, you get IPv6, because of that, programs that are IPv6-capable will start asking for AAAA records As the DNS Server is broken it will not reply to the request. Thus it looks like things are slow.

If the other PCs are using that DNS server, but not asking for AAAA records, then all will be fine and you won't notice the slowness indeed.

[rsoulliere (rsoulliere)]

The quickest way I found to work around this issues outside of firefox was to use the opendns solution found at:
https://store.opendns.com/setup/device/ubuntu/

This allowed me to use aptitude to download software and updates. This is the easiest solution for joe user and grandmothers to get started with Karmic without editing conf files. Eventually, there may be a better and more permanent solution for all users.

[Markus Thielmann (thielmann)]

Joeren: Well, my suggestion was to *switch* the nameserver entries, not to replace them. I already suspected that Karmic changed the order of the DNS requests, in fact asking the last nameserver in /etc/resolv.conf first, while Jaunty asks the first nameserver.

While changing the order again wouldn't solve any problem here, but people with a single broken DNS server wouldn't notice the change as a regression, since it worked for them in Jaunty and it might still work for them with Windows.

[Markus Thielmann (thielmann)]

Joeren: BTW, man resolv.conf states: "If there are multiple servers, the resolver library queries them in the order listed.", which seems not the case for karmic.

[JLotspeich (jlotspeich34)]

I disabled ipv6 in firefox, and lspci indicates ipv6 is turned off in
karmic. The problem still persists in the case that I'm using a linksys
router as the DNS relay. If the router is removed from the DNS list,
everything speeds up.

Before anyone tells me to get a new router, this problem didn't occur in
Jaunty, just Karmic, and I've done everything possible to confirm that
IPV6 is disabled on my machine. So it's one of two things:

1) Karmic uses a different library than Jaunty to do DNS lookups, and
that library has introduced some kind of bug (I'm guessing it's a
sequence of queries, not just one since it takes a few minutes of very
slow surfing before the router gives up entirely)

2) Karmic is lying about ipv6 being disabled and is insisting on using
it anyway.

I attempted to report this as a separate bug because I completely
eliminated IPV6 from being the culprit, but here we are.

[Taralyn Darkchylde (hellion0)]

The procedure in #59 helped my problem - I was having the same issues, DNS lookup was delayed horribly under Karmic but was spot-on under Jaunty.

[Sennaista (sennaista)]

I have the same problem. Firefox, Thunderbird, Synaptic, etc are all very slow and surprisingly disabling IPv6 in Firefox doesn't speed it up either, contrary to what others are reporting.

[jordan.sc (jordanjsc)]

I have the same problem with 2 desktops and a laptop. I've tried the laptop wirelessly on 3 different networks and the severe slowdown continues. The workaround below fixes the slowdown by disabling IPv6 on bootup (it's worked on all 3 computers):

start a Terminal session and type:

gksu gedit /etc/default/grub ....then change

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

to

GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1 quiet splash"

then

sudo update-grub

Reboot and network speed should be back to normal.

I hope that a fix for this disconcerting bug is provided ASAP. Thanks.

[Michael Hasenstein (hasenstein)]

@jorden.sc: I had already done this manually in menu.lst, I now also tried the above - but even though I let update-grub even generate a new menu.lst, the ipv6 param is missing!!!!
I had to add it manually again.

Also, disabling ipv6 this way does solve my firefox DNS problem (I reenabled ipv6 in the firefox prefs), BUT my Ruby on Rails (2.3.4) plus apache and passenger development environment now takes MUCH longer to respond than before. Now my normally much slower laptop (with an identical dev. environment) suddenly is much faster than my PC. Some network lib is still causing a pause before each request - and it is not the apache/passenger side.

It is my mysql connection!! How do I know: On my laptop the vmware-internal network host-only network did not work any longer after the 9.10 upgrade. So I moved the mysql server from the laptop's Windows into the Ubuntu inside Vmware and gave it some more RAM. It now works as before. On my PC - also running Windows and Ubuntu inside VMware - I left the database on Windows and Rails connects over the network, there I don't use host-only networking (the "global" network adapter is always on, unlike on the laptop where I often work without offline any network attached to the laptop).

[Johan (deberghes-johan)]

When I test this command :
for i in `cat /etc/resolv.conf | grep ^nameserver | cut -f2 -d' '`; do dig @$i www.microsoft.com AAAA; done
The response is instantaneous. But when I do this lookup :
for i in `cat /etc/resolv.conf | grep ^nameserver | cut -f2 -d' '`; do dig @$i www.speakeasy.net AAAA; done
This is very slow, and the delay is there.
Notice that I have no IPv6 connectivity, and that the IPv6 stack is activated, as the default.
This is the delay that we are talking about. And I think this is a misbehaviour from the involved authoritative DNS servers :
http://www.ietf.org/rfc/rfc4074.txt
The explanation is that the DNs server should return an error when there is no AAA record. But in this case it returns nothing. It doesn't follow the RFC. So the resolver wait for the answer, and then time out.
I think that the fact of changing one DNS server to another resolved the problem is due to the fact that the newly configured DNS servers does cache the "no response" state", or something like that. Just a thought. Don't know enough sfuss about it.

One solution is to wait for the involved DNS to update and support AAA records. But it may happen in years.
Another solution is : Disable AAA address resolution when there is no IPv6 routable address configured on any interface, or when Network manager has his IPv6 parameters set to "ignore" (which is the default in Karmic, and is a new feature that is not present in Jaunty ). The getaddrinfo() function is the one that should not ask for AAAA records when there is no IPv6 connectivity.
It seems that the Red hat team has already encountered this problem :
http://kbase.redhat.com/faq/docs/DOC-17904

The wrapper library seems to be a solution for that. What do you think of all this ?

[Jeroen Massar (massar)]

The solution is to use a working DNS server.

A couple of ways to accomplish this:
 - use your ISP DNS servers directly
 - use OpenDNS or a similar provider
 - local caching resolver: put 127.0.0.1 as the nameserver (/etc/resolv.conf) and install pdns-recursor

The latter can be done per default in Ubuntu and would always work and require no setting changes.
The only negative side-effect is that the caching effect of DNS might be less used, but then again, most sites use DNS loadbalancing now, thus this is a non-issue.

[Christian Kosmowski (ckosmowski)]

This is no solution. This is just a workaround. All this worked well in Intrepid, Hardy and so on. There should be an package update so that installing updates fixes the problem.

[walec51 (me-adamwalczak)]

@rsoulliere thank you very much, your link in #66 solved this issue for me.

Personally I regret trying to upgrade to Karmic. I had a severe bug in Jaunty that freezed my system once a day. When I upgraded to Karmic my system began to be unbootable because of an sergv in mountall. When I reinstalled I had no net :/

For those that say that the problem is in the way routes and some dns servers handle ip6:
So Karmic was released to show use how bad network equipment we bought ? to teach us a lesson ?

This bug should be a show stopper, at least for the feature change caused it. Ubuntu is becoming more unusable release after release. I'll try to do more beta and alpha tests my self in the future but I can see here beta testers comments didn't help.

[Jeroen Massar (massar)]

@cosmo #76, of course it is "just" a work around. The problem lies in your DSL/cable modem, not in Ubuntu. Thus ubuntu can only work around this issue. You didn't see the issue before as you where not using all the features of the Internet (IPv6 ;)

[jordan.sc (jordanjsc)]

@Jeroen Massar #78: You wrote "the problem lies in your DSL/cable modem, not in Ubuntu". I'd like to make a few points.

1) No IPv6 slowdowns in Jaunty or Mint 7 (which both have IPv6 enabled) anywhere I've tried. Which "features" of the internet am I not using in Jaunty but am using in Karmic?

2) Serious slowdowns with Karmic at a local coffeehouse, my place of business and the airport where I have no control over their modems.

3) OpenDNS, etc. remove some features of Firefox such as "automatically" determining which site I'd like to go to when I type key words into the address bar. If I enter "ubuntu forums", Firefox will take me to http://ubuntuforums.org/. OpenDNS will not do that.

4) Bottom line is that this needs to be fixed/patched/whatever so that most anybody trying Ubuntu for the first time will have a good experience. Currently, for many, Ubuntu cannot provide such an experience.

[Sennaista (sennaista)]

This is very stupid and I should have tried it before contributing to this bug. I have been using a Belkin wireless adapter on Intrepid and Jaunty with no issues so when I upgraded to Karmic and internet was slow I didn't think it could be problem with the adapter support. Well, I'm still not sure whether that is the problem or not but, having seen that disabling IPv6 in Firefox didn't do any good, I decided to give it a try and change to a wired connection and every thing is back to normal now, fast internet access across the whole system.

Maybe it's worth a try for all of you who had this problem using wireless.

[Michael Hasenstein (hasenstein)]

It is not a wireless-related issue, a least not for me: I (unfortunately) upgraded two VMware Linux systems.

[Cruxic (cruxic)]

re #59:

I reversed the order in /etc/resolv.conf and saw no improvement, however, commenting out the "domain" and "search" lines gave a vast improvement:

   # Generated by NetworkManager
   #domain domain.actdsltmp
   #search domain.actdsltmp
   nameserver 192.168.0.1
   nameserver 205.171.2.65

I'm not sure what that means. Disabling ipv6 in firefox works too for me.

[Bernard Bou (bbou)]

How long will this bug be confirmed but more or less denied ("the problem lies in your ..., not in Ubuntu" #78) ? #79-1 alone stands as the killer argument.

[Per Heldal (heldal)]

Another issue in addition to ipv6 is how the resolver-lib handles search domains. I found I could avoid the delay by removing the "domain" and "search" statements from /etc/resolv.conf. In this particular case there's a broadband-router which put either the providers domains or a configured local alternative in the DHCP-response. There's no way to make it drop the options altogether when it's not used properly. Thus a solution (in addition to a local caching resolver etc described above) is to remove these options (domain-name and domain-search) from the list of options the dhcp-client (/etc/dhcp3/dhclient.conf) asks for. This problem is easily identified as described in #82 above.

[Per Heldal (heldal)]

Wrt #84, is it a correct observation that the resolver-library in karmic always tries to add search-domains to the queried string even when the original query from the application is for a FQDN? The first DNS-request from a karmic client looking for "www.google.com" in a browser is for "www.google.com.localdomain" when "domain" or "search" is specified in /etc/resolv.conf. I thought the search-strings only were appended when there was no domain-part specified, and/or if there was no response/hit on the first request.

[Per Heldal (heldal)]

The reason for the delay described in #84 and #85 seems to be a firmware-bug in a wlan router (D-Link DIR-655). It doesn't seem to pass NXDOMAIN-responses to the clients when it is set up to relay requests. The upstream servers respond immediately with NXDOMAIN when queried directly. However, this problem wasn't discovered before so the behaviour of the client resolver has obviously changed in karmic.

[Jonzer (jonzer)]

Just to add to the voices being affected by this bug. Everyone in Ireland using an Eircom router (00,000's of customers) will encounter this issue. It is crazy, I logged on here to read about it and I can't believe it was pointed out weeks ago and nothing was done about it.

No point in blaming 'the router' or anything upstream. All I know is everything was fine in Hardy, everything is fine in Windows 7. Hell, if I load up a Windows XP VM in Virtualbox within Karmic, DNS lookups work flawlessly from the same router!!

Very disappointing.

[Christian Kosmowski (ckosmowski)]

This is exactly my opinion Jonathan!

In Germany it's the Router of the Provider "Alice" which has this issue. And i think it could be, that it maybe not talking in the correct standard way but it worked in Hardy and Intrepid and where is the use of forcing standard behaviour and killing functionality?

Everyone here is posting "reasons" and "workarounds" but that's not the point. That's exactly the point why all this windows-user guys say "Linux is to complicated". In an Operating System called "Ubuntu" (we all now what the name means and what it stands for) which aims to be an OS for every user it is absolutely impossible to accept "workarounds". And all this worked fine in the last versions.

[Laurent Bigonville (bigon)]

This has nothing todo with network-manager or linux kernel. And not even libc as dig doesn't use libc functions to resolve names.

I think that this can be worked around with tweaking /etc/gai.conf or by blacklisting the ipv6 kernel module

[John O'Dwyer (johnrory-odwyer)]

Work around in #66 worked for me. Before applying this workaround my resolv.conf file read as follows:

# Generated by NetworkManager
nameserver 192.168.1.254

After applying the workaround my resolv.conf file reads as follows:

# Generated by NetworkManager
nameserver 208.67.222.222
nameserver 208.67.220.220

[jordan.sc (jordanjsc)]

@Laurent - I'm not familiar with the way confirmed Ubuntu bugs are dealt with, but since you changed the assignee to nobody, does this mean that the issue will not be addressed? And that many thousands or more of Karmic users will have the equivalent of dial-up speeds unless they are tech savvy and know how to tweak conf files or blacklist modules?

[Philipp Kern (pkern)]

One could argue that the resolver could refrain from issuing IPv6 queries when no IPv6 connectivity is available. (I.e. just link-local addresses on the interfaces.) It might be too expensive to check this on each and every query but it would relieve the problem a bit.

Disabling IPv6 in Firefox by default is a very bad solution as we want to achieve IPv6 support out of the box. I also encountered those broken resolvers in the wild, though, and it resulted in me configuring sane DNS servers for the single Alice access point through NetworkManager, which works just fine. If there would be an anycasted DNS resolving offer out in the wild we could use that instead, but even then you usually rely on using the local DNS servers in case they give you different views in the corporate environment (to see local hosts or route requests differently).

[Laurent Bigonville (bigon)]

@jordan.sc: IMHO assigning bugs to people or team like that is a good habit, you can subscribe them instead

I also think that ipv6 support must be on by default

[Christian Kosmowski (ckosmowski)]

I'll give it up... posting, confirming assigning bugs in this senseless bugtracking system with every ignorant hacker freak telling some stupid workarounds about howto edit some stupid config file. iMho A bugtracking System should keep track of bugs for people SOLVING Bugs. This one here should be called Workarounding System.

I mean i always told my Family and Friends to use Ubuntu because its EASY. Because its just WORKING unlike Windows. I told them "trhough your windows copys away you dont need them if you ever tried ubuntu".

And now? When i tell those guys aka "Users" or "Mortal" - hey the whole community knows about the Problem that your internet connection is as slow as in the early days back when you had a 56k Modem. But its ok for THEM because THEY are SELFISH and able to edit config files. And guys this is pretty important for IPV6 ???

I made the assignation to get someone to look at the PROBLEM not to disable ipv6. the should REPAIR this. So i assign it again. Will do this 1000 times more if i need to.

[Kevin Otte (nivex)]

I've been experiencing similar slowdowns at work on my laptop using dnsmasq for caching. As soon as I bypass my local dnsmasq and start using the provided DNS servers directly, the slowdowns go away.

[Michael Hasenstein (hasenstein)]

@cosmo et al., I reported that it also slows down my VMware host-only network communication when my laptop is without any connection to an outside network, so THERE IS NO INTERNET AND NO NAMESERVER AT ALL. Yet, I had to move mySQL from the host OS into the virtual (Ubuntu 9.10) machine in order to be able to continue working - because every single mySQL request, and there are a lot when you develop a website, took a few seconds longer than before the upgrade.

Also, I disabled IPv6 in grub (ipv6.disable=1) and while it helps Firefox (I re-enabled ipv6 in Firefox and it still worked) it did NOT help other applications: my ssh connection to a server on the Internet takes a loooooong time, but when I use "ssh -4" (tell ssh to use ipv4 only) it's immediate. AFTER I DISABLED IPV6?!

So the case is NOT as simple as some people want to make us believe, and it certainly is NOT a problem "of your router" (i.e. mine). See above.

I've been "doing Linux" since 1995, I even did kernel development (networking/firewall/network address translation) up to kernel 2.4. And I say this is a bug in Ubuntu!!! Okay, any number of exclamation marks, i.e. yelling, does not make me any more right, but... oh well.

But who cares, the only reason I'm still here is that SuSE's and Fedora's new releases are still two weeks away.

[Martin Olsson (mnemo)]

Don't assign bugs to persons or groups, it won't make anyone fix anything faster. And stop posting workarounds, the bug report should focus only on a SRU bugfix (use forums for workarounds). Only assign it to yourself if you intend to work on this bug and have the necessary skills. Many many bugs spiral out of control with rants and complaints and then the useful info gets buried and it takes longer to fix it.

The best thing you can do if you want this bug fixed faster is find someone on IRC who has got the skills to fix this bug and then post the LP url to them and explain why it's important.

[Gregory Burd (gregburd)]

This issue is serious, it affects a large percentage of the Ubuntu population and, as is obvious by this long discussion, is both a) annoying and b) hard to understand/diagnose/fix properly.

But, if you filter this conversation there is both a good diagnosis (#4) and a good, tested workaround (#74) implemented by engineers at RedHat. The one issue with the workaround is that it essentially disables IPv6 AAAA DNS record lookups when calling getadderinfo() with AF_UNSPEC.

In the README.txt of the RedHat workaround called "libwgetaddrinfo.so" you find:
"IPv6 lookup avoidance by LD_PRELOAD=libwgetaddrinfo.so. getaddrinfo will perform IPv4 and IPv6 lookups when using AF_UNSPEC." They also state that this technically breaks RFC2553 compliance, which is bad.

Right now, in the short term, we're faced with a "lesser of two evils" decision:
1. Adopt the RedHat solution and in the process break RFC2553 while fixing 100% of this problem.
2. Annoy and confuse 80-90% of the Ubuntu population by doing nothing.

Don't get me wrong, I am an IPv6 zealot, but it is counterproductive to the IPv6 cause to force people to disable/fix/workaround IPv6 by hand. That kind of deep mental scaring will end up generating negative PR (google: ipv6 sucks) which will only slow adoption.

I suggest that we adopt option #1 (the RedHat workaround) ASAP and deploy it before this becomes an even bigger firestorm (like an article in eWeek talking about Window 7's superior networking speeds). That's good for no one.

In the mean time we (that's the "royal we") should find some way to un-break the RFC2553 compliance perhaps by only enabling the workaround when the network interface is routed over a broken network (one that doesn't properly do AAAA DNS record lookups). But this will take time and some creativity and we can't afford to wait for that when there is a perfectly good solution available now.

I hope this suggestion is helpful and moves this issue toward resolution.

regards,

-greg

[JoeKlein (jsklein)]

There is another option, and that is to enable IPv6 on the host, and
properly configure the local network and edge router to support IPv6.

Joe Klein

On Tue, Nov 3, 2009 at 2:35 PM, Martin Olsson <email address hidden> wrote:
> Don't assign bugs to persons or groups, it won't make anyone fix
> anything faster. And stop posting workarounds, the bug report should
> focus only on a SRU bugfix (use forums for workarounds). Only assign it
> to yourself if you intend to work on this bug and have the necessary
> skills. Many many bugs spiral out of control with rants and complaints
> and then the useful info gets buried and it takes longer to fix it.
>
> The best thing you can do if you want this bug fixed faster is find
> someone on IRC who has got the skills to fix this bug and then post the
> LP url to them and explain why it's important.
>
> ** Changed in: linux (Ubuntu)
>     Assignee: IPv6 Task Force (ipv6) => (unassigned)
>
> ** Changed in: network-manager (Ubuntu)
>     Assignee: IPv6 Task Force (ipv6) => (unassigned)
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a member of IPv6 Task
> Force, which is a bug assignee.
>
> Status in “linux” package in Ubuntu: Confirmed
> Status in “network-manager” package in Ubuntu: Confirmed
>
> Bug description:
> Binary package hint: firefox-3.5
>
> By default, when Firefox 3.5 is installed, web pages are loaded slowly and it looks like IPv6 DNS lookups is the cause. However, you can go to about:config and change network.dns.disableIPv6 to true, the problem gets fixed.
>
> ProblemType: Bug
> Architecture: i386
> Date: Sun Aug 23 12:06:13 2009
> DistroRelease: Ubuntu 9.10
> Package: firefox-3.5 3.5.2+nobinonly-0ubuntu2
> ProcEnviron:
>  LANG=en_US.UTF-8
>  SHELL=/bin/bash
> ProcVersionSignature: Ubuntu 2.6.31-6.26-generic
> SourcePackage: firefox-3.5
> Uname: Linux 2.6.31-6-generic i686
>

[Michael Hasenstein (hasenstein)]

@JoeKlein: No it is not. It would be - if this were a Debian bugsite. However this is Ubuntu. It is getting a little tiring my friend, but I'm sure you too know who the target users of Ubuntu are vs. Debian or CentOS.

[Philipp Kern (pkern)]

On Tue, Nov 03, 2009 at 06:39:06PM -0000, Michael Hasenstein wrote:
> Also, I disabled IPv6 in grub (ipv6.disable=1) and while it helps
> Firefox (I re-enabled ipv6 in Firefox and it still worked) it did NOT
> help other applications: my ssh connection to a server on the Internet
> takes a loooooong time, but when I use "ssh -4" (tell ssh to use ipv4
> only) it's immediate. AFTER I DISABLED IPV6?!
>
> So the case is NOT as simple as some people want to make us believe, and
> it certainly is NOT a problem "of your router" (i.e. mine). See above.

It's a problem with the nameserver on the router, not with the router
itself. If the libc does AAAA requests that are not properly answered
which it doesn't when you force it to (through -4) that's hardly the
fault of the C lib.

> I've been "doing Linux" since 1995, I even did kernel development
> (networking/firewall/network address translation) up to kernel 2.4. And
> I say this is a bug in Ubuntu!!! Okay, any number of exclamation marks,
> i.e. yelling, does not make me any more right, but... oh well.
>
> But who cares, the only reason I'm still here is that SuSE's and
> Fedora's new releases are still two weeks away.

Oh great. Rest assured that I experienced the same problem with Fedora, too.

Kind regards,
Philipp Kern

[Michael Hasenstein (hasenstein)]

@Philipp: Since you even quoted my message, could you please explain how it can *only* be a nameserver issue if I have an issue with only a laptop-internal VMware network???

[camper365 (camper365)]

Right now I am seriously considering switching over to Fedora or openSUSE and scrapping Ubuntu. I am perfectly able to tweak conf files and apply workarounds, but I don't want to do it every minute of every day. I have work to do other than get the system running.
It seems to me that the management and developers of Ubuntu are lazy and trying to deny the problem. The problem is there whether they are a victim of it or not.
Articles saying how slow Ubuntu networking is compared to Windows 7 is certainly not going to help the Linux community at all. I know the developers put a lot of work into this release, but it won't be much more to apply a workaround. This bug has existed for over two months and that is plenty of time to apply a workaround and potentially a fix.

[FiremanEd (firemaned)]

My work around of this issue in Karmic is to actually bypass the dhcp option in network connections and manually enter them. This worked even on the PowerPC Karmic install on my PowerBook G4.

[dave b. (d+b)]

So i thought that nothing changed in this since 9.04 when a lot of applications were ipv6 enabled....

The issue is that AAAA records are useful for computers which have ipv6 routes (on the internet).

So is this bug is new or valid....?. If some one can reproduce this and provide a tcpdump showing some evidence of extra look ups that would be interesting.

However, if the system does not have a valid ipv6 route i would hope that it does not try to connect to an ipv6 non-route-able host..... I know there is a problem when a *false* ipv6 route is advertised. In this case, there may be a problem. Unlike ipv4, ipv6 is not widely deployed so checking if a route is valid *may* be a good idea(throw it away if there is a known working ipv4 route). I am not sure what networking-manager does.

If there is a library waiting for the response of an ipv6 request and there is no ipv6 route.... that is an issue.

[Christian Kosmowski (ckosmowski)]

Thank you Gregory Burd this is what me and 80-90% of the Ubuntu population wanted to hear.

[Andy Langdon (andy-langdon)]

I also approve of Gregory Burds suggestion #98

[Laurent Bigonville (bigon)]

and why not blacklisting the loading of the ipv6 module instead of replacing system functions like that?
In that way we don't even break RFC's

[Antonio Costantino (anto-costantino)]

Suggestion #98 seems to be the most reasonable so far...

[Martin Pitt (pitti)]

The RH workaround with a preload library isn't something that we could sensibly put into an SRU, since you'd have to preload it for every single program that does DNS resolution (and there are thousands in the archive).

Just to ensure that we are all talking about the same thing, this is what I see:

$ time host www.ubuntu.com
www.ubuntu.com has address 91.189.90.40 <- appears immediately
[20 second pause here]
;; connection timed out; no servers could be reached
;; connection timed out; no servers could be reached

real 0m20.371s
user 0m0.000s
sys 0m0.050s

> and why not blacklisting the loading of the ipv6 module instead of replacing system functions like that?

IPv6 is built into our kernel, there are no modules. But perhaps we could modularize those in the kernel? Kernel team, does the Jaunty kernel have a different IPv6 setup than Lucid? We didn't have this problem in Jaunty, and I don't believe it's a glibc regression (since a jaunty chroot on karmic kernel has the same problem).

[Bernhard Schmidt (berni)]

I strongly disagree, breaking IPv6 globally just because some people sit on broken resolvers is extremely bad. I suspect the IPv6 users group is at least as big as the one sitting on broken resolvers, and while the IPv6 group is getting larger every day the broken resolvers group should stagnate or even get smaller.

Earlier Ubuntu versions had a patched libc6 that disabled AAAA lookups for AF_UNSPEC _IF_ no global IPv6 addresses were configured on the system. This could be a way forward and would be in line what Windows is doing today.

[Jeremy Visser (jeremy-visser)]

I'm sorry, but blacklisting IPv6 is unacceptable. The 'apocalypse' is drawing near — current projections point to 2012 as the most likely date.

So it's almost certain that by 2011 the majority of Internet users will have an IPv6 connection.

And then this begs the question: will Ubuntu 10.04 systems be in production in 2012. Of *course*!

Therefore anybody with the slightest bit of common sense cannot possibly for a moment consider disabling IPv6 out-of-the-box. Having IPv6 work out-of-the-box is extremely important, despite the issues people are having now (which, to be honest, are due to broken IPv6 connectivity or broken DNS resolvers; not Ubuntu's fault, and not something we should fix).

[Jonzer (jonzer)]

If this is the attitude it is a sad day.

Two very high profile OS launches around the same time. Windows 7, and Ubuntu 9.10. Plug a Windows 7 box into my router, and everything is fine. Plug an Ubuntu 9.10 box into my router, and my internet experience is dog slow. But it's ok, i'm sure the general public will understand that Ubuntu 9.10 is still superior, it's actually Windows 7 that is broken, believe it or not!

Ubuntu 8.04 and 9.04 are also "broken", so if you want a good internet experience, perhaps downgrade to one of those fine distributions

I presume the official Ubuntu solution then is not to use routers with crap DNS resolvers on them. Everyone should be petitioning their ISP to send out routers to all customers that have built in DNS resolvers that correctly support ipv6! I'm sure the ISPs will definitely want to help out here, I'm sure it won't cost them much to replace all of these routers in Germany, France, the USA, Ireland, etc.

*sigh*

[Gregory Burd (gregburd)]

@ Martin Pit - Excellent points and I see where the RH solution is not the
best for some obvious technical reasons (Doh! I didn't think that one
completely through, my mistake). Thanks for pointing that out to me.
@ Bernhard Schmidt - I too would like to gently move people forward, to have
ISPs and users replace "broken" equipment and to transition everyone to
IPv6. The reality is that this is a transition period and we need to deal
with a situation where we have a foot in both worlds. Clearly this
situation causes pain on both sides of the fence. The libc6 patch you
mention seems like a reasonable approach, maybe there is even a way to
improve it, and one that might get us through this intermediate stage. Once
IPv6 capable hardware is in the majority and I can call up any old ISP and
ask for IPv6 service (or better yet, not even have to make that call) then
we're past the tipping point and we don't have to keep the workaround patch
around. We're not there today.
@ Jeremy Visser - You are passionate and persuasive in your arguments. This
is not an discussion about who's at fault or how much time we have before
"the apocalypse", we need to separate passion from this discussion and move
on to fixing the problem. I'm simply trying to find a way to bring people
over to the IPv6 world without creating and equally dispassionate anti-IPv6
crowd along the way (take @ Jonzer's response as an example).

There is no need to be absolutists on this issue one way or the other.
 There has to be a technical solution that lets us live in the middle
ground, lets just engineer around this and get past the political bickering.
 :-)

I hope this helps,

-greg

[Martin Pitt (pitti)]

Jonzer [2009-11-04 14:35 -0000]:
> I presume the official Ubuntu solution then is not to use routers with
> crap DNS resolvers on them.

It's not the official Ubuntu solution.

[Lasse Karstensen (lasse-karstensen)]

It will be very short sighted to disable IPv6 by default. Please do not consider it. If it still is feasible, +1 for the solution described in #111.

[Jonzer (jonzer)]

@ martin

Thanks for the clarification, apologies for the emotive nature of my contributions, but I find some of the other contributions here exasperating!

I presume Ubuntu 8.04 had IPV6 out of the box and it was fine, if it used the solution as suggested in #111 then that would appear to be the best way forward.

[Laurent Bigonville (bigon)]

Well when I was talking about blacklisting the ipv6 module (that is not a module anymore, but anyway) it was talking about a configuration option (like on RH system) to easily prevent the loading of the module on boot, but I was not talking about blacklisting it _by default_ (as I'm also a big IPv6 fervent too, die NAT, die!)

[arjun (ontont)]

Is this bug affecting every user of 9.10? I am not experiencing any lag. I have ipv4 on my home DSL.

[Gregory Burd (gregburd)]

So, it seems that there is growing consensus for #111 if not enabled by
default. But I wonder, is there no way to detect this situation and switch
between the two behaviors automatically? What if after a few failed AAAA
lookups the code says, "Ah ha! Network interface foo is on a link which
isn't quite ready for IPv6 because AAAA requests keep timing out which must
be annoying the heck out of the user of this system by now. From now on,
requests on the foo interface will skip the AAAA record lookup and go
directly for the A record instead." Do this after requests for AAAA records
continually timeout (it won't take long to see the pattern) and requests for
A records always succeed.

Note, this is just a crazy idea off the top of my head so feel free to tell
me, "Nope, sorry. Not a good solution." But if you do, could you also
explain to me your reasoning?

Again, just trying to find a happy middle ground, :-)

-greg

[Martin Pitt (pitti)]

Ah, Tollef shed some light on this. Ubuntu's glibc up to early Karmic had a patch applied which disabled unnecessary IPv6 DNS lookups (http://err.no/patches/glibc-only-lookup-ipv6-if-it-makes-sense.diff). This was dropped in Karmic to fix some IPv6 lookup issues (bug 239701, bug 374674), but also caused this regressions.

Mithrandir| so I suspect somebody should take my patch, refine it so it doesn't just reject v6 addresses (try again after processing if there no hits, allowing ipv6 then, or something like that)

I confirm that in my dapper to jaunty chroots "normal" applications like wget, apt-get, firefox etc. work just fine. I was just misled to think it'd be a kernel issue because I used "host" (which also times out).

http://bugs.debian.org/435646 is related to this, BTW.

[Martin Pitt (pitti)]

Would it be reasonably to reapply the patch as it is for karmic-updates (reopening the other two bugs) and refine the patch for lucid?

[Mark Schouten (mark-prevented)]

On Wed, Nov 04, 2009 at 06:09:41PM -0000, Martin Pitt wrote:
> Would it be reasonably to reapply the patch as it is for karmic-updates
> (reopening the other two bugs) and refine the patch for lucid?

Would by the best solution, if you ask me.

!mithrandir++

--
Mark Schouten <email address hidden>

[jordan.sc (jordanjsc)]

I inadvertently changed the status from "triaged" to "fix committed" and can't seem to change it back. I apologize for that. Can someone please correct? Thanks.

[Tollef Fog Heen (tfheen)]

]] Martin Pitt

| Ah, Tollef shed some light on this. Ubuntu's glibc up to early Karmic
| had a patch applied which disabled unnecessary IPv6 DNS lookups
| (http://err.no/patches/glibc-only-lookup-ipv6-if-it-makes-sense.diff).
| This was dropped in Karmic to fix some IPv6 lookup issues (bug 239701,
| bug 374674), but also caused this regressions.
|
| Mithrandir| so I suspect somebody should take my patch, refine it so it
| doesn't just reject v6 addresses (try again after processing if there no
| hits, allowing ipv6 then, or something like that)

If you want to emulate a broken DNS server (regardless of whether you
have access to one), add something like the following iptables rule:

sudo iptables -A OUTPUT -p udp --dport 53 \! -f -m u32 --u32 "0 >> 22 & 0x3C @ 8 >> 11 & 0x1F = 0 && 0 >> 22 & 0x3C@ 17 & 0xFF @ 18 & 0xFF @ 21 & 0xFF = 0x1c" -j DROP

then try to look up sixxs.net or any other second-level domain. It does
not matter whether this actually has AAAA records or not. Assuming you
don't have any IPv6 address with scope >= site, this should be slow on
9.10 and fast on 7.04 through 9.04. If you have any IPv6 address with
scope >= site, it will be slow on all variants. (The reason for the
two-level limitation is due to limitations in the u32 classifier.)

--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

[renegat (rozbujnik)]

By now probably the best solution is to modify grub as stated in
http://www.webupd8.org/2009/11/how-to-disable-ipv6-in-ubuntu-910.html

I don't know if blacklisting / disabling ipv6 is good or bad idea and I don't really care. I only want for my browser work as it is supposed to. I know that if edit grub and disable ipv6 everything works smoothly and "clean" installation don't let me browse internet in peace.

[no1lunchbox (rabbithattrick-deactivatedaccount)]

I'm a regular internet user, not a techie like most of you. I reported this problem as bug #472586 before I stumbled across this bug report. I liked Ubuntu - before the upgrade. Now I'm screwed and can't use it, and I'm going back to Windows. Why was this upgrade released with a problem like this? For a regular person like me this is totally wrong. I had to put in a lot of time and effort to learn how to install Ubuntu, configure it, use it, etc... For every regular person who wants to try and use Ubuntu this is strong message: Ubuntu is for techies only; regular people are not invited.

[robert leleu (robert-jean-leleu)]

Indeed the opendns is a good workaround.....however once applied, my
Thunderbird was no longer able to connect to smtp-msa.orange.fr, I had
to give it the numeric address 193.252.22.72

[Martin Pitt (pitti)]

For testing I reactivated the IPv6 patch in eglibc and upload it to my PPA:

  https://launchpad.net/~pitti/+archive/ppa/+packages

eglibc (2.10.1-0ubuntu15.0test1) karmic; urgency=low

  * debian/patches/series: Apply any/local-ipv6-lookup.diff again to fix
    painfully long timeouts on DNS resolution, if routers do not send an
    answer to AAAA (IPv6) DNS queries. With this patch those DNS requests are
    not sent in the first place if there are no routable IPV6 interfaces.
    (LP: #417757) This reopens LP #239701, #374674, so a full solution would
    need to fix the patch properly.

This should at least provide a bandaid for the people subscribed here to get sanity again.

It works well for me. However, due to reopening the other two bugs this isn't a complete solution, so the patch needs refinement.

[Martin Pitt (pitti)]

Oh, forgot: The amd64 packages are ready, i386 is still building. Should be available in < 1 hour.

[Kendall Price (kpauburn)]

Just wanted to say #19 fixed it for me.

[PsionicBOOM (psionicboom)]

Will anyone testing the patch mentioned in comments #121-130 please let us know how things are going? (=
Can anyone give a guesstimate as to when this patch will be released to the general public through karmic updates?
I've been trying to get my family and friends into Ubuntu, but I cannot recommend it to them until this very serious problem is resolved. I don't know anybody who would be fine to purchase a new modem/router for the sake of IPv6 in Ubuntu, at least not when everything, except Karmic, appears to be working fine. I believe no1lunchbox's comment (#127) hits the nail on the head.

[Phillip Spencer (phillip-spencer)]

In #132 SonicBOOM wrote "I don't know anybody who would be fine to purchase a new modem/router for the sake of IPv6 in Ubuntu, at least not when everything, except Karmic, appears to be working fine."

Like Bernard Bou (post #9) I am with Orange in France. I actually installed to a new Livebox last week and made sure everything was working fine in 9.04 before "upgrading" to 9.10 on three machines so even a brand new router does not help.

I realise the technically savvy posting above have put a lot of work into hacks and work-arounds but I endorse the comments in posts #14, #87, #88, #127 & #132. An average user wants things to work out of the box and 9.10 should not have been released like this. I am astounded this is not rated a "critical" bug.

[Bernhard Schmidt (berni)]

Pessimistic estimates say that the problem we suspect (a DNS proxy that
cannot handle AAAA queries) affects about 0.5% of the global userbase.
50% of those can easily be fixed with a firmware upgrade of their
router, which is usually a very good idea anyway.

I'm sure there are regions with higher rates if a large provider gave
out broken systems, but generally saying that all Ubuntu 9.10
installations are affected and everyone has to purchase new routers is
pure FUD.

For the record, this bug can easily be tested with the following tools

$ time dig -t a noc.sixxs.net +nofail | grep -e status -e A -e time

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26324
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 0
;noc.sixxs.net. IN A
;; ANSWER SECTION:
noc.sixxs.net. 85356 IN A 213.197.29.32
;; AUTHORITY SECTION:
;; Query time: 1 msec

real 0m0.025s
user 0m0.016s
sys 0m0.004s

$ time dig -t aaaa noc.sixxs.net +nofail | grep -e status -e AAAA -e time

; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14731
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 0
;noc.sixxs.net. IN AAAA
;; ANSWER SECTION:
noc.sixxs.net. 85342 IN AAAA 2001:838:1:1:210:dcff:fe20:7c7c
;; AUTHORITY SECTION:
;; Query time: 4 msec

real 0m0.024s
user 0m0.012s
sys 0m0.004s

You are only affected _if_ the first query is fast, but the second query
is very slow and/or shows something else than NOERROR in the first line
(SERVFAIL or REFUSED for example). If the DNS query is fine you should
not be affected.

[jordan.sc (jordanjsc)]

@Bernhard #134 wrote
"Pessimistic estimates say that the problem we suspect (a DNS proxy that
cannot handle AAAA queries) affects about 0.5% of the global userbase.
50% of those can easily be fixed with a firmware upgrade of their
router, which is usually a very good idea anyway."

Even if a firmware upgrade of my router fixed this issue (and it did not), I cannot, nor am I authorized to do firmware upgrades at the local coffee shop, at the airport and at my place of business, all of which are locations in which the IPv6 bug slows my network connections to a crawl. My network speeds are quite fast when I use Ubuntu 9.04, PCLinuxOS 2009.2, Windows XP, Vista and 7 on my "broken" router and at the other locations I mentioned above.

Please, let's get this issue resolved within Ubuntu as none of us has the power or influence or wealth to fix all the world's "broken" routers and ISPs.

[Guinnevere (marijke)]

I have a Livebox too, but the newest firmware has just been auto-installed, and I still experienced the same problem.

So, following up on #13 and #32 worked for me.

[Andy Langdon (andy-langdon)]

Do we actually know which routers are causing problems?
I have a Netgear WGR614v6 patched with latest firmware V2.0.19_1.0.19
Does this problem only affect a few routers that do not support IPv6, if so which ones work?
Which home routers support IPv6 at the moment? I seem to find it hard to find any. If I was to buy a new router (and it is about time) I would want it to be IPv6 enabled.

[Jeroen Massar (massar)]

@ Jordan.sc #135

> I cannot, nor am I authorized to do firmware upgrades at the local coffee shop
> at the airport and at my place of business

And generally these locations have neutered DNS already anyway, eg they only allow HTTP/HTTPS and generally only after authenticating (after being hijacked using DNS to their portal thingy). These locations (coffeeshops, hotels, airports, other generic hotspots etc etc) don't provide true internet anyway and are thus breaking already way too much protocol wise that there is nothing you can do about it.

The only thing you can do in those locations is to VPN out if you want to resolve that problem.

In the cases where DNS requests are not blocked (like in hotels and coffeeshops), you can setup pdns-recursor and use 127.0.0.1 in your resolv.conf

[Filofel (filofel)]

@Bernhard #134 wrote
"Pessimistic estimates say that the problem we suspect (a DNS proxy that
cannot handle AAAA queries) affects about 0.5% of the global userbase.

@Bernhard #134 wrote
50% of those can easily be fixed with a firmware upgrade of their
router, which is usually a very good idea anyway."

Do you have any verifiable and opposable reference on those figures?
Where is the study that produced those "pessimistic estimates" (...)
"0.5%" (...) "50% of those"(...) ?
What was the methodology used to produced those figures?
What was the sample?
How was it picked worldwide?

[damaan (jon-ekdahl)]

Thanks Jeroen. The pdns-recursor work around solved my issues with DNS-timeouts during package installation.

[Brownout (brownout)]

@Filofel #139
https://sites.google.com/site/ipv6implementors/conference2009/agenda/10_Lees_Google_IPv6_User_Measurement.pdf

[Per Heldal (heldal)]

This is as Jeroen and others have pointed out repeatedly, primarily an issue with broken resolvers/forwarders often part of cheap broadband/wlan routers. I just want to repeat that this isn't specific to IPv6, and emphasise that changes in karmic have made things worse.

Karmic always first tries to append the local/search-domain to requests. That is an important change from previous versions where search-domains are appended only when the client application only specifies the host-part.

In the last couple weeks I've come across number of different broadband CPE's (multiple brands) which have a DNS-forwarder that fails to forward nxdomain responses to the clients. In the past these clients would only experience delays for non-existent hosts. After upgrading to karmic they get a 20sec delay for every request. I.e. the user types "ubuntu.com" in the browser, karmic first tries "ubuntu.com.searchdomain" and sits waiting as the nxdomain-response is lost.

[Kendall Price (kpauburn)]

Is there a list of routers that work?

[Kendall Price (kpauburn)]

or what can a consumer look for to find a router that works?

[Kendall Price (kpauburn)]

Would there be any advantage to installing DD-WRT or some such to a problem router?

[Per Heldal (heldal)]

Slight update to #142.

I may have gotten protocol orders wrong in my analysis, or the dns-lookup-mechanism has been altered in the last libc update. The net result however, is still that every attempt by an application to connect to a v4-only host is preceded by a dns-request that result in a lost nxdomain response. Example: capturing DNS packets while connecting to a webserver with no v6-support reveal the following order of DNS queries and responses:

1. The client searches for an AAAA record. The DNS-server returns SOA with noerror.

2. The client tries the local variant of the AAAA record. The DNS-server returns NXDOMAIN which is lost in the faulty DNS-forwarder.

3. After timeout (20sec) the client goes looking for an A-record and succeeds.

Wouldn't it be preferable to prioritise differently? The client could send concurrent requests for A and AAAA, and drop the attempt on the local variant of AAAA when it gets one or more valid a-records.

[Jeroen Massar (massar)]

@ Per Heldal / #43

glibc 2.10 does AAAA+A queries in parallel. I don't know if it does that also for the ones in the search path, which seems to be the problem you are describing above.

Nevertheless, search path should (afaik) only be applied to non-FQDN hostnames (eg 'www' or 'hostname') not to anything containing already a domain (eg 'www.domain'). When this is done and users type eg 'ubuntu.org' then the local search path does not come into play.

The problem that I have seen with recursors (even one on an old dd-wrt as even dnsmasq had this issue quite some time ago, but if you don't update it doesn't get fixed ;) was that it simply completely silently dropped AAAA queries. It didn't even try to forward them. No response nothing. Even for www.ubuntu.org etc. Which is another error case from the scenario you describe, which is even funnier, as NXDOMAIN should definitely be handled properly (if they don't how the heck does that DNS recursor even work when somebody mistypes a URL ? :).

[Cromartie (cromartie)]

This is an Ubuntu bug, plain and simple. My DNS resolver didn't suddenly grow issues because I updated my OS. This is completely unacceptable for an operating system that touts itself as being "Linux for Humanbeings", where is the user-friendliness in breaking Internet connectivity for a lot of people? By all means, change it but give a fair warning; "If you are not using a very modern router do not upgrade as we don't care about you" and provide a one-click button after install and fixes the issue, rather than having people chasing their tails, disabling IPv6 etc. etc. Especially since most of those "fixes" are voodoo magic posted by clueless idiots -- fixing Firefox and no other application is not a fix.

[Jeroen Massar (massar)]

WORKING FIX (run as root of course):

apt-get install pdns-recursor resolvconf
echo "nameserver 127.0.0.1" >> /etc/resolvconf/resolv.conf.d/base
resolvconf -u

1) This will effectively install pdns_recursor which is a DNS recursor that simply works(tm) and it installs the resolvconf framework (which you most likely already have)
2) this echo sets the nameserver to 127.0.0.1, aka pdns
3) update resolv.conf with the above info

You should now have "nameserver 127.0.0.1" in /etc/resolv.conf and all your DNS queries should be super duper fast again (also because they get locally cached a bit ;)

Of course if you have a firewall between your host and the Internet where DNS servers exist the above might not work when DNS gets blocked...

I would almost suggest that Ubuntu make the above the default. The problem is though that little firewall thing, if that is there or you visit some netcafe or other network where only HTTP/HTTPS works, then indeed it does not work...

[Martin Pitt (pitti)]

Cromartie [2009-11-09 13:31 -0000]:
> This is an Ubuntu bug, plain and simple.

Folks, please calm down. As the bug status shows, it is an
acknowledged and open bug report, and there was no official statement
like "your router sucks, bad luck, go away". There is a new glibc
package for testing, and several workarounds were proposed, and
eventually we'll fix this in karmic-updates properly.

> Especially since most of those "fixes" are voodoo magic posted by
> clueless idiots -- fixing Firefox and no other application is not a fix.

Watch your language. The other people subscribed here are trying to
help! This conduct is unacceptable.

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[strel (d-mestetskiy)]

Thanks Jeroen, I will try this tonight, when I get home. I don't really care
if it comes as an update or a work around, as long as it works. So thanks
for the tip.

On Mon, Nov 9, 2009 at 7:08 AM, Martin Pitt <email address hidden> wrote:

> Cromartie [2009-11-09 13:31 -0000]:
> > This is an Ubuntu bug, plain and simple.
>
> Folks, please calm down. As the bug status shows, it is an
> acknowledged and open bug report, and there was no official statement
> like "your router sucks, bad luck, go away". There is a new glibc
> package for testing, and several workarounds were proposed, and
> eventually we'll fix this in karmic-updates properly.
>
> > Especially since most of those "fixes" are voodoo magic posted by
> > clueless idiots -- fixing Firefox and no other application is not a fix.
>
> Watch your language. The other people subscribed here are trying to
> help! This conduct is unacceptable.
>
> Martin
> --
> Martin Pitt | http://www.piware.de
> Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second
> delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>
> Status in “glibc” package in Ubuntu: Confirmed
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “glibc” source package in Lucid: Confirmed
> Status in “network-manager” source package in Lucid: Invalid
> Status in “glibc” source package in Karmic: Confirmed
> Status in “network-manager” source package in Karmic: Invalid
>
> Bug description:
> In Karmic, DNS lookups take a very long time with some routers, because
> glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no
> (non-loopback) IPv6 interfaces configured. Routers which do not repond to
> this cause the lookup to take 20 seconds (until the IPv6 query times out).
>

--
Dmitriy Mestetskiy

[strel (d-mestetskiy)]

I think the workaround worked, at least it looks like its gotten much
better.

On Tue, Nov 10, 2009 at 7:02 AM, Carropa <email address hidden> wrote:

> ** Changed in: glibc (Ubuntu Karmic)
> Status: Confirmed => Fix Released
>
> ** Changed in: glibc (Ubuntu Karmic)
> Status: Fix Released => Confirmed
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second
> delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>
> Status in “glibc” package in Ubuntu: Confirmed
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “glibc” source package in Lucid: Confirmed
> Status in “network-manager” source package in Lucid: Invalid
> Status in “glibc” source package in Karmic: Confirmed
> Status in “network-manager” source package in Karmic: Invalid
> Status in “glibc” package in Fedora: Confirmed
>
> Bug description:
> In Karmic, DNS lookups take a very long time with some routers, because
> glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no
> (non-loopback) IPv6 interfaces configured. Routers which do not repond to
> this cause the lookup to take 20 seconds (until the IPv6 query times out).
>

--
Dmitriy Mestetskiy

[Ricardo Fernández (koshrf)]

I've seen this behavior in quite few installation I have done; and saying "install a DNS server/Edit sysctl.conf" should not be the answer to fix the problem, not everybody have the skills required to do that, and it should not be a requirement at all. I don't think this could be fixed for karmic at this point but it should be fixed for lucid, otherwise the "common" users (not experts) will get tired of using "software" that doesn't work (They can't browse the web the way they like). So far I just disable IPv6 in firefox after installing ( network.dns.disableIPv6 ), thats quite enough to make most users happy.

[strel (d-mestetskiy)]

I think that given the fact that Ubuntu is free, its pretty good for an
operating system, even though sometimes it does come with hick ups. But I am
sure thats something, Ubuntu developers will strive to get ironed out in the
future. But like I said, its pretty good for being free.

On Sun, Nov 15, 2009 at 9:39 PM, Ricardo Fernández <email address hidden> wrote:

> I've seen this behavior in quite few installation I have done; and
> saying "install a DNS server/Edit sysctl.conf" should not be the answer
> to fix the problem, not everybody have the skills required to do that,
> and it should not be a requirement at all. I don't think this could be
> fixed for karmic at this point but it should be fixed for lucid,
> otherwise the "common" users (not experts) will get tired of using
> "software" that doesn't work (They can't browse the web the way they
> like). So far I just disable IPv6 in firefox after installing (
> network.dns.disableIPv6 ), thats quite enough to make most users happy.
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second
> delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>
> Status in “glibc” package in Ubuntu: Confirmed
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “glibc” source package in Lucid: Confirmed
> Status in “network-manager” source package in Lucid: Invalid
> Status in “glibc” source package in Karmic: Confirmed
> Status in “network-manager” source package in Karmic: Invalid
> Status in “glibc” package in Fedora: Confirmed
>
> Bug description:
> In Karmic, DNS lookups take a very long time with some routers, because
> glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no
> (non-loopback) IPv6 interfaces configured. Routers which do not repond to
> this cause the lookup to take 20 seconds (until the IPv6 query times out).
>

--
Dmitriy Mestetskiy

[enz (markus-enzenberger)]

I am experiencing the same problem. I am using one of Germany's largest Internet provider (T-Online) with a recent DSL/router combo that is handed out by them to their customers (Speedport W 303V, got is 6 months ago, updated to the newest firmware). So I think that probably a larger number of people than you think are affected by this bug (at least in Germany). I was able to bring my system into a usable state with the workaround of disabling IPv6 in the kernel by editing grub config file. But non-technical people, who cannot do things like that, will have a very bad experience and probably stop using Ubuntu, because the network is unusably slow for them. Please take this bug serious. I would say it is critical.

[Nech (gerard-guadall)]

#155 you have rightly.

It's a critical bug

[Wladimir J. van der Laan (laanwj)]

I have the same problem with my WRT54G router. I know this is a bug in the firmware not in Linux, but I do see this as a regression simply because I did not have the problem before. I simply don't need IPV6 yet, when I do, I'll think about upgrading my routers.

I now worked around it by installing the pdns-precursor, and setting 127.0.0.1 as DNS, but It would be very nice if a user-friendly workaround was added. Also, I can't use the internal DHCP-to-DNS functionality of my router anymore.

[khaktus (khaktus)]

The same issue for me on DSL. I was working on 9.04 for a few months without problem, now absolutely no internet connection on 9.10 - neither in Mozilla, nor in Synaptic. However, as I run dual boot with XP, on Windows, network connection is fine. Lucky me...

There are so many bugs launched and so many forums with dozens of workarounds for experts, creating new bugs... Can somebody write a step by step solution for newbie?

0. The network is not working. (Don't tell me to download something).
1. What to do/rewrite/edit/launch to make a temporary workaround - network somehow working (mozilla, synaptic, or whatever)
2. What to do to download/get somehow some official fix (not creating new bugs) ... if it already exists.

I'm not satisfied with answers like "upgrade NM with ppa" or "try pppoeconf"... it doesn't tell me much.

Please, someone, make a clear summary.

Do not send me another links to read endless forums... I have spent hours reading them... I'm used to the plurality of responses and solutions while working with Ubuntu, but having such a major problem, I'm loosing my patience. It is a major issue and newbies are not able to spend days reading forums, that offer so many workarounds, that need to be topped by other workarounds... I need to see what to do right now and not to read what hundreds of people tried... Where is Ubuntu's official solution?

Thank you

[strel (d-mestetskiy)]

This has been posted by Jeroen earlier, like days ago:

WORKING FIX (run as root of course):

apt-get install pdns-recursor resolvconf
echo "nameserver 127.0.0.1" >> /etc/resolvconf/resolv.conf.d/
base
resolvconf -u

1) This will effectively install pdns_recursor which is a DNS recursor that
simply works(tm) and it installs the resolvconf framework (which you most
likely already have)
2) this echo sets the nameserver to 127.0.0.1, aka pdns
3) update resolv.conf with the above info

You should now have "nameserver 127.0.0.1" in /etc/resolv.conf and all
your DNS queries should be super duper fast again (also because they get
locally cached a bit ;)

Of course if you have a firewall between your host and the Internet
where DNS servers exist the above might not work when DNS gets
blocked...

I would almost suggest that Ubuntu make the above the default. The
problem is though that little firewall thing, if that is there or you
visit some netcafe or other network where only HTTP/HTTPS works, then
indeed it does not work...

On Wed, Nov 18, 2009 at 4:45 AM, khaktus <email address hidden> wrote:

> The same issue for me on DSL. I was working on 9.04 for a few months
> without problem, now absolutely no internet connection on 9.10 - neither
> in Mozilla, nor in Synaptic. However, as I run dual boot with XP, on
> Windows, network connection is fine. Lucky me...
>
> There are so many bugs launched and so many forums with dozens of
> workarounds for experts, creating new bugs... Can somebody write a step
> by step solution for newbie?
>
> 0. The network is not working. (Don't tell me to download something).
> 1. What to do/rewrite/edit/launch to make a temporary workaround - network
> somehow working (mozilla, synaptic, or whatever)
> 2. What to do to download/get somehow some official fix (not creating new
> bugs) ... if it already exists.
>
> I'm not satisfied with answers like "upgrade NM with ppa" or "try
> pppoeconf"... it doesn't tell me much.
>
> Please, someone, make a clear summary.
>
> Do not send me another links to read endless forums... I have spent
> hours reading them... I'm used to the plurality of responses and
> solutions while working with Ubuntu, but having such a major problem,
> I'm loosing my patience. It is a major issue and newbies are not able to
> spend days reading forums, that offer so many workarounds, that need to
> be topped by other workarounds... I need to see what to do right now and
> not to read what hundreds of people tried... Where is Ubuntu's official
> solution?
>
> Thank you
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second
> delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>
> Status in “glibc” package in Ubuntu: Confirmed
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “glibc” source package in Lucid: Confirmed
> Status in “network-manager” source package in Lucid: Invalid
> Status in “glibc” source package in Karmic: Confirmed
> Status in “network-manager” source package in Karmic: Invalid
> Status in “glibc” package in Fedora: Confi...

[Ricardo Fernández (koshrf)]

#159

That is not a fix, it is a "Workaround", and maybe it will work at the house, but in a secure network (like a company network) it will not work and it will make Ubuntu useless.

The only real fix to this is disabling IPv6 support (for desktop version). In the real world pretty much no one uses IPv6, I'm not quite sure why Ubuntu would want this enabled by default, maybe they can do it when everybody start using IPv6, meanwhile, it is worthless.

[Laurent Bigonville (bigon)]

Well on company network they don't use home routers with buggy resolver...

[JoeKlein (jsklein)]

Ricardo,

> version). In the real world pretty much no one uses IPv6, I'm not quite
> sure why Ubuntu would want this enabled by default, maybe they can do it
> when everybody start using IPv6, meanwhile, it is worthless.

Actually, based on my studies, over 4 million people use IPv6
transport on the IPv6 Internet. Many are unaware of the fact. Based on
a study of the DNS root servers, there are 10-20 million systems
making AAAA request, without IPv6 transport. Lastly, there are about
250-400 million systems shipped in the last 4 years with IPv6 services
enabled.

On the security side, if your firewalls and routers (and other IA
gear) are unable to process IPv6 packets, they are also unable to
protect against native and tunneled IPv6 attacks. So again the issue
is not that Ubuntu needs IPv6 disabled, the issue is people need to
update their infrastructure to IPv6 enabled devices to properly
protect their networks.

Link to presentations: http://sites.google.com/site/ipv6security/
Link to Video: http://www.ustream.tv/recorded/2504950/

Joe Klein

On Wed, Nov 18, 2009 at 3:19 PM, Ricardo Fernández <email address hidden> wrote:
> #159
>
> That is not a fix, it is a "Workaround", and maybe it will work at the
> house, but in a secure network (like a company network) it will not work
> and it will make Ubuntu useless.
>
> The only real fix to this is disabling IPv6 support (for desktop
> version). In the real world pretty much no one uses IPv6, I'm not quite
> sure why Ubuntu would want this enabled by default, maybe they can do it
> when everybody start using IPv6, meanwhile, it is worthless.
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a member of IPv6 Task
> Force, which is a direct subscriber.
>
> Status in “glibc” package in Ubuntu: Confirmed
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “glibc” source package in Lucid: Confirmed
> Status in “network-manager” source package in Lucid: Invalid
> Status in “glibc” source package in Karmic: Confirmed
> Status in “network-manager” source package in Karmic: Invalid
> Status in “glibc” package in Fedora: Confirmed
>
> Bug description:
> In Karmic, DNS lookups take a very long time with some routers, because glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no (non-loopback) IPv6 interfaces configured. Routers which do not repond to this cause the lookup to take 20 seconds (until the IPv6 query times out).
>

[Ricardo Fernández (koshrf)]

#162

Yeah I'll tell my ISP to upgrade their Hardware because my Ubuntu doesn't work. That will surely work. *sigh*

Also they are over 600+ million internet users around the world (maybe more?), 4 millions using IPv6 is around 0.7% of the internet users.. so yeah, pretty much no one. And how many of thoses 4 millions use Ubuntu ? let me guess it should be below 0.1%, so, making IPv6 enabled by default will only be used by 0.1% (below that number to be honest). Default should be for the other 99.9%.

Yes they are a lot of systems shipped with IPv6, the point is thoses systems _works_ and Ubuntu 9.10 doesn't (not the whole distro of course, just this "cant use internet thing").

You can't ask everybody to update their whole network hardware to IPv6 to make Ubuntu works, thats quite hard to do, it is easier just to switch distro (Like Fedora, Debian, anything) or just Windows 7, and thats what most people will do when they try to use their favorite distro (Ubuntu) and notice the slow internet.

On the security side, just because a firewall/router doesn't process IPv6 it doesn't mean you can use some sort of attack, you can't attack something that doesn't understand you, at most you can just try to DoS it, so I'm quite sure no one is afraid of a IPv6 attack over a IPv4 network.

[Laurent Bigonville (bigon)]

From libc.NEWS file:
  Starting with version 2.9-8, unified IPv4/IPv6 lookup have been enabled
  in the glibc's resolver. This is faster, fixes numerous of bugs, but is
  problematic on some broken DNS servers and/or wrongly configured
  firewalls.

  If such a DNS server is detected, the resolver switches (permanently
  for that process) to a mode where the second request is sent only when
  the first answer has been received. This means the first request will
  be timeout, but subsequent requests should be fast again. This
  behaviour can be enabled permanently by adding 'options single-request'
  to /etc/resolv.conf.

Could you try adding 'options single-request' in /etc/resolv.conf ?

[Laurent Bigonville (bigon)]

an other option will be to install nscd (apt-get install nscd) that will cache dns requests locally (windows and macos does the same IIRC)

[JoeKlein (jsklein)]

Ricardo,

Actually, if you are having the problem with your ISP, I suspect a
large number of technical and non-technical people on the same ISP are
having timeout problems with other operating systems and applications.
The difference is that you are a knowledgeable user that has
identified and knows how to solve the problem. E-mail me directly with
the ISP name and I would be willing to contact them about the problem.

Actually my studies show that 75% of device on the internet today have
or could have IPv6 enabled. I was only sharing the 'conservative
numbers'. The biggest jump in IPv6 users in the last 12 months was due
to BitTorrent clients on windows system enabling IPv6 tunneling
(Teredo).

Over the next 4 years, all of the major ISP's are expected to move
quickly to dual stack and IPv4 over IPv6 transport. Comcast should be
the first, followed the the crowd of cable provides then telcom
providers. So 0.1% might be small today, but should get to +1% by the
end of 2010.

Again on the security side, there are many attacks which have been and
are accruing on unsuspecting networks which have IPv6 enable but do
not have proper IPv6 security controls (IPv6 capable firewalls,
routers, IDS,...). The first attack was back in 2002 on a debian
distro that did not have IPv6 firewall enabled, but did have the IPv4
enabled. So continue to believe there are no IPv6 security attacks,
the same way as many people said back in 2002, there are no wifi
attacks and WEP is secure!

BTW, I am not having the problem, I loaded Miredo on my Ubuntu system
and it seem to be fine. With that in mind, have we considered just
having Miredo installed by default on the Distro?

Joe

On Wed, Nov 18, 2009 at 5:39 PM, Ricardo Fernández <email address hidden> wrote:
> #162
>
> Yeah I'll tell my ISP to upgrade their Hardware because my Ubuntu
> doesn't work. That will surely work. *sigh*
>
> Also they are over 600+ million internet users around the world (maybe
> more?), 4 millions using IPv6 is around 0.7% of the internet users.. so
> yeah, pretty much no one. And how many of thoses 4 millions use Ubuntu ?
> let me guess it should be below 0.1%, so, making IPv6 enabled by default
> will only be used by 0.1% (below that number to be honest). Default
> should be for the other 99.9%.
>
> Yes they are a lot of systems shipped with IPv6, the point is thoses
> systems _works_ and Ubuntu 9.10 doesn't (not the whole distro of course,
> just this "cant use internet thing").
>
> You can't ask everybody to update their whole network hardware to IPv6
> to make Ubuntu works, thats quite hard to do, it is easier just to
> switch distro (Like Fedora, Debian, anything) or just Windows 7, and
> thats what most people will do when they try to use their favorite
> distro (Ubuntu) and notice the slow internet.
>
> On the security side, just because a firewall/router doesn't process
> IPv6 it doesn't mean you can use some sort of attack, you can't attack
> something that doesn't understand you, at most you can just try to DoS
> it, so I'm quite sure no one is afraid of a IPv6 attack over a IPv4
> network.
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second delays by de...

[Ricardo Fernández (koshrf)]

#166

8.04, 8.10, 9.04 (Ubuntu), Windows XP, Windows Vista, Windows 7, Fedora 9/10/11, Debian Lenny (and sid), all thoses Distro/OS works like a charm in my network, Ubuntu 9.10 (desktop-clients in my network) have a real slow "internet", so calling an ISP because a Linux distro doesn't work is not a real solution.

Please understand this is an Ubuntu 9.10 problem it is not an ISP or router or hardware or anything else but Ubuntu, simple as that. This isn't a forum about how cool IPv6 is and why we should use it, this is a bug tracking system where people report "Hey my internet doesn't work", so asking those people to upgrade their external hardware/software isn't a solution or a fix.

[Laurent Bigonville (bigon)]

Well actually it IS an issue with the modem/router.

The previous version of ubuntu (windows and maybe OSX caches dns request or work differently) workaround this router bug. I seems that the workaround was causing other issues (see the 2 bugs mentioned earlier).

Did you try the solutions I proposed yesterday?

> Could you try adding 'options single-request' in /etc/resolv.conf ?

or

> an other option will be to install nscd (apt-get install nscd)

See http://udrepper.livejournal.com/20948.html (DNS NSS improvement paragraph)

[Ricardo Fernández (koshrf)]

#168

I tried "options single-request' in /etc/resolv.conf" yesterday and so far it doesn't work for me, but i need 2-3 days to get feedback from my ubuntu users in the network to see how it goes. The problem with this, is that the resolv.conf will change every time i change dhcp server, so I need to make the option permanent, I can't ask my users to edit system files to fix this, they have no idea how to use vi or sudo (to make the option permanent, it is not only editing /etc/resolv.conf).

I haven't tried the nscd one, I only use it to cache the ldap info, and I'm not quite sure I would like to have a local cache of my dns in each pc but thats me.

[Carlin (carlin-public)]

What is the status of this bug? Above it looks like it was fixed; "status: Confirmed → Fix Released", but at the top it only says the status is "Confirmed". If it is fixed, is the fixed package available now via Update Manager? Or do I have to enable other repos (Testing, Proposed, et al.) to get the fixed package?

I've used so many different work-arounds that I'm sure what worked and what didn't. Overall I'd like to fix this properly so does anyone know of a router or a list of routers that don't suffer from this problem? I am using a DSL-G604T on wireless.

[Martin Pitt (pitti)]

Carlin [2009-11-20 7:01 -0000]:
> What is the status of this bug? Above it looks like it was fixed;
> "status: Confirmed → Fix Released", but at the top it only says the
> status is "Confirmed".

It was wrongly closed and reopened.

> If it is fixed, is the fixed package available now via Update
> Manager? Or do I have to enable other repos (Testing, Proposed, et
> al.) to get the fixed package?

As I wrote earlier, there is a package from a PPA which reapplies a
patch to glibc to fix this. But it reopens two other bugs. Laurent's
suggestion seems better, but I can't test it here (I'm in Dallas at
UDS). I can test it next week when I'm back home, or someone else test
Laurent's suggestion first.

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[dhenry (tfc-duke)]

"options single-request" in /etc/resolv.conf works, but it has two drawbacks:

1) The first time you want to access a website, it will be slow.

2) when the process terminates, i.e. when you close the application or when it ends by itself (web browser, apt-get, etc.), you lose all your "fast DNS resolving" stuff. When you reopen the browser, it will be slow again. Take apt-get: you run it for updating or installing some packages, and it will close itself at the end of the operation, and all is lost. Next time you run apt-get, it has to resolve everything again with the slow method...

[muhalifsirin (alperense)]

this bug also cripples my system as well. i don't know why none of the workarounds gave me a consistent solution. They fix the system for a while, but then for some reason I go back to square one(I guess I install something, or something changes configuration files). This is the biggest problem I had with ubuntu in 2 and a half years I have been using. Disabling ipv6 must not be the solution.
The only thing that works well now for me is torrents. Updating repositories is a pain now

[muhalifsirin (alperense)]

Here is one more thing:
Because of this bug I have gone back to jaunty. There, when I added a repository for firefox, it upgraded my xulrunner and installed sth called xulrunner-gnome-support. Then I began to have the same problem. After downgrading xulrunner to whatever in the official repos my internet became OK.
Ping times went back from 2500 ms to 60ms again

[Andy Langdon (andy-langdon)]

I've now installed Linux Mint 8.0 and I have had no problems with browser speed or DNS lookups.

[skyydiver (mfarmee)]

I'm really surprised not to have seen a real fix for this yet.

[Schreiber (schreiber07)]

I could not use google-earth on kubuntu 9.10 64 - fresh install. The program starts but I get a server-connect-error. I tried for days different solutions. But nothing works.

Problem completely solved by specifying Opendns as my DNS servers (https://store.opendns.com/setup/device/ubuntu/). I did not have this problem when running Jaunty or hardy.

[Jonzer (jonzer)]

It is nothing short of incredible that this issue still exists.

Very very sad.

[Mike Cummings (mikec7x)]

I am using the workaround - adding "ipv6.disable=1" to the GRUB_CMDLINE_ LINUX_DEFAULT line in /etc/default/grub- and this works well for me. After the recent updates to the LIBIND packages , i tried removing this entry, and the problem still exists, so those were no help. Still waiting for a resolution.

[Mark Schouten (mark-prevented)]

On Fri, 2009-12-11 at 11:30 +0000, Jonzer wrote:
> It is nothing short of incredible that this issue still exists.
>
> Very very sad.
>

I agree with everyone that this issue should be fixed in Ubuntu,
somehow. But has anyone even bothered to complain at their ISP or modem
provider, who are the actual cause of this issue?

DNS Resolution isn't some small part of a operating system, you cannot
'just fix this here a bit' and see what happens.

So, give the developers some time to fix this issue and do it right, and
don't forget to complain at your own suppliers.

[robert leleu (robert-jean-leleu)]

Could someone write a template letter (I would translate to french).
because to be effective such a letter should presumably refer to some
norms, standards or conventions, which most of us are not aware of.

 Mark Schouten skribis (esperanto estas la unua internacia lingvo):
> On Fri, 2009-12-11 at 11:30 +0000, Jonzer wrote:
>> It is nothing short of incredible that this issue still exists.
>>
>> Very very sad.
>>
>
> I agree with everyone that this issue should be fixed in Ubuntu,
> somehow. But has anyone even bothered to complain at their ISP or modem
> provider, who are the actual cause of this issue?
>
> DNS Resolution isn't some small part of a operating system, you cannot
> 'just fix this here a bit' and see what happens.
>
> So, give the developers some time to fix this issue and do it right, and
> don't forget to complain at your own suppliers.
>

[Mark Schouten (mark-prevented)]

On Sat, 2009-12-12 at 09:51 +0000, robert leleu wrote:
> Could someone write a template letter (I would translate to french).
> because to be effective such a letter should presumably refer to some
> norms, standards or conventions, which most of us are not aware of.

I think a complaint from a 'normal' user (not a nerd or professional
ipv6 evangalist) might work better. The nerd always wine, the response
is always 'we never get requests from normal users'.

[Filofel (filofel)]

That might do it on the (very) long term, and if this is what the problem is, it has to be done anyway.

But I don't see how that would solve the problem at hand *NOW*.
I'm experiencing the problem and I'm connected behind a recent Linksys BEFVP41 VPN router, downstream my DSL box, that I use to isolate / protect me from the network. Linksys is Cisco, btw, the king of the routing world. And the box is new, bought in 2009. Latest firmware, april 2009.
http://www.linksysbycisco.com/US/en/support/BEFVP41/download
And I do have the problem.
There are myriads of people and small and large businesses in the same situation.
Expecting that all those are going to rush buying a new router (and for many, finding someone to config it for them) because Linux misbehaves (even if it "rightly" misbehaves) seems a wild goose chase to me.
Many would rather conclude that Windows is still the king of the hill after all and change the piece of software that reveals the problem for the one that doesn't.

[Sune Woeller (sune-woeller)]

This should be moved to critical - makes karmic unusable.

[Pau Iranzo (paugnu)]

Exactly, I couldn't agree more, this should be moved to critical. We're having serious problems at the office and are thinking to change to OpenSUSE or other linux distro.

[Antonio Navarro (anavarrog)]

Critical please.

2009/12/17 Pau Iranzo <email address hidden>

> Exactly, I couldn't agree more, this should be moved to critical. We're
> having serious problems at the office and are thinking to change to
> OpenSUSE or other linux distro.
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second
> delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “glibc” package in Ubuntu: Confirmed
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “glibc” source package in Lucid: Confirmed
> Status in “network-manager” source package in Lucid: Invalid
> Status in “glibc” source package in Karmic: Confirmed
> Status in “network-manager” source package in Karmic: Invalid
> Status in “glibc” package in Fedora: Confirmed
>
> Bug description:
> In Karmic, DNS lookups take a very long time with some routers, because
> glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no
> (non-loopback) IPv6 interfaces configured. Routers which do not repond to
> this cause the lookup to take 20 seconds (until the IPv6 query times out).
>
> *** PLEASE DO NOT COMMENT ON THIS BUG unless you have something
> constructive to say. Everything that can be said has already been said, and
> if you comment, you are just adding noise. Please let those that actually
> know what they are doing concentrate on fixing this bug from now on. ***
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/417757/+subscribe
>

[Pconfig (thomas9999)]

For your interest. I tried Opensuse 11.2 on the same hardware and it seems like they're suffering from the same issue. They deliver Firefox with disableipv6=true by default though.

Here is their bugreport: https://bugzilla.novell.com/show_bug.cgi?id=539869

[jclere (jclere-redhat-bugs)]

I have change the dns proxy option to off in my router (Netopia-3000) that fixes the problems on my boxes (f11 and f12).

[Matthias Klose (doko)]

fixed in lucid

[rossco (ross-jemima)]

Here is (hopefully) a "constructive" suggestion. Since IPv6 is going to be the future of the internet it makes sense to try and support it. Obviously thought there is going to be a transition period where support for it is not going to be always available (as is the case at the moment where most home routers do not support it). So would it not make sense for Ubuntu to check for IPv6 connectivity/services either at boot time or as network interfaces go up and down or periodically? If there is IPv6 connectivity/services it should use it otherwise revert to IPv4. That way this problem might be more future proof.

[Martin Pitt (pitti)]

Accepted eglibc into karmic-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[cheerios_ (jussiava)]

I noticed using torrent software like qbittorrent (1.5.6) on Karmic (but not on Jaunty) would kill all network functionality for _other_ software for some minutes, and when up afterwards resolving connections the network would be very sluggish. No such problem with Transmission (better default settings?). This coupled with the ipv6 dns problems...

[Martin Pitt (pitti)]

I tested the karmic-proposed packages on my wife's karmic PC.

I removed the nameserver workaround, rebooted, and confirmed that I got the long delays back. Then I upgraded to the new libc, rebooted again, and confirmed that name resolution was fast again.

[Bernhard Schmidt (berni)]

The karmic-proposed package also seems to work as expected when you have had flawless IPv6 before. DNS resolution and connecting to IPv6-enabled services works fine.

[cheerios_ (jussiava)]

I tried the Karmic patch, then an upgrade to Lucid. DNS is fast when it works, but I often lose all DNS functionality for all applications for several minutes randomly. At startup a simple wget in shell will work fine, yet opening chrome/firefox (with multiple tabs) will block dns calls for a long time.

This fix is towards the right direction, but not complete to mitigate all DNS-issues by any means based on my usage (ZyXEL router).

[Martin Pitt (pitti)]

cheerios_ [2010-01-04 20:06 -0000]:
> This fix is towards the right direction, but not complete to mitigate
> all DNS-issues by any means based on my usage (ZyXEL router).

It's the very same patch that we have carried since dapper or even
earlier. Did you have such problems before? Perhaps you still have
another workaround in place which now interferes with the new patch?

[Brad Peters (thefubar)]

I tested the karmic-proposed package on my work machine, however, I'm still having intermittent issues with this. Since this is a work machine, I'm at the mercy of our IT department as far as routers and DNS resolution goes. I've disabled all workarounds, installed the new package and rebooted. It worked for about five minutes before going back to the same behavior I was seeing previously.

All throughout this bug report, a lot of people (developers, I'm assuming) have commented that the routers are broken and it shouldn't be on Ubuntu to fix the problem. I'm sorry, but if I go from a working version of Ubuntu (Jaunty) and upgrade, I expect everything to work equally as well, if not better, than the previous release.

[Mark Schouten (mark-prevented)]

On Tue, 2010-01-05 at 12:51 +0000, Brad Peters wrote:
> I tested the karmic-proposed package on my work machine, however, I'm
> still having intermittent issues with this. Since this is a work
> machine, I'm at the mercy of our IT department as far as routers and DNS
> resolution goes. I've disabled all workarounds, installed the new
> package and rebooted. It worked for about five minutes before going
> back to the same behavior I was seeing previously.

Wouldn't tcpdumps come in handy in these cases so we see what actually
happens on the network stack?

[tai133 (tairus166)]

FWIW:
1. I'm having this problem intermittently
2. When it occurs on my Karmic box, it also occurs on a Windows 7 laptop
(wireless) on my home network.
3. Other computers on the network (XP both wired and wireless) remain
functional DNS wise, except that sometimes my web request to the router
config page is forwarded to the Karmic box (see 4b below). Router reboot
required to fix.
4. Upgrading my router (D-Link DIR-655) firmware seemed to help, but
after three days of flawless function, the problem occurred again. I
found that I had not checked the (new with latest firmware) 'Advanced
DNS' checkbox in the router config. I checked that and rebooted the
router. All is well since then, <24 hours.
4a. Because of domestic issues, I shut down all computers and the home
network at night and reboot each morning.
4b. Port 80 is forwarded from the router to apache on my Karmic box.
5. My upgrade from Jaunty to Karmic (Kubuntu) is seriously broken. Only
terminal shutdowns work (Bug#418509) K3b ruins disks but doesn't burn
them and no media players function properly except vlc and totem.
6. A work machine with similar configuration upgraded to Karmic without
incident.

Brad Peters wrote:
> I tested the karmic-proposed package on my work machine, however, I'm
> still having intermittent issues with this. Since this is a work
> machine, I'm at the mercy of our IT department as far as routers and DNS
> resolution goes. I've disabled all workarounds, installed the new
> package and rebooted. It worked for about five minutes before going
> back to the same behavior I was seeing previously.
>
> All throughout this bug report, a lot of people (developers, I'm
> assuming) have commented that the routers are broken and it shouldn't be
> on Ubuntu to fix the problem. I'm sorry, but if I go from a working
> version of Ubuntu (Jaunty) and upgrade, I expect everything to work
> equally as well, if not better, than the previous release.
>

--
Tai Sines
"Share your strengths, not your weaknesses." -- Yogi Bhajan

[Martin Pitt (pitti)]

Reopening for lucid, this regressed with the recent eglibc merge.

[Launchpad Janitor (janitor)]

This bug was fixed in the package eglibc - 2.10.1-0ubuntu16

---------------
eglibc (2.10.1-0ubuntu16) karmic-proposed; urgency=low

  [ Martin Pitt ]
  * debian/patches/series: Apply any/local-ipv6-lookup.diff again to fix
    painfully long timeouts on DNS resolution, if routers do not send an
    answer to AAAA (IPv6) DNS queries. With this patch those DNS requests are
    not sent in the first place if there are no routable IPV6 interfaces.
    (LP: #417757) This reopens LP #239701, #374674, so a full solution would
    need to fix the patch properly.

  [ Matthias Klose ]
  * patches/any/cvs-malloc-check.diff: new patch from upstream to fix bugs
    with MALLOC_CHECK. Closes: #557158. LP: #425723.
 -- Matthias Klose <email address hidden> Thu, 24 Dec 2009 11:16:36 +0100

[Brad Peters (thefubar)]

Can anyone confirm this is now fixed? I reverted back to Jaunty at work until this was confirmed fixed. I see the fix was released, but I want to be certain before I incur more downtime on my box.

[robert leleu (robert-jean-leleu)]

I use Karmic, and to overcome the bug I used the four numbers IP of my FAI SMTP
I just checked that returning to the normal smtp name I obtain a correctly fast response.

Also for internet browsing I had disabled the ipv6 in the configuration of the wifi lan.
I just activated it again, and checked some pages: seems normal.....

Checking in the package manager I find that eglibc-source is not installed , and that the version of this package is proposed as 2.10.1-0ubuntu15

So I just understand nothing to what is happening.....

[dhenry (tfc-duke)]

It seems to be OK for me too, so I think that yes, it's really fixed :)

[robert leleu (robert-jean-leleu)]

I concluded too fast.....I just experimented slow connection to smtp, so I put back the "IP" smtp, which worked fine......
I stay with ipv6 activated, and will report if I experiment delays in browsing....

[jordan.sc (jordanjsc)]

I' ve been experiencing intermittent slow browsing in Firefox 3.5 with IPv6 enabled. It's much better than it was but there are periodic 3 to 4 second delays in loading web pages. Additionally, preset radio stations in Rhythmbox sometimes take 3 to 7 seconds to load. Update manager is slower to start as well. When I switch over to Jaunty (with Firefox 3.5 installed) on the same machine, I see none of these issues.

[RobertL (robert-loehning)]

My Internet connection via cable and router is still as slow as before. Downloads are about 100kps were the very same machine (Fujitsu Siemens Amilo A notebook) running Windows XP does about 600kps.
Which information can I provide to help you help me?

[Martin Pitt (pitti)]

RobertL [2010-01-12 21:20 -0000]:
> My Internet connection via cable and router is still as slow as
> before. Downloads are about 100kps were the very same machine
> (Fujitsu Siemens Amilo A notebook) running Windows XP does about
> 600kps.

This is totally unrelated. The symptom of this bug is that it takes
some 10 seconds to establish a connection (due to slow name
resolving). It does not affect download/upload speed at all.

[jordan.sc (jordanjsc)]

I suspected user (as in me) error after my report yesterday. I ran all updates on my laptop running Ubuntu Netbook Remix 9.10 (Karmic) and tested browsing with ipv6 enabled, playing the radio in Rhythmbox and checking for updates. Speeds seemed normal and similar to what I was seeing using 9.04. I apologize for yesterday's erroneous results.

[jordan.sc (jordanjsc)]

To clarify my terminology per Martin Pitt, I should have said that there are no longer any delays when browsing, using Rhythmbox and checking for updates. The bug appears fixed on my netbook running UNR 9.10/Karmic. I will test again on my desktop machine running Karmic.

[Filofel (filofel)]

After applying the updates and removing the workarounds I had used before, the problem seems to be gone on my machines.
Thanks, folks!

[Martin Pitt (pitti)]

Please don't change bug states without explanation. As I explained earlier, it seems that a later lucid version dropped the patch again. I get the problem again in current lucid.

[Matthias Klose (doko)]

no, the patch is not dropped. Martin, I would appreciate it if you could have a look at this again.

[Micah Gersten (micahg)]

Please don't change bug states without explanation.

[Michael McLeish (michael-mcleish)]

This problem is not fixed on my computer, unfortunately. I'm a first time poster here - been watching this bug for a long time. It appeared to be fixed for a day or two, but I still have the long delays on page requests. Also, I've tried downloading large files via firefox and I've also been experiencing delays in the start of the downloads, as well as long interruptions. Watching this on the system monitor confirms, when this bug acts up, there is zero incoming transmissions via wireless. The only work around I've been able to get to work is to open a new window in firefox, and hit stop + refresh until it 'unlocks' itself and the incoming transmissions restart.

Also, I don't seem to have the "eglibc" package installed on my version of Karmic (Desktop, 64 bit). Is this embedded in another package? I've found "eglibc-source" in the repos - am I supposed to install this?

Sorry for the questions, but this has been a most frustrating bug and I'd like to see some resolution. Thanks for all your hard work folks!

[Jeremy Visser (jeremy-visser)]

On Mon, 2010-01-18 at 23:31 +0000, Michael McLeish wrote:
> This problem is not fixed on my computer, unfortunately. I'm a first
> time poster here - been watching this bug for a long time. It appeared
> to be fixed for a day or two, but I still have the long delays on page
> requests.

Perhaps delayed AAAA requests are not your only problem. If you're
adventurous, you could have a look at what seems to be holding it up in
Wireshark. (Though that's for another bug report.)

> Also, I don't seem to have the "eglibc" package installed on my version
> of Karmic (Desktop, 64 bit). Is this embedded in another package?

eglibc is just a source package name: the binary packages that result
from it that appear in your package manager are not necessarily (but
usually are, but in this case, are not) named the same.

http://packages.ubuntu.com/source/eglibc

So the binary version of eglibc is called libc6 in your package manager.

[Deebs (deebs)]

I have also been watching this bug for a long time and just chirping in to say this issue is now fixed for me on karmic.

Thanks guys.

[robert leleu (robert-jean-leleu)]

Generally speaking this bug is not solved.
I just had a general crash, and installed a new Ubuntu Karmic.....Whaouh! If I were a newbie I'd erased the whole stuff....

I, very slowly, googled to patches, choose the "Grub" patch......and it works for Firefox, Thunderbird and apt-get.....

But the Ubuntu version now distributed has to be considered as bugged......and since installation are usually made from downloaded CD, and requires connection to internet, why not write the "good" line in Grub config, and at the first connection check the internet provider against a list of providers, and deliver a "good new" message such as:

"Hello, you're happy to have such an internet provider....he uses uptodate technology, and we can set your Ubuntu to use it. Paay attention if you change your provider, or connect from other sites: you may experience slow connection. Please consult such a page for further explanations", and automatically modify the Grub configuration to use ipv6

[cheerios_ (jussiava)]

I couldn't get wireless to work properly on my box after Jaunty->Karmic upgrade. Fresh Karmic install works better. I still get long "Resolving host..." timeouts (even against google.com) now and then, but mostly things work. Not sure what went wrong during the upgrade to cause all the problems.

[Yermo (yml-yml)]

I can confirm that there is something else, beyond IPV6 lookups, that is causing timeouts.

Kubuntu 9.10 with "libc6-i686 2.10.1-0ubuntu16 (i386)" which, if I'm not mistaken, contains the IPV6 fix.

Dell Nseries desktop box. Completely stock.

Using fixed IP behind a D-Link 707 consumer grade firewall. DNS server running on a CentOS 5 box on the other side of the firewall.

IPV6 disabled in /etc/default/grub using
   GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1 quiet splash"

IPV6 disabled in firefox as well.

Regardless of whether I use a local caching nameserver, pdns-recursor, or a name server on a CentOS 5 box, the timeout phenomenon is the same. From a gut feel point of view, I'd say it's worse when using the local caching nameserver.

Running WinXP in VMWare Workstation 7 on the same physical machine with no timeouts. (i.e. While it's timing out in firefox I switch into XP and pull the same page. Comes up instantly.)

My feeling is we're dealing with some kind of race condition in the resolver library because it happens when doing a number of simultaneous DNS requests.

Open 20 or so tabs in firefox to various websites with varying load times. Close firefox. Reopen firefox and at the same time open Thunderbird. In this scenario Thunderbird will timeout 100% of the time as will the majority of tabs in firefox.

It's intermittent. In FF, I can have one tab that's loading and other that's timing out. Sometimes it will pull the main html page but timeout on graphics or CSS.

I would like to be able to run Kubuntu instead of having to fallback to CentOS 5. I am willing to lend a hand to track this down if you would like to give me some tests to run. I can reproduce this problem consistently.

[Yermo (yml-yml)]

My bad. It looks like what I described above is covered by this bug: https://bugs.launchpad.net/ubuntu/+source/nss-mdns/+bug/94940

Editing /etc/nsswitch.conf and changing

   hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4

to

   hosts: files dns

looks like it "fixes" this issue.

[manu (manojkg3)]

I think it is better to notify that "Ubuntu is not suitable for wireless Network" in home page of Ubuntu. So it help to avoid the installation for the wireless users.
Thank you....

[dhenry (tfc-duke)]

It's fixed in Karmic. But still present on Lucid. Please fix it before Lucid release, not like for Karmic (fix was too late).

@manu: it has nothing to do with wireless. It's the same problem with wired connection.

[manu (manojkg3)]

Dear friend,

I think that I am using ubuntu vedio studio 9.10 is Karmic. So how can I fix
this? using “eglibc” package....

manu

On 7 February 2010 14:59, dhenry
<<email address hidden><tfc.duke%<email address hidden>>
> wrote:

> It's fixed in Karmic. But still present on Lucid. Please fix it before
> Lucid release, not like for Karmic (fix was too late).
>
> @manu: it has nothing to do with wireless. It's the same problem with
> wired connection.
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second
> delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “eglibc” package in Ubuntu: Triaged
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “eglibc” source package in Lucid: Triaged
> Status in “network-manager” source package in Lucid: Invalid
> Status in “eglibc” source package in Karmic: Fix Released
> Status in “network-manager” source package in Karmic: Invalid
> Status in “glibc” package in Fedora: Confirmed
>
> Bug description:
> In Karmic, DNS lookups take a very long time with some routers, because
> glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no
> (non-loopback) IPv6 interfaces configured. Routers which do not repond to
> this cause the lookup to take 20 seconds (until the IPv6 query times out).
>
> *** PLEASE DO NOT COMMENT ON THIS BUG unless you have something
> constructive to say. Everything that can be said has already been said, and
> if you comment, you are just adding noise. Please let those that actually
> know what they are doing concentrate on fixing this bug from now on. ***
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/417757/+subscribe
>

[dhenry (tfc-duke)]

I thought it was in karmic-updates, or maybe karmic-proposed repository.
Which version of libc6 package do you have? I have 2.10.1-0ubuntu16 from karmic-updates.

[manu (manojkg3)]

how can I find the version?

manu

On 7 February 2010 18:52, dhenry
<<email address hidden><tfc.duke%<email address hidden>>
> wrote:

> I thought it was in karmic-updates, or maybe karmic-proposed repository.
> Which version of libc6 package do you have? I have 2.10.1-0ubuntu16 from
> karmic-updates.
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second
> delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “eglibc” package in Ubuntu: Triaged
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “eglibc” source package in Lucid: Triaged
> Status in “network-manager” source package in Lucid: Invalid
> Status in “eglibc” source package in Karmic: Fix Released
> Status in “network-manager” source package in Karmic: Invalid
> Status in “glibc” package in Fedora: Confirmed
>
> Bug description:
> In Karmic, DNS lookups take a very long time with some routers, because
> glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no
> (non-loopback) IPv6 interfaces configured. Routers which do not repond to
> this cause the lookup to take 20 seconds (until the IPv6 query times out).
>
> *** PLEASE DO NOT COMMENT ON THIS BUG unless you have something
> constructive to say. Everything that can be said has already been said, and
> if you comment, you are just adding noise. Please let those that actually
> know what they are doing concentrate on fixing this bug from now on. ***
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/417757/+subscribe
>

[Ashkan_Akhavein (ashkan-akhavein)]

Hello everyone,

a rather basic question:

How can I update (or install the new) eglibc package?

I don't want to use the update manager since I haven't installed
any updates after the fresh installation of Karmic(I use Karmic on an USB drive)
and updating ruins my system(it just won't boot after updating. I asked
a question about this update problem elsewhere in Launchpad :
https://answers.launchpad.net/ubuntu/+source/yelp/+question/99635 )

Maybe I can use Synaptic or use a command from terminal
or download something directly from the net and install it?

Thank you for your help.

[Manuel Bärenz (turion)]

1. Do you have the package source "karmic-updates" enabled? If no, enable it in Synaptic just for one time. After enabling it, do an update (but not an upgrade!).
2. Look for the libc6 package and choose the version (Ctrl + E, or in the "Package" Menu). Choose the newest which should be 2.10.1-0ubuntu16 from karmic-updates and click the green checkmark ;) Should work.

[Ashkan_Akhavein (ashkan-akhavein)]

>>> @Turion

Thank you Tourion. I Installed the new eglibc package ( libc6 2.10.1-0ubuntu16).

I don't think it has fully resolved the problem.(but sure my browsing gets less delays now)
I still have the delay problem with some websites(even the with the google.com !).
Well, I certainly do not want to "add more noise" to this discussion.

Thank you all again.

P.S. I really hope that the Lucid Lynx will be released with no such bug.

[Yermo (yml-yml)]

From what I am observing here, the IPV6 problem is not the sole cause of slow lookups and connection speed. Despite turning off IPV6, running my own name server (even a local caching one), modifying /etc/nsswitch.conf, tweaking settings in ethtool, etc. etc. etc. I still get stalls and failed connections so often that the distribution is on the verge of unusable.

To reproduce this problem open 15 tabs or so in firefox. close firefox. Open it again and open another network app like Thunder bird at the same time. More often than not many tabs will fail to load and you'll get a failed to connect error from Thunderbird.

As has been reported, this does not happen, for instance, in WinXP running in VMware on the same physical machine.

I am behind a D-Link DI-707 switch.

Unfortunately, it looks like I am going to have to bail on Ubuntu. It's too bad. The distribution has a lot of promise.

[Laurent Bigonville (bigon)]

@Yermo:

Could you post the result of

ip -6 addr and ip -6 route please?

[Yermo (yml-yml)]

@Laurent As I mentioned, I have turned off ipv6 in grub:

root@humility:~# ip -6 addr
root@humility:~#

root@humility:~# ip -6 route
172.16.38.0/24 dev vmnet8 proto kernel scope link src 172.16.38.1
192.168.194.0/24 dev vmnet1 proto kernel scope link src 192.168.194.1
192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.5
169.254.0.0/16 dev eth0 scope link metric 1000
default via 192.168.0.1 dev eth0 metric 100

I am running VMWare Workstation 7 on this machine.

I run my own nameserver which is on a CentOS 5 machine on the other side of a DLink DI-707 firewall gadget.

I would be more than happy to run any tests to help track this down.

[Yermo (yml-yml)]

@Laurent Ok, I spent some more time researching this, playing with various scenarios.

1. Fedore Core 12 running in VMWare exhibits the same problem as Ubuntu 9.10 running natively.

2. WinXP running in VMWare does /not/ exhibit the problem.

3. Cent OS 5.4 running in VMWare does exhibit the problem. Something I was not expecting.

4. Ubuntu 8.04 did /not/ exhibit this problem. It started with 8.10 and appeared to be much worse in 9.10.

Based on some discussions that I remember reading (but no longer have a link to) I decided to try to upgrade the firmware on my DLink 707 firewall gadget to version 2.57 from version 2.51.

It's too early to tell for sure, but my initial feeling is that network performance is much improved. My standard test of loading multiple large pages simultaneously is not causing the problem.

If this turns out to be the "fix", the question would be why did Ubuntu 8.04 work flawlessly and later versions have problems?

[Laurent Bigonville (bigon)]

I think this has really nothing todo with this bug (aka slow dns queries)

Did you try with something else than firefox? Like wget for example?

Could you try to disable ipv6 support in firefox (about:config in the url bar and then looking for network.dns.disableipv6 key)?

[Yermo (yml-yml)]

@Laurent yes. /ALL/ network access regardless of application is affected. Please re-read the report I wrote. Yes, this has nothing to do with the IPv6 DNS lookup problem. IPV6 AAAA record lookup was one issue that would cause the slow connections so many people are experiencing. My point is that while the IPv6 lookup issue has been resolved, it is not the only cause of the slow dns lookup/connection symptoms people are experiencing.

As I have said, IPV6 is /disabled/ on my machine via grub. IPV6 is disabled in firefox. IPV6 IS NOT THE SOLE REASON FOR SLOW DNS AND NETWORK CONNECTION IN UBUNTU. i have verified this here. Maybe it should be listed as a separate bug, but none of the other bugs I have found (of which there are many) seem to match the symptoms I'm seeing here exactly.

Slow dns lookups. DNS lookup failure 30% of the time or so. Connection failure 30% of the time or so. Extremely slow net connectivity over a wired connections. I've tried every combination of work around I could find on the net to no avail.

Then I upgraded the firmware on my DI-707 router and suddenly the problem disappears. The confusing thing is that Ubuntu 8.04, WinXp and EVERY SINGLE DISTRIBUTION BEFORE had no issue with this router. It seems to be restricted to recent 2.6 kernels.

I had read somewhere, I'm sorry I have forgotten where, that there was some TCP/IP standards issue that someone had decided to implement too strictly in recent kernels which caused connections through many legacy routers/switches/firewalls to fail (which is why I tried the firmware upgrade).

At this point, for me, the problem seems to be solved, but I have the feeling that there are many Ubuntu users out there that are still suffering from slow/failing dns lookups and the IPV6 issue does not explain all of them.

There is still some network level problem.

But like I said, the firmware upgrade on my DI-707 seems to have worked around whatever problem Ubuntu has.

I'm reporting this in an attempt to be helpful; too often in my own software business customers who are running into trouble with our software simply don't report their experiences so we have no idea that there's a problem.

[Laurent Bigonville (bigon)]

Sorry I lost track in all these comments.

I think it could be interesting to open a new bug report for that then so we don't mix everything, but it will be quite hard to track down the issue as it seems fixed for you :/

Anyway thanks for reporting bug as it helps to make Ubuntu (and GNU/Linux) better

[Yermo (yml-yml)]

@Laurent Yes, I was thinking the same thing since it's now fixed for me.

However, if you do some searches, even in the Ubuntu forums I think, you'll find dozens if not hundreds of posts from people saying they turned off IPV6 but still had slow lookups/connectivity. There was one thread somewhere where it was mentioned that this was due to Ubuntu/Linux now implementing some standard (I don't know which one) more rigidly than many router/switch manufacturers. That was the reason I thought to upgrade the firmware in the DI-707.

Doing a quick search I find:

http://superuser.com/questions/67686/internet-very-slow-when-upgrading-to-ubuntu-9-10 (3rd comment)

http://ubuntuforums.org/showthread.php?p=8758456 (5th post)

[Laurent Bigonville (bigon)]

Are you connected by wifi?

[Yermo (yml-yml)]

@Laurent No. Wired connection exclusively.

[strel (d-mestetskiy)]

I have a wired connection. And I recently changed my modem to a much newer
one. I've been having issues recently, which seemed to spawn from an updated
vbox. I have ubuntu running as quest on windows 7. After the update of vbox,
my connection would be very strange, at start up of Firefox, everything
would load. After couple seconds or so, the connection would grind to a
halt. Recently, I activate IPV6 on Firefox, and the problem has gotten much
much better, but not completely alleviated. Some pages are constantly trying
to load, but never get there. But thats much better than before. Before I
switched the flag in Firefox, pretty much all the pages would be stuck in
load mode, it was so frustrating.

Specs:
Windows 7 with ubuntu guest through vbox
wired connection straight to the modem, nothing in between.

Windows 7 seems to work fine
Ubuntu seems like there are some issues still.

On Mon, Feb 22, 2010 at 5:19 PM, Yermo <email address hidden> wrote:

> @Laurent No. Wired connection exclusively.
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second
> delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>
> Status in “eglibc” package in Ubuntu: Triaged
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “eglibc” source package in Lucid: Triaged
> Status in “network-manager” source package in Lucid: Invalid
> Status in “eglibc” source package in Karmic: Fix Released
> Status in “network-manager” source package in Karmic: Invalid
> Status in “glibc” package in Fedora: Confirmed
>
> Bug description:
> In Karmic, DNS lookups take a very long time with some routers, because
> glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no
> (non-loopback) IPv6 interfaces configured. Routers which do not repond to
> this cause the lookup to take 20 seconds (until the IPv6 query times out).
>
> *** PLEASE DO NOT COMMENT ON THIS BUG unless you have something
> constructive to say. Everything that can be said has already been said, and
> if you comment, you are just adding noise. Please let those that actually
> know what they are doing concentrate on fixing this bug from now on. ***
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/417757/+subscribe
>

--
Dmitriy Mestetskiy

[cheerios_ (jussiava)]

I kept having issues even after the fix was released until poking into NetworkManager settings for the wireless connection in use
(in my case: /etc/NetworkManager/system-connections/Auto\ godel), and changing all the settings below method=ignore to 'true'. Somehow method=ignore wasn't enough to disable ipv6 lookups.

BEFORE:
[ipv6]
method=ignore
ignore-auto-routes=false
ignore-auto-dns=false
never-default=false

AFTER:
[ipv6]
method=ignore
ignore-auto-routes=true
ignore-auto-dns=true
never-default=true

[Ricardo Fernández (koshrf)]

This bug is far from fixed, there was a fix committed but it didn't do anything, they are still lot of people complaining (including myself of course), and it seems it still happening in 10.04.

I don't know how you guys are testing this, but so far, I'm still getting DNS delays and the browser takes a lot sometimes to resolve and open a webpage.

[Antonio Navarro (anavarrog)]

I agree with Ricardo Fernández and Yermo. This problem is not fixed. I'm still experiencing it in every wireless network i try to connect any of my laptops using ubuntu/kubuntu karmic. Airports, libraries, museums... always the same problem. I've stopped using linux in laptops because of this crap.

[Ashkan_Akhavein (ashkan-akhavein)]

@ anyone concerned/responsible/capable :

PLEASE DO NOT LET THIS GET INTO THE FINAL RELEASE OF LUCID.

Thank you very much.

P.S. I think "karma" have dealt the "Koala" a losing hand.
        Hope something better for the "Lynx".

[Martin Pitt (pitti)]

Please pretty please STOP changing the status of this bug!

[Ricardo Fernández (koshrf)]

I was wondering something, If a Fix was Released, and if it doesn't work (as reported by almost all users since the release), what we should do? Open a new bug ticket?

I'm not changing the status or anything, but it is kinda lame that the status can't be reverted because the fix doesn't work at all. After the Fix was "Released" no one at Ubuntu have been trying to reach the users to see what is going on, they just check that the bug is "fixed" and thats it; is there no need to check the feedback?

This Bug is _NOT_ fixed at all, check the comments from the people, it didn't do the job, they are still delays on resolving names and the browser takes few seconds before opening webpages, this same is happening on 10.04 right now.

I guess this bug will be like the VNC client on Ubuntu (the one installed by default on gnome, vinillo), 4 releases of Ubuntu and still doesn't work at all, no one can even use it, and still there was a "Fix Released".

*sigh*

[Mark Schouten (mark-prevented)]

On Wed, 2010-03-03 at 13:36 +0000, Ricardo Fernández wrote:
> I was wondering something, If a Fix was Released, and if it doesn't work
> (as reported by almost all users since the release), what we should do?
> Open a new bug ticket?

No, you should try to determine if the problem you are experiencing is
the same problem as the problem described in this bug.

If you still have issues (which you shouldn't have, since *this* bug is
fixed), try to debug and find which problem you are experiencing.

This problem is about trying to get AAAA-records on machines that don't
even have any IPv6 connectivity. (devs, please correct me if needed). If
you tcpdump your interface, and see no AAAA-queries, this bug is not
your problem, but there might be another problem.

So, if you still experience issues. Try using wired networks instead of
wireless (I myself have crappy wireless performance with Karmic and a
Linksys interface), tcpdump your interface to see what happens. Etc etc
etc.

> I guess this bug will be like the VNC client on Ubuntu (the one
> installed by default on gnome, vinillo), 4 releases of Ubuntu and still
> doesn't work at all, no one can even use it, and still there was a "Fix
> Released".

Never heard of Vinillo, but ok. That's not what this bug is about. :)

Regards,

Mark

[jordan.sc (jordanjsc)]

I have 3 PCs (1 wired, 2 wireless) on my local home network running Karmic that were all affected by the IPv6 issue described here. The released fix worked for all 3 PCs. There are no more delays - performance is comparable to 9.04, Mandriva with IPv6 turned off, and Windows XP/7. I know of several other folks who had this issue and the fix worked for them as well. My wireless cards are Atheros and Ralink, and my router is a Netgear RP614.

[Hilton Shumway (hillshum)]

This bug is about Ubuntu dealing with bad DNS servers that don't understand
AAAA (IPv6) queries. Before claiming this isn't fixed, please make sure that
bad DNS servers are in fact the issue. If disabling IPv6 AAAA lookups in
Firefox fixes the issue, then you are dealing with this bug. If switching to
good DNS servers like those of openDNS or Google fixes the issue, then you
are dealing with this bug. If switching from wireless to wired* on the same
LAN* fixes the issue, then you are not dealing with this bug.

Please don't change the status of this bug unless you are sure that you have
issues with IPv6 DNS lookups.
Hilton

--
It's bad civic hygiene to build technologies that could someday be used to
facilitate a police state." -- Bruce Schneier

On Wed, Mar 3, 2010 at 7:09 AM, jordan.sc <email address hidden> wrote:

> I have 3 PCs (1 wired, 2 wireless) on my local home network running
> Karmic that were all affected by the IPv6 issue described here. The
> released fix worked for all 3 PCs. There are no more delays -
> performance is comparable to 9.04, Mandriva with IPv6 turned off, and
> Windows XP/7. I know of several other folks who had this issue and the
> fix worked for them as well. My wireless cards are Atheros and Ralink,
> and my router is a Netgear RP614.
>
> --
> [karmic regression] all network apps / browsers suffer from multi-second
> delays by default due to IPv6 DNS lookups
> https://bugs.launchpad.net/bugs/417757
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “eglibc” package in Ubuntu: Triaged
> Status in “network-manager” package in Ubuntu: Invalid
> Status in “eglibc” source package in Lucid: Triaged
> Status in “network-manager” source package in Lucid: Invalid
> Status in “eglibc” source package in Karmic: Fix Released
> Status in “network-manager” source package in Karmic: Invalid
> Status in “glibc” package in Fedora: Confirmed
>
> Bug description:
> In Karmic, DNS lookups take a very long time with some routers, because
> glibc's DNS resolver tries to do IPv6 (AAAA) lookups even if there are no
> (non-loopback) IPv6 interfaces configured. Routers which do not repond to
> this cause the lookup to take 20 seconds (until the IPv6 query times out).
>
> *** PLEASE DO NOT COMMENT ON THIS BUG unless you have something
> constructive to say. Everything that can be said has already been said, and
> if you comment, you are just adding noise. Please let those that actually
> know what they are doing concentrate on fixing this bug from now on. ***
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/417757/+subscribe
>

[Marius (mm473)]

I had this bug in Karmic, and the fix definitely worked for me. However, I installed Lucid Alpha 3 and the bug is BACK. I know its the same bug because all of the workarounds for Karmic (Firefox disable IPv6, Grub disable IPv6, etc...) fix it in Lucid.

I think people are frustrated because a) this bug is really annoying, b) it took a long time to even acknowledge that it was something that needed fixing, and c) we don't want to see TWO consecutive release version of Ubuntu have unusable internet out-of-the-box.

[Ricardo Fernández (koshrf)]

I'm still having the same issue while trying to use any web browser, takes forever to resolve a name, have to reload every page twice until I get it.

I have ipv6 disable:

cat /proc/sys/net/ipv6/conf/all/disable_ipv6
1

Also Firefox have IPv6 Disabled.

But tcpdump still report IP6 stuff:

04:34:16.007477 IP6 fe80::c1f:dc31:a82c:435e.57503 > ff02::c.1900: UDP, length 146
04:34:19.913100 IP6 fe80::c1f:dc31:a82c:435e.57503 > ff02::c.1900: UDP, length 146
04:34:20.409483 IP6 fe80::226:69ff:fe65:da00 > ip6-allrouters: ICMP6, router solicitation, length 16
04:38:04.515848 IP6 fe80::c1f:dc31:a82c:435e.546 > ff02::1:2.547: dhcp6 solicit

I don't know if they are related.
This happen with WiFi and Wired cable.

I don't know what else to do to post more info about this bug, and as some other are reported in this same bug we need some directions, in my opinion it isn't Fixed. I don't have this problem in debian, fedora and centos (the other 3 distros i use), it is only happening in Ubuntu 9.10 and 10.04 (Same Laptop, tested with all the other distros), 8.04, 8.10, 9.04 doesn't have this problem.

[Philipp Kern (pkern)]

On Sat, Mar 06, 2010 at 09:13:38AM -0000, Ricardo Fernández wrote:
> 04:34:16.007477 IP6 fe80::c1f:dc31:a82c:435e.57503 > ff02::c.1900: UDP, length 146
> 04:34:19.913100 IP6 fe80::c1f:dc31:a82c:435e.57503 > ff02::c.1900: UDP, length 146
> 04:34:20.409483 IP6 fe80::226:69ff:fe65:da00 > ip6-allrouters: ICMP6, router solicitation, length 16
> 04:38:04.515848 IP6 fe80::c1f:dc31:a82c:435e.546 > ff02::1:2.547: dhcp6 solicit

At least the c1f one does not look like EUI-64 and thus rather like
Windows using IPv6 privacy addresses. For the third line it would be
helpful to see the output of "ip -6 addr", but it's also possible that
it's not the Linux box but another one on the segment.

Kind regards,
Philipp Kern

[Ricardo Fernández (koshrf)]

2010/3/6 Philipp Kern <email address hidden>:
> On Sat, Mar 06, 2010 at 09:13:38AM -0000, Ricardo Fernández wrote:
>> 04:34:16.007477 IP6 fe80::c1f:dc31:a82c:435e.57503 > ff02::c.1900: UDP, length 146
>> 04:34:19.913100 IP6 fe80::c1f:dc31:a82c:435e.57503 > ff02::c.1900: UDP, length 146
>> 04:34:20.409483 IP6 fe80::226:69ff:fe65:da00 > ip6-allrouters: ICMP6, router solicitation, length 16
>> 04:38:04.515848 IP6 fe80::c1f:dc31:a82c:435e.546 > ff02::1:2.547: dhcp6 solicit
>
> At least the c1f one does not look like EUI-64 and thus rather like
> Windows using IPv6 privacy addresses.  For the third line it would be
> helpful to see the output of "ip -6 addr", but it's also possible that
> it's not the Linux box but another one on the segment.

I disabled IPv6 at the kernel source, make a new kernel without IPv6
at all, and I'm still getting those messages, so my guess is that
those are other machines as you suggest (my home network).

The browsers (any of them, firefox, chrome, epiphany, lynx, elinks)
still takes to much to resolv a name and download the content of it,
apt get stuck sometimes while updating, haven't tried anything else.

Anything else I can do to debug this problem ? so far it seems it is
this bug, but the release fixed doesn't work for me (and all the other
installation I've done over my work network).

[Yermo (yml-yml)]

@Ricardo Fernández As I mention above, in my case, after disabling IPV6, I still had the same symptoms. Terribly long delays making any kind of network connections (all wired). I noticed here that it seemed to be related to making more than one network connection at a time.

Based on forum posts elsewhere, I upgraded the firmware on my D-Link 707 switch/firewall and then suddenly all network problems disappeared.

Of course, CentOS, WinXP, Win2K and Ubuntu 8.04 never had a problem with the router, so I'm not sure why this fixed the issue.

But unfortunately, I've wasted too much time on Ubuntu 9.10 problems. It's been by far the buggiest least reliable error prone distribution I've ever come across (been developing software using Linux professionally since early 1993). This is also the very first time I've been unable to find good information on how work around issues that come up despite having put in some serious effort. (Sound, anyone?) It's open source after all and I've offered to help track these issues down, but got no useful response.

So I've bailed. I'm running Fedora Core 12 now. Everything works as it should.

[Hilton Shumway (hillshum)]

On Sat, Mar 6, 2010 at 12:51 PM, Yermo <email address hidden> wrote:

> @Ricardo Fernández As I mention above, in my case, after disabling IPV6,
> I still had the same symptoms. Terribly long delays making any kind of
> network connections (all wired). I noticed here that it seemed to be
> related to making more than one network connection at a time.
>
>

If disabling IPv6 fixes the issue, you aren't dealing with this bug. Please
open a new one if need be

[Yermo (yml-yml)]

@Hillshum re-read what I wrote. Disabling IPV6 /does not/ resolve the symtpom.

But for me it no longer matters. I've bailed and switched to Fedora Core 12. It works.

[Ricardo Fernández (koshrf)]

>> @Ricardo Fernández As I mention above, in my case, after disabling IPV6,
>> I still had the same symptoms. Terribly long delays making any kind of
>> network connections (all wired). I noticed here that it seemed to be
>> related to making more than one network connection at a time.
>>
>>
>
> If disabling IPv6 fixes the issue, you aren't dealing with this bug. Please
> open a new one if need be

What part of "disabling IPv6 doesn't fix this issue" you didn't read?
seriously, I'm starting to get tired of saying this didn't fix the bug
for me, and few others keep saying the same, are you guys really not
reading the posts? Just because the status was changed to "Fixed"
doesn't mean it is really fixed if people are still getting the same
problems.

Please tell me tools to give you more info, tell me what to do to help
the developers to find answers, tell me what do I do need to do to
debug the info you people need.

And don't tell me it is fixed because otherwise i wouldn't be posting
here don't you think ?

Really c'mon guys, why it is so hard? If it is fixed for you, nice!
congratulations! now move along, we have people here that still have
problems (including myself).

No offense, but dealing with Fedora bugzilla and people there is more
easy than dealing with the ubuntu community and developers, here i
have to say around 50 times why it isn't fixed for me and people still
don't belive it, thats just amaizing.

[Yermo (yml-yml)]

@Ricardo: Amen.

Ubuntu is a nice "end-user-friendly" distribution. I respect what they are trying to do. But there comes a point where bailing is the only sensible option. Fedora Core 12 works like a champ. Networking works. Sound works. Package management works. Java works. OpenOffice works. etc. And I've noticed the same thing you have. The quality of technical posts for Fedora Core is definitely much higher than what I've seen in Ubuntu. I'm not sure why. It's all Linux after all. I guess Fedora attracts a more technical crowd and they seem, at least for the moment, to be much more focused on solving problems.

[Laurent Bigonville (bigon)]

For those who still affected by slowdown did you try what is proposed at comment 134 ?

time dig -t a noc.sixxs.net +nofail | grep -e status -e A -e time
time dig -t aaaa noc.sixxs.net +nofail | grep -e status -e AAAA -e time

And also

time getent hosts www.sixxs.net

And post the result?

[Steve Langasek (vorlon)]

On Sun, Mar 07, 2010 at 07:01:53AM -0000, Ricardo Fernández wrote:
> What part of "disabling IPv6 doesn't fix this issue" you didn't read?
> seriously, I'm starting to get tired of saying this didn't fix the bug
> for me, and few others keep saying the same, are you guys really not
> reading the posts? Just because the status was changed to "Fixed"
2> doesn't mean it is really fixed if people are still getting the same
> problems.

Conversely, having the same symptoms (and as much as anyone has shown
here since the fix for this bug was verified is that "their network is
slow", which is a very broad symptom) doesn't mean this bug isn't fixed.

You say you've disabled ipv6. How did you do this? Do you have network
traces showing that eglibc is still sending the wrong DNS queries?

I have personally retested the karmic version of libc6 in response to your
message here. The bug described in this report is fixed - a system with no
routable ipv6 addresses will not generate AAAA DNS queries with libc6
2.10.1-0ubuntu16 installed. So whatever problem you're seeing should be
tracked in a separate bug report.

> And don't tell me it is fixed because otherwise i wouldn't be posting
> here don't you think ?

No offense, but experience shows that with high-profile bugs where the
symptom is described in such general terms as is the case here, users *very
frequently* follow up when the problem they're experiencing has nothing to
do with the bug report in question.

> Really c'mon guys, why it is so hard? If it is fixed for you, nice!
> congratulations! now move along, we have people here that still have
> problems (including myself).

If it's fixed for them, then it's almost 100% certain that people still
experiencing problems have a different bug. Different bug -> different bug
report.

(If you're reading this because you *did* file a separate bug and your bug
was marked as a duplicate of this one, then that was a mistake and we should
correct that status. Unfortunately, Launchpad's UI doesn't make it easy to
figure out which commenters in a bug report filed which duplicates, and
encourages users to all follow up to the master bug.)

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

[cheerios_ (jussiava)]

Still getting slow Internet (hardly anything responds), on Karmic (even with OpenDNS -- not a DNS issue!): http://pastebin.org/107680 All tips and pointers appreciated on how to troubleshoot. Frustrating part is, sometimes connections work fine for a while, then all stops responding.

[B.B. Lauret (bblauret)]

Got this issue after upgrading to lucid today. On karmic I did not have this problem.

I've attached the output of
time dig -t a noc.sixxs.net +nofail | grep -e status -e A -e time
time dig -t aaaa noc.sixxs.net +nofail | grep -e status -e AAAA -e time
time getent hosts www.sixxs.net

[eris23 (jdkatz23)]

Still a problem in the Ubuntu 10.04 Beta 1 live dvd. I had to disable ipv6 in Firefox.

[Mack (mark-msschuurmans)]

Fix for karmic doesn't work for me.

[cdahlin (cdahlin-redhat-bugs)]

This is reliable. Some programs have perfect DNS, some don't work at all.

[sadmac@foucault coding]$ ping edge.launchpad.net
PING edge.launchpad.net (91.189.89.225) 56(84) bytes of data.
64 bytes from wildcard-launchpad-net.banana.canonical.com (91.189.89.225): icmp_seq=1 ttl=42 time=101 ms
64 bytes from wildcard-launchpad-net.banana.canonical.com (91.189.89.225): icmp_seq=2 ttl=42 time=102 ms
64 bytes from wildcard-launchpad-net.banana.canonical.com (91.189.89.225): icmp_seq=3 ttl=42 time=102 ms
64 bytes from wildcard-launchpad-net.banana.canonical.com (91.189.89.225): icmp_seq=4 ttl=42 time=102 ms
^C
--- edge.launchpad.net ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3816ms
rtt min/avg/max/mdev = 101.601/101.983/102.238/0.235 ms
[sadmac@foucault coding]$ bzr co lp:libnih libnih-error
bzr: ERROR: Connection error: Could not resolve 'edge.launchpad.net' [Errno -2] Name or service not known
[sadmac@foucault coding]$

[cdahlin (cdahlin-redhat-bugs)]

Above is on latest F12

[Lucious Daniels Jr (ladmatic)]

I'm still having this issue. Though I don't think it has to do with ipv6. I've tried every fix I've come across...editing nsswitch, disabling ipv6, open dns, still no luck. I even tried putting another distro on the laptop (arch) and it too suffers the same problem, which makes me think it's an issue with the newer kernel and/or the driver for my particular wireless adapter. No problems at all on jaunty with the same machine.

[Michael Schnupp (michas)]

As you have both a working and a not working configuration, this makes you the perfect candidate for giving more details on that problem. Please give more details: What *exactly* is the problem you are talking about?

Is it a problem while resolving a name to an IP address? (DNS)
Is it a problem connecting to the other host? (Syn, Ack, etc.)
Or is it a problem while beeing connected in some kind of TCP session? (bandwidth, etc.)

If you are not sure, please use wireshark to monitor your connection. (All phases above are visible there and their timing.)
Please do so on the "working" and "not working" system and report the difference.

But since you are pretty sure it has *nothing* to do with IPv6, it is almost certainly another bug.
So *please* open a new bug and give only a pointer to the new bug here.

Hopefully this will finally bring some light on the cause of that issue. :)

[Matt Giuca (mgiuca)]

Sorry to add to the noise a bit, but I just want to get the scope of this bug clear. I certainly have *a bug* but I can't figure out if it is this one or not. In the description, it says
"If disabling IPv6 or using good DNS servers like openDNS fixes the problem, you are not dealing with this bug."

Does "not dealing with this bug" mean "you have another bug which is unrelated to this", or does it mean "you have this bug, but that isn't a valid fix for it." Maybe we can get this text cleared up to avoid all this confusion about who has the bug and who doesn't.

It also says
"Routers which do not repond [sic] to this cause the lookup to take 20 seconds"

The latter quote seems to imply that the bug will appear for certain DNS servers (e.g., a router), but not others (e.g., proper Internet DNS servers). Therefore, if "using a good DNS server like openDNS fixes the problem", why am I not dealing with the bug? The bug only shows up on certain DNS servers; changing my DNS server is a workaround, but surely it implies I have this same bug.

I have the iiNet BoB router, which is giving me 20 second DNS lookup delays. Changing my DNS settings (in resolv.conf) to a public DNS server (in this case, using my ISP's DNS server) fixes the problem immediately in all apps. So is this the bug, or should I be reporting another one?

[Michael Schnupp (michas)]

Ok, you have a bug. But are you sure it is a bug in Ubuntu?
Do you have any other OS working correctly with the same router?

Please have a look at your network traffic. (Wireshark is nice for this.)

*This* bug goes like this:
Ubuntu: Hey router, what is IPv6 address of example.com?
Router: (ignores that question and does not answer at all)
Ubuntu: (waits for an answer for some seconds and finally...)
Ubuntu: Ok, I assume you don't have one. Then do you have any IPv4 address of example.com?
Router: (answering immediately) Sure, here you are.

With disabled IPv6 Ubuntu does only ask for a IPv4 address and the problem is no longer visible.
If your bug does not go away after disabling IPv6 it is *not* this bug.

Please investigate where your delay comes from. Is it really the router not answering at all? Does it really involve IPv6?
Is there really something Ubuntu can do to fix this? (i.e. are there other OS which do?)

[Matt Giuca (mgiuca)]

My comment wasn't really to add to the bug report, but mostly to try and clarify the bug's description. I am very confused about whether this is my bug, and the bug description doesn't really help, as it contains a confusing statement which makes me unsure about whether I have the bug.

I understand that this is pretty much a bug in the *routers*, not in Ubuntu. The reason it's reported here on Launchpad and not in a thousand routers is because you're planning to implement a workaround in Ubuntu, right?

What you are saying is, if I switch to another DNS server, then it *may not* be the same bug, but it *might* be (if my router is ignoring IPv6 DNS requests, but the other DNS server is good). But it could be other problems with my router's DNS server besides IPv6.

But if I disable IPv6 entirely, and the bug does not go away, then it is not this bug. (I haven't been able to disable IPv6 yet).

I should have said in my last comment that this bug is new to me in Lucid Beta. It was working fine in Karmic with the exact same setup (same laptop, same router). It works fine in Windows XP. It still works fine with any other router. So it's clearly a combination of this router and Lucid that is a problem *for me*. I'm not asking for help here (since I've already worked around the issue), just trying to figure out if I am having the same bug (so I can be useful, and provide you with socket dumps) or if it isn't (so I can get out of your hair). I will experiment with disabling IPv6 and looking at Wireshark tonight.

[Lucious Daniels Jr (ladmatic)]

Okay. I just gave wireshark a try. I'm not exactly sure what to post, but to be clear on my problem. I click on a random link in my browser and I get a "Waiting for.." dialog in the status. This does not happen every time, but is fairly regular, especially on specific sites, like deviantart.com. Using wireshark during one of these hangs I see a lot of:

[TCP Retransmission] GET....

Then I see the ARP protocol will display [Who has 192.168.1.1 Tell 192.168.1.123] Which is my router IP and Laptop IP respectively.

[jordan.sc (jordanjsc)]

This bug has been officially fixed for Karmic but not for Lucid. When can we expect a fix for Lucid which is just 16 days from final release?

[Tollef Fog Heen (tfheen)]

]] Matt Giuca

Hi,

| I understand that this is pretty much a bug in the *routers*, not in
| Ubuntu. The reason it's reported here on Launchpad and not in a thousand
| routers is because you're planning to implement a workaround in Ubuntu,
| right?

Yes (to both questions).

| What you are saying is, if I switch to another DNS server, then it *may
| not* be the same bug, but it *might* be (if my router is ignoring IPv6
| DNS requests, but the other DNS server is good). But it could be other
| problems with my router's DNS server besides IPv6.

If it works fine with Karmic and works fine with Lucid with IPv6
disabled, but is slow at resolving DNS with IPv6 enabled, that is this
bug. If it's slow with IPv6 disabled, that's really interesting as
well.

| But if I disable IPv6 entirely, and the bug does not go away, then it is
| not this bug. (I haven't been able to disable IPv6 yet).

In grub, add «ipv6.disable=1» after the line with loads the kernel (this
one presumably ends with «quiet splash»).

Can you include the output of running «ip a» in your reply as well?
Preferably both before and after disabling IPv6.

--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

[Matthias Klose (doko)]

There's a eglibc testbuild prepared; it's tested to install and survive a reboot; if you want to test it, please add to /etc/apt/sources.list:

  deb http://ppa.launchpad.net/ubuntu-toolchain/ppa/ubuntu lucid main

run apt-get update, apt-get install libc6 and report results here.

[Seth Hikari (sethhikari)]

I tried the new eglibc not really sure if it is working I see resolving host last less time. Nothing exploded

[Martin Pitt (pitti)]

Tollef Fog Heen [2010-04-14 20:15 -0000]:
> If it works fine with Karmic and works fine with Lucid with IPv6
> disabled, but is slow at resolving DNS with IPv6 enabled, that is this
> bug.

Oh, interesting. I have such a buggy router as well, and in karmic it
was fixed with local-ipv6-lookup.diff.

ipv6.disable=1 does not make any difference for me in Lucid (I haven't
tested that with karmic, since the eglibc in karmic-updates fixed it)

> In grub, add «ipv6.disable=1» after the line with loads the kernel (this
> one presumably ends with «quiet splash»).

The option was recognized, dmesg says "IPv6: Loaded, but
administratively disabled", and "ip a" dropped the "inet6" lines as
well. But the 20 second DNS resolution timeout still happens.

> Can you include the output of running «ip a» in your reply as well?
> Preferably both before and after disabling IPv6.

Before:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 00:1d:72:df:71:c3 brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:17:c4:53:a9:f8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.109/24 brd 192.168.2.255 scope global wlan0
    inet6 fe80::217:c4ff:fe53:a9f8/64 scope link
       valid_lft forever preferred_lft forever

After:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 00:1d:72:df:71:c3 brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:17:c4:53:a9:f8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.109/24 brd 192.168.2.255 scope global wlan0

Matthias Klose [2010-04-16 0:16 -0000]:
> There's a eglibc testbuild prepared; it's tested to install and survive
> a reboot; if you want to test it, please add to /etc/apt/sources.list:
>
> deb http://ppa.launchpad.net/ubuntu-toolchain/ppa/ubuntu lucid main

Makes no difference here. Is that still the same patch as
local-ipv6-lookup.diff in karmic-updates?

The only workaround which works fine for me is to change DNS server. I
have a script /etc/network/if-up.d/0nameserver which overwrites
/etc/resolv.conf to use my providers' DNS servers instead of my
router.

[Lucious Daniels Jr (ladmatic)]

Apparently there is a regression in the ath5k driver affecting my card. I was able to fix my problem by using the Windows driver and ndiswrapper. Maybe worth a shot for anyone still having problems.

[Tollef Fog Heen (tfheen)]

]] Lucious Daniels Jr

| Apparently there is a regression in the ath5k driver affecting my card.
| I was able to fix my problem by using the Windows driver and
| ndiswrapper. Maybe worth a shot for anyone still having problems.

That is another bug than this one; _please_ don't clutter this bug with
reports about hardware and driver problems.

--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

[Tollef Fog Heen (tfheen)]

]] Martin Pitt

| ipv6.disable=1 does not make any difference for me in Lucid (I haven't
| tested that with karmic, since the eglibc in karmic-updates fixed it)

That's interesting, since eglibc does iterate over configured network
interfaces and only sets seen_ipv6 if you have a non-IPv4 address which
is also not loopback

This suggests it's taking another code path from what I was assuming
from reading the code.

| Matthias Klose [2010-04-16 0:16 -0000]:
| > There's a eglibc testbuild prepared; it's tested to install and survive
| > a reboot; if you want to test it, please add to /etc/apt/sources.list:
| >
| > deb http://ppa.launchpad.net/ubuntu-toolchain/ppa/ubuntu lucid main
|
| Makes no difference here. Is that still the same patch as
| local-ipv6-lookup.diff in karmic-updates?

It's a somewhat simplified version which I believe should have worked,
but apparently doesn't.

--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

[Lucious Daniels Jr (ladmatic)]

@Tollef Fog Heen # 276

My apologies, but I only comment on this bug because my original was deleted (marked as a duplicate), and I was automatically (and wrongfully) subscribed to this one. I thought it would be beneficial to share my information with anyone else that was wrongfully subscribed to this bug.

[Derek (bugs-m8y)]

I've noticed multisecond delays since switching to Lucid.
I have two boxes here. Identical resolv.conf entries for local DNS.
The Karmic machine responds instantaneously, the Lucid one takes several seconds for uncached addresses.

If I run:
ltrace wget -O/dev/null http://domainwebserver/url/

And I haven't hit this webserver on our domain recently, I can see it hang for several seconds on getaddrinfo.
It then hangs again resolving the full name (domainwebserver.ourdomain.local) if a redirect is issued.

So far I've tried:
disabling ipv6 in sysctl.conf and as kernel param in grub.
replacing the hosts line in nsswitch.conf w/ only "files dns" (the full hosts line in lucid works fine in karmic).

Uninstalling libnss-mdns.

Despite all this I repeatedly get hangs on getaddrinfo - other things respond quickly, such as dig and nslookup.

The only improvement I've experienced (and this was also new to Lucid) was that after I removed:
mdns4_minimal [NOTFOUND=return] I no longer got complete failure to perform lookups.
That line appears to work fine in Karmic, with the exact same servers.

I've also tried adding or removing wins from the hosts line in nsswitch.conf in case that was why Karmic was fast.
No difference. Whether wins was there or not, Karmic remained fast, and Lucid slow.

[Derek (bugs-m8y)]

Oh. One last note. It appears that it isn't just local machines that are slow.
ltrace wget -O/dev/null http://www.ubuntu.com

Also hangs for several seconds on getaddrinfo

[Derek (bugs-m8y)]

Ok. I am pleased to say this has finally been resolved.
The last few things I tried.
http://www.debian-administration.org/article/Disabling_IPv6_under_a_2.6_kernel

Commenting out all references to ipv6 in hosts (shouldn't be necessary but I was desperate)

#Disable IPv6
net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.all.accept_ra=0
net.ipv6.conf.etho.accept_ra=0

In sysctl.conf - although sysctl didn't recognise any of 'em so I doubt that was solution

I think it was that first link that recommends:
To disable IPv6 it should be as simple as adding the lines

alias net-pf-10 off
alias ipv6 off

to /etc/modprobe.d/00local (creating the file if it exists).

[Derek (bugs-m8y)]

FWIW, I spoke too soon, I can only imagine the couple of sites I tested must have been cached by something else hitting 'em on login (ubuntu.com perhaps due to ubuntu one, and local dev site due to a cron). I'm still getting slow lookups :(

[Johan (deberghes-johan)]

I confirm that this bug is back in Lucid RC ...

[triage (triage-redhat-bugs)]

This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we may not be able to fix it before Fedora 11 is end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora please change the 'version' of this
bug to the applicable version. If you are unable to change the version,
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

[Derek (bugs-m8y)]

BTW. If anyone at all has any idea how to stop Lucid from doing AAAA lookups (besides the many things tried above), that'd be lovely.
Despite all the attempts I've made to disable IPv6 in the comments above, I still get a ton of AAAA lookups from our local DNS.

Karmic? All A.

[Derek (bugs-m8y)]

Have also tried:
[ipv6]
method=ignore
ignore-auto-routes=true
ignore-auto-dns=true
never-default=true

In network manager config and options single-request in resolv.conf

Nothing seems to stop it from trying AAAA despite ip -6 reporting no ipv6 routes or interfaces.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=435646#49
I haven't tried that yet, for one thing his site is down, for another seems rather old so I have no idea if it applies.

Basically, seems there is no way to make Lucid not try AAAA

[tytus64 (tytus64-redhat-bugs)]

Just like Casey I am experiencing the same problem on F12.

Clipper:~ $ ping peach.mycompany.com
PING peach.mycompany.com (10.26.1.61) 56(84) bytes of data.
64 bytes from peach.mycompany.com (10.26.1.61): icmp_seq=1 ttl=63 time=0.267 ms
64 bytes from peach.mycompany.com (10.26.1.61): icmp_seq=2 ttl=63 time=0.311 ms
^C
--- peach.mycompany.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1371ms
rtt min/avg/max/mdev = 0.267/0.289/0.311/0.022 ms
Clipper:~ $ ssh peach.mycompany.com
ssh: Could not resolve hostname peach.mycompany.com: No address associated with hostname

Here is the output from Wireshark when ssh command is issued:

24921 4305.188943 10.4.1.236 10.1.1.151 DNS Standard query A peach.mycompany.com
24922 4305.188983 10.4.1.236 10.1.1.151 DNS Standard query AAAA peach.mycompany.com
24923 4305.189460 10.1.1.151 10.4.1.236 DNS Standard query response A 10.26.1.61
24924 4305.189475 10.1.1.151 10.4.1.236 DNS Standard query response

The only way for me to fix this problem is to put the following line in /etc/hosts
10.26.1.61 peach

I should mention that this problem is _not_ unique to this network - the F12 machine is a laptop and I can see this problem at work as well as at home. It does not happen all the time but often enough to be annoying. Not sure what's the trigger.

Clipper:~ $ rpm -qa |grep glibc
glibc-2.11.1-4.i686
glibc-2.11.1-4.x86_64
glibc-headers-2.11.1-4.x86_64
glibc-common-2.11.1-4.x86_64
glibc-devel-2.11.1-4.x86_64
glibc-debuginfo-2.11.1-1.x86_64

Let me know if I can help in anyway to debug it.

[Neil (goofandfroggie)]

Hi
Sorry this more just a question
I have been having same/similar problem, Firefox most of the time.. times out... some times I can get google and even use it to search but never go to a web page. and the problem is with Thunderbird also, can not get emails only occasionally, it too times out.
am I the only one with this, is this all related with the ipv6?
O I can use google chrome and Opera with no problems, just Firefox is a problem if that is any help to anyone.
I have the problem with the bata (Which I did te upgrade) and the official release.
 again sorry I have no real input, but getting very frustrated with it.
Neil

[Derek (bugs-m8y)]

Neil, I would have e-mailed you privately to not repeat something brought up already in the bug, but you use Hotmail which has some particularly stupid blocking policies so my mail would not have gotten to you.

Firefox, go to the url about:config
Search for:
network.dns.disableIPv6

And set to true.

Firefox is using IPv6 since Lucid is not correctly handing the lack of ipv6 interfaces and ipv6 support in servers.

[Derek (bugs-m8y)]

I think I've finally fixed things to my satisfaction, system-wide:
$ cat getaddrinfo_wrap.c
// getaddrinfo wrapper
#include <dlfcn.h>
#define getaddrinfo _foo
#include <netdb.h>
#undef getaddrinfo

int getaddrinfo(const char *node, const char *service, struct addrinfo *hints, struct addrinfo **res)
{
    typedef int (*FP_getaddrinfo)(const char*, const char*, struct addrinfo*, struct addrinfo **);
    FP_getaddrinfo org_getaddrinfo = dlsym(((void *) -1), "getaddrinfo");
    hints->ai_flags|=AI_ADDRCONFIG;
    return org_getaddrinfo(node, service, hints, res);
}

Running:
gcc -fPIC -c -Wall getaddrinfo_wrap.c getaddrinfo_wrap.o
gcc -shared getaddrinfo_wrap.o -ldl -lstdc++ -o getaddrinfo_wrap.so
sudo cp getaddrinfo_wrap.so /usr/local/lib/
sudo chown root:root /usr/local/lib/getaddrinfo_wrap.so
sudo sh -c "echo /usr/local/lib/getaddrinfo_wrap.so >> /etc/ld.so.preload"

There. No more problems.

BTW, I do have ipv6 interfaces after stripping all my ipv4 hacks from above, that seems utterly irrelevant to this finally working.
I of course have no idea what the results of this could be on various apps, but at least the ones I use seem happy.

[Derek (bugs-m8y)]

With regards to Firefox, see also:
https://bugzilla.mozilla.org/show_bug.cgi?id=467497

[Derek (bugs-m8y)]

FYI, I changed:
hints->ai_flags|=AI_ADDRCONFIG;
to
if(hints) hints->ai_flags|=AI_ADDRCONFIG;

For obvious reasons :-/

[Derek (bugs-m8y)]

http://sourceware.org/bugzilla/show_bug.cgi?id=4599

This is kind of related to the last few comments. If you decide to force AI_ADDRCONFIG, until those fixes are in place, you should watch out for IPv6 in your hosts file.

[Derek (bugs-m8y)]

oh, and that came from:
https://bugzilla.mozilla.org/show_bug.cgi?id=467497#c9
and the following two comments.

I do wish Launchpad allowed anchors to comment numbers in the context of the whole page

[omair (omair-hafiz)]

Are there any updates with regards to this bug? I've been waiting patiently for some kind of fix (I check proposed updates everyday in the hope that there is some mention of this). My system is becoming frankly unusable since 90% of what I do is on the net. I've seen other bugs that have been open for five years or so and I fear that this one is going the same route. No updates whatsoever, even the discussion on this list have stopped.

It's a pity that I'm thinking of installing something else (even a windows 7 installation at this point) even though I've been thoroughly satisfied with 10.04 in all other respects.

[omair (omair-hafiz)]

Also what I don't understand is why DOCKY bugs are assigned as "critical" in the ubuntu bug list but this one is only of 'high' importance!

[Flurin Rindisbacher (flurin-deactivatedaccount)]

omair: did you try the pdns-recursor workaround?
maybe this can help you until there is a fix.

install pdns-recursor via synaptic or apt-get.
then edit /etc/resolv.con (sudo gedit /etc/resolv.conf) and set nameserver to 127.0.0.1

if your problem is only in firefox you can disable ipv6: enter about:config in the addressbar and confirm the warning. then enter ipv6 as a filter and double click on network.dns.disableIPv6 (after this the value must be enabled)

hope this stops you thinking about windows 7 ;-)

[smonsarr (smonsarr-junk)]

For me using openDNS works fine as a workaround.

[Derek (bugs-m8y)]

omair, if you can't change your DNS, I've found that forcing AI_ADDRCONFIG as noted in #288, #289 and (importantly) #290
works nicely for me.

Also, if it causes trouble for you, you can just remove or comment out the ld.so.preload line.

I've applied it on 5 computers here at work w/ no issues and immediate improvements, where DNS changes are simply not an option.

I do also set:
network.dns.disableIPv6;true

in Firefox as well. But all the other apps on the system are now working nicely (wget, ssh etc).

[omair (omair-hafiz)]

hello,

flurin & derek: thanks for the information. I'll definitely try the pdns-recursor workaround. the firefox workaround didn't work for me, unfortunately. but i'll try changing the dns and forcing AI_ADDRCONFIG. Lets see what happens.

As for windows 7, I took out my installation CDs for Windows Vista that came with my Lenovo T400. Installation, with all the crap customizations, took about 2 hours? (I just phased out and started playing GTA4 after a certain point). After that the goddamn updates took literally 12 hours (with all the restarts and my 2 MB shared connection). I ended up with a system that booted up in a minute, with a fingerprint reader that didnt work and reintroduced me to the general slowness that drove me to linux in the first place. So now I'm sitting here with lucid back on and checking proposed updates. I have horrible internet, but atleast I don't want to throw my laptop out the window.

By the way, I had a Live CD of openSuse 11.2 lying around (KDE) and I can confirm that I had no issues whatsoever with my internet when I installed that on my system as was the case with Jaunty. Fedora 13 and the latest PCLinuxOS, however, suffer from the same issue. Additionally, the problem is curiously confined only to my internet connection at home and not at work. I initially had the same router (Linksys WRT54G) at home and at work. I changed the one at home thinking that it may have been a router problem and got a DLink Wireless N router instead. That did not work. I have the same ISP at home and work but different modems: a ZyXEL P-600 at work and an Alcatel SpeedTouch Home at home. The modem at home is considerably older then the one at work. Additionally, I have also found that if the internet on my Lucid box is acting up at home, it slows down the internet for everyone else connected to the router. So the lucid lynx is not only managing to annoy me but also other windows users (my wife) as well!

I hope that these workarounds work - Lucid is actually the best OS I've used in a very long time.

[triage (triage-redhat-bugs)]

Fedora 11 changed to end-of-life (EOL) status on 2010-06-25. Fedora 11 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

[Szabolcs (szhorvat)]

It took forever for this to get fixed for Karmic, and now, after upgrading to Lucid, the bug is back. This is absolutely ridiculous. And no, most of us are not in a position to buy a new router or switch ISPs because Ubuntu gets randomly broken with every upgrade.

[Jeremy Visser (jeremy-visser)]

On the contrary, Ubuntu is not a position to deviate from pushing forward with IPv6 just because some of you have broken hardware.

[Derek (bugs-m8y)]

Jeremy. Member of the IPv6 taskforce eh.

Well, it is fortunate for me that the code snippet I posted in #288 and #290 worked, because otherwise Ubuntu's pushing forward would have pushed it right off our (large) corporate network.

We have 0 control over that infrastructure. So it was either eliminate the slow and steady introduction of Linux and more open services in general, or find a workaround for this *bug*.

There's ideology, and then there's pragmatism.

It may not work for everyone, but it'd be nice if something equiv to defaulting to AI_ADDRCONFIG without the need for that preload trick was made available in some alternate package that people could add, see if it works for them, and remove once transitions were complete.

[Jeroen Massar (massar)]

Dear Derek, there is a way to fix this problem in your large corporate network, like we did for that small corporate network that I am using: fix the resolvers. As you are claiming to have a large corporate network, you most likely have only a handful of recursors but you might have a 100k clients, lets see which ones are easier to upgrade, 100k clients which are all over the place or that 10 max or so recursors.... easy pick I would say.

The thing you most likely are forgetting is the fact that the DNS recursors that you are using are not only broken for AAAA records, but most likely for every single other address. Thus, by resolving this issue you will solve other magical problems too.
You can directly move on to support DNSSEC too for that matter if you are busy anyway.

Yes, the problem is annoying, no there is not much that Ubuntu or any other OS can do about this. Thus fix the problem in the right spot.

[Derek (bugs-m8y)]

This is where pragmatism comes in.
We have absolutely no control over those resolvers, and even if we had any influence whatsoever with those who did, corporate networks are very slow to change. Ubuntu is the outsider. The Windows machines work. Your solution is not a pragmatic one.

So, while being "pure" is good, I thought Ubuntu stayed out of such things.

That's why Ubuntu offers easy integration of binary drivers for ATI and nVidia, why Ubuntu has a restricted-extras for convenient meta.

Simply because IPv6 is *better* doesn't mean you should sacrifice adoption for the ideal.

Using AI_ADDRCONFIG is simple enough, and as noted browsers like Firefox and Chrome have adopted that.

What would be nice would be a simple package that forces it across the board, simply as an option for broken networks.

[tai133 (tairus166)]

I have to agree with Derek. With due respect to all the techs who do the
hard work of keeping Ubuntu (and especially Kubuntu in my case) so
great, I find that, as technical folks, we sometimes get overly focused
on the technical side and forget about the larger world in which that
exists. Sometimes the technically correct solution is not the right
solution in the real world, at least not at first.

Because of all the issues with Karmic, this should have been anticipated
and accounted for in Lucid. By this I mean that clear documentation,
fixes and workarounds should have been provided - if the problem could
not be accounted for silently in code. This problem is a huge hassle for
users who aren't up to speed on the technical side of connecting to the
internet. Those who are may look down on those who are not, but that's
no way to run an operating system.

Derek wrote:
> This is where pragmatism comes in.
> We have absolutely no control over those resolvers, and even if we had any influence whatsoever with those who did, corporate networks are very slow to change. Ubuntu is the outsider. The Windows machines work. Your solution is not a pragmatic one.
>
> So, while being "pure" is good, I thought Ubuntu stayed out of such
> things.
>
> That's why Ubuntu offers easy integration of binary drivers for ATI and
> nVidia, why Ubuntu has a restricted-extras for convenient meta.
>
> Simply because IPv6 is *better* doesn't mean you should sacrifice
> adoption for the ideal.
>
> Using AI_ADDRCONFIG is simple enough, and as noted browsers like Firefox
> and Chrome have adopted that.
>
> What would be nice would be a simple package that forces it across the
> board, simply as an option for broken networks.
>

--
Tai Sines
"Share your strengths, not your weaknesses." -- Yogi Bhajan

[JG (jg+launchpad)]

Thanks Derek. Your patch worked for me. I had already disabled IPv6 via sysctl and took all IPv6 addresses off my interfaces. The about:config solves firefox, but mutt and ssh were still a problem.

I'm a bit surprised at some of the suggested workarounds. I wouldn't really blame the resolvers - things shouldn't be doing AAAA lookups if ipv6 is disabled in the first place. It might be possible to blame the authors of virtually every network-aware app, but that isn't realistic.

Most of us running ubuntu in corporate networks with broken Microsoft resolvers are doing so completely unsupported. If you open a ticket, you'll be lucky if ignoring it is the worst that happens. More likely you'll be told to use a supported environment and just give them another reason why linux users are an expensive problem. "Go fix your resolvers" is just not a reasonable response. Using other DNS servers doesn't work in this case either, because they don't have access the intranet zones.

[alfredo (alacis)]

Hi, Folks.

I set up the function "getaddrinfo()" as specified in #288 & #290 above, but then lost connectivity to Samba shares on other machines in the local LAN.

When I commented-out the line "/usr/local/lib/getaddrinfo_wrap.so" in the file "/etc/ld.so.preload", instantly my Samba shares returned.

What now?

Alf

[Derek (bugs-m8y)]

Yeah, dunno what to say. WFM w/ my samba shares.
mount.cifs //intranetdev/wwwroot /home/nemo/Shares/intranet

That sorta thing.

Guess you're out of luck on that fix. Here's hoping something else works. Sorry.

[Derek (bugs-m8y)]

Well, kind of a good news/bad news situation.
Bad news. A little while ago my solution stopped working for me. Drove me absolutely batty.
I tried all the other things too, disable.ipv6=1 as a kernel parameter, various options in sysctl.conf that used to work, blacklisting any possible modules (not that they were loaded), and of course stripping down nsswitch.conf since all that mdns stuff had always caused unresolvable here.

Heck, I also tried:
        hints->ai_flags|=AI_ADDRCONFIG;
        hints->ai_flags|=AI_V4MAPPED;
        hints->ai_flags&=!AI_ALL;
in my wrapper, even though I have no idea if those would work, and specifying a struct addrinfo newhints; if hints were null.

Despite trying every conceivable way of saying NO I DO NOT WANT IPV6 CAUSE MY NETWORK SUCKS.
I still saw:
sudo tcpdump -an | grep 192.168.1.100
17:15:07.744171 IP 192.168.1.2.34702 > 192.168.1.100.53: 25164+ AAAA? reddit.com. (28)

(for a wget of reddit.com)

Anyway.
The happy ending is that recently a 3rd DNS resolver was added. The two broken ones are still broken, but so long as I explicitly specify only the new one in network settings and disable resolv.conf setup from DHCP, I'm fine.

I still have no idea what changed, but at least mine is working.

My sympathies for those of you still stuck in this situation.

[Tore Anderson (toreanderson)]

There is no question that the underlying problem here is defective DNS resolvers that choke on perfectly legitimate AAA queries. That said, there are a couple of issues present in software shipped by Ubuntu that cause the problem to manifest itself as slowdowns noticeable by end users:

1) When called with the AI_ADDRCONFIG flag, libc's getaddrinfo() function does not disregard link-local IPv6 addresses when determining whether or not the local host has usable IPv6 connectivity. Since every IPv6-capable OS will have link-local IPv6 addresses assigned to all interfaces - regardless of any external connectivity being available or not - this essentially makes AI_ADDRCONFIG on Linux useless for the purpose of suppressing AAAA queries when they're not useful.

I've submitted a bug to the GNU libc upstream about this issue at <http://sourceware.org/bugzilla/show_bug.cgi?id=12377>.

getaddrinfo() on other operating systems (such as Apple Mac OS X and Microsoft Windows) does disregard link-local IPv6 addresses when called with AI_ADDRCONFIG, which is why the problem appears to affect GNU/Linux distributions more than other operating systems.

2) Many applications do not set the AI_ADDRCONFIG flag when calling getaddrinfo(). This includes, notably, Mozilla Firefox. However, a patch to correct this has recently been committed to the mozilla-central developement repo and will likely be part of Firefox 4.0 beta 11 (hopefully also 3.6.15), see <https://bugzilla.mozilla.org/show_bug.cgi?id=614526>. Microsoft Windows enables the use of AI_ADDRCONFIG as the system-wide default, as far as I know, which explains why it is able to cope better with those broken middleware boxes. Mac OS X does not set AI_ADDRCONFIG by default, however it has an extremely short timeout waiting for AAAA responses after the A response has been answered (around 125ms), which in turn hides the problem from most end users. Additionally, most major browsers (except Firefox) do set AI_ADDRCONFIG explicitly, which suppress the problematic AAAA queries in the first place.

So what Ubuntu could to avoid this problem is 1) to develop and include a patch to glibc that makes getaddrinfo() ignore link-local addresses for AI_ADDRCONFIG purposes, and 2) to back-port the NSPR patch already committed to mozilla-central to the version of Firefox shipped (or wait until Mozilla releases a new version with the patch already included).

Tore

[tore (tore-redhat-bugs-1)]

There is no question that the underlying problem here is defective DNS resolvers that choke on perfectly legitimate AAA queries. That said, there are a couple of issues present in software shipped by Fedora that cause the problem to manifest itself as slowdowns noticeable by end users:

1) When called with the AI_ADDRCONFIG flag, libc's getaddrinfo() function does not disregard link-local IPv6 addresses when determining whether or not the local host has usable IPv6 connectivity. Since every IPv6-capable OS will have link-local IPv6 addresses assigned to all interfaces - regardless of any external connectivity being available or not - this essentially makes AI_ADDRCONFIG on Linux useless for the purpose of suppressing AAAA queries when they're not useful.

I've submitted a bug to the GNU libc upstream about this issue at <http://sourceware.org/bugzilla/show_bug.cgi?id=12377>.

getaddrinfo() on other operating systems (such as Apple Mac OS X and Microsoft Windows) does disregard link-local IPv6 addresses when called with AI_ADDRCONFIG, which is why the problem appears to affect GNU/Linux distributions more than other operating systems.

2) Many applications do not set the AI_ADDRCONFIG flag when calling getaddrinfo(). This includes, notably, Mozilla Firefox. However, a patch to correct this has recently been committed to the mozilla-central developement repo and will likely be part of Firefox 4.0 beta 11 (hopefully also 3.6.15), see <https://bugzilla.mozilla.org/show_bug.cgi?id=614526>. Microsoft Windows enables the use of AI_ADDRCONFIG as the system-wide default, as far as I know, which explains why it is able to cope better with those broken middleware boxes. Mac OS X does not set AI_ADDRCONFIG by default, however it has an extremely short timeout waiting for AAAA responses after the A response has been answered (around 125ms), which in turn hides the problem from most end users. Additionally, most major browsers (except Firefox) do set AI_ADDRCONFIG explicitly, which suppress the problematic AAAA queries in the first place.

So what Fedora could to avoid this problem is 1) to develop and include a patch to glibc that makes getaddrinfo() ignore link-local addresses for AI_ADDRCONFIG purposes, and 2) to back-port the NSPR patch already committed to mozilla-central to the version of Firefox shipped (or wait until Mozilla releases a new version with the patch already included).

Tore

[Johan (deberghes-johan)]

At last nomebody has understood the problem ! Well done !

I totally agree with your solution no 1, which is don't consider link-local
adresses (the ones which start with fe80:: ) as IPv6 adresses that can
resolve AAAA DNS records because that never happen and never will by design

[Neil (goofandfroggie)]

I'm glad to see it's not just me having this problem still.
I was give this little fix and works great. maybe a help, for give me if this has been posted already, there is a lot to read though.

sudo gedit /etc/resolv.conf

change: nameserver 10.1.1.1 (numbers maybe different on yours)
to: nameserver 8.8.8.8 and then save.

I would be interested if it works for other people.
this is only the way I can use Google earth Firefox & Thunderbird with out changing the ipv6 settings in Ff & T/bird. I can not use earth at all unless I change the nameserver, then all is good.
But I have to do this each time I start up.

[Martin Pitt (pitti)]

Hello,

Neil [2011-02-01 7:32 -0000]:
> I would be interested if it works for other people.

Yes, for me as well.

> But I have to do this each time I start up.

I created a script for that:

$ cat /etc/network/if-up.d/0nameserver
#!/bin/sh
grep -q Speedport_W_303V_Typ_B /etc/resolv.conf || exit 0
cat <<EOF > /etc/resolv.conf
nameserver 217.0.43.81
nameserver 217.0.43.65
EOF

The first line checks if I'm in my "home" network, as I only want to apply this
workaround when I'm at home.

(The script needs to be executable)

--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Mechanical snail (replicator-snail)]

>change: nameserver 10.1.1.1 (numbers maybe different on yours)
>to: nameserver 8.8.8.8 and then save.

I think this is just switching from your ISP's to Google's DNS server. Admittedly many ISPs' servers are broken, but changing the default warrants more discussion.

[Tore Anderson (toreanderson)]

FYI, today Mozilla released Firefox 4.0 beta 11, which now calls getaddrinfo() with the AI_ADDRCONFIG flag. You get it from http://www.firefox.com/beta/.

This solves half of the problem. The remaining piece is now to make glibc ignore link-local IPv6 addresses when called with the AI_ADDRCONFIG flag. (This is how all other major operating systems behave already.) I have a bug open in the glibc bugzilla at http://sourceware.org/bugzilla/show_bug.cgi?id=12377 - it would be really great if the Ubuntu glibc developers could help out by writing a suitable patch and attach it to the bug report. I don't think it should be very hard (just extend the already existing logic that ignores loopback addresses). Unfortunately, I'm not much of a programmer myself...

Tore

[Tore Anderson (toreanderson)]

Here's one half of the solution - it's a patch to glibc that makes getaddrinfo() ignore link-local addresses when called with the AI_ADDRCONFIG flag set. This makes getaddrinfo() avoid querying for AAAAs when the host has no IPv6 connectivity, provided that the AI_ADDRCONFIG flag is set.

Tore

[Tore Anderson (toreanderson)]

Here's the second half of the solution. It's a patch that makes Mozilla Firefox use AI_ADDRCONFIG when calling getaddrinfo(). Note that the Mozilla release drivers have already approved this patch for inclusion on the 3.6.x branch, and it has already been commited to Firefox 4.0 (it's included in beta11).

Tore

[tore (tore-redhat-bugs-1)]

Created attachment 478268
Solution 1/2: Make getaddrinfo()+AI_ADDRCONFIG ignore link-locals

[tore (tore-redhat-bugs-1)]

Created attachment 478270
Solution 2/2: Make Mozilla Firefox use AI_ADDRCONFIG when calling getaddrinfo()

[tore (tore-redhat-bugs-1)]

The two patches I've just attached solves this problem for most users:

The first makes getaddrinfo() ignore link-local addresses when called with the AI_ADDRCONFIG flag set. This makes getaddrinfo() avoid querying for AAAAs when the host has no IPv6 connectivity, provided that the AI_ADDRCONFIG flag is set. This brings glibc's getaddrinfo() behaviour in line with Mac OS X and Windows.

The second makes Mozilla Firefox use AI_ADDRCONFIG when calling getaddrinfo(). Note that the Mozilla release drivers have already approved this patch for inclusion on the 3.6.x branch, and it has already been commited to Firefox 4.0 (it's included in beta11).

Please apply.

(Of course, there might be applications other than Mozilla Firefox that does not set AI_ADDRCONFIG as well, which would require similar patches. However, Mozilla Firefox is the obvious one and likely the source of most user complaints.)

Tore

[gene (eugenios)]

Unbelievable, this bug still manages to bug people on the latest and fully updated Ubuntu 11.04!
The strangeness of the situation is as follows:

ubuntu 11.04, uname -a:
Linux 3.0.0-mine #3 SMP Thu Jul 28 14:03:44 CDT 2011 i686 i686 i386 GNU/Linux

Where, with firefox 5.0, epiphany, chromium some websites take very long time to load, while everything else involving connection is fast. E.g., w3m, lynx and especially elinks are extremely fast! At the same time on!!!! On the

ubuntu 10.04:
 uname -a
Linux 2.6.35.13-mine #1 SMP Fri May 6 00:20:57 CDT 2011 x86_64 GNU/Linux
Exact same browsers are almost as fast as their text-based brethren.

So, IMHO, the problem resides not with the actual browser(s) but with something else.
This is getting ridiculous!

[gene (eugenios)]

Forgot to mention, that neither disabling ipv6 completely, nor "playing" with the /etc/nsswitch.conf works.
Now since this bug is filed against Karmic, I wonder do I have to make my bug a duplicate? In case if I see it on my machine.

[Neil (goofandfroggie)]

Yet still some problems for me to but I must say only with Google earth and Ubuntu Tweek g/earth wont "connect" and tweek cant get the updates. but if I set sudo gedit /etc/resolv.conf and set the servername to 8.8.8.8 they will work. as I said earlier in comment #312

[Joel Eidsath (jeidsath)]

I have this issue with ssh in Ubuntu 11.10. Installing the power-dns resolver as mentioned in an earlier comment worked for me.

To install pdns-resolver, I I set my nameserver to 127.0.0.1 in /etc/resolv.conf and followed these instructions:
http://www.thatfleminggent.com/2009/08/09/getting-a-powerdns-recursor-up-and-going-fast

[Javier Vilalta (jvilalta)]

I'm not sure if this is the same bug I'm experiencing, but if I try to access a domain without IPv6 address, I get this on tshark:

  0.000000 192.168.2.103 -> 192.168.2.254 DNS 74 Standard query AAAA one.ubuntu.com
  0.074500 192.168.2.254 -> 192.168.2.103 DNS 135 Standard query response
  0.074682 192.168.2.103 -> 192.168.2.254 DNS 95 Standard query AAAA one.ubuntu.com.internal.eudemo.info
  0.075854 192.168.2.254 -> 192.168.2.103 DNS 95 Standard query response, No such name
  0.075991 192.168.2.103 -> 192.168.2.254 DNS 74 Standard query A one.ubuntu.com
  0.147486 192.168.2.254 -> 192.168.2.103 DNS 106 Standard query response A 91.189.89.219 A 91.189.89.218

As you can see, between the AAAA and the A resolution there's a wrong query with my local domain added: this is the one which takes a few seconds to fail (not in this case because I have setup my dnsmasq with local=/internal.eudemo.info/ to get a fast response)
I have tested it with both Firefox and Chrome and both do the same, so I assume is a system problem. Is this the same problem or I need to open a separate bug report (or something is wrong with my setup)?

[Stéphane Graber (stgraber)]

New patches have been proposed a few days ago on redhat's bugtracker at https://bugzilla.redhat.com/show_bug.cgi?id=505105

[Tore Anderson (toreanderson)]

Stéphane, the same patch was posted in this bug as well, see comment #316. (The one in #317 is no longer necessary, as it's been included in the NSPR upstream code for a long time now.)

Tore

[JoeKlein (jsklein)]

LinkedIn
------------

Bug,

I'd like to add you to my professional network on LinkedIn.

- Joe

Joe Klein
Security Researcher at IPv6 Cyber Security Forum
Washington D.C. Metro Area

Confirm that you know Joe Klein:
https://www.linkedin.com/e/vie2u9-h1zn0mwv-2f/isd/7022549566/G7SZOZHv/?hs=false&tok=3uKWta_34LVlc1

--
You are receiving Invitation to Connect emails. Click to unsubscribe:
http://www.linkedin.com/e/vie2u9-h1zn0mwv-2f/Vwq1u_xoiZpYl-DoyfGWOkloZaoeXvNdJwvY8FM/goo/417757%40bugs%2Elaunchpad%2Enet/20061/I2403245545_1/?hs=false&tok=3LBymgQqwLVlc1

(c) 2012 LinkedIn Corporation. 2029 Stierlin Ct, Mountain View, CA 94043, USA.

[dhenry (tfc-duke)]

This LinkedIn invitation is a bit odd : Bug can't reply to you :)

[JoeKlein (jsklein)]

LinkedIn
------------

Bug,

I'd like to add you to my professional network on LinkedIn.

- Joe

Joe Klein
Security Researcher at IPv6 Cyber Security Forum
Washington D.C. Metro Area

Confirm that you know Joe Klein:
https://www.linkedin.com/e/vie2u9-h8qdlhrc-u/isd/7022549566/G7SZOZHv/?hs=false&tok=2v3uvX5CP0E5s1

--
You are receiving Invitation to Connect emails. Click to unsubscribe:
http://www.linkedin.com/e/vie2u9-h8qdlhrc-u/Vwq1u_xoiZpYl-DoyfGWOkloZaoeXvNdJwvY8FM/goo/417757%40bugs%2Elaunchpad%2Enet/20061/I3098597473_1/?hs=false&tok=3e7QtgmV70E5s1

(c) 2012 LinkedIn Corporation. 2029 Stierlin Ct, Mountain View, CA 94043, USA.

[tore (tore-redhat-bugs-1)]

Okay, so this is still a problem. What happens is:

1) the user enters some host name into his web browser or other application of choice running on a machine connected to an IPv4-only Ethernet network
2) the application kicks of an getaddrinfo() call for the host name, using AF_UNSPEC and AI_ADDRCONFIG
3) getaddrinfo() transmits both IN A (IPv4) and IN AAAA (IPv6) DNS queries to the upstream resolver
4) The upstream resolver, which is typically some cheapo home gateway or something, don't understand the IN AAAA queries and either doesn't respond to them at all, or screw them up somehow
5) getaddrinfo() doesn't get a valid answer for the IN AAAA queries (valid answer could include NXDOMAIN or NODATA status codes), retransmits them, sits around waiting
6) user wonders why the web page or whatever takes "forever" to load, goes to submit/comment on bugs such as this one
7) getaddrinfo() finally times out the IN AAAA queries, returns IPv4 results to the application
8) lather rinse repeat

AI_ADDRCONFIG *should* have solved this issue, by suppressing IN AAAA queries from IPv4-only machines. However, the auto-configured IPv6 link local addresses on all Ethernet interfaces, causes getaddrinfo() to consider that the machine has IPv6, and therefore it won't suppress IN AAAA queries anymore. More info here:

https://fedoraproject.org/wiki/Networking/NameResolution/ADDRCONFIG#Problem_2:_IN_AAAA_DNS_query_suppression_from_Ethernet-connected_IPv4-only_hosts

[psimerda (psimerda-redhat-bugs)]

I see no reason why this shouldn't be fixed. We are working on solutions, all information here:

https://fedoraproject.org/wiki/Networking/NameResolution/ADDRCONFIG

Related fedora feature page:

https://fedoraproject.org/wiki/Features/DualstackNetworking

Adding to the 'dualstack' tracker bug and modified the summary.

[psimerda (psimerda-redhat-bugs)]

*** Bug 697149 has been marked as a duplicate of this bug. ***

[Pavel Šimerda (pavlix-a)]

I would like to add new information and research that has been done in the Fedora project:

https://fedoraproject.org/wiki/Networking/NameResolution/ADDRCONFIG

It links to related fedora bug reports which in turn link to upstream bug reports. It contains enough information about what is required to solve dualstack getaddrinfo() problems.

We are working on this and invite anyone from the community to help us get rid of dualstack-related name resolution problems. Feel free to contact us. Contacts at the Fedora feature page:

https://fedoraproject.org/wiki/Features/DualstackNetworking

Or contact me directly:

https://fedoraproject.org/wiki/User:Pavlix

[psimerda (psimerda-redhat-bugs)]

*** Bug 459756 has been marked as a duplicate of this bug. ***

[fedora-admin-xmlrpc (fedora-admin-xmlrpc-redhat-bugs)]

This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.

[bcotton (bcotton-redhat-bugs)]

This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

[markzzzsmith (markzzzsmith-redhat-bugs)]

I disagree with making getaddrinfo() consider a host with only IPv6 link-local addresses to not have IPv6 connectivity. It does, it has IPv6 connectivity to all other IPv6 hosts on the directly attached links which also all have link-local addresses. This is actually the point of hosts automatically configuring link-local addresses on all interfaces all the time, and the IPv6 Addressing Architecture specifying that all interfaces must have link-local addresses - so that they can at a minimum always reach their on-link neighbors via link-local addressing. This is also why protocols such as IPv6 neighbor discovery, Multicast Listener Discovery and routing protocols such as OSPF use link-local addresses as source and/or destination addresses to reach their neighbors.

Combine autoconfigured IPv6 link-local addresses with a service discovery protocol such as Multicast DNS or SSDP, and you have Zero Configuration networking without any user intervention. Compare that with IPv4, where support for 169.254.0.0/16 is patchy, because it is done in userspace via DHCPv4. IPv6 will universally and reliably provide zero configuration networking.

I think it is reasonable to hosts more robust against broken devices in the network, but ignoring IPv6 link-local connectivity and then suppressing AAAA queries is not the solution. Happy Eyeballs (RFC6555) and IPv6 source and destination selection (RFC6724) are.

[tore (tore-redhat-bugs-1)]

(In reply to comment #64)
> I disagree with making getaddrinfo() consider a host with only IPv6
> link-local addresses to not have IPv6 connectivity.

That's not what this bug report is about. It's about suppressing DNS "IN AAAA" queries if the host only has link-local addresses and AI_ADDRCONFIG is supplied.

> Combine autoconfigured IPv6 link-local addresses with a service discovery
> protocol such as Multicast DNS or SSDP,

This bug report is specifically about DNS. It's not about MDNS, SSDP, /etc/hosts, or any other NSS backends.

> I think it is reasonable to hosts more robust against broken devices in the
> network, but ignoring IPv6 link-local connectivity and then suppressing AAAA
> queries is not the solution.

How is it useful for a host with only link-local addresses to perforn "IN AAAA" DNS queries? Keep in mind that in order to use a link-local address, you need to supply an interface scope (e.g., "fe80::1%eth0"), and that DNS cannot supply this information.

Tore

[markzzzsmith (markzzzsmith-redhat-bugs)]

(In reply to comment #65)
> (In reply to comment #64)
> > I disagree with making getaddrinfo() consider a host with only IPv6
> > link-local addresses to not have IPv6 connectivity.
>
> That's not what this bug report is about. It's about suppressing DNS "IN
> AAAA" queries if the host only has link-local addresses and AI_ADDRCONFIG is
> supplied.
>

I understand that.

The AI_ADDRCONFIG flag does not preclude link-local addresses. From RFC3493:

" If the AI_ADDRCONFIG flag is specified, IPv4 addresses shall be
   returned only if an IPv4 address is configured on the local system,
   and IPv6 addresses shall be returned only if an IPv6 address is
   configured on the local system. The loopback address is not
   considered for this case as valid as a configured address."

Note that loopback addresses are, so the designers specifically thought about exclusion of addresses types.

> > Combine autoconfigured IPv6 link-local addresses with a service discovery
> > protocol such as Multicast DNS or SSDP,
>
> This bug report is specifically about DNS. It's not about MDNS, SSDP,
> /etc/hosts, or any other NSS backends.
>

getaddrinfo() is not a DNS protocol specific API call, and is used in front of all those NSS backends, so that applications don't have to be exposed to how the address information was determined. For example, I run MDNS at home, and when I enable it, all of my IPv6 applications automatically work with it.

Here is what RFC3493 describes it as:

6.1 Protocol-Independent Nodename and Service Name Translation

   Nodename-to-address translation is done in a protocol-independent
   fashion using the getaddrinfo() function.

getaddrinfo() can return all the information necessary to use a link-local address i.e. both the address, and in the interface index, via the sin6_scope_id field of the sockaddr_in6 structure that is returned via the ai_addr field.

By making AI_ADDRCONFIG ignore link-local addresses, the getaddrinfo() call becomes broken for NSS backends that can provide both the link-local address and the corresponding interface index, such as MDNS or any other future ones.

Perhaps the DNS NSS backend could provide it, by returning the interface index of the interface it received the response on if a link-local address is returned. On the common single-homed host, this is likely to be the correct interface index for the link-local address.

> > I think it is reasonable to hosts more robust against broken devices in the
> > network, but ignoring IPv6 link-local connectivity and then suppressing AAAA
> > queries is not the solution.
>
> How is it useful for a host with only link-local addresses to perforn "IN
> AAAA" DNS queries? Keep in mind that in order to use a link-local address,
> you need to supply an interface scope (e.g., "fe80::1%eth0"), and that DNS
> cannot supply this information.
>

Something else outside of DNS could provide the interface information, and the application combines them. Specifying a hostname (perhaps in /etc/hosts) and an interface will be much simpler than specifying literal link-local addresses because getaddrinfo() won't lookup IPv6 addresses when the host only has link-local addresses.

T...

[horsley1953 (horsley1953-redhat-bugs)]

I thought the obvious problem with this was addressed way up near the top in comment 20 when someone pointed out that using the same port for the IPv4 and IPv6 queries gave firewalls fits. Ulrich Drepper had one of his standard "purity over practicality" tantrums and refused to change it to use two different ports to accommodate dumb firewalls, but since Ulrich is gone now, perhaps saner heads could revisit that? (And perhaps revisit all other bug fixes rejected over the years by Ulrich? :-).

[psimerda (psimerda-redhat-bugs)]

(In reply to comment #66)
> The AI_ADDRCONFIG flag does not preclude link-local addresses. From RFC3493:
>
>
> " If the AI_ADDRCONFIG flag is specified, IPv4 addresses shall be
> returned only if an IPv4 address is configured on the local system,
> and IPv6 addresses shall be returned only if an IPv6 address is
> configured on the local system. The loopback address is not
> considered for this case as valid as a configured address."
>
> Note that loopback addresses are, so the designers specifically thought
> about exclusion of addresses types.

We are not discussing the designers' virtues but the technical issues. The RFC is (1) INFORMATIONAL and (2) wrong. For more detailed information, see:

https://fedoraproject.org/wiki/Networking/NameResolution/ADDRCONFIG

(any comments of technical value welcome)

> For example, I run MDNS at home,
> and when I enable it, all of my IPv6 applications automatically work with it.

This is not true. Just try mDNS with link-local addresses (which you mentioned) and you will realize that this feature is absent with the current glibc and nss-mdns.

> getaddrinfo() can return all the information necessary to use a link-local
> address i.e. both the address, and in the interface index, via the
> sin6_scope_id field of the sockaddr_in6 structure that is returned via the
> ai_addr field.

getaddrinfo() can, while the NSS backends cannot. Therefore currently getaddrinfo() would only return scope_id for IPv6 literals, not mDNS nor any similar protocol.

> By making AI_ADDRCONFIG ignore link-local addresses, the getaddrinfo() call
> becomes broken for NSS backends

Currently false. You can't break a feature that is absent.

> Perhaps the DNS NSS backend could provide it,

You don't need scope_id for global addresses and you don't need this information with DNS responses at all.

> Something else outside of DNS could provide the interface information, and
> the application combines them.

I don't see the need for that. DNS returns global addresses. Global addresses don't need scope_id.

> Specifying a hostname (perhaps in /etc/hosts)

Not sure whether /etc/hosts can be used to provide scope_id.

> and an interface will be much simpler

There is currently no standard way to do that. And I don't think it is valuable enough to seek standardization for that.

> than specifying literal link-local
> addresses because getaddrinfo() won't lookup IPv6 addresses when the host
> only has link-local addresses.

There's a much easier solution. Just don't apply the same rules to mDNS you apply to DNS. I believe all of this is already described in:

https://fedoraproject.org/wiki/Networking/NameResolution/ADDRCONFIG

> The "Happy Eyeballs" technique (RFC6555) wasn't just intended to be applied
> to web browsers, according to this draft from Fred Baker:
>
> "Happier Eyeballs"
> https://www.ietf.org/id/draft-baker-happier-eyeballs-00.txt
>
> and could probably be applied to the DNS "application". For example, off the
> top of my head:
>
> 1) issue a standard DNS query including both A and AAAA queries.

In the case described by this bug report, there's no need to query AAAA as global routing is not available anywa...

[tore (tore-redhat-bugs-1)]

Pavel already responded to most of your points, so I'll try to avoid just repeating his points.)

(In reply to comment #66)

> getaddrinfo() is not a DNS protocol specific API call, and is used in front
> of all those NSS backends, so that applications don't have to be exposed to
> how the address information was determined. For example, I run MDNS at home,
> and when I enable it, all of my IPv6 applications automatically work with it.

Well, again, this isn't about MDNS. Nobody is suggesting to make the MDNS backend ignore link-locals if called from getaddrinfo() w/AI_ADDRCONFIG. This bug is specifically about the DNS backend's behaviour; MDNS is out of scope.

> By making AI_ADDRCONFIG ignore link-local addresses, the getaddrinfo() call
> becomes broken for NSS backends that can provide both the link-local address
> and the corresponding interface index, such as MDNS or any other future ones.

See above, this is about the DNS backend *only*.

> Perhaps the DNS NSS backend could provide it, by returning the interface
> index of the interface it received the response on if a link-local address
> is returned. On the common single-homed host, this is likely to be the
> correct interface index for the link-local address.

This is a flawed assumption, even in the single-homed host case. One obvious example: If you run a local caching resolver (which I believe NetworkManager has native support for doing these days), you'll end up with all the returned link-local addresses being scoped to the "lo" interface, which is probably not what you want.

> Something else outside of DNS could provide the interface information, and
> the application combines them. Specifying a hostname (perhaps in /etc/hosts)
> and an interface will be much simpler than specifying literal link-local
> addresses because getaddrinfo() won't lookup IPv6 addresses when the host
> only has link-local addresses.

It would appear to me that the proper thing for such an application to do is to simply not use AI_ADDRCONFIG. However, does such an application actually exist, or are you inventing it just to support your position?

> 1) issue a standard DNS query including both A and AAAA queries.

Not possible. There is no single DNS query that requests both A and AAAA responses. (If, by any chance, you want to say "ANY" right now - don't, it doesn't do what you think it does.)

> 2) if no response is received after 400ms (roughly half way around the
> world), issue two individual queries, one for an A and one for an AAAA.

Two individual queries is what's being done today, and that's the only thing you can do. Also, 400ms, even multiple seconds, is too short a timeout - the major part a DNS lookup isn't the single RTT to the resolver listed in /etc/resolv.conf - it's waiting for that resolver to actually find the record in question. This is the sum of all RTTs to all the authoritative name servers in the delegation chain, potentially including timeouts and retransmits at some of the steps.

The only way to get Happy Eyeballs-ish behaviour using getaddrinfo() is if you run an IPv4-only thread with getaddrinfo(AF_INET)->connect(AF_INET), and a similar one for AF_INET6. You can't do it wit...

[markzzzsmith (markzzzsmith-redhat-bugs)]

(In reply to comment #69)
> Pavel already responded to most of your points, so I'll try to avoid just
> repeating his points.)
>
> (In reply to comment #66)
>
> > getaddrinfo() is not a DNS protocol specific API call, and is used in front
> > of all those NSS backends, so that applications don't have to be exposed to
> > how the address information was determined. For example, I run MDNS at home,
> > and when I enable it, all of my IPv6 applications automatically work with it.
>
> Well, again, this isn't about MDNS. Nobody is suggesting to make the MDNS
> backend ignore link-locals if called from getaddrinfo() w/AI_ADDRCONFIG.
> This bug is specifically about the DNS backend's behaviour; MDNS is out of
> scope.
>

There were no qualifiers on your described behaviour. You may have been talking about DNS, but the description of the change of behaviour to AI_ADDRCONFIG did not specify that it was limited to the DNS backend.

> > By making AI_ADDRCONFIG ignore link-local addresses, the getaddrinfo() call
> > becomes broken for NSS backends that can provide both the link-local address
> > and the corresponding interface index, such as MDNS or any other future ones.
>
> See above, this is about the DNS backend *only*.
>

Again, you had no qualifiers.

> > Perhaps the DNS NSS backend could provide it, by returning the interface
> > index of the interface it received the response on if a link-local address
> > is returned. On the common single-homed host, this is likely to be the
> > correct interface index for the link-local address.
>
> This is a flawed assumption, even in the single-homed host case. One obvious
> example: If you run a local caching resolver (which I believe NetworkManager
> has native support for doing these days), you'll end up with all the
> returned link-local addresses being scoped to the "lo" interface, which is
> probably not what you want.
>

Then the cache is broken. It should be caching all the information that would be returned in the sockaddr structure returned to getaddrinfo(), not just the returned IP addresses.

> > Something else outside of DNS could provide the interface information, and
> > the application combines them. Specifying a hostname (perhaps in /etc/hosts)
> > and an interface will be much simpler than specifying literal link-local
> > addresses because getaddrinfo() won't lookup IPv6 addresses when the host
> > only has link-local addresses.
>
> It would appear to me that the proper thing for such an application to do is
> to simply not use AI_ADDRCONFIG. However, does such an application actually
> exist, or are you inventing it just to support your position?
>

I don't know if an application like this exists, but I doubt you know absolutely that it doesn't exist. Where are the restrictions that say such an application can't exist? The definition of AI_ADDRCONFIG didn't prohibit them, or even make recommendations against them.

You're asserting that link-locals aren't being stored in DNS. How do you know that? Have you queried all the DNS space in the world?

There has been no prohibitions on link-local addresses being put in DNS, and now you are creating one, and are asserting that as you've...

[tore (tore-redhat-bugs-1)]

(In reply to comment #70)

> There were no qualifiers on your described behaviour. You may have been
> talking about DNS, but the description of the change of behaviour to
> AI_ADDRCONFIG did not specify that it was limited to the DNS backend.

The title of this bug is:

«getaddrinfo() with AI_ADDRCONFIG doesn't suppress ****AAAA DNS queries**** on IPv4-only networks»

(emphasis mine)

If that's not a crystal clear qualifier, I don't know what is.

> Then the cache is broken. It should be caching all the information that
> would be returned in the sockaddr structure returned to getaddrinfo(), not
> just the returned IP addresses.

glibc speaks to a caching resolver (e.g. dnsmasq) on 127.0.0.1 or ::1, using the regular DNS protocol. The DNS protocol has no means of communicating an interface scope ID. So how exactly would this work?

> You're asserting that link-locals aren't being stored in DNS. How do you
> know that? Have you queried all the DNS space in the world?

No, but I am asserting that storing link-locals in DNS is completely pointless, as it cannot possibly work, because there is no way the DNS protocol can communicate a scope ID to the querier.

> There has been no prohibitions on link-local addresses being put in DNS, and
> now you are creating one, and are asserting that as you've never seen reason
> to do so, nobody else has either.
>
> Here is a realistic scenario where link-local addresses would usefully be
> stored in DNS.
>
> An organisation may want to create organisation wide unique static
> link-local addresses, assigning them to their routers' interfaces. This
> would make the link-local addresses independent of the MAC addresses of the
> routers interfaces, and would also make the use of link-local addresses as
> e.g., static route next hops, simpler and less error prone because there are
> no intentional duplicates. e.g., their first router's first configured
> interface would have fe80::1, their e.g. 10th router's first configured
> interface might be fe80::15, depending on how many interfaces the other
> routers have.
>
> To document the static link local addresses the following sorts of DNS
> records are created (using router10, interface eth0 as an example)
>
> eth0.rtr10.example.com. IN AAAA fe80::15
> eth0.rtr10.example.com. IN TXT "Ethernet 0 on Router 10, MAC addr
> 02:00:00:00:00:01"
>
> 5.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa.
> IN PTR eth0.rtr10.example.com.

Red herring. Making getaddrinfo() with AI_ADDRCONFIG suppress IN AAAA queries have nothing to do with this use of DNS as essentially a documentation tool, and would not "prohibit" this in any way.

What you can't do, though, is e.g. "ssh eth0.rtr10.example.com" - regardless of presence of IPv4 addresses on the host, IPv6 addresses on the host, and whether or not the ssh application uses AI_ADDRCONFIG.

> The happy eyeball behaviour would be within the DNS backend, hidden from the
> application behind the getaddrinfo(,AI_ADDRCONFIG) call.

Well, you can say that this behaviour is already present within getaddrinfo(). If called without AI_ADDRCONFIG (or with on a host that has the required global addresses configur...

[markzzzsmith (markzzzsmith-redhat-bugs)]

(In reply to comment #71)
> (In reply to comment #70)
>
> > There were no qualifiers on your described behaviour. You may have been
> > talking about DNS, but the description of the change of behaviour to
> > AI_ADDRCONFIG did not specify that it was limited to the DNS backend.
>
> The title of this bug is:
>
> «getaddrinfo() with AI_ADDRCONFIG doesn't suppress ****AAAA DNS queries****
> on IPv4-only networks»
>
> (emphasis mine)
>
> If that's not a crystal clear qualifier, I don't know what is.
>

You continue to miss the point. *Your* proposal on how to *fix* the problem was to change the behaviour of AI_ADDRCONFIG, regardless of the backend, because you didn't *specify* what backend your solution applied to.

Even then, the title is not actually saying what the problem is. The actual problem is CPE or DNS servers that did not handle IPv6 AAAA queries correctly.

> > Then the cache is broken. It should be caching all the information that
> > would be returned in the sockaddr structure returned to getaddrinfo(), not
> > just the returned IP addresses.
>
> glibc speaks to a caching resolver (e.g. dnsmasq) on 127.0.0.1 or ::1, using
> the regular DNS protocol.

It can also speak to a caching resolver directly, as it does with nscd. That is the cache that I though you were talking about, because it is part of glibc.

> The DNS protocol has no means of communicating an
> interface scope ID. So how exactly would this work?
>
> > You're asserting that link-locals aren't being stored in DNS. How do you
> > know that? Have you queried all the DNS space in the world?
>
> No, but I am asserting that storing link-locals in DNS is completely
> pointless,

Well, as you saw, I pointed out a valid and reasonable use for storing link-locals in DNS, so it isn't completely pointless.

> as it cannot possibly work, because there is no way the DNS
> protocol can communicate a scope ID to the querier.
>

It doesn't need to, that information can be gleaned from some other source, such as a command line option, or a configuration file. In this instance, DNS is still useful, because it is providing a much simpler and easier to type name for an IPv6 link-local address, even though in itself it isn't enough information to use the returned link-local address by itself.

> > There has been no prohibitions on link-local addresses being put in DNS, and
> > now you are creating one, and are asserting that as you've never seen reason
> > to do so, nobody else has either.
> >
> > Here is a realistic scenario where link-local addresses would usefully be
> > stored in DNS.
> >
> > An organisation may want to create organisation wide unique static
> > link-local addresses, assigning them to their routers' interfaces. This
> > would make the link-local addresses independent of the MAC addresses of the
> > routers interfaces, and would also make the use of link-local addresses as
> > e.g., static route next hops, simpler and less error prone because there are
> > no intentional duplicates. e.g., their first router's first configured
> > interface would have fe80::1, their e.g. 10th router's first configured
> > interface might be fe80::15, depending on how many int...

[tore (tore-redhat-bugs-1)]

(In reply to comment #72)

> You continue to miss the point. *Your* proposal on how to *fix* the problem
> was to change the behaviour of AI_ADDRCONFIG, regardless of the backend,
> because you didn't *specify* what backend your solution applied to.

My very first response to you in this thread, in comment #65, began like this:

«That's not what this bug report is about. It's about suppressing DNS "IN AAAA" queries [...]»

So how you can claim that I am not clear about talking specifically about DNS is beyond me, to be honest. It should in any case be clear by now, I hope.

> Even then, the title is not actually saying what the problem is. The actual
> problem is CPE or DNS servers that did not handle IPv6 AAAA queries
> correctly.

Such CPEs and DNS servers are buggy, true. However, it is in our users' best interest to help them avoid tickling these bugs, because it leads to crappy user experiences and bug reports with a huge number of subscribers:

https://bugzilla.redhat.com/show_bug.cgi?id=459756
https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/417757

It's sucks extra, because this is perceived to be a Linux-specific problem. MS Windows and Apple Max OS X does interpret the AI_ADDRCONFIG flag in the proposed way (i.e., it will suppress IN AAAA queries if the host only has link-local addresses configured. (I haven't verified that this behaviour still is in place in the latest versions of those operating systems though.)

> > glibc speaks to a caching resolver (e.g. dnsmasq) on 127.0.0.1 or ::1, using
> > the regular DNS protocol.
>
> It can also speak to a caching resolver directly, as it does with nscd. That
> is the cache that I though you were talking about, because it is part of
> glibc.

NSCD is *not* a resolver. NSCD knows nothing of AAAA queries or the DNS protocol at all. The only thing NSCD can do is to cache results that came from NSS backends, such as - you guessed it - DNS.

> Well, as you saw, I pointed out a valid and reasonable use for storing
> link-locals in DNS, so it isn't completely pointless.

This is still a red herring. If you use DNS as a documentation tool like you've outlined, there's no reason why you'd use AI_ADDRCONFIG when extracting the records. Otherwise the "documentation" would look different when read on a computer with no IPv6 addresses (not even link-locals), or on a Mac/Windows computer with IPv6 link-locals, than it would when read on a machine with global IPv6 (or a Linux machine with IPv6 link-locals). I find it far-fetched that anyone would use getaddrinfo() for "reading" such "DNS documentation" to begin with, as you cannot retrieve your TXT records with it, for example.

> > as it cannot possibly work, because there is no way the DNS
> > protocol can communicate a scope ID to the querier.
>
> It doesn't need to, that information can be gleaned from some other source,
> such as a command line option, or a configuration file.

FWIW, it simply doesn't work in a fully updated Fedora 18:

tore@wrath:~$ host ll.fud.no
ll.fud.no has IPv6 address fe80::230:1bff:febc:7f23
tore@wrath:~$ ssh ll.fud.no%eth0
ssh: Could not resolve hostname ll.fud.no%eth0: Name or service not known
tore@wrath:~$ ssh f...

[psimerda (psimerda-redhat-bugs)]

(In reply to comment #73)
> My very first response to you in this thread, in comment #65, began like
> this:
>
> «That's not what this bug report is about. It's about suppressing DNS "IN
> AAAA" queries [...]»
>
> So how you can claim that I am not clear about talking specifically about
> DNS is beyond me, to be honest. It should in any case be clear by now, I
> hope.

I think it is crystal clear and if anyone wants to have a good summary, there's still:

https://fedoraproject.org/wiki/Networking/NameResolution/ADDRCONFIG

> > Even then, the title is not actually saying what the problem is. The actual
> > problem is CPE or DNS servers that did not handle IPv6 AAAA queries
> > correctly.
>
> Such CPEs and DNS servers are buggy, true. However, it is in our users' best
> interest to help them avoid tickling these bugs, because it leads to crappy
> user experiences and bug reports with a huge number of subscribers:

+1

> It's sucks extra, because this is perceived to be a Linux-specific problem.
> MS Windows and Apple Max OS X does interpret the AI_ADDRCONFIG flag in the
> proposed way (i.e., it will suppress IN AAAA queries if the host only has
> link-local addresses configured.

Please keep being specific whether you're talking about DNS or generally. I don't think the Apple folks would break their link-local name resolution deliberately.

> NSCD is *not* a resolver.

+1

Using NSCD with network name resolution and AI_ADDRCONFIG sounds dangerous to me.

> > Well, as you saw, I pointed out a valid and reasonable use for storing
> > link-locals in DNS, so it isn't completely pointless.

This is irrelevant to the problem in this bug report, as NSS backends currently don't convey scope_id at all.

With that in mind, I think we should stop polluting this bug report with link-local in DNS as it's irrelevant in the current situation. Start a new bug report and link it from here, if you're still interested, and describe your use case there.

> The standard CLI frontend for getaddrinfo(), "getent ahosts", *doesn't* use
> AI_ADDRCONFIG, for a very good reason.

+1

According to my own micro-research, AI_ADDRCONFIG only good for one specific purpose which is a loop over getaddrinfo results with connect() in each step.

https://fedoraproject.org/wiki/Networking/NameResolution#Connecting_to_services_using_getaddrinfo.28.29

> AI_ADDRCONFIG is counter-productive if your ultimate goal is to learn what
> address records are in DNS, because it would arbitrarily hide records from
> you depending on the machine you're running it on - IN AAAA records would be
> hidden on IPv4-only machines, and IN A records would be hidden on IPv6-only
> machines.

+1

> AI_ADDRCONFIG is only useful if the main goal isn't to dump the addresses,
> but to actually get an address that in turn will be used as a destination
> for communication with. As in, when getaddrinfo() is just a mandatory step
> towards the ultimate goal making a connect() somewhere.

Exactly. Any other discussions than those related to getaddrinfo()+connect() should be kept off this bug report.

> host tore@wrath:~$ host ll.fud.no
> ll.fud.no has IPv6 address fe80::230:1bff:febc:7f23
> tore@wrath:~$ ssh ll...

[Faye Salwin (faye-salwin)]

In the hope that this helps someone. I spent most of today fighting this and found a solution.

d-i preseed/early_command string grep -q options /etc/resolv.conf || echo "options single-request" >> /etc/resolv.conf ;

and then

d-i preseed/late_command string grep -q options /etc/resolvconf/resolv.conf.d/tail || echo "options single-request" >> /etc/resolvconf/resolv.conf.d/tail ;

The difference in speed of install is marked.

[Faye Salwin (faye-salwin)]

oops, that late_command doesn't work, but you get the picture. It's missing in-target, but I'm not sure if I can in-target redirect.

[bcotton (bcotton-redhat-bugs)]

This message is a notice that Fedora 19 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 19. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 19 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

[oliver.henshaw (oliver.henshaw-redhat-bugs)]

"options single-request-reopen" in /etc/resolv.conf seems to be an effective workaround for a broken DNS resolver, even for applications that don't use AI_ADDRCONFIG.

Maybe this behaviour could be chosen unconditionally on nodes where the only IPv6 addresses are link-local? Or is this best discussed in another bug?

[horsley1953 (horsley1953-redhat-bugs)]

(In reply to Phil Oester from comment #9)
> But the question remains, WHY did the behavior change? Originally, glibc
> DID use unique ports for the AAAA and A queries. From a "predictability"
> perspective, that is a more secure approach, no? Similar to how ISNs are
> now randomized in TCP.
>
> It seems many people's problems would be solved by going back to the
> (arguably more secure) method of using distinct ports for the A and AAAA
> queries.

Since Ulrich is no longer around to defend to the death indefensible decisions, maybe it is time to just go ahead and put back the separate ports, the elimination of which caused all the problems in the first place.

[codonell (codonell-redhat-bugs)]

(In reply to Tom Horsley from comment #77)
> (In reply to Phil Oester from comment #9)
> > But the question remains, WHY did the behavior change? Originally, glibc
> > DID use unique ports for the AAAA and A queries. From a "predictability"
> > perspective, that is a more secure approach, no? Similar to how ISNs are
> > now randomized in TCP.
> >
> > It seems many people's problems would be solved by going back to the
> > (arguably more secure) method of using distinct ports for the A and AAAA
> > queries.
>
> Since Ulrich is no longer around to defend to the death indefensible
> decisions, maybe it is time to just go ahead and put back the separate
> ports, the elimination of which caused all the problems in the first place.

The glibc community is consensus driven. Someone needs to write up a plan and drive it forward. The glibc team can do this, but this particular issue is lower on the overall priority list for stub resolver fixes. Principally we have no way to test this easily, so we're trying to build out our testing infrastructure to get coverage. In the past this was all tested by hand, and we can see how badly that turned out.

[oliver.henshaw (oliver.henshaw-redhat-bugs)]

Testing on the F21 live image, I don't have a problem.

Probsbly this is https://sourceware.org/git/?p=glibc.git;a=commit;h=16b293a7a6f65d8ff348a603d19e8fd4372fa3a9 in glibc 2.20. I wonder if this resolves all broken DNS resolver issues - is there anyone on F21 who still has problems with bad routers and AAAA DNS queries in getaddrinfo()?

[Rolf Leggewie (r0lf)]

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".

[jkurik (jkurik-redhat-bugs)]

This bug appears to have been reported against 'rawhide' during the Fedora 23 development cycle.
Changing version to '23'.

(As we did not run this process for some time, it could affect also pre-Fedora 23 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 23 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora23

[Rolf Leggewie (r0lf)]

@doko, this ticket is marked as fix released even for karmic, yet remains in an open state for the development release. Is there anything left to do?

[Martin Pitt (pitti)]

Right, closing the floating task.

[bcotton (bcotton-redhat-bugs)]

This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora 'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 23 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

[oliver.henshaw (oliver.henshaw-redhat-bugs)]

Seems like no-one has reported a problem since the F21 release.

[fweimer (fweimer-redhat-bugs)]

(In reply to Oliver Henshaw from comment #79)
> Testing on the F21 live image, I don't have a problem.
>
> Probsbly this is
> https://sourceware.org/git/?p=glibc.git;a=commit;
> h=16b293a7a6f65d8ff348a603d19e8fd4372fa3a9 in glibc 2.20. I wonder if this
> resolves all broken DNS resolver issues - is there anyone on F21 who still
> has problems with bad routers and AAAA DNS queries in getaddrinfo()?

Agreed. We have not seen further reports of the issue, so closing this bug.

[cpanceac (cpanceac-redhat-bugs)]

However, i've seen sometimes when a web page takes many seconds to load, but since i've not investigated the problem. the root cause may be completely different.