The timeout is no longer correct

[Matthieu Baerts]

Binary package hint: wget

Hello,

With the new release of Karmic it seems that the timeout of wget is no longer correct.
In man pages we can see "-T seconds" but if I try :
   $ wget my_file -T 5
It doesn't work ! But no problem with :
   $ wget my_file -T 20
My ping is less than 60ms

In fact wget is used by Cairo-Dock and now there is an error: (sorry, it's in French)
   $ wget "http://themes.cairo-dock.org/clock/BlackEye/BlackEye.tar.gz" -O "/tmp/cairo-dock-net-file.test" -t 2 -T 5
   --2009-10-30 21:13:45-- http://themes.cairo-dock.org/clock/BlackEye/BlackEye.tar.gz
   Résolution de themes.cairo-dock.org... échec: Connexion terminée par expiration du délai d'attente.
   wget: unable to resolve host address `themes.cairo-dock.org'
 => Failed

   $ wget "http://themes.cairo-dock.org/clock/BlackEye/BlackEye.tar.gz" -O "/tmp/cairo-dock-net-file.test" -t 2 -T 20
   --2009-10-30 22:22:43-- http://themes.cairo-dock.org/clock/BlackEye/BlackEye.tar.gz
   Résolution de themes.cairo-dock.org... 87.98.128.158
   Connexion vers themes.cairo-dock.org|87.98.128.158|:80... connecté.
   requête HTTP transmise, en attente de la réponse... 200 OK
   Longueur: 16452 (16K) [application/x-gzip]
   Saving to: `/tmp/cairo-dock-net-file.test'

   100%[===========================================>] 16 452 --.-K/s in 0,08s

   2009-10-30 22:22:53 (194 KB/s) - « /tmp/cairo-dock-net-file.test » sauvegardé [16452/16452]

=> It's ok but we can see "200 OK" and in the command we had -T 20. So I suppose that now -T 1 = 10ms

[Micah Cowan (micahcowan)]

The timeout in the first case was a DNS timeout. This has nothing to do with ping times, which may be much faster than a given DNS resolution, especially if the primary DNS server configured for your box is down. This is not at all unusual. Nothing has changed related to timeouts in Wget in a long time.

A better way to demonstrate that the timeout is less than it's claimed, would be to run wget under the system "time" command, which will spit out how long wget ran (and so, for a single-file fetch, determine about how long wget waited before bailing).

[Matthieu Baerts (matttbe)]

Thanks for your answer.
I'm sure you are right: it's not a bug due to wget.
But maybe can you help me to find the responsible library (or libraries )? Because all was ok on Jaunty and not on Karmic with the same command!

[Micah Cowan (micahcowan)]

I don't think it's a bug with any library or command; probably just a temporary network failure.

Does wget fail _every_ time you specify -T 5 (and is it always during name resolution)? If so, and if you're able to get immediate pings at 60ms, then that _would_ indicate a problem with Wget (or something it relies on). If not, then it was most likely a temporary network problem.

[Matthieu Baerts (matttbe)]

It happens every time but only on Ubuntu Karmic.
I'm not at home now but I can test tomorrow with other website, etc.

But if you're on Ubuntu Karmic (or maybe on Debian Sid) can you test this command:
   $ wget "http://themes.cairo-dock.org/clock/BlackEye/BlackEye.tar.gz" -O "/tmp/cairo-dock-net-file.test" -t 2 -T 5

Thanks

[Micah Cowan (micahcowan)]

Sid's using a newer (just-released) version of Wget than Karmic. I'm not running Karmic yet, but inspecting the sources, Karmic hasn't changed anything to Wget 1.11.4 except a security fix for https that's not related to timeouts.

[Matthieu Baerts (matttbe)]

Yes I'd a look to these changes and this is the reason that I think now this bug isn't due to Wget but something it relies on.

On the bug #472920 I've told if the reporter have all lists of themes (which are downloaded with wget) and he said no. Two others people on our forum have this problem too with '-T 5'. So it seems that there a bug but we don't know which package is the responsible.
But I've tested on Xubuntu 9.10 and there is no bug!
I will test tomorrow with other servers and with Gnome on Karmic.
Thanks for your help.

[Matthieu Baerts (matttbe)]

Hello Micah
I think I have found the responsible : #417757
It seems IPv6 DNS lookups is the cause.
What do you think about it?

[Micah Cowan (micahcowan)]

So, in summary, wget _is_ taking the full 5 seconds before timing out, and the timeouts are caused by slow IPv6 lookups? (If so, one workaround would be to add "-4" to wget's options).

[Matthieu Baerts (matttbe)]

After some tests with other people it seems that there is a bug only with computers, routers and their ISP which support IPv6.

The download works only if we wait for approximately 10 sec (so it works only with -T 11 or more).
I don't know if the better workaround would be to add "-10" or to force IPv4 if it's possible.
Thanks.

[Matthieu Baerts (matttbe)]

Hello,
There isn't any problem if we force the use of IPv4 (with -4).
It's for me the best workaround for Karmic.

[Micah Cowan (micahcowan)]

Matthieu,

Since you've indicated that this isn't a problem with wget, it's probably best to leave "wget" set to Invalid, or assign this to a more appropriate package (or more likely, mark it as a duplicate of the core issue's bug report).

You may also want to check behavior if you use --prefer-family=ipv4 rather than -4, as this way wget isn't forced to ignore ipv6 when it's the only option available for a given address (very rarely the case, in practice, but...)

Once you determine which option works best for your needs, I'd recommend editing /etc/wgetrc to set "prefer_family = ipv4" or "inet4_only = on", as needed, until Karmic's issue with less-than-perfect DNS providers(?) is resolved.

[Matthieu Baerts (matttbe)]

Thanks for this tip about '.wgetrc' !
I hope that this bug will be fixed on Karmic and not on Lucid (if we know quickly which package(s) or some ISP are the responsible)

PS : Why not change the status to 'Won't Fix' ?

[Micah Cowan (micahcowan)]

Because "Won't Fix" implies that it's a wget bug, but we've decided not to address it, rather than a real bug with some other package/infrastructure.