Installer/wget is unable to retrieve files via a proxy

[Jeff Balderson]

Binary package hint: debian-installer

Installer is unable to retrieve files from apt-proxy/apt-cacher style proxies using "hostname:port", despite the fact that the dialogs say that it's valid.

1) Netbooted from /ubuntu/hardy/main/installer-sparc/20070308ubuntu31/images/sparc64/netboot/2.6/boot.img
2) manually set "apt-proxy.company.com:9999" and "/ubuntu/" at the prompt for which repository to use

The problem seems to stem from the fact that the installer's version of wget appears to be hobbled in some fashion:

This is from a running i386 Hardy system on the same network:
jbalders@testbox:~$ wget -q http://apt-proxy.company.com:9999/ubuntu//dists/hardy/Release -O - | grep ^Suite: | cut -d' ' -f 2
hardy

So, the proxy is definitely working, and none of my other machines have problems accessing it.

This is from the shell on the netbooted system:
~ # wget -q http://apt-proxy.company.com:9999/ubuntu//dists/hardy/Release -O - | grep ^Suite: | cut -d' ' -f 2
wget: no response from server
~ # wget -q http://192.168.1.20:9999/ubuntu//dists/hardy/Release -O -
wget: no response from server

It can get to the server from the netbooted system if I drop the port:
~ # wget -q http://192.168.1.20/ubuntu//dists/hardy/Release -O -
wget: server returned error 404: HTTP/1.1 404 Not Found
(note that port 80 is just a plain web sever, no Ubuntu packages, so the 404 is normal)

[Jeff Balderson (jbalders)]

By default, apt-cacher can also be accessed via Apache2 (if you have it installed).

As a workaround, you can set the mirror to:

"apt-cacher.company.com" and "/apt-cacher/ubuntu"

Although I didn't used to have to do this with Gutsy and prior.

[Colin Watson (cjwatson)]

I wonder if this is due to endianness confusion in busybox's wget implementation. Could you try using port 8738 (which happens to be 0x2222, and so the same in either host or network byte order), and see if that works?

[Colin Watson (cjwatson)]

Also, could you drop the -q from your manual tests? I'd like to see what the "Connecting to" message says.

[Jeff Balderson (jbalders)]

This test is with the apt-cacher server still running on port 9999
--------------------------------
~ # wget http://apt-proxy.company.com:9999/ubuntu//dists/hardy/Release -O -
wget: no response from server
~ # wget http://apt-proxy.company.com:8738/ubuntu//dists/hardy/Release -O -
wget: Unable to connect to remote host (192.168.1.20): Connection refused

This is with the apt-cacher server running on 8738
--------------------------------
~ # wget http://apt-proxy.company.com:8738/ubuntu//dists/hardy/Release -O -
wget: no response from server

This is via Apache (apt-cacher install sets up "Alias /apt-cacher /usr/share/apt-cacher/apt-cacher.pl")
--------------------------------
~ # wget http://apt-proxy.company.com:80/apt-cacher/ubuntu//dists/hardy/Release -O - | grep ^Suite: | cut -d' ' -f 2
hardy

It appears to me that the problem isn't an endianness problem, but rather that the busybox implementation of wget had a regression between Gutsy and Hardy such that it no longer works with apt-cacher or apt-proxy since it doesn't believe it's getting a response from the server.

I ran tcpdump on the server. I ran "wget http://apt-proxy.company.com:8738/ubuntu//dists/hardy/Release -O -" from the installer's busybox. The request from the client:
GET /ubuntu//dists/hardy/Release HTTP/1.1

Host: apt-proxy.company.com

User-Agent: Wget

Connection: close

Then there's an ack back to the client, and then a FIN/ACK back to the client.

From an installed client, the request looks like this:
GET /ubuntu//dists/hardy/Release HTTP/1.0

User-Agent: Wget/1.10.2

Accept: */*

Host: apt-proxy.company.com:8738

Connection: Keep-Alive

HTTP/1.1 200 OK
Connection: Keep-Alive

Accept-Ranges: bytes
Content-Length: 65852
Content-Type: text/plain; charset=UTF-8
Last-Modified: Fri, 18 Apr 2008 16:46:10 GMT
...

[Jeff Balderson (jbalders)]

It should also be noted that I found a new version of the netboot installer had been released. All of the above tests were performed booted from http://us.archive.ubuntu.com/ubuntu/dists/hardy/main/installer-sparc/20070308ubuntu39/images/sparc64/netboot/2.6/boot.img

[MikeM (michaelm)]

Hi,

I have also encountered this problem. the busybox wget works against an apt-cacher via apache but not directly against the apt-cache daemon. I'm not sure what the root cause of this is and if it is due to the busybox wget not including a complete HTTP request header or if its a bug in apt-cacher's HTTP header handling. Eitherway, I have managed to patch apt-cacher to workaround/fix this issue.

[Thomas Hotz (thotz-deactivatedaccount)]

Marking as confirmed because a patch is there.