HTTPS sources fail to update

[Thom May]

Using an https source line (deb https://ubuntu.xxx/archive xxx/) , we see:

 Fetched 565B in 11s (48B/s)
 Failed to fetch https://ubuntu.xxx/archive/xxx/
Packages.gz transfer closed with outstanding read data remaining
 Reading package lists... Done

y.y.y.y - - [11/Apr/2007:12:50:38 +0200] "GET /archive/xxx/Packages.gz HTTP/1.1" 416 431 "-" "Debian APT CURL/1.0
(0.6.46.4ubuntu10build2)"

HTTP response 416 is: (from rfc2616) 10.4.17 416 Requested Range Not Satisfiable so I guess curl is just sending out an incorrect Range request header when it has a preexisting list.
Deleting the lists from /var/lib/apt/lists and /var/lib/apt/lists/partial and re-running apt-get update fixes the issue.

[Thom May (thombot)]

with Debug::Acquire::https set to true:

> GET /archive/xxx/Packages.gz HTTP/1.1
Range: bytes=12696-
User-Agent: Debian APT-CURL/1.0 (0.6.46.4ubuntu10build2)
Host: ubuntu.xxx
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: max-age=0

< HTTP/1.1 416 Requested Range Not Satisfiable
< Date: Mon, 23 Apr 2007 15:18:22 GMT
< Server: Apache/2.2.3 (Ubuntu) DAV/2 mod_ssl/2.2.3 OpenSSL/0.9.8b
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=iso-8859-1
* transfer closed with outstanding read data remaining
* Closing connection #0
Err https://ubuntu.xxx xxx/ Packages
  transfer closed with outstanding read data remaining
Fetched 563B in 1s (469B/s)

[Thom May (thombot)]

Ah, the range request is a red herring. What's actually going on is that the partial file isn't getting removed on a successful download, so curl tries to generate a range request from the end of the file.
Commenting the range request option out results in the file doubling in size each update.

[Todd Kover (kovert)]

The range can still an issue. If the list file changes in such a way that it can't be resumed the next time an apt-get update is run (such as a reordering of the contents), I believe it will fail in the same way. The https method probably needs to handle the case of the web server ignoring the range request for situations where it can't honor it for this circumstance, or someone will have to go in and remove the partial file for things to start working again.

No?

-Todd

[Robert Coup (rcoup)]

Another related debian bug, with a patch:

apt-transport-https: doesn't reset curl options between files or timestamp downloaded files
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=437150

[Michael Vogt (mvo)]

This should be fixed in gutsy now (+ the debian patch is added to the report too).

[stiV (stefan-wehinger)]

I have this probem w. ubuntu feisty on multiple machines, and although deleting lists from /var/lib/apt/lists and /var/lib/apt/lists/partial works, I'd love to have the working version of apt-transport-https in feisty too.
I'm using my own mirror and changing to gutsy is not an option at the moment. I can build the package myself, but I'd need a patch for the version feisty uses and can't find one ... anyone able to help?

[dan_linder (dan-linder)]

This is still happening on my Gutsy release. I had to remove the file in /var/lib/apt/lists/partial, and the update worked. I'm running apt 0.7.6ubuntu14.1, and I see the link (by Robert Coup) shows this being fixed in apt 0.7.7 - when is 0.7.7 getting pushed into the Gutsy distribution?

Dan