apt-helper download fails to download ttf-mscorefonts with certain redirects (blanks in URI)

[Matthias Andree]

apt-helper does not process redirects properly, breaking ttf-mscorefonts-installer downloads, related bugs:
https://bugs.launchpad.net/ubuntu/+source/msttcorefonts/+bug/1654623
https://bugs.launchpad.net/ubuntu/+source/msttcorefonts/+bug/1654855

This doesn't happen with cURL or wget from the command line.

This trace reveals a bug in the generation of URIs through what appears to me improper processing of escaped space and unescaped URI encoded versions (%20), look for "GET /project/corefonts/the fonts/final/andale32.exe HTTP/1.1" below - this should arguably be "the%20fonts" again, not the processed version.

This trace is generated with:

$ cat /etc/apt/apt.conf
Debug {
    Acquire {
 http "TRUE";
 https "TRUE";
    };
};

Failure trace:

$ LANGUAGE= LANG= LC_ALL=C /usr/lib/apt/apt-helper download-file http://downloads.sourceforge.net/corefonts/andale32.exe /tmp/andale32.exe
0% [Working]GET /corefonts/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.15)

Answer for: http://downloads.sourceforge.net/corefonts/andale32.exe
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Sun, 08 Jan 2017 15:02:59 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
Location: http://downloads.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 178

0% [Working]GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.15)

GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.15)

Answer for: http://downloads.sourceforge.net/project/corefonts/the fonts/final/andale32.exe
HTTP/1.1 302 Found
Server: nginx
Date: Sun, 08 Jan 2017 15:03:00 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
content-disposition: attachment; filename="andale32.exe"
Set-Cookie: sf_mirror_attempt="corefonts:netcologne:the%20fonts/final/andale32.exe"; expires=120; Path=/
Location: https://netcologne.dl.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 166

0% [Working]* Trying 2001:4dd0:1234:6::5f...
* Connected to netcologne.dl.sourceforge.net (2001:4dd0:1234:6::5f) port 443 (#0)
* found 169 certificates in /etc/ssl/certs/ca-certificates.crt
* found 690 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: netcologne.dl.sourceforge.net (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: CN=netcologne.dl.sourceforge.net
* start date: Sun, 06 Nov 2016 19:27:00 GMT
* expire date: Sat, 04 Feb 2017 19:27:00 GMT
* issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
* compression: NULL
* ALPN, server did not agree to a protocol
> GET /project/corefonts/the fonts/final/andale32.exe HTTP/1.1
Host: netcologne.dl.sourceforge.net
User-Agent: Debian APT-CURL/1.0 (1.2.15)
Accept: */*
Cache-Control: max-age=0
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT

< HTTP/1.1 302 Moved Temporarily
< Server: nginx/1.11.8
< Date: Sun, 08 Jan 2017 15:03:00 GMT
< Content-Type: text/html
< Content-Length: 161
< Connection: keep-alive
< Content-Range: bytes */198384
* Failed writing header
* Closing connection 0
Err:1 http://downloads.sourceforge.net/corefonts/andale32.exe
  The HTTP server sent an invalid Content-Range header
E: Failed to fetch https://netcologne.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe The HTTP server sent an invalid Content-Range header

E: Download Failed

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: apt 1.2.15ubuntu0.2
ProcVersionSignature: Ubuntu 4.8.0-32.34~16.04.1-generic 4.8.11
Uname: Linux 4.8.0-32-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.4
Architecture: amd64
CurrentDesktop: GNOME-Flashback:Unity
Date: Sun Jan 8 16:10:07 2017
SourcePackage: apt
UpgradeStatus: No upgrade log present (probably fresh install)

[Julian Andres Klode (juliank)]

Duplicate of bug 1651923 - and the others too, of course.