mirror: protocol failing again

[Stuart Bishop]

Bug #1020995 seems to have regressed under saucy.

If sources.list contains lines like:
    deb mirror://mirrors.ubuntu.com/mirrors.txt saucy main restricted

sudo apt-get update fails with messages like:

    Err mirror://mirrors.ubuntu.com saucy/main amd64 Packages
      No mirror file '/var/lib/apt/mirrors/mirrors.ubuntu.com_mirrors.txt' found [Mirror: ]

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: apt-cacher-ng 0.7.18-1
ProcVersionSignature: Ubuntu 3.11.0-11.17-generic 3.11.3
Uname: Linux 3.11.0-11-generic x86_64
NonfreeKernelModules: nvidia
ApportVersion: 2.12.5-0ubuntu1
Architecture: amd64
Date: Wed Oct 9 21:56:51 2013
EcryptfsInUse: Yes
InstallationDate: Installed on 2013-02-26 (225 days ago)
InstallationMedia: Ubuntu 13.04 "Raring Ringtail" - Alpha amd64 (20130225)
MarkForUpload: True
SourcePackage: apt-cacher-ng
UpgradeStatus: Upgraded to saucy on 2013-09-24 (14 days ago)

[Stuart Bishop (stub)]

Not flagged as a regression, as the bug was claimed fixed in 0.7.13-1 but raring has an earlier version.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Paul Annesley (launchpda)]

I believe the problem is erroneous URL encoding/escaping.

With `deb mirror://mirrors.ubuntu.com/mirrors.txt` in sources.list, apt-cacher-ng's logs show this:

403 Forbidden file type or location: http://mirrors.ubuntu.com/mirrors.txt?arch=amd64&dist=saucy-updates

Adding anything matching /mirrors.txt/ to acng's volatile file list (VfilePattern) changes this from a 403 Forbidden (generated by acng) to a 404 (proxied from mirrors.ubuntu.com).

However hitting http://mirrors.ubuntu.com/mirrors.txt?arch=amd64&dist=saucy-updates directly is 200 OK.

Using mitmproxy I can see that apt-cacher-ng is URL-escaping the question mark and ampersands:

>> GET http://mirrors.ubuntu.com/mirrors.txt%3farch%3damd64%26dist%3dsaucy-updates
       ← 404 text/html 327B 303.73kB/s

So that explains the 404, and it probably also explains the 403; once URL-escaped, it's probably not matching the default VfilePattern regexp.