dpkg-offline Local repository is lower-priority than remotes in sources.list

[Daniel Manrique]

A test system with 12.04.4 server was installed on a Virtualbox VM, choosing only openssh server in the task selection. Once the install is completed, a dpkg-offline-produced tarball of checkbox-certification-server is uploaded to the server, then the server is immediately taken offline (cable disconnected). Then the usual procedure to install the tarball is followed.

add-ofline-repository -u results in a bunch of W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/precise/Release.gpg Temporary failure resolving 'us.archive.ubuntu.com' for obvious reasons. This should be OK as the local repo should provide required packages.

When apt-get installing checkbox -certification-server, however, one instance reported about 1700 failures to download packages:

Failed to fetch http://us.archive.ubuntu.com/ubuntu/pool/main/n/nmap/nmap_5.21-1.1ubuntu1_amd64.deb Temporary failure resolving 'us.archive.ubuntu.com'
Failed to fetch http://us.archive.ubuntu.com/ubuntu/pool/universe/p/pybootchartgui/pybootchartgui_0+r141-0ubuntu3_amd64.deb Temporary failure resolving 'us.archive.ubuntu.com'
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

Interestingly, most of these packages are provided in the tarball.

One other instance also reported these failures to download, *but* was able to install checkbox-cert-server (and all required packages were indeed installed, from the local repository clearly since there's no remote file).

apt-cache policy reports the remote repo has priority:

$ apt-cache policy stress
stress:
  Installed: 1.0.1-1build1
  Candidate: 1.0.1-1build1
  Version table:
 *** 1.0.1-1build1 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise/universe amd64 Packages
        500 file:/var/local-apt-repositories/apt-repo-ubuntu-12.04.4-server-amd64.tar.gz-20140207-checkbox-certification-server_0.18ppabzr589201401280223ubuntu12.04.1_all/ ./ Packages
        100 /var/lib/dpkg/status

Some research indicates it's not possible for a repository placed in a different apt.list.d file to have higher priority if it provides the *same* package and version as one appearing before it:

http://askubuntu.com/questions/135339/assign-highest-priority-to-my-local-repository

Thus there's no clever way to bump our priority to make the local package install prioritarily.

Two options are left:
1- Instead of using an apt.list.d file, add our dpkg lines directly at the *top* of /etc/apt/sources.list.
2- the apt-add-repository script should comment out all repositories in /etc/apt/sources.list (or maybe we can detect and comment out only *remote* repositories).

Option 2 seems simpler, since it will use just sed to comment out lines. Option 1 would involve more complicated parsing of sources.list, which further complicates our uninstall process (which currently just deletes a file).

In the meanwhile, the suggested workaround is to manually comment all entries in sources.list, before installing the tarball. For the dpkg_offline use case this should be OK, since the system is most likely offline.

Still, for use on systems with some sort of connectivity, we should allow control over which sources.list entries are munged.

[Daniel Manrique (roadmr)]

Yech :( So I did the work to ensure our local repo has priority:

$ apt-cache policy freepats
freepats:
  Installed: (none)
  Candidate: 20060219-1
  Version table:
     20060219-1 0
        500 file:/var/local-apt-repositories/apt-repo-trusty-desktop-amd64.iso-20140210-git_13a1.9rc1-1_amd64_stress_1.0.1-1ubuntu1_amd64_openttd_1.3.3-1build1_amd64/ ./ Packages
        500 http://archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

The package is available from the local repo first, then the remote one. Still, by disabling connectivity through specifying an invalid proxy, it still insists on trying to download the remote packages:

$ sudo apt-get install freepats
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  freepats
0 upgraded, 1 newly installed, 0 to remove and 21 not upgraded.
Need to get 29.0 MB of archives.
After this operation, 34.0 MB of additional disk space will be used.
Err http://archive.ubuntu.com/ubuntu/ trusty/universe freepats all 20060219-1
  Could not connect to badproxy:1328 (badproxy). - connect (111: Connection refused)
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/f/freepats/freepats_20060219-1_all.deb Could not connect to 10.153.104.60:1328 (10.153.104.60). - connect (111: Connection refused)

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

Looks like we will have to resort to disabling all existing repositories :(

[Daniel Manrique (roadmr)]

What happens is that given two sources for the same version of a package, apt-get prefers the ones that can authenticate a package. Since the local-repo packages are not signed, apt-get falls through to the remote ones.

Passing --allow-unauthenticated makes the local packages take precedence.

OK, more options :) the easiest way is to reconfigure apt-get to allow unauth'd packages to be installed. This is very easy to do but leaves users (at least the ones with systems that are connected) at risk to download unauthenticated packages from other sources.

The second option (I need to research this) would be to authenticate our packages, which would have to be done with some sort of local key (to continue working entirely offline).