Minimize installed packages from -proposed

[Martin Pitt]

It sometimes happens that a package P in -proposed gets stuck because one of its dependencies D also gets uploaded and is broken. Right now we always run all tests against the entirety of -proposed, which will cause P's tests to fail due to the broken D.

However, in many cases P's dependencies will be satisfiable in -release, so if we would run tests only against P's -proposed binaries and only use those packages from -proposed which are necessary to satisfy the dependencies (to catch library transitions or packages with versioned Depends: which need to land in lockstep) we could land P and still keep D in -proposed.

This needs fleshing out, and we need to figure out how to do that with apt pinning.

Once we do that, we need to become strict in britney about always respecting the recorded triggering package in the result logs. We must then not consider newer test results which were run against a different trigger Q as an updated test result for a failing test against P. We currently only do this results-per-trigger tracking for kernels, but not for normal packages as the triggering package currently doesn't influence the test results there.

[Steve Langasek (vorlon)]

Logic for apt pinning:

 - take the list of binary packages produced by the triggering package
 - give these binary packages a pin priority of 900
 - give wily-proposed a pin priority of 100
 - give wily a pin priority of 900

Example (taken from a current package in update_excuses, libmatio, that lists a depends on another package in -proposed, hdf5):

 Package: libmatio-dev libmatio2 libmatio2-dbg libmatio-doc
 Pin: release a=wily-proposed
 Pin-Priority: 900

 Package: *
 Pin: release a=wily
 Pin-Priority: 900

 Package: *
 Pin: release a=wily-proposed
 Pin-Priority: 100

I have confirmed that with the above pin, if you do 'apt-get install libhdf5-dev', you get libhdf5-dev from wily; and if you do 'apt-get install libmatio-dev', you get libmatio-dev plus the necessary hdf5 deps from wily-proposed.

Caveat: you will get libhdf5-10 installed, which is only available in -proposed. You will *not* get the -proposed version of libhdf5-dev installed, since the version of libhdf5-dev in wily satisfies the dependency of libmatio-dev. So instead you'll get libhdf5.so (or whatever) pointing at libhdf5.so.8, while libmatio.so.2 is linked against libhdf5.so.10. This should *usually* not be a problem in practice. If it were a problem, I don't see a way to automatically fix it anyway with pinning, because you can't transitively pin things.

[Martin Pitt (pitti)]

Changes to autopkgtest:
 - Extend --apt-pocket=<pocketname> to also accept --apt-pocket=<pocketname>=binary1,binary2,src:src1,...
   This would add the additional apt sources as usual, but additionally create an /etc/apt/prefererences.d/01autopkgtest like in comment #1. all "src:src1" entries are expanded to all binary packages of src1 in <pocketname>.
 - An integration test for this requires some work: Add functionality to create a set of fake debs, then create a fake -release and -proposed archive (with apt-ftparchive), and test various combinations.

Changes to autopkgtest-cloud:
 - Replace "--apt-pocket=proposed" with comma separated params['triggers'], prefixed with src:
 - If no test triggers are given (which happens with manual test requests that don't specify a trigger and thus should apply to and run against all -proposed package), continue to use "--apt-pocket=proposed" as before.

Changes to britney:
 - No changes are necessary for installing only the -proposed packages for the trigger for requesting an individual test run.
 - Request individual test runs for every trigger, instead of just one test run for all triggers. This is what we already do for kernels, but now we need to do that for all packages as a test of p for a trigger t1 does not apply to a different trigger t2 any more. (NB that this will cause a lot of additional test runs, particularly in scenarios like "new KDE version mass upload")

[Martin Pitt (pitti)]

@Steve: I just tried this on current wily-proposed with http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#obconf :

---- /etc/apt/preferences.d/01autopkgtest ----
Package: obconf
Pin: release a=wily-proposed
Pin-Priority: 900

Package: *
Pin: release a=wily
Pin-Priority: 900

Package: *
Pin: release a=wily-proposed
Pin-Priority: 100
------------------------------------------------------------

# apt-get install -s obconf
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 obconf : Depends: libobrender32 (>= 3.6.0) but it is not going to be installed
          Depends: libobt2 (>= 3.6.0) but 3.5.2-8 is to be installed
          Recommends: openbox but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

It seems it's not considering its dependencies from -proposed, but tries to install everything else from wily? Only if I bump the priority in the third clause from 100 to 900 it works again (but then of course installs all deps from -proposed).

[Martin Pitt (pitti)]

What does work is to not specify "obconf" itself, but its dependencies, i. e. the binaries of "openbox".

  Package: openbox libobt2 libobrender29 openbox-dev gnome-panel-control openbox-gnome-session openbox-kde-session

This is actually the main use case that we want for autopkgtest -- i. e. we update a dependency, we run a package (from -release, as much as possible) against the new dependency in -proposed.

However, the use case in comment #5 also must work -- e. g. right now we have obconf itself in -proposed so we must be able to test that in isolation as well. In this case it should take obconf (pinned explicitly) plus the openbox deps (as necessary for resolving dependencies) from -proposed.

[Steve Langasek (vorlon)]

The issue here is that libobrender32 is a new library in wily-proposed, but libobt2 is an existing library in wily and libobrender32 needs the newer version of the library from wily-proposed. That falls under the 'caveat' in my analysis.

Perhaps in this case we should fall back to installing with no pins, at least as a first approximation?

[Martin Pitt (pitti)]

Steve Langasek [2015-10-13 18:27 -0000]:
> Perhaps in this case we should fall back to installing with no pins, at
> least as a first approximation?

Unless mvo has a clever idea, I guess that's what we'll do then.

[Martin Pitt (pitti)]

autopkgtest now has a series of commits which implement and test --apt-pocket=proposed=binpkg1,src:srcpkg2,... .

[Martin Pitt (pitti)]

I locally did the britney change, but I won't push/roll this out on a Friday evening.

[Martin Pitt (pitti)]

I (locally) ran this for a few packages in current xenial-proposed, found/fixed one more bug, and this looks good now.

britney change to request one test per trigger: http://bazaar.launchpad.net/~ubuntu-release/britney/britney2-ubuntu/revision/525

autopkgtest-cloud worker change to select triggering package from -proposed and use the rest from -release: https://git.launchpad.net/~ubuntu-release/+git/autopkgtest-cloud/commit/?id=f4fa3badf7a

[Launchpad Janitor (janitor)]

This bug was fixed in the package autopkgtest - 3.18

---------------
autopkgtest (3.18) unstable; urgency=medium

  Behaviour changes/improvements:
  * adt-virt-lxc: Drop --eatmydata option. Conceptually this isn't specific to
    the LXC runner, it should always be used for package installation (when
    available) with all runners, but never for tests.
  * setup-commands/cloud-vm-setup, tools/adt-setup-vm: Install and use
    eatmydata.
  * Use eatmydata for --apt-upgrade and installing test dependencies, if
    available.
  * ssh-setup/nova: Provide custom implementation of "wait-reboot" using "nova
    reboot --poll". This works more reliably on some clouds.
  * ssh-setup/nova: Call "nova boot" with --debug to better be able to
    diagnose failures.
  * Extend --apt-pocket=POCKET option to accept an additional
    "=pkgname1,src:srcname1,..." package list. An entry "src:srcname" expands
    to all binary packages built by that source. If given, set up apt pinning
    to use only those packages from POCKET, so that package updates in that
    pocket can be tested independently from each other for better isolation.
    (LP: #1503150)
  * Add ssh-setup/maas: Acquire and Deploy a machine via MAAS to use as an
    autopkgtest testbed. It assumes that MaaS is already set up, machines are
    commissioned to it, and you added your ssh key to it.

  Bug fixes:
  * ssh-setup/nova: Drop "Warning: PoC", we've been using this in production
    for a long time.
  * tools/adt-setup-vm: Check if we already have a configuration for the
    specific network interface that we are configuring, not just any
    interface.
  * tools/adt-buildvm-ubuntu-cloud: Counter-hack the Ubuntu livecd-rootfs
    workaround for LP #1510345 by creating a configuration for ens3 and
    removing the one for eth0.

 -- Martin Pitt <email address hidden> Wed, 11 Nov 2015 10:11:48 +0100