precise → trusty upgrade: packages fail due to python:any dependency

[Martin Pitt]

During automatic upgrade tests from precise to trusty (like http://d-jenkins.ubuntu-ci:8080/view/Upgrade/job/upgrade-ubuntu-precise-trusty-desktop-i386_vm/2/) several packages fail due to the new python:any dependency:

dpkg: dependency problems prevent configuration of python-aptdaemon:
 python-aptdaemon depends on python:any (>= 2.7.1-0ubuntu2).
 python-aptdaemon depends on python-apt (>= 0.8.5~ubuntu1); however:
  Version of python-apt on system is 0.8.3ubuntu7.1.
[...]
dpkg: dependency problems prevent configuration of unity:
 unity depends on python:any.
dpkg: error processing unity (--configure):
 dependency problems - leaving unconfigured

or in my manual test:

Setting up python-defer (1.0.6-2) ...
dpkg: dependency problems prevent configuration of python-dirspec:
 python-dirspec depends on python:any (>= 2.7.1-0ubuntu2).

dpkg: error processing package python-dirspec (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of python-protobuf:
 python-protobuf depends on python:any (>= 2.7.1-0ubuntu2).

dpkg: error processing package python-protobuf (--configure):
 dependency problems - leaving unconfigured
[...]
and lots more.

In some upgrades dpkg has not been upgraded yet, but at the point where apt-get dist-upgrade finally fails it is upgraded. However, python2.7 is not upgraded yet:

  ii dpkg 1.17.5ubuntu2 i386 Debian package management system
  ii python 2.7.3-0ubuntu2 i386 interactive high-level object-oriented language (default version)
  ii python-minimal 2.7.3-0ubuntu2 i386 minimal subset of the Python language (default version)
  ii python2.7 2.7.3-0ubuntu3.4 i386 Interactive high-level object-oriented language (version 2.7)
  ii python2.7-minimal 2.7.3-0ubuntu3.4 i386 Minimal subset of the Python language (version 2.7)

and trying to clean up still fails:

$ sudo dpkg --configure -a
[...]
dpkg: dependency problems prevent configuration of python-pkg-resources:
 python-pkg-resources depends on python:any (>= 2.7).
 python-pkg-resources depends on python:any (<< 2.8).

dpkg: error processing package python-pkg-resources (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of duplicity:
 duplicity depends on python:any (>= 2.7.1-0ubuntu2).

So it seems apt needs to be nudged to upgrade python/python2.7 before upgrading any python:any reverse dependency?

[Colin Watson (cjwatson)]

Isn't this basically http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=723586 ? We could backport that.

[Martin Pitt (pitti)]

Most likely a duplicate of that Debian bug, thanks Colin! I'll build apt with that fix and test again.

[Martin Pitt (pitti)]

http://anonscm.debian.org/gitweb/?p=apt/apt.git;a=commitdiff;h=b4140ecf1

[Martin Pitt (pitti)]

Trusty's apt already has that fix. So supposedly what happens is that apt isn't upgraded early enough. Perhaps ubuntu-release-upgrader should take care to update dpkg and apt first, so that fixes like these are actually applied?

[Martin Pitt (pitti)]

I discussed that with Michael Vogt. It seems that installing trusty's apt first is hard to revert if the upgrade can't be calculated after that, or the user wants to abort. For lucid → precise we used to backport the current apt (release-upgrader-apt), but that's a rather intrusive step. If we have an easy fix for that for precise proper, SRUing apt would be the best course.

I just tried to backport http://anonscm.debian.org/gitweb/?p=apt/apt.git;a=commitdiff;h=b4140ecf1 to our precise apt, which applies fairly well. But this alone introduces some regression, apt-get update or many of the tests then say

E: Problem parsing dependency Depends
E: Error occurred while processing python3-gnupg (NewVersion2)
E: Problem with MergeList /tmp/tmp.kzDn7m7XFL/rootdir/var/lib/apt/lists/_tmp_tmp.kzDn7m7XFL_aptarchive_dists_stable_main_binary-amd64_Packages
E: The package lists or status file could not be parsed or opened.

So this will require another backported patch to go along with this, if we want to go with the SRU route.

[Martin Pitt (pitti)]

I tried many times with different scenarios, and I've never been able to reproduce this in a schroot or manually upgrading in an up-to-date precise VM. Until now, the referenced upgrade tests were run on a rather outdated precise VM (from May 2013, 12.04.2), and I refreshed it now. The upgrade test succeeds now.

Also, precise has python 2.7.3-0ubuntu2 which is >= 2.7.1-0ubuntu2, so even if python does not get upgraded until later it ought to suffice. So if anything, we need to backport said apt commit, but until we have some stronger evidence that it is actually required and causes real upgrade failures I'll close this bug.