apt-get dist-upgrade and apt full-upgrade not reporting held back package

[Colin Law]

I have the situation where a package cannot be upgraded because of missing dependencies. However
sudo apt-get dist-upgrade
and
sudo apt full-upgrade
and
sudo apt upgrade
all show
0 to upgrade, 0 to newly install, 0 to remove and 0 not to upgrade.

Only
sudo apt-get upgrade
shows
0 to upgrade, 0 to newly install, 0 to remove and 1 not to upgrade.

I believe all of the above commands should show that 1 cannot be upgraded.

This bug is not about the particular package I am trying to install, but about the way apt reports the problem.
The details, for reference, are that I am trying to upgrade mosquitto from the mosquitto ppa. After apt-get update I see
$ apt-cache policy mosquitto
mosquitto:
  Installed: 1.4.8-1build1
  Candidate: 1.4.9-0mosquitto1
  Version table:
     1.4.9-0mosquitto1 500
        500 http://ppa.launchpad.net/mosquitto-dev/mosquitto-ppa/ubuntu xenial/main amd64 Packages
 *** 1.4.8-1build1 500
        500 http://gb.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
        100 /var/lib/dpkg/status

but if I try to manually install mosquitto I get

The following packages have unmet dependencies.
 mosquitto : Depends: libwebsockets3 (>= 1.2) but it is not installable

and

$ apt-cache policy libwebsockets3
libwebsockets3:
  Installed: (none)
  Candidate: (none)
Due to the fact that libwebsockets3 is not available for Xenial.
As I said this bug is not about the problem with this particular package but about the fact that apt does not report that there is a package to be upgraded, but it cannot be done.

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: apt 1.2.12~ubuntu16.04.1
ProcVersionSignature: Ubuntu 4.4.0-22.40-generic 4.4.8
Uname: Linux 4.4.0-22-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.1
Architecture: amd64
CurrentDesktop: Unity
Date: Sun Jun 5 08:45:33 2016
InstallationDate: Installed on 2014-10-21 (592 days ago)
InstallationMedia: Ubuntu 14.10 "Utopic Unicorn" - Alpha amd64 (20141017)
SourcePackage: apt
UpgradeStatus: Upgraded to xenial on 2016-03-06 (90 days ago)

[Julian Andres Klode (juliank)]

Hi, thanks for your bug report. I'm not sure I understand it, though:

First you write that "sudo apt upgrade" shows 0 upgrades, then it's the only one that shows 1.

Could you fix this please?

I myself have never seen this AFAIK.

[Colin Law (colin-law)]

Sorry, it is apt-get upgrade that shows 1 not to upgrade.
Description corrected.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Adam Conrad (adconrad)]

Simple reproducer:

# cat /etc/apt/sources.list.d/test.list
deb [trusted=yes] copy://home/adconrad/apt-test ./
# cat /home/adconrad/apt-test/Packages
Package: apt
Priority: important
Section: admin
Architecture: amd64
Version: 99:1
Depends: test (>= 99:1)
# apt-get update
# apt-get dist-upgrade | grep upgraded
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
# apt-get autoremove | grep upgraded
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
# apt-get upgrade | grep upgraded
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
# apt-get install hello | grep upgraded
0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.

Note that only "dist-upgrade" appears to fail to report the package being held because it's uninstallable.

[Adam Conrad (adconrad)]

Tested precise, trusty, xenial, and yakkety. Looks like this bug was introduced between precise and trusty (that is, it works correctly in precise's 0.8.16~exp12ubuntu10, but not trusty's 1.0.1ubuntu2 or later).

[Adam Conrad (adconrad)]

(To test in older versions, change the copy:// to a file:/ because derp)

[David Kalnischkies (donkult)]

That is a regression of sorts caused by a sleight of hand used to avoid another bug. The commit in question would be https://anonscm.debian.org/cgit/apt/apt.git/commit/?id=446551c8ffd2c9cb9dcd707c94590e73009f7dd9 although the involved code changed in the mean time the general idea remains the same: We change the candidate (which we also do in another case: M-A:same version screws) which means later parts do not realize that there ever was a chance to upgrade the package.

In other words: Not that easy to fix…