linux-cloud-tools version specific packages are being removed by unattended-upgrade's Remove-Unused-Dependencies

Bug #1698159 reported by Ian Gordon on 2017-06-15
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
apt (Ubuntu)
Undecided
Unassigned
Trusty
Undecided
Unassigned
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned
unattended-upgrades (Ubuntu)
Undecided
Unassigned
Trusty
Undecided
Unassigned
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned

Bug Description

[Impact]
When running kernel version 4.4.0-78 (for example) and the unattended-upgrade packages installs a new kernel 4.4.0-79 (for example) the linux-cloud-tools-4.4.0-78 and linux-cloud-tools-4.4.0-78-generic packages are removed by the Remove-Unused-Dependencies rule, because unlike "normal" kernel packages they are not excluded by /etc/apt/apt.conf.d/01autoremove-kernels.

The linux-cloud-tools package has the hyper-v integration services daemons in it, so it's deletion leaves some of the integration not working.

Could /etc/apt/apt.conf.d/01autoremove-kernels be updated so other hyper-v users aren't caught out by this?

Thanks,

Ian.

[Test case]

Make sure that linux-cloud-tools for protected kernels are listed in APT::NeverAutoRemove, for example, on cosmic:

   "^linux-cloud-tools-4\.17\.0-9-generic$";
   "^linux-cloud-tools-4\.18\.0-7-generic$";

Those are in 01autoremove-kernels

[Regression Potential]
some linux-cloud-tools will linger around, so it takes more space.

[Other info]

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: unattended-upgrades 0.90ubuntu0.6
ProcVersionSignature: Ubuntu 4.4.0-79.100-generic 4.4.67
Uname: Linux 4.4.0-79-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.6
Architecture: amd64
Date: Thu Jun 15 15:27:35 2017
InstallationDate: Installed on 2016-04-25 (415 days ago)
InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.3)
PackageArchitecture: all
SourcePackage: unattended-upgrades
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.apt.apt.conf.d.10periodic: [modified]
modified.conffile..etc.update-motd.d.90-updates-available: [deleted]
modified.conffile..etc.update-motd.d.98-fsck-at-reboot: [deleted]
modified.conffile..etc.update-motd.d.98-reboot-required: [deleted]
mtime.conffile..etc.apt.apt.conf.d.10periodic: 2016-04-25T16:28:36.640000

Ian Gordon (ian-gordon) wrote :
Balint Reczey (rbalint) wrote :

Yes, indeed, the packages should be kept but 01autoremove-kernels is generated by /etc/kernel/postinst.d/apt-auto-removal which is shipped by apt.
Reassigning to apt.

Changed in unattended-upgrades (Ubuntu):
status: New → Confirmed
status: Confirmed → Triaged
affects: unattended-upgrades (Ubuntu) → apt (Ubuntu)
Balint Reczey (rbalint) on 2018-02-14
Changed in unattended-upgrades (Ubuntu):
status: New → Triaged
tags: added: rls-bb-incoming
Balint Reczey (rbalint) wrote :

I'm testing the fix for u-u right now that will rely on apt's one. The u-u fix will land soon in the git repository.

Changed in unattended-upgrades (Ubuntu):
status: Triaged → In Progress
Steve Langasek (vorlon) on 2018-02-22
tags: removed: rls-bb-incoming
tags: added: id-5a8ef52aa60dcd08c82dd6d1
Changed in apt (Ubuntu Bionic):
status: Triaged → In Progress
Julian Andres Klode (juliank) wrote :
Changed in apt (Ubuntu Bionic):
status: In Progress → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package unattended-upgrades - 1.0ubuntu1

---------------
unattended-upgrades (1.0ubuntu1) bionic; urgency=medium

  * Merge from Debian unstable
    - Remaining changes:
      - unattended-upgrades: Do not automatically upgrade the development
        release of Ubuntu unless Unattended-Upgrade::DevRelease is true.
    - Dropped changes, included in Debian:
      - Run upgrade-between-snapshots only on amd64.
        The test exercises only unattented-upgrade's Python code and uses
        dependencies from the frozen Debian snapshot archive thus running
        it on all architectures would provide little benefit.

unattended-upgrades (1.0) unstable; urgency=medium

  [ Simon Arlott ]
  * Revert sending mails on WARNINGS when in MailOnlyOnError mode"
  * Consider conffile prompts to be errors (Closes: #852465)
    Flag packages that have to be upgraded manually because of a conffile
    prompt and consider this to be an error when sending email or exiting.

  [ Simon McVittie ]
  * Add python, python3, setuptools, DistutilsExtra to Build-Depends.
    They are needed for `clean`, so Build-Depends-Indep is not enough.
  * Add .gitignore and debian/.gitignore
  * Remove bzr configuration.
    This is unnecessary now that u-u is in git.

  [ Michael Vogt ]
  * unattended-upgrades: tweak mail-on-warnings PR
  * unattended-upgrade: extract is_autoremove_valid helper

  [ Balint Reczey ]
  * Run upgrade-between-snapshots only on amd64.
    The test exercises only unattented-upgrade's Python code and uses
    dependencies from the frozen Debian snapshot archive thus running
    it on all architectures would provide little benefit.
  * Clean up processes started for getting md5 sums
  * Don't keep /var/lib/dpkg/status open multiple times
  * Adjust candidates in UnattendedUpgradesCache.open()
  * Perform autoremovals in minimal steps, too.
    Also add check to remove only the set of packages selected for autoremoval.
    Without that check unattended-upgrades when (by default) configured to
    remove newly unused packages could also remove auto removable packages
    which were unused before starting starting the upgrade step.
  * Remove unused automatically installed kernel packages
    (LP: #1357093, #1624644, #1675079, #1698159)
  * Stop including Python syntax in the report (Closes: #876796)
  * Do not auto remove packages related to the running kernel (LP: #1615381)
  * Check packages to be autoremoved against blacklists, whitelists.
    Also check if the packages are held.
  * Report package removals in the summary email (Closes: #876797)
  * Run upgrade-between-snapshots test with debugging enabled
  * Don't create new UnattendedUpgradesCache for checking for autoremovals
    .open() refreshes the state in each cache_commit(), this is enough
  * Update .pot and .po files
  * Update .travis.yml to actually build and test u-u from the repo
  * Run only a simple installation test on Travis, the system upgrade
    test was always failing

 -- Balint Reczey <email address hidden> Thu, 01 Mar 2018 17:29:33 +0700

Changed in unattended-upgrades (Ubuntu Bionic):
status: In Progress → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 1.6~beta1

---------------
apt (1.6~beta1) unstable; urgency=medium

  [ David Kalnischkies ]
  * allow the apt/lists/auxfiles/ directory to be missing (Closes: 887624)
  * add apt-helper drop-privs command…
  * restore gcc visibility=hidden for apt-private
  * ensure correct file permissions for auxfiles
  * allow the apt/lists/auxfiles/ directory to be missing (Closes: 887624)
  * add apt-helper drop-privs command…
  * restore gcc visibility=hidden for apt-private
  * ensure correct file permissions for auxfiles

  [ Julian Andres Klode ]
  * indexcopy: Copy uncompressed indices from cdrom again (LP: #1746807)
  * Work around test-method-mirror failure by setting umask at start
  * Check that Date of Release file is not in the future
  * apt.conf.autoremove: Add linux-cloud-tools to list (LP: #1698159)
  * indexcopy: Copy uncompressed indices from cdrom again (LP: #1746807)
  * Work around test-method-mirror failure by setting umask at start
  * Check that Date of Release file is not in the future
  * apt.conf.autoremove: Add linux-cloud-tools to list (LP: #1698159)

  [ Chris Leick ]
  * German manpage translation update
  * German manpage translation update

 -- Julian Andres Klode <email address hidden> Mon, 26 Feb 2018 13:14:13 +0100

Changed in apt (Ubuntu Bionic):
status: Fix Committed → Fix Released
Julian Andres Klode (juliank) wrote :

Cherry-picked the change into xenial's 1.2.y branch for the next SRU. Will do that in trusty too.

Changed in apt (Ubuntu Xenial):
status: New → Triaged
Changed in apt (Ubuntu Trusty):
status: New → Triaged
Łukasz Zemczak (sil2100) wrote :

We would need some SRU information here. Since this is part of the xenial apt SRU, we'd need to have reliable testing steps for verification of the update. Thank you!

description: updated
Łukasz Zemczak (sil2100) wrote :

Thanks! Would it be maybe possible to actually check if the cloud-tools packages aren't auto-removed by unattended-upgrades as well besides just looking at the APT::NeverAutoRemove rules?

Changed in apt (Ubuntu Xenial):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-xenial

Hello Ian, or anyone else affected,

Accepted apt into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/1.2.28 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Ian Gordon (ian-gordon) wrote :

Hi,

I installed apt 1.2.28 from proposed on a xenial server with the linux-cloud-tools packages installed and it added linux-cloud-tools to the VersionedKernelPackages section of /etc/apt/apt.conf.d/01autoremove.

It did not add linux-cloud-tools lines to the 01autoremove-kernels file. This did not occur until I upgraded the kernel to 4.4.0-137. After doing this the 01autoremove-kernels contains:

  "^linux-cloud-tools-4\.4\.0-134-generic$";
  "^linux-cloud-tools-4\.4\.0-137-generic$";

I confirmed that linux-cloud-tools package was also not auto removed.

So the new version of apt fixes the problem.

Regards,

Ian.

tags: added: verification-done-xenial
removed: verification-needed-xenial
Brian Murray (brian-murray) wrote :

Hello Ian, or anyone else affected,

Accepted apt into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/1.2.29 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

tags: added: verification-needed-xenial
removed: verification-done-xenial

Marking verification-done-xenial again, this got reset by 1.2.29.

FWIW; I also retested: after upgrading apt to 1.2.29 in the xenial container, and rerunning the kernel hook that creates the file, I also see

   "^linux-cloud-tools-4\.18\.0-8-generic$";

in the list.

tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers