Full backport SRU for unattended-upgrades

[Matt Bearup]

[Impact]

* I would like to propose a one-off full backport of unattended-upgrades
1.2ubuntu1 to Bionic, Artful, Xenial and possibly Trusty releases because selectively backporting fixes for crashes and for issues that made u-u unreliable would be more risky thanks to the huge number of fixes and the
inter-dependencies between them.

[Test Case]

* Since this backport involves fixing several bugs and this bug itself covers the full backport this is an overview of all LP bugs the upload fixes:

- most important bugs (feel free to add more)
   + LP: #1615381 : apt-get autoremove may remove current kernel

All LP bugs fixed by the backport, comments are welcome:
   + LP: #1230246 could use SRU template but seems straightforward
   + LP: #1260041 SRU template ok
   + LP: #1396787 SRU template ok
   + LP: #1446552 SRU template ok
   + LP: #1455097 SRU template ok
   + LP: #1458204 could use SRU template but seems straightforward
   + LP: #1577215 could use SRU template but seems straightforward
   + LP: #1602536 SRU template ok
   + LP: #1615381 could use SRU template but seems straightforward
   + LP: #1624644 SRU template ok
   + LP: #1649709 not affecting xenial
   + LP: #1654070 could use SRU template but seems straightforward
   + LP: #1654600 SRU template ok
   + LP: #1675079 SRU template ok
   + LP: #1680599 could use SRU template but seems straightforward
   + LP: #1686470 SRU template ok
   + LP: #1690980 SRU template ok
   + LP: #1698159 SRU template ok
   + LP: #1702793 this bug
   + LP: #1714019 just a merge request
   + LP: #1718419 just a merge request
   + LP: #1719630 SRU template ok
   + LP: #1722426 just a merge request
   + LP: #1737442 SRU template ok
   + LP: #1737635 SRU template ok
   + LP: #1737637 SRU template ok
   + LP: #1737717 SRU template ok
   + LP: #1741579 SRU template ok
   + LP: #1764797 just a merge request
   + LP: #1773033 SRU template ok
   + LP: #1775292 SRU template ok
   + LP: #1775307 SRU template ok
   + LP: #1778219 SRU template ok
   + LP: #1778800 SRU template ok
   + LP: #1779157 SRU template ok
   + LP: #1781176 SRU template ok
   + LP: #1781183 SRU template ok
   + LP: #1781446 SRU template ok
   + LP: #1781586 SRU template ok
   + LP: #1785093 SRU template ok
   + LP: #1789637 SRU template ok
   + LP: #1803749 SRU template ok

[Regression Potential]

* Due to this update covering the full backport unattended-upgrades can regress in any imaginable way including failing to install, upgrade, run, or removing essential packages from the system. Those are unlikely.* There are open bugs about u-u being slower than in the past, thus this may be a likely regression but IMO the pending speed optimizations should not be blocking the backport because the reliability issues are more important to fix and speed optimizations can be cherry-picked later.

[Other Info]

I asked for an exception for the package in following the SRU process:
https://lists.ubuntu.com/archives/ubuntu-release/2018-May/004479.html

I'm preparing the backport in ppa:rbalint/scratch and also run
autopkgtests on it in addition to testing it manually in VMs.

[Original Bug Text]

Changes to support day-of-week patching and logging to syslog were added to upstream (https://github.com/mvo5/unattended-upgrades) over a year ago. These changes are not present in the latest Xenial nor Trusty packages (0.90 and 0.82.1) - requesting that these changes be pulled from upstream.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Brian Murray (brian-murray)]

We don't generally backport new features to stable release, http://wiki.ubuntu.com/StableReleaseUpdates, could you give some justification as to why these should be added to a stable release? Will these be fixing specific bug reports?

[Eric Desrochers (slashd)]

There is one bug in particular I'm interested in LP: #1615381 , where a running kernel may be removed by 'uu'

# /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
...
Removing linux-image-4.13.0-39-generic (4.13.0-39.44~16.04.1) ...^M
WARN: Proceeding with removing running kernel image.^M

[Eric Desrochers (slashd)]

Any update/progress on the full backport of 'uu' ?

[Balint Reczey (rbalint)]

I'm collecting the fixes in Bionic's u-u for Xenial and it is close to be ready for the full backport, but fixing LP: #178116 (or fixing all cases where python-apt losses the lock) is a prerequisite for the backport if we want to avoid an increased crash rate on Xenial.

[Brian Murray (brian-murray)]

Its worth mentioning that for bugs with links to Error Tracker crash buckets that those buckets will probably not receive crash reports from the new version of the package even if it is "the same crash" because bucketing is done using python line numbers which are likely to change given the scope of changes. The point being a search should be done of crash reports in the Error Tracker about Ubuntu 16.04 with the new package version for any bugs that have a known crash bucket to really ensure that they are still not occurring.

[Brian Murray (brian-murray)]

Hello Matt, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.0 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Matt, or anyone else affected,

Accepted unattended-upgrades into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/unattended-upgrades/1.1ubuntu1.18.04.7~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.