17.04 to 17.10 dist upgrade switched me to lowlatency kernel b/c of aufs-tools

[James]

[SRU Justification]
The combination in 17.10 of a new Recommends: on aufs-dkms from aufs-tools, with a new Provides: aufs-dkms on each of the kernel packages causes apt to choose a kernel at random to satisfy this package relationship on upgrade from 17.04.

Randomly installing a kernel is bad mmmkay.

[Test case]
1. Install aufs-tools and linux-image-generic on a 17.04 system.
2. Configure the apt sources to point to artful.
3. Run sudo apt update && sudo apt dist-upgrade.
4. Confirm that apt wants to install linux-image-4.13.0-16-lowlatency.
5. Cancel the upgrade.
6. Enable artful-proposed in sources.list.
7. Run sudo apt update && sudo apt dist-upgrade.
8. Confirm that apt no longer offers to install linux-image-4.13.0-16-lowlatency.

[Regression potential]
This will cause a behavior change in that currently, installing aufs-tools into a chroot or container will pull in a kernel package with it by default, and after this SRU it will no longer do so.

This is not a regression.

If a user is running a non-Ubuntu kernel which does not provide (explicitly or logically) aufs-dkms, after this SRU, installing aufs-tools will not cause an Ubuntu kernel package to be pulled in as a recommends.

This is uninteresting, as the user would discover the lack of kernel aufs support the first time they tried to use the utilities; and it is unlikely that their preferred solution to the problem would be to stop using a non-Ubuntu kernel.

[Original description]
When I upgraded from 17.04 to 17.10, I was automatically switched from the generic kernel to the lowlatency kernel. I did not have the lowlatency kernel installed prior to the dist upgrade (I have verified this by inspecting my /var/log/apt/history.log* files, all upgrades prior to the dist upgrade only show updating *-generic kernel packages, if I had lowlatency installed it would show that as well.) I'm not sure why I was switched to lowlatency, I don't see anything in the release notes saying that there was a switch to lowlatency as the default, and other users who have just done a dist upgrade weren't switched to lowlatency. So I think it is probably a bug that I was switched to lowlatency - perhaps there's a package that I have installed that is incorrectly depending on the lowlatency kernel in 17.10? I'm not sure how to find out if that's the case.

To be explicit, after the dist-upgrade, I had both the 4.13.0-16-generic and 4.13.0-16-lowlatency kernels installed, and by default 4.13.0-16-lowlatency was what was booted into.

This also caused a problem when I did the upgrade - perhaps this is a separate bug that I should report, let me know and I'll do that. After the dist-upgrade, wifi didn't work. I have a BCM4352 wifi card, whose driver is provided by the bcmwl-kernel-source package, which uses DKMS to compile it. Presumably during the dist-upgrade, the driver was compiled for the 4.13.0-16-generic kernel, so when I manually selected that kernel on boot, it worked, but it wasn't compiled for the 4.13.0-16-lowlatency kernel, which is why wifi didn't work out of the box after the dist-upgrade. The solution was of course straight forward, I had to reinstall the bcmwl-kernel-source package, but a non working system after an upgrade requiring reinstalling packages or booting into a different kernel to fix is presumably a bug.

If the switch from generic to lowlatency was a feature, not a bug, then such a switch should also be accompanied by compiling all the kernel modules that were compiled against the generic kernel against the new lowlatency kernel, so as to ensure the system is in a working state after completing a dist upgrade. Also, if the switch is a feature, then I would expect it to be documented in the release notes with a brief explanation or link to more information on why the switch was done.

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1728760/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Brian Murray (brian-murray)]

Thanks for taking the time to report this bug and helping to make Ubuntu better. Could you please add the log files from '/var/log/dist-upgrade/' to this bug report as separate attachments? Thanks in advance.

[Travis (travissmith90)]

This is exactly what happened to me as well. Computer is rather powerful, so not necessarily an obvious candidate for ll kernel: 16GB, core i7, NVMe M.2 SSD.

There is nothing that I can ascertain as to why the system switched to the low-latency kernel after the upgrade to 17.10.

[James (j-james)]

Sorry for not responding sooner. I've attached what seem to be the most relevant files, if there's any others of interest let me know. This log message seems to be relevant:

Installing linux-image-4.13.0-16-lowlatency as Recommends of aufs-tools

[Brian Murray (brian-murray)]

Thanks for digging into this, this is the result of a change in the aufs-tools package.

+aufs-tools (1:4.1+20160919-2) unstable; urgency=medium
+
+ * debian/control:
+ - Move aufs-dkms from Depends to Recommends. (Closes: #838632)
+
+ -- Jan Luca Naumann <email address hidden> Sun, 25 Sep 2016 20:44:43 +0200

[Steve Langasek (vorlon)]

It's not the demotion of the aufs-dkms relationship from depends to recommends that triggers this, but that the 17.04 aufs-tools had no dependency at all and 17.10 adds the recommends.

I think aufs-tools needs to drop this Recommends in SRU and until after 18.04 release, since all of our kernels logically satisfy the relationship and expressing it in apt does bad things to the upgrade.

[Brian Murray (brian-murray)]

Hello James, or anyone else affected,

Accepted aufs-tools into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/aufs-tools/1:4.1+20161219-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Jesse Glick (jesse-glick)]

After upgrading from Zesty (with Docker installed) to Artful, I noticed the switch to the lowlatency kernel (since it caused me a little DKMS pain with the VirtualBox package), but otherwise paid it little mind. Later I realized that my laptop was constantly freezing up (could not even move the mouse pointer) when just one process was consuming most of a single CPU core, which had never been true before. After finding this issue, I tried booting into the generic kernel (ESC to get the Grub menu) and then removing the lowlatency kernel (image & headers). So far so good. Unclear to me whether updating `aufs-tools` at this point would make any difference, since it sounds like this change would only affect new upgrades.

[Brian Murray (brian-murray)]

I followed the test case in the bug description and can confirm that after enabling artful-proposed in my /etc/apt/sources.list file the lowlatency kernel is no longer offered to be installed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package aufs-tools - 1:4.1+20161219-1ubuntu0.1

---------------
aufs-tools (1:4.1+20161219-1ubuntu0.1) artful; urgency=medium

  * Remove the Recommends: on aufs-dkms; our kernels in 17.10 have started
    to provide this virtual package, but this means apt on upgrade will pick
    a kernel at random to satisfy the Recommends. Since all of our kernel
    packages logically satisfy this relationship, the explicit declaration
    only causes trouble. This change can be dropped for the 18.10 release.
    (LP: #1728760)

 -- Steve Langasek <email address hidden> Tue, 07 Nov 2017 10:58:54 -0800

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for aufs-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.