Support post install enablement of OEM-enabled devices

[Iain Lane]

[ Description ]

The Ubuntu installer (ubiquity), working together with ubuntu-drivers, will install an "OEM metapackage" for the platform being installed, if there is one which matches.

This means that if Canonical has performed enablement for a device, users will receive the same experience if they purchase hardware with Ubuntu preinstalled or if it has another OS and they later install Ubuntu.

However, if the hardware was enabled post-release and the user is offline when installing Ubuntu, the installer will not know that there is any enablement that it should install. Similarly if the enablement happens after Ubuntu has been installed. In these cases we need a way inside the installed session for the same enablement to be provided.

We're adding the capability for update-manager to install these packages. They themselves install a sources.list.d snippet referring to an "OEM archive" specific to the device, so update-manager needs to know to update (as in `apt update`) and then upgrade (`apt upgrade`) a second time after installing oem-foo-meta from the Ubuntu archive.

update-manager will be consuming a file provided by update-notifier to know if the device needs an oem metapackage or not.

NOTE NOTE NOTE NOTE NOTE NOTE: The OEM metapackages are LTS only, so the intention is that this change is effectively a no-op on hirsute. Therefore we are proposing NOT to SRU to groovy, as there is no chance of a regression for groovy users.

[ QA ]

= On a certified device =

1. Update the system to the latest without focal-proposed.
2. Enable focal-proposed, and then only install update-manager/1:20.04.10.2 and update-notifier/3.192.30.4 from focal-proposed.
3. Reboot the system, login the desktop and wait for a while.
The notification will pop up and it will show "Improved hardware support" on the certified machines that has the OEM metapackage support.

= On a non certified device =

1. Update the system to the latest without focal-proposed.
2. Enable focal-proposed, and then only install update-manager/1:20.04.10.2 and update-notifier/3.192.30.4 from focal-proposed.
3. Reboot the system, login the desktop and wait for a while.
The notification will pop up but it won't show "Improved hardware support" on non certified machines.

[ What could go wrong ]

In this update we rework transaction handling. If this is wrong, then the progress bar or terminal could stop working.

If there's a bug in the way we install / update / upgrade the OEM metapackages then we could break installing any update.

If we accidentally apply this logic to non OEM systems then we could break updating for everybody.

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-manager - 1:21.04.2

---------------
update-manager (1:21.04.2) hirsute; urgency=medium

  * Make pycodestyle happy
  * Make pyflakes happy, fixes a bug too

 -- Julian Andres Klode <email address hidden> Mon, 14 Dec 2020 12:13:51 +0100

[Timo Aaltonen (tjaalton)]

Hello Iain, or anyone else affected,

Accepted update-manager into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-manager/1:20.04.10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello Iain, or anyone else affected,

Accepted update-notifier into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.30.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Shih-Yuan Lee (fourdollars)]

I didn't see $XDG_RUNTIME_DIR/ubuntu-drivers-oem.package-list after installing update-manager/1:20.04.10.2 and update-notifier/3.192.30.4 from focal-proposed on Dell XPS 13 9300.
But I do see oem-somerville-melisa-meta by `ubuntu-drivers list-oem` and `ubuntu-drivers list`.

[Julian Andres Klode (juliank)]

I'm not convinced that this verification is sufficient to conclude failure.

You have to get an update notification and launch update-manager via update-notifier, as update-notifier writes the file before it launches update-manager.

[Shih-Yuan Lee (fourdollars)]

I found a way to verify this.

1. Update the system to the latest without focal-proposed.
2. Enable focal-proposed, and then only install update-manager/1:20.04.10.2 and update-notifier/3.192.30.4 from focal-proposed.
3. Reboot the system and wait for a while.
The notification will pop up and we can see "Improved hardware support" on the certified machines.

[Shih-Yuan Lee (fourdollars)]

This is from Dell XPS 13 9300.

[Julian Andres Klode (juliank)]

@fourdollars So, I guess you meant to set this as verification-done now, or do you want to do more testing?

[Shih-Yuan Lee (fourdollars)]

Dell Inspiron 5391 doesn't have the OEM metapackage.

[Shih-Yuan Lee (fourdollars)]

I am testing more.

[Shih-Yuan Lee (fourdollars)]

The system of Dell Inspiron 5391 is installed by the stock Ubuntu 20.04.1 ISO without any update but only with update-manager/1:20.04.10.2 and update-notifier/3.192.30.4 from focal-proposed.

It keeps crashing on Dell Inspiron 5391.

[Łukasz Zemczak (sil2100)]

Hello Iain, or anyone else affected,

Accepted update-manager into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-manager/1:20.04.10.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Shih-Yuan Lee (fourdollars)]

It works fine on Dell Inspiron 5391 with update-manager/1:20.04.10.3 and update-notifier/3.192.30.4 from focal-proposed.

[Shih-Yuan Lee (fourdollars)]

I am checking Dell XPS 13 9300 with update-manager/1:20.04.10.3 and update-notifier/3.192.30.4.

[Shih-Yuan Lee (fourdollars)]

It works fine on Dell XPS 13 9300 with update-manager/1:20.04.10.3 and update-notifier/3.192.30.4.

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.30.4

---------------
update-notifier (3.192.30.4) focal; urgency=medium

  * List OEM metapackages before invoking update-manager. Before invoking
    update-manager, run a helper script to figure out if there are any
    oem-*-meta packages which apply to this system. These are enablement
    packages maintained by Canonical which install any tweaks required by a
    particular piece of hardware. The script takes care to exit quickly if
    it's been run before or if the system is already enabled. It uses
    ubuntu-drivers' python interface, so we need to add a dependency to make
    sure that is available.There will be a companion change in
    update-manager to consume this list. (LP: #1908050)

 -- Iain Lane <email address hidden> Mon, 14 Dec 2020 11:22:00 +0000

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for update-notifier has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-manager - 1:20.04.10.3

---------------
update-manager (1:20.04.10.3) focal; urgency=medium

  [ Julian Andres Klode ]
  * UpdateManager/backend/__init__.py: Fix unbound local variables regression
    introduced by the previous SRU (See LP 1910556), and correct comment

  [ William Wilson ]
  * UpdateManager/Core/UpdateList.py: change to a regex from a static list
    of packages to be grouped under Ubuntu Base (LP: #1902025)

update-manager (1:20.04.10.2) focal; urgency=medium

  [ Iain Lane & Julian Andres Klode ]
  * Support post install enablement of OEM-enabled devices (LP: #1908050)

 -- Brian Murray <email address hidden> Wed, 13 Jan 2021 08:35:58 -0800

[Brian Murray (brian-murray)]

While the version of ubuntu-drivers-common in the release pocket for Ubuntu 20.04 LTS has code for "system_device_specific_metapackages" it seems that some users[1] have strange versions of ubuntu-drivers-common installed and are receiving Tracebacks with an attribute error. So it seems like the update-notifier SRU would have benefited from a versioned dependency on ubuntu-drivers-common.

[1] https://errors.ubuntu.com/problem/13cbbbaf8da4338aa930bf2b22960d807b38ae70

[Iain Lane (laney)]

On Fri, Jan 22, 2021 at 01:02:56AM -0000, Brian Murray wrote:
> While the version of ubuntu-drivers-common in the release pocket for
> Ubuntu 20.04 LTS has code for "system_device_specific_metapackages" it
> seems that some users[1] have strange versions of ubuntu-drivers-common
> installed and are receiving Tracebacks with an attribute error. So it
> seems like the update-notifier SRU would have benefited from a versioned
> dependency on ubuntu-drivers-common.
>
> [1]
> https://errors.ubuntu.com/problem/13cbbbaf8da4338aa930bf2b22960d807b38ae70

Some of these people are running derivatives which appear to be taking
Ubuntu's SRUs directly but also modifying some packages - that seems a
bit risky.

But not all of them.

I can see that this could maybe happen if update-notifier runs when
you're in the middle of a release upgrade from bionic, if
ubuntu-drivers-common gets updated first and then the old
update-notifier runs before it itself gets updated. (side note: if that
can happen, it seems undesirable - we don't want update-notifier running
when there's other apt operations ongoing including release upgrading)

Do you think we should follow up with an update to set a version on this
dep?

Cheers,

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]