[SRU] Add support for release upgrading

Bug #2038958 reported by Simon Quigley
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
lubuntu-update-notifier (Ubuntu)
Fix Released
Medium
Simon Quigley
Jammy
Fix Committed
Medium
Simon Quigley
Lunar
Fix Released
Medium
Simon Quigley
Mantic
Fix Released
Medium
Simon Quigley

Bug Description

[ Impact ]

The lubuntu-update-notifier package is a PyQt 5-based application which automatically checks for updates and helps the user install them. It is the main update notifier system in Lubuntu, using update-notifier-common to check for updates and apply them using the aptdaemon library. It was originally written by Hans Möller, a former Lubuntu Member, but maintained by the Lubuntu team.

In bug 2002255, a precedent was set regarding updates to this software. When initially shipped, it was a rather primitive application with rough edges, especially around error handling. We have since polished it, adding necessary information (for example, debconf prompts).

This bug is regarding the default user experience around upgrading to the latest version of Lubuntu. There is currently no graphical way in which the user is prompted for or allowed to proceed with the upgrade. In our notes, we typically ask them to run `sudo do-release-upgrade`, which does do the job at a basic level, but does not integrate it with Lubuntu's existing, light, Qt-based UI.

The upload at hand fixes that. When the user is prompted for updates, they will be asked afterwards about upgrading to the latest release (if available). This evaluation is done via `do-release-upgrade -c`, not by re-implementing any existing logic. If they choose to accept, the existing Qt frontend is launched, via `lxqt-sudo do-release-upgrade -m desktop -f DistUpgradeViewKDE`. When that process completes, the update notifier cleanly exits. I have added a hard dependency on ubuntu-release-upgrader-qt to ensure this process is successful.

Noble is not affected by this, because a complete rewrite is now in the release pocket. This uses C++/Qt 5 and already implements this functionality.

[ Technical Details ]

The update notifier is started by `lubuntu-upg-notifier.sh`, which is ran as an autostart application within LXQt (I have tested patches pending for next cycle which moves this to a systemd user timer). This patch starts off by calling `do-release-upgrade -c` and capturing the exit code via `$?`. That exit code is directly fed into lubuntu-notifier.py as an argument.

In the Python file, I added an argument via ArgParse which converts that exit code into a boolean (for more Pythonic use later). That boolean is then passed (through a few functions and class declarations) to `Dialog.initUI()`.

If the dialog is launched without any packages to upgrade but with a release to upgrade to, it will skip right to that function. (Setting the title to "Upgrade Lubuntu" is declared twice, once for a window without any upgrades, and another for the "after-upgrade" step.)

This is where the additional import of QThread (etc.) comes in; while it's not the original bug, when an additional window pops up, the original update notifier window freezes. To fix that, I created a special class which inherits QThread, and used that to call subprocess (I tried to use the Python threading library, but the freeze was still happening).

Both `Dialog.on_upgrade_finished()` and `Dialog.initUI()` call `Dialog.call_release_upgrader()`, which uses `RunUpgradeThread()` as well. You'll notice I created an additional `self.thread2` variable; if I were to re-define `self.thread` within that function, that kills its own parent (in the case of upgrades), which makes the additional thread entirely non-functional.

When that QThread process is done, a clean `app.exit()` is performed (via an existing function).

You might notice `self.buttonBox.clicked.disconnect(self.call_upgrade)` - this is required because otherwise the connect below it simply *adds* to the one performed in `__init__()`, causing two password popups.

Additionally, if Launchpad can be queried, a link to our release announcement is generated.

[ Test Plan ]

 (Jammy only)
 * Run `sudo sed -i "s/lts/normal/" /etc/update-manager/release-upgrades`

 * Log out and log back in (to restart the notifier).
 * Install any updates (if there are any to install).
   - There's two sub-testcases here. The first is a system where updates are being applied, and when prompted, the user goes ahead with the upgrade. The second is a system where all updates are already applied, and on first launch it prompts the user to upgrade.
 * Click the link to ensure it brings you to the correct release notes.
 * Go through the upgrade process, ensuring the window smoothly closes.

[ Where problems could occur ]

 * If `do-release-upgrade` decides to return a non-integer exit code, or the shell changes in an unexpected way where that code is no longer being captured correctly, it will cause an ArgParse exception.
 * If the output of `do-release-upgrade -c` changes drastically, the version will not be detected.
 * All Launchpad error handling is pretty bluntly done; if it can't access it for any reason, the fallback is no link.
 * Existing Qt bugs could very well manifest themselves in this application. By importing an additional Qt library, you're technically increasing the surface area in which errors can occur.
 * This relies on the increasingly-bitrotting ubuntu-release-upgrader-qt, which needs some cleanup work. It's our best shot for now, since re-implementing it should be done for all Qt flavors.

Simon Quigley (tsimonq2)
Changed in lubuntu-update-notifier (Ubuntu Jammy):
importance: Undecided → Medium
Changed in lubuntu-update-notifier (Ubuntu Lunar):
importance: Undecided → Medium
Changed in lubuntu-update-notifier (Ubuntu Mantic):
importance: Undecided → Medium
Changed in lubuntu-update-notifier (Ubuntu Jammy):
status: New → In Progress
Changed in lubuntu-update-notifier (Ubuntu Lunar):
status: New → In Progress
Changed in lubuntu-update-notifier (Ubuntu Mantic):
status: New → In Progress
Changed in lubuntu-update-notifier (Ubuntu Jammy):
assignee: nobody → Simon Quigley (tsimonq2)
Changed in lubuntu-update-notifier (Ubuntu Lunar):
assignee: nobody → Simon Quigley (tsimonq2)
Changed in lubuntu-update-notifier (Ubuntu Mantic):
assignee: nobody → Simon Quigley (tsimonq2)
Revision history for this message
Brian Murray (brian-murray) wrote :

Sometime ago[1] ubuntu-release-upgrader was modified so that sudo is called with the `-E` so that environment variables are passed along to do-release-upgrade. While this was initially done for a specific environment variable it is also important that other environment variables are passed to do-release upgrade e.g. XDG_SESSION_TYPE. Looking at the diff I don't see `-E` but then again I don't know if lxqt-sudo supports that argument.

Regardless, you could provide some details about whether or not environment variables are passed to do-release-upgrade?

[1] https://git.launchpad.net/ubuntu-release-upgrader/commit/?id=f849580210947308892c42e8a5c12a1d8b039c26

Changed in lubuntu-update-notifier (Ubuntu Lunar):
status: In Progress → Incomplete
Revision history for this message
Simon Quigley (tsimonq2) wrote :

These are the environment variables explicitly allowlisted by lxqt-sudo: https://github.com/lxqt/lxqt-sudo/blob/master/sudo.cpp#L99

I've attached three files; they are the output of env with `qterminal` / `sudo qterminal` / `lxqt-sudo qterminal`.

If we need to allowlist more variables, or skip that filtering entirely, say the word, I'll have an lxqt-sudo upload incoming.

Thanks.

Revision history for this message
Simon Quigley (tsimonq2) wrote :
Revision history for this message
Simon Quigley (tsimonq2) wrote :
Simon Quigley (tsimonq2)
Changed in lubuntu-update-notifier (Ubuntu Lunar):
status: Incomplete → In Progress
Revision history for this message
Steve Langasek (vorlon) wrote :

+ lxqt-sudo (>= 1.3.0-0ubuntu2),

Wrong version; the version accepted into mantic-proposed is 1.3.0-0ubuntu1.1.

Revision history for this message
Steve Langasek (vorlon) wrote :

SRU blocked on availability for fix for the devel series

Changed in lubuntu-update-notifier (Ubuntu):
status: In Progress → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lubuntu-update-notifier - 1.0.0~alpha1

---------------
lubuntu-update-notifier (1.0.0~alpha1) noble; urgency=medium

  * Replace the old update notifier with Lubuntu Update.
  * Change maintainer to Lubuntu Developers.
  * Update copyright file.

 -- Aaron Rainbolt <email address hidden> Thu, 11 Jan 2024 03:21:20 +0000

Changed in lubuntu-update-notifier (Ubuntu):
status: Incomplete → Fix Released
Simon Quigley (tsimonq2)
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Simon, or anyone else affected,

Accepted lubuntu-update-notifier into mantic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lubuntu-update-notifier/0.6.2~23.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-mantic to verification-done-mantic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-mantic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in lubuntu-update-notifier (Ubuntu Mantic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-mantic
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Simon, or anyone else affected,

Accepted lubuntu-update-notifier into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lubuntu-update-notifier/0.6.0~23.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in lubuntu-update-notifier (Ubuntu Lunar):
status: In Progress → Fix Committed
tags: added: verification-needed-lunar
Changed in lubuntu-update-notifier (Ubuntu Jammy):
status: In Progress → Fix Committed
tags: added: verification-needed-jammy
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Simon, or anyone else affected,

Accepted lubuntu-update-notifier into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lubuntu-update-notifier/0.5.1~22.04.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Simon Quigley (tsimonq2) wrote :

Lunar is good. Here are my specific steps:
 1. Install Lubuntu 23.04.
 2. Enable -proposed, only upgrade lubuntu-update-notifier 0.6.0~23.04.3 which brings in lxqt-sudo (previous SRU is already good).
 3. Restart the system entirely.
 4. Run all system updates. Reboot again, as a result of these updates.
 5. I'm notified of a new release being available. When I click "Apply," it successfully launches the graphical release upgrader.

Success.

tags: added: verification-done-lunar
removed: verification-needed-lunar
Revision history for this message
Simon Quigley (tsimonq2) wrote :

In testing this for Mantic, I realized something important: we can't actually test it quite yet, because Noble hasn't been released! If we modified the package further, that will require another round of SRU + testing.

I asked myself, how exactly does do-release-upgrade get this information? Well, it gets it from https://changelogs.ubuntu.com/meta-release (when set to "normal," of course.) If I could simply MITM changelogs.ubuntu.com in my sandbox environment, that will simulate Noble being released. Unlike plasma-distro-release-notifier, our notifier system doesn't care about ubuntu.csv. It'll notify whenever the release is ready. This is the only way I can possibly see to test this without actually changing the package in any way, or adding UX for devel upgrades, which is outside the scope of this SRU entirely.

Here's my MITMProxy script (consider it as DWTFYW-licensed):
# cat test.py
class AddHeader:
    def response(self, flow):
        replacement_content = ""
        with open("/tmp/meta-release", "r") as file:
            replacement_content = file.read()
        flow.response.text = replacement_content

addons = [
    AddHeader()
]

This successfully tricked do-release-upgrade into thinking there's a new release available, which therefore triggered this upgrade.

In terms of lubuntu-update-notifier, which is what we're specifically testing for, verification is successful, with version 0.6.2~23.10.1 in mantic-proposed. I followed the same instructions as Lunar, precisely.

tags: added: verification-done-mantic
removed: verification-needed-mantic
Revision history for this message
Brian Murray (brian-murray) wrote :

You could also add an /etc/hosts entry for changelogs.ubuntu.com pointing to as server in your control and add noble to meta-release on that server.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lubuntu-update-notifier - 0.6.2~23.10.1

---------------
lubuntu-update-notifier (0.6.2~23.10.1) mantic; urgency=medium

  * Add support for release upgrading, when all updates are applied (LP: #2038958).

 -- Simon Quigley <email address hidden> Tue, 10 Oct 2023 14:06:19 -0500

Changed in lubuntu-update-notifier (Ubuntu Mantic):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for lubuntu-update-notifier has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lubuntu-update-notifier - 0.6.0~23.04.3

---------------
lubuntu-update-notifier (0.6.0~23.04.3) lunar; urgency=medium

  * Add support for release upgrading, when all updates are applied (LP: #2038958).

 -- Simon Quigley <email address hidden> Tue, 10 Oct 2023 14:06:19 -0500

Changed in lubuntu-update-notifier (Ubuntu Lunar):
status: Fix Committed → Fix Released
Revision history for this message
Simon Quigley (tsimonq2) wrote :

> You could also add an /etc/hosts entry for changelogs.ubuntu.com pointing to as server in your control and add noble to meta-release on that server.

That would work if do-release-upgrade didn't explicitly use SSL when querying changelogs.ubuntu.com :)

Revision history for this message
Aaron Rainbolt (arraybolt3) wrote (last edit ):

Sadly, I can't say this is verification-done for Jammy. I hit a number of bugs trying to test it.

* There doesn't appear to be any line wrapping at all on the main update installation screen anymore. This may seem petty, but this is the updater some of our users are going to use for the next year (and maybe even the next three years if they use Lubuntu Jammy beyond its end-of-life). This looks bad enough to warrant fixing that.
* More to the point, on a fresh Lubuntu Jammy installation, my first attempt at running the upgrade bombed out.
- Installed Lubuntu 22.04.
- Used the sed command in the test plan.
- Enabled -proposed and ran sudo apt update.
- Installed the updated lubuntu-update-notifier.
- Disabled -proposed and ran sudo apt update.
- Rebooted.
- Used the new updater to install all system updates.
- Got the offer to upgrade and accepted it.
- Entered my password.
- All windows vanished, no upgrade was started.
* I then logged out and back in, in an attempt to trigger the upgrade. I was immediately offered the ability to upgrade, but with the peculiar message "An upgrade to Lubuntu 23.10 is available. Would you like to install it? Reboot required". This looks like some things are getting combined that shouldn't be combined with the "reboot required" bit.
* Notwithstanding the above oddity, I confirmed that I did want to upgrade, whereupon I was asked for my password. I provided it... and was immediately asked for my password again. I provided that too, which caused a normal *update* installation window to appear, not the do-release-upgrade window. It finished doing whatever it did very quickly, I clicked "Close", and then everything vanished. No upgrade was started.
* Finally, I rebooted the VM, and was again offered the upgrade. I accepted, input my password (twice again - it asked me for it two times), and *that* finally triggered the do-release-upgrade window. But for some reason, the background "Upgrade Lubuntu" window didn't change to say "Upgrading..." or anything similar like I had seen it do before. It remained active! This looks like a user could very easily start the upgrader twice, or potentially kill an in-progress upgrade on accident.

This needs more work before we can push it into an existing LTS. I'll look into it hopefully soon.

tags: added: verification-failed-jammy
removed: verification-needed-jammy
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.