[SRU] Software Sources is not compatible with deb822

[Leó Kolbeinsson]

[ Impact ]

Ubuntu 24.04 now uses deb822 to represent software sources for deb packages, instead of /etc/apt/sources.list. The complete rationale for this change can be found here: https://discourse.ubuntu.com/t/spec-apt-deb822-sources-by-default/29333

software-properties has two graphical frontends for editing these software sources, one is GTK-based and the other is Qt-based. When the underlying change to deb822 sources was made, the GTK-based frontend and the DBus backend interface gained support.

The original bug report shown below informed us of the lack of deb822 support in the Qt frontend, which results in a broken, non-functional interface when attempting to view or edit most values on the first two tabs. The fix for this is to implement a Qt dialog for deb822 sources that somewhat matches the existing GTK interface.

[ Test Plan ]

Install software-properties-qt 0.99.48 or earlier.

Reproducing the bug:
 1. From the menu, open Software Sources. Alternatively, run `sudo -E software-properties-qt` in a terminal.
 2. Observe that all four checkboxes on the Ubuntu Software tab (main, universe, restricted, and multiverse) are unchecked.
 3. Move to the Other Software tab and observe that there are no items in the list.

Upgrade to software-properties-qt 0.99.49.

Intended functionality:
 1. From the menu, open Software Sources. Alternatively, run `sudo -E software-properties-qt` in a terminal.
 2. Observe that one or more checkboxes on the Ubuntu Software tab (representing main, universe, restricted, or multiverse) are checked. Toggle one or more of the checkboxes.
 3. Using a text editor of some kind (perhaps Vim), confirm that /etc/apt/sources.list.d/ubuntu.sources reflects your changes.
 4. Switch to the Other Software tab, there should be one item in the list.
 5. Select that item, then click Edit at the bottom of the dialog. Make a modification to every field on the screen, or a combination of fields you would like to test. All fields should work.
 6. Click the OK button at the bottom of the dialog to confirm the changes, then click Edit for that item again. The changes you made should be reflected (meaning, the changes were saved.)
 7. Using a text editor of some kind (perhaps Vim), confirm that /etc/apt/sources.list.d/ubuntu.sources reflects your changes.

[ Where problems could occur ]

This is well-tested code. That being said, the following small edge cases exist:
 - If you create a very large amount of rows, it infinitely expands the dialog without a scroll wheel. This is the same functionality as the GTK frontend.
 - Error validation is not performed on the deb822 lines, they are simply written to the ubuntu.sources file. This is good as a starting point, but also relies on the appropriate update notifier handling these errors. This is the same functionality as the GTK frontend.

In a future update, this functionality could be broken by one or more of the following underlying changes:
 - Changes to the software-properties backend which are only implemented for the GTK frontend.
 - Changes to the location or formatting of the ubuntu.sources file.
 - Regressions in the Python bindings for Qt 5, which are rare but possible.
 - Regressions in Python itself.

[ Other Info ]

Ubuntu Studio's live ISO and installed system are affected by this. Only Lubuntu's installed system is affected by this.

[ Original Report ]

Upgrading Lubuntu Jammy to Lubuntu Noble using TUI results in "Software sources" not correctly updated after upgrade. Prompt= shows normal and not LTS

Used command sudo do-release-upgrade -d

Otherwise the upgrade is good

see attached screenshot

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: software-properties-common 0.99.46
ProcVersionSignature: Ubuntu 6.8.0-22.22-generic 6.8.1
Uname: Linux 6.8.0-22-generic x86_64
ApportVersion: 2.28.0-0ubuntu1
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: LXQt
Date: Sat Apr 13 11:44:28 2024
InstallationDate: Installed on 2024-04-13 (0 days ago)
InstallationMedia: Lubuntu 22.04.4 LTS "Jammy Jellyfish" - Release amd64 (20240331)
PackageArchitecture: all
SourcePackage: software-properties
UpgradeStatus: Upgraded to noble on 2024-04-13 (0 days ago)
mtime.conffile..etc.init.d.apport: 2024-02-22T14:20:00

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/2061214

[Leó Kolbeinsson (leok)]

Tested again 18-04-2024 upgarde from Lubuntu Jammy to Lubuntu Noble and this still not correct. As in above screenshot the "software sources" shows unchecked boxes (downloadable software )under "Ubuntu Software". However Prompt now shows LTS which is correct.

[Leó Kolbeinsson (leok)]

Further .. looking a bit closer - "automatic updates" are checked and maybe that should suffice?

[Julian Andres Klode (juliank)]

Please attach a tarball of your /etc/apt/sources.list and /etc/apt/sources.list.d

[Leó Kolbeinsson (leok)]

@juliank Julian

attached is tarbell as requested

[Leó Kolbeinsson (leok)]

This is also present after installing latest Lubuntu Noble iso dated 20240418

my test results here :

https://iso.qa.ubuntu.com/qatracker/milestones/450/builds/299825/testcases/1701/results

[Leó Kolbeinsson (leok)]

This also occurs after upgrading Lubuntu Mantic to Lubuntu Noble

Test results:

https://iso.qa.ubuntu.com/qatracker/milestones/450/builds/299391/testcases/1310/results

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

Just did a fresh install of Lubuntu with Noble 20240419 and I can confirm that the "Ubuntu Software" pane is entirely unchecked but "Prompt=lts" as expected.

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

I also did a fresh install of Kubuntu with Noble 20240419 and the same results are to be found as with Lubuntu above.

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

I'm confident this has to do with the switch to deb822 sources. With Noble, the deb822-compliant /etc/apt/sources.list.d/ubuntu.sources exists, but /etc/apt/sources.list does not. I created it with some relevant lines (not deb822-compliant, of course) and reopened software-properties-qt and found that the relevant lines in the first panel were checked as expected.

What's curious is that it doesn't appear to affect software-properties-gtk. I removed the sources.list file and opened it up and everything was checked as expected.

[Julian Andres Klode (juliank)]

The last comment made me realize we are talking about the Qt frontend here, and yes, sure, we only ever implemented deb822 for the Gtk frontend and the Dbus backend.

The Qt frontend needs to gain a deb822 entry editor dialog, possibly some rendering fixes for deb822 source entries, and swap on deb822=True in the SoftwareProperties.__init__() supercall to enable it.

[Leó Kolbeinsson (leok)]

Tested Lubuntu Noble ISO dated 24-04-2024 and the bug no longer present.

Also tested upgrades from Lubuntu Jammy and Lubuntu Mantic to Lubuntu Noble using GUI and also TUI and bug not present.

[Chris Guiver (guiverc)]

Booted an install made earlier today on hp_compaq_dc7700 of lubuntu noble (20240424) ; https://iso.qa.ubuntu.com/qatracker/milestones/453/builds/300235/testcases/1701/results/

sudo apt update; sudo apt full-upgrade & no updates are available. (software-properties-qt 0.99.48.1)

Following the "Intended Functionality" listed in the Bug.Description at the top of this bug report and behavior is what I'd expect.

The `/etc/apt/sources.list` file contained comments only ; with all subsequent changes being reflected in `/etc/apt/sources.list.d/ubuntu.sources` and looked appropriate to me.

However I'm tired, and fear I've missed something (that I can't 'put my finger on'), but I've repeated steps 1..7 twice & didn't locate what 'irks' me sorry.

[Julian Andres Klode (juliank)]

Yeah sorry folks this was a bit awkward, to avoid respinning other images we temporarily spun out software-properties-qt into its own package (0.99.48.1) and fixed it there, and hence there was no bug closure or anything. This will fold back into the main package in a zero-day SRU in 0.99.49.

[theofficialgman (theofficialgman)]

I do not think the GTK version is fixed.

I am seeing the issue as seen in the screenshot.

edit: the comments showing is likely intended functionality and not a bug even though it looks strange to me

`sources.list`
```
# Ubuntu sources have moved to the /etc/apt/sources.list.d/ubuntu.sources
# file, which uses the deb822 format. Use deb822-formatted .sources files
# to manage package sources in the /etc/apt/sources.list.d/ directory.
# See the sources.list(5) manual page for details.
```

`ubuntu.sources`
```
```
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

## Ubuntu distribution repository
##
## The following settings can be adjusted to configure which packages to use from Ubuntu.
## Mirror your choices (except for URIs and Suites) in the security section below to
## ensure timely security updates.
##
## Types: Append deb-src to enable the fetching of source package.
## URIs: A URL to the repository (you may add multiple URLs)
## Suites: The following additional suites can be configured
## <name>-updates - Major bug fix updates produced after the final release of the
## distribution.
## <name>-backports - software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
## Components: Aside from main, the following components can be added to the list
## restricted - Software that may not be under a free license, or protected by patents.
## universe - Community maintained packages.
## Software from this repository is only maintained and supported by Canonical
## for machines with Ubuntu Pro subscriptions. Without Ubuntu Pro, the Ubuntu
## community provides best-effort security maintenance.
## multiverse - Community maintained of restricted. Software from this repository is
## ENTIRELY UNSUPPORTED by the Ubuntu team, and may not be under a free
## licence. Please satisfy yourself as to your rights to use the software.
## Also, please note that software in multiverse WILL NOT receive any
## review or updates from the Ubuntu security team.
##
## See the sources.list(5) manual page for further settings.
Types: deb
URIs: http://ports.ubuntu.com/ubuntu-ports/
Suites: noble noble-updates noble-backports
Components: main universe restricted multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

## Ubuntu security updates. Aside from URIs and Suites,
## this should mirror your choices in the previous section.
Types: deb
URIs: http://ports.ubuntu.com/ubuntu-ports/
Suites: noble-security
Components: main universe restricted multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
```

[ԜаӀtеr Ⅼарсһуnѕkі (wxl)]

@theofficialgman this is specifically about the qt interface, so i've you've found a bug against the gtk one, it is something different. please file a separate bug. thanks!

[Steve Langasek (vorlon)]

Hello Leó, or anyone else affected,

Accepted software-properties into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/software-properties/0.99.49 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted software-properties (0.99.49) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

software-properties-qt/unknown (amd64, arm64, armhf, i386, ppc64el, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#software-properties

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!