[FFe][SRU] Repository profiles only act on .list files and do not affect .source files

[Mark Cunningham]

[Impact]

  * Currently, as reported above, Landscape repository profiles only act upon .list files while .sources remain in effect. As Noble transitioned from the default source definition being the traditional .list files to DEB822 .sources files, this bug manifested when a customer noticed sources that they wanted disabled were still appearing in apt update.

  * The determined solution was to add pattern matching for .sources extensions as well as .list files. Merged PR here: https://github.com/canonical/landscape-client/pull/298

  * The first LTS to support DEB822 .sources was Xenial, however, the first LTS where this format became the default was noble. The proposed SRU is to bring this change back to Focal

  * additionally, .sources and .list files do not get restored when repository profiles are unapplied. This was discovered during verification of the first proposed fix for this bug.

[Testing]

  * To reproduce the problem, apply a repository profile on an affected ubuntu system and observe that it does not migrate .sources files to .sources.save by listing /etc/apt/sources.list.d/, and running apt update to see that the sources are still being used

  * Upgrade to the landscape-client packages from proposed and apply a Landscape profile, noting that .sources files successfully move to .sources.save and no longer appear in the output of apt update

  * Ensure that .list files are also renamed to .list.save (as this is the current behaviour)

The above steps succeeded during verification for proposed versions 23.02-0ubuntu1~22.04.5 (jammy), 24.02-0ubuntu5.4 (noble), and 24.08-0ubuntu2.2 (oracular). These additional steps failed, and should be tested with any new proposed versions:

  * following the above steps, unapply the Landscape repository profile and ensure that .sources.save and .list.save files are renamed back to .sources and .list, respectively

[What can go wrong]

  * The change is rather localized and brief, so it is not expected to introduce any regressions or problems. Applying a profile may take more time due to extra pattern matching and checking - particularly is the filesystem has many .sources files

  * .sources or .list files might be moved and fail to get restored. This can result in the ubuntu archive sources no longer being considered by apt, or a system that has no active sources or sourceparts.

[Original Description]

Somewhat related to LP:2062561, landscape-client currently only appends .save to source lists ending in .list, which leaves 822 style lists using the .sources extension intact.

This means that applying a repository profile to any machine using the new .sources files will still have their public repository sources enabled after a repository profile is applied.

https://github.com/canonical/landscape-client/blob/main/landscape/client/manager/aptsources.py#L144

This was reported 2024-11-08 internally, and now also reported by a customer in case:00400550

[Bryan Fraschetti (bryanfraschetti)]

The customer that reported the issue internally is consuming the landscape-client/jammy 24.04-0landscape0 package from the self-hosted PPA https://launchpad.net/~landscape/+archive/ubuntu/self-hosted-24.04/+packages, attached here is the debdiff against that package

[Bryan Fraschetti (bryanfraschetti)]

As in my other LP comment, this bug is affecting both self-hosted landscape and off-premise landscape. This debdiff is patched against the noble self-hosted package

[Bryan Fraschetti (bryanfraschetti)]

Fixed DEP-3 nit and removed cosmetic changes from the upstream commit per Ubuntu Sponsor's advice

[Bryan Fraschetti (bryanfraschetti)]

Fixed DEP-3 nit and removed cosmetic changes from the upstream commit per Ubuntu Sponsor's advice

[Dave Jones (waveform)]

Targetting to affected series based on associated merge requests. This bug is not yet fixed in production, so I'm looking into that first. We're well into the freeze period now, so this needs to be have an FFe, but the SRU template already filled out can largely serve this purpose; I'll amend the bug title accordingly.

[Dave Jones (waveform)]

It appears Andreas' comments in the plucky branch have been addressed, but I'm going to build a PPA version for autopkgtest just to double check this. If that succeeds, I'll sponsor for plucky (but the SRU's should wait for this to land in devel first).

[Dave Jones (waveform)]

Unfortunately it appears there are still some autopkgtest failures to work-around, specifically relating to the https? squid proxy appearing (or not) where it's unexpected in the results:

amd64: https://autopkgtest.ubuntu.com/results/autopkgtest-plucky-waveform-landscape-client/plucky/amd64/l/landscape-client/20250407_152430_0fc08@/log.gz

arm64: https://autopkgtest.ubuntu.com/results/autopkgtest-plucky-waveform-landscape-client/plucky/arm64/l/landscape-client/20250407_151842_36820@/log.gz

[Graham Inggs (ginggs)]

This sounds more like a bug fix than a feature to me, but with all the SRU paperwork in place, consider this FFe approved

[Launchpad Janitor (janitor)]

This bug was fixed in the package landscape-client - 24.12-0ubuntu2

---------------
landscape-client (24.12-0ubuntu2) plucky; urgency=medium

  [ Bryan Fraschetti ]
  * d/p/2087852-feat-manage-ubuntu-sources-glob.patch: include DEB822 formatted sources
    when managing apt sources by regex matching .sources files (LP: #2087852)

  [ Mitch Burton ]
  * Fix FTBFS (LP: #2106263)
    - d/p/unittest-makeSuite-deprecation.patch: fix tests on python versions
      where unittest.makeSuite has been removed
    - d/rules: don't make root:root owner of installed executables
    - d/tests/control: set environment variables for autopkgtest user and
      build environment
    - d/p/allow-http-proxy-in-tests.patch: fix tests by making them
      insensitive to http/https proxy environment variables

 -- Mitch Burton <email address hidden> Wed, 26 Mar 2025 17:47:28 -0700

[Lukas Märdian (slyon)]

Sponsored for Oracular & Noble. The backport for Jammy & Focal is a bit different and needs to be double-checked.

[Lukas Märdian (slyon)]

Also confirmed and sponsored Jammy & Focal.

[Andreas Hasenack (ahasenack)]

Please amend the test plan with a check to confirm that the existing *.list files are also renamed, not just the *.sources types.

[Andreas Hasenack (ahasenack)]

Hello Mark, or anyone else affected,

Accepted landscape-client into oracular-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/landscape-client/24.08-0ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-oracular to verification-done-oracular. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-oracular. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello Mark, or anyone else affected,

Accepted landscape-client into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/landscape-client/24.02-0ubuntu5.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello Mark, or anyone else affected,

Accepted landscape-client into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/landscape-client/23.02-0ubuntu1~22.04.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

I didn't accept it for focal yet because there is an upload in focal-proposed already[1], and looks like it's ready to be released. Given the holidays ahead, I won't release it today, but it should be good to go next week.

1. https://bugs.launchpad.net/landscape-client/+bug/2020064

[Mitch Burton (mitchburton)]

Tested on jammy using landscape-client version 23.02-0ubuntu1~22.04.5.

I added the proposed source using a .sources file so that I could test it was moved to a .save file after repository profile application.

Registered with landscape-server and applied a repository profile. The resulting state on the client:

ubuntu@thorough-albacore:~$ ls /etc/apt/sources.list.d/
landscape-2087852-test.list proposed.sources.save

ubuntu@thorough-albacore:~$ cat /etc/apt/sources.list
# Landscape manages repositories for this computer
# Original content of sources.list can be found in sources.list.save

I then removed the repository profile from the instance and verified that the state was restored to the original state:

ubuntu@thorough-albacore:~$ ls /etc/apt/sources.list.d/
landscape-2087852-test.list.save proposed.sources.save

ubuntu@thorough-albacore:~$ cat /etc/apt/sources.list
## Note, this file is written by cloud-init on first boot of an instance
## modifications made here will not survive a re-bundle.
## if you wish to make changes you can:
## a.) add 'apt_preserve_sources_list: true' to /etc/cloud/cloud.cfg
## or do the same in user-data
## b.) add sources in /etc/apt/sources.list.d
...

So while the .sources file did get moved to a .save file, it did not get moved _back_ when the repository profile was removed.

I think this fails validation, even though technically the original issue where the .sources files were not moved to .save is fixed.

[Bryan Fraschetti (bryanfraschetti)]

I happened to be testing the SRU against Jammy as well and I see the same behaviour that you do - the .sources files move to .sources.save but don't move back to .sources when the repository profile is removed.

Though, I don't believe this is actually related to the patch. I see all files in sources.list.d/ affected by this issue in the current version of the package as well, without consuming from -proposed. For example, I attached my pro token to the client so my sources.list.d/ had the following list files:
ubuntu-esm-apps.list ubuntu-esm-infra.list

After applying the profile they moved to
ubuntu-esm-apps.list.save ubuntu-esm-infra.list.save
which is expected.

Then after removing the profile, those also don't revert to their original state.

I think we're hitting a separate issue related to the fact that the reversion of sources to their original state only happens specifically on the sources.list file (see here https://github.com/canonical/landscape-client/blob/main/landscape/client/manager/aptsources.py#L139 which checks for the existence of sources.list.save and moves it back). There is no equivalent mechanism (as far as I could tell) that reinstates the sources.list.d directory, so none of its contents are restored to their previous state

Maybe we could add a change that pattern matches for *.save in sources.list.d and restores all of them in the same conditional block that reverts sources.list.save? Though, this might open up a new can of worms since previously applied repository profiles will be present in that directory as profile.list.save, which we wouldn't want to restore.

I see how it could constitute a failure if we want to find a way to selectively revert the contents of sources.list.d within the scope of this issue. But I do just want to mention that's not the current behaviour anyways so I don't think it's a regression. I'd be happy to hear your thoughts on how we should go about this

[Mitch Burton (mitchburton)]

On jammy, this is fine, as the main sources are all in sources.list. On noble and newer, the main sources are in source.list.d/ubuntu.sources, and therefore wouldn't be restored when repo profile is unapplied.

This can definitely be considered a currently-existing bug in >= noble and should be fixed. I would prefer this be fixed as part of this SRU as it's intimately related to the changes already present here.

Basically what I'm saying is that I'm okay with the "doesn't un-save files in sources.list.d" on focal and jammy, but I'm not okay with that behaviour in noble, plucky, and later versions.

In other words, I think this passes validation for focal and jammy but fails validation for noble and plucky (and technically oracular, but it's out-of-support now, right).

[Bryan Fraschetti (bryanfraschetti)]

I'm happy with that assessment. So the SRU for Focal (once verified) and Jammy will proceed, while noble will be blocked on this?

Also, I'm not familiar with the SRU process for the self-hosted PPA. Presumably the sponsorship and verification occurs independently? Or will it consume from -updates and apply self-hosted specific patches on top of that

[Andreas Hasenack (ahasenack)]

> So the SRU for Focal (once verified) and Jammy will proceed, while noble will be blocked on this?

I'm afraid no. We can't release a fix for focal and jammy, and NOT release the fix for noble and later. The reason is that jammy users (for example) upgrading to noble would then regress behavior, because noble doesn't have the fix yet. And the fix we are talking about here is handling deb822 source files.

[Andreas Hasenack (ahasenack)]

What all this means, is that even releasing landscape-client for focal to fix LP: #2020064 is now blocked on this bug here.

[Mitch Burton (mitchburton)]

Okay, so LP: #2020064 is a blocked on this, and therefore this has highest priority for resolution, from my perspective.

Landscape team had a discussion yesterday regarding what the "correct" behaviour should be. We'll be producing an upstream fix sometime in the next few days. I'll take care of producing patches for every supported ubuntu version, plus focal. These patches will supersede whatever is currently in-flight for this bug.

Let me know if it sounds like there's any problems with that.

[Bryan Fraschetti (bryanfraschetti)]

Quite the tangled web...

That sounds like a good plan, though. Thanks for discussing solutions to the sources.list.d/ problem with the rest of the Landscape team as well

[Andreas Hasenack (ahasenack)]

+1

[Mitch Burton (mitchburton)]

>I'll take care of producing patches for every supported ubuntu version, plus focal. These patches will supersede whatever is currently in-flight for this bug.

I've finished with this. I've also created a MP for plucky (instead of oracular).

[Mitch Burton (mitchburton)]

I've created new MPs for the two series that failed verification (noble and oracular) as well as for all other supported series.

[Julian Andres Klode (juliank)]

I'm unable to actually upload the proposed branches at this time, as while the branch is fine, the bug has not been updated in terms of where problems could occur or anything to mention the patch that has gone wrong and the fixing a second time.

The focal release has reached its end of standard support and as such the update likely should be removed from proposed, and further updates be delivered via esm-infra-updates, given this does not fit the past-EOSS Pro enablement policy for future archive updates.

[Bryan Fraschetti (bryanfraschetti)]

Thanks Julian for the suggestion. I've just updated the [What could go wrong] section of the bug report to reflect the issue that was encountered in the first iteration of this SRU and that although the new patch attempts to address this that the problem could potentially recur in this iteration. Let me know if there's anything else I can improve or if this suffices

[Mitch Burton (mitchburton)]

I've updated the bug description to include the additional issue identified, as well as mentioning the failed verification of the first patch.