apt 2.0.5 in focal causes autopkgtest failures for update-manager

[Brian Murray]

======================================================================
ERROR: Test that updates from the security node go in
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest.0o0kua/build.B7h/src/tests/test_update_list.py", line 30, in setUp
    self.cache = MyCache(apt.progress.base.OpProgress(),
  File "/tmp/autopkgtest.0o0kua/build.B7h/src/UpdateManager/Core/MyCache.py", line 70, in __init__
    apt.Cache.__init__(self, progress, rootdir)
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 170, in __init__
    self.open(progress)
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 232, in open
    self._cache = apt_pkg.Cache(progress)
apt_pkg.Error: W:Unable to read /tmp/autopkgtest.0o0kua/build.B7h/src/tests/aptroot-update-list-test/etc/apt/preferences.d/ - DirectoryExists (2: No such file or directory), E:Regex compilation error - Invalid range end

Full test results can be found here:

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-focal/focal/amd64/u/update-manager/20210416_180839_df210@/log.gz

Downgrading apt to version 2.0.4 causes the test to pass again.

[Julian Andres Klode (juliank)]

Can confirm it's broken:

^['linux-.*'-5\.4\.0-72-generic$|^ 'kfreebsd-.*'-5\.4\.0-72-generic$|^ 'gnumach-.*'-5\.4\.0-72-generic$|^ '.*-modules'-5\.4\.0-72-generic$|^ '.*-kernel'-5\.4\.0-72-generic$|^ 'linux-.*'-5\.4\.0-72-generic$|^ 'kfreebsd-.*'-5\.4\.0-72-generic$|^ 'gnumach-.*'-5\
.4\.0-72-generic$|^ '.*-modules'-5\.4\.0-72-generic$|^ '.*-kernel'-5\.4\.0-72-generic$|^ 'linux-.*'-5\.4\.0-72-generic$|^ 'kfreebsd-.*'-5\.4\.0-72-generic$|^ 'gnumach-.*'-5\.4\.0-72-generic$|^ '.*-modules'-5\.4\.0-72-generic$|^ '.*-kernel']-5\.4\.0-72-generic$
23:03

Seems like the test is broken

APT::VersionedKernelPackages "['linux-.*', 'kfreebsd-.*', 'gnumach-.*', '.*-modules', '.*-kernel', 'linux-.*', 'kfreebsd-.*', 'gnumach-.*', '.*-modules', '.*-kernel', 'linux-.*', 'kfreebsd-.*', 'gnumach-.*', '.*-modules', '.*-kernel']";

This code sequence is wrong:

        # override the kernel autoremove regex
        real_versioned_kernel_pkgs = apt.apt_pkg.config.value_list(
            "APT::VersionedKernelPackages")
        apt.apt_pkg.config.set("APT::VersionedKernelPackages", "linux-.*")
        self.addCleanup(
            lambda: apt.apt_pkg.config.set("APT::VersionedKernelPackages",
                                           str(real_versioned_kernel_pkgs)))

It reads a list, then sets a value option instead of the list, and then after exit, overrides the value option with a string representation of the python list.

[Julian Andres Klode (juliank)]

What you want is define

def set_versioned_kernel_pkgs(real_versioned_kernel_pkgs):
  apt_pkg.config.clear("APT::VersionedKernelPackages")
  for k in real_versioned_kernel_pkgs:
    apt_pkg.config.set("APT::VersionedKernelPackages::", k)

and then use

        real_versioned_kernel_pkgs = apt.apt_pkg.config.value_list()
        set_versioned_kernel_pkgs(["linux-.*"])
        self.addCleanup(lambda: set_versioned_kernel_pkgs(real_versioned_kernel_pkgs))

[Julian Andres Klode (juliank)]

Also add a Depends: apt in debian/tests/control so the test is triggered automatically by apt!

[Brian Murray (brian-murray)]

Given that we are also adding apt to debian/test/control I don't think this should stay in -proposed after verification even though it is only a test fix.

[Łukasz Zemczak (sil2100)]

Okay, I think once in a while it's possible to release test-only fixes like these. Let's see after 7 days.

[Łukasz Zemczak (sil2100)]

Hello Brian, or anyone else affected,

Accepted update-manager into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-manager/1:21.04.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Brian, or anyone else affected,

Accepted update-manager into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-manager/1:20.10.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Brian, or anyone else affected,

Accepted update-manager into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-manager/1:20.04.10.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted update-manager (1:21.04.10) for hirsute have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-release-upgrader/1:21.04.11 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/hirsute/update_excuses.html#update-manager

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Brian Murray (brian-murray)]

Using the update_excuses report as a reference we can see the test pass for Focal.

update-manager (1:20.04.10.6 to 1:20.04.10.7)

    Migration status for update-manager (1:20.04.10.6 to 1:20.04.10.7): BLOCKED: Needs an approval (either due to a freeze, the source suite or a manual hint)
    Issues preventing migration:
    Not touching package due to block request by freeze (please contact ubuntu-release if update is needed)
    Additional info:
    0 days old

[Brian Murray (brian-murray)]

And they also passed for Hirsute.

update-manager (1:21.04.8 to 1:21.04.10)

    Migration status for update-manager (1:21.04.8 to 1:21.04.10): BLOCKED: Needs an approval (either due to a freeze, the source suite or a manual hint)
    Issues preventing migration:
    Not touching package due to block request by freeze (please contact ubuntu-release if update is needed)
    Additional info:
    0 days old

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-manager - 1:21.10.1

---------------
update-manager (1:21.10.1) impish; urgency=medium

  * UpdateManager/Core/MetaRelease.py: use -proposed in the variable which
    corresponds to it.
  * debian/control: drop the build-dep on apt-clone which was missed during
    the separtion of u-r-u and update-manager.

 -- Brian Murray <email address hidden> Wed, 28 Apr 2021 12:37:49 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-manager - 1:21.04.10

---------------
update-manager (1:21.04.10) hirsute; urgency=medium

  * tests/test_update_list.py: rewrite how APT::VersionedKernelPackages is set.
    (LP: #1924809)

 -- Brian Murray <email address hidden> Fri, 16 Apr 2021 15:32:48 -0700

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for update-manager has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Łukasz Zemczak (sil2100)]

...what about groovy? This feels verified there as well!

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-manager - 1:20.10.6

---------------
update-manager (1:20.10.6) groovy; urgency=medium

  * tests/test_update_list.py: rewrite how APT::VersionedKernelPackages is set.
    (LP: #1924809)
  * debian/tests/control: add apt to the list of Depends.

 -- Brian Murray <email address hidden> Fri, 16 Apr 2021 16:21:15 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-manager - 1:20.04.10.7

---------------
update-manager (1:20.04.10.7) focal; urgency=medium

  * tests/test_update_list.py: rewrite how APT::VersionedKernelPackages is set.
    (LP: #1924809)
  * debian/tests/control: add apt to the list of Depends.

 -- Brian Murray <email address hidden> Fri, 16 Apr 2021 16:25:21 -0700