linux-lts and linux-hwe autopkgtests fail due to version mismatch

Bug #1723223 reported by Dan Streetman on 2017-10-12
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Status tracked in Disco
Trusty
Medium
Unassigned
Xenial
Medium
Unassigned
Bionic
Medium
Unassigned
Cosmic
Medium
Unassigned
Disco
Medium
Unassigned

Bug Description

the autopkgtests for the kernel include 'ubuntu-regression-suite' which contains this test:

sver=`dpkg-parsechangelog -SVersion`
read x rver x </proc/version_signature

flavour=${rver#*-*-}
rver=${rver%-$flavour}

echo "Source Package Version: $sver"
echo "Running Kernel Version: $rver"

if [ "$sver" != "$rver" ]; then
    echo "ERROR: running version does not match source package" 1>&2
    exit 1
fi

However, when testing a -lts or -hwe kernel build, the running kernel will not match because the running kernel is the stock version while the building/testing kernel is the newer -lts/-hwe version.

This failure results in autopkgtest errors for all other packages that invoke linux autopkgtests - for example, initramfs-tools, systemd, ... This requires unnecessary work to investigate the autopkgtest errors every time, only to find it's a autopkgtest error.

Tags: sts Edit Tag help
Dan Streetman (ddstreet) wrote :

For example, here is a autopkgtest failure that was flagged by my update to initramfs-tools package:

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-trusty/trusty/ppc64el/l/linux-lts-xenial/20171012_183054_740b6@/log.gz

the failure is simply due to the ubuntu-regression-suite test:

autopkgtest [18:30:28]: test ubuntu-regression-suite: [-----------------------
Source Package Version: 4.4.0-97.120~14.04.1
Running Kernel Version: 3.13.0-133.182
ERROR: running version does not match source package

Changed in linux (Ubuntu):
assignee: nobody → Dan Streetman (ddstreet)
importance: Undecided → Low
status: New → In Progress
Dan Streetman (ddstreet) on 2017-10-12
Changed in linux (Ubuntu Trusty):
status: New → In Progress
Changed in linux (Ubuntu Xenial):
status: New → In Progress
Changed in linux (Ubuntu Trusty):
importance: Undecided → Low
Changed in linux (Ubuntu Xenial):
importance: Undecided → Low
assignee: nobody → Dan Streetman (ddstreet)
Changed in linux (Ubuntu Trusty):
assignee: nobody → Dan Streetman (ddstreet)
Changed in linux (Ubuntu):
status: In Progress → Invalid
Eric Desrochers (slashd) on 2017-10-16
description: updated
Dan Streetman (ddstreet) wrote :

the failing test was added by bug 1385330

Dan Streetman (ddstreet) wrote :

This unfortunately is not a simple fix.

To start with, the core problem is that the 'ubuntu-regression-suite' assumes that the currently installed kernel == the built kernel, but that is not guaranteed by the test, because it defines 'Depends:' values, but does not include the @ dependency (which autopkgtest resolves to all the binaries defined by this source package).

Now we get into deeper problems:

Unfortunately, we can't simply add @ to the test Depends: entries, because autopkgtest includes the dbgsym ddebs files but isn't able to actually find the ddeb file (likely because it ends with 'ddeb' instead of 'deb'). This is work-around-able by adding the line 'Package-type: ddeb' (or anything not 'deb') to the dbgsym entry in the control file.

That gets us past the missing dbgsym ddeb, but autopkgtest can't tell that the kernel is built in different 'flavours', and the kernel currently only builds the first 'flavour' during autopkgtest runs, assuming that testing all flavours isn't necessary. But, adding @ to Depends: adds the binary files from all flavours, not just the one that was built. So, autopkgtest is not able to find the (e.g.) -lowlatency flavoured files, because only (e.g.) -generic flavoured files have been built.

The rules file could be changed to build all flavours, even if only running autopkgtests, but that would be a significant increase in time and computing power, just to work around an annoying issue.

Instead of adding @ to the ubuntu-regression-suite, we could add the specific linux-image-VERSION and linux-image-extra-VERSION files that are built for this specific kernel version source package; but the test control file is currently static, at debian/tests/control. That would need to be changed to be auto-created similar to the debian/control file.

autopkgtest may also be fixable to do a better job of replacing @ with only the files that were actually built.

or, the 'ubuntu-regression-suite' test could simply be skipped if the package under test isn't 'linux' - assuming that its regression test suite only really needs to be applied to the normal 'linux' source package, not any of the lts/hwe/hwe-edge backports. That's certainly the easiest way to fix this.

Dan Streetman (ddstreet) on 2019-02-12
Changed in linux (Ubuntu Cosmic):
importance: Undecided → Medium
Changed in linux (Ubuntu Trusty):
importance: Low → Medium
Changed in linux (Ubuntu Disco):
status: Invalid → New
Changed in linux (Ubuntu Xenial):
status: In Progress → New
Changed in linux (Ubuntu Trusty):
status: In Progress → New
Changed in linux (Ubuntu Disco):
assignee: Dan Streetman (ddstreet) → nobody
Changed in linux (Ubuntu Xenial):
assignee: Dan Streetman (ddstreet) → nobody
Changed in linux (Ubuntu Trusty):
assignee: Dan Streetman (ddstreet) → nobody
Changed in linux (Ubuntu Disco):
importance: Low → Medium
Changed in linux (Ubuntu Bionic):
importance: Undecided → Medium
Changed in linux (Ubuntu Xenial):
importance: Low → Medium

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1723223

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Changed in linux (Ubuntu Bionic):
status: New → Incomplete
Changed in linux (Ubuntu Cosmic):
status: New → Incomplete
Changed in linux (Ubuntu Trusty):
status: New → Incomplete
Changed in linux (Ubuntu Xenial):
status: New → Incomplete
Eric Desrochers (slashd) on 2019-02-12
Changed in linux (Ubuntu Disco):
status: Incomplete → In Progress
assignee: nobody → Eric Desrochers (slashd)
tags: added: sts
Eric Desrochers (slashd) wrote :

https://people.debian.org/~mpitt/autopkgtest/README.package-tests.html

@builddeps@ will be replaced by the package's Build-Depends:, Build-Depends-Indep:, and build-essential. This is useful if you have many build dependencies which are only necessary for running the test suite and you don't want to replicate them in the test Depends:. However, please use this sparingly, as this can easily lead to missing binary package dependencies being overlooked if they get pulled in via build dependencies.

Eric Desrochers (slashd) wrote :

After a discussion with vorlon, his proposal is to blacklist the "special" kernels so they're ignored for autopkgtests.

Eric Desrochers (slashd) on 2019-03-17
Changed in linux (Ubuntu Disco):
assignee: Eric Desrochers (slashd) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers