openafs 1.6.20-2ubuntu2 ADT test failure with linux 4.12.0-6.7

[Seth Forshee]

Testing failed on:
    armhf: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-artful-canonical-kernel-team-unstable/artful/armhf/o/openafs/20170712_202748_ead20@/log.gz
    s390x: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-artful-canonical-kernel-team-unstable/artful/s390x/o/openafs/20170712_200145_ead20@/log.gz

[Seth Forshee (sforshee)]

From armhf logs:

Setting up openafs-fileserver (1.6.20-2ubuntu2) ...
Created symlink /etc/systemd/system/multi-user.target.wants/openafs-fileserver.service → /lib/systemd/system/openafs-fileserver.service.
Job for openafs-fileserver.service failed because the control process exited with error code.
See "systemctl status openafs-fileserver.service" and "journalctl -xe" for details.
invoke-rc.d: initscript openafs-fileserver, action "start" failed.
● openafs-fileserver.service - OpenAFS file and database server manager
   Loaded: loaded (/lib/systemd/system/openafs-fileserver.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2017-07-12 20:01:36 UTC; 34ms ago
  Process: 1460 ExecStop=/usr/bin/bos shutdown localhost -wait -localauth (code=exited, status=1/FAILURE)
  Process: 1458 ExecStart=/usr/sbin/bosserver $DAEMON_ARGS (code=exited, status=0/SUCCESS)

Jul 12 20:01:36 autopkgtest-lxd-vunnbu systemd[1]: openafs-fileserver.service…ed
Jul 12 20:01:36 autopkgtest-lxd-vunnbu systemd[1]: Starting OpenAFS file and …..
Jul 12 20:01:36 autopkgtest-lxd-vunnbu bos[1460]: bos: could not find entry (…y)
Jul 12 20:01:36 autopkgtest-lxd-vunnbu bos[1460]: [1B blob data]
Jul 12 20:01:36 autopkgtest-lxd-vunnbu systemd[1]: openafs-fileserver.service…=1
Jul 12 20:01:36 autopkgtest-lxd-vunnbu systemd[1]: Failed to start OpenAFS fi…r.
Jul 12 20:01:36 autopkgtest-lxd-vunnbu systemd[1]: openafs-fileserver.service…e.
Jul 12 20:01:36 autopkgtest-lxd-vunnbu systemd[1]: openafs-fileserver.service…'.
Hint: Some lines were ellipsized, use -l to show in full.
dpkg: error processing package openafs-fileserver (--configure):
 subprocess installed post-installation script returned error exit status 1

[Anders Kaseorg (andersk)]

From the same logs:

autopkgtest [19:57:44]: testbed running kernel: Linux 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 14:58:00 UTC 2017

Setting up openafs-modules-dkms (1.6.20-2ubuntu2) ...
Loading new openafs-1.6.20 DKMS files...
It is likely that 4.4.0-64-generic belongs to a chroot's host
Building for 4.10.0-26-generic and 4.12.0-6-generic

modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-64-generic/modules.dep.bin'
modprobe: FATAL: Module openafs not found in directory /lib/modules/4.4.0-64-generic
Failed to load openafs.ko. Does it need to be built?
grep: /lib/modules/4.4.0-64-generic/modules.dep: No such file or directory

This test cannot possibly work in a chroot. What is responsible for running these tests? (It’s not the openafs package, which has no debian/tests.)

[Seth Forshee (sforshee)]

There's some kind of special rules for dkms packages in ADT which makes them get tested so that we know that the module builds with the kernel being used. I don't think the lines you pointed out actually result in the test failing however.

The test is passing on armhf now, so we can probably just close this bug. We have another failure from ADT that you might be interested in though, bug 1711835.

[Anders Kaseorg (andersk)]

Where do you get the idea that the test is passing on armhf now? It’s still marked as a “regression” on armhf and s390x at http://people.canonical.com/~ubuntu-archive/proposed-migration/artful/update_excuses.html#openafs, and this has been holding back the sync of 1.6.21-1 (bug 1705850) for weeks, as well as the update to 1.6.21-1ubuntu1.

[Anders Kaseorg (andersk)]

It seems you are right that the lines I quoted aren’t responsible for the failure. I had forgotten about bug 1625091, which shows the same openafs-fileserver errors you quoted and no openafs-modules-dkms errors. (Perhaps this should be marked as a duplicate.)

> Hint: Some lines were ellipsized, use -l to show in full.

Can autopkgtest be made to use -l here? I still don’t see how to debug this without knowing what the truncated log messages actually say.

[Seth Forshee (sforshee)]

I've generally been looking at the results triggered by kernel packages, for example the results with the current artful-proposed kernel which did pass.

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-artful/artful/armhf/o/openafs/20170818_155126_2985b@/log.gz

Not that things seem to be going perfectly in that log either.

I'm not all that familiar with the ADT infrastructure, but I believe for armhf and s390x the tests use containers for confinement instead of kvm, which perhaps helps explain the chroot messages (I note that it's complaining about a 4.4 kernel when that's not a version that has ever been used in artful).

If it is the case that openafs is always going to fail with the ADT container confinement we may be able to get a member of the release team to hint these as expected failures so that they will not block package promotion.