ubuntu_ecryptfs tests failed on Xenial s390x instances

[Po-Hsu Lin]

In the ubuntu_ecryptfs test suite, it constantly fail with 28 test cases on all s390x instances (zVM / zKVM / Ubuntu on LPAR) with Xenial kernel.

 miscdev-bad-count.sh.xfs 3 sec Failed
  extend-file-random.sh.xfs 5 sec Failed
  trunc-file.sh.xfs 3 sec Failed
  directory-concurrent.sh.xfs 2 sec Failed
  file-concurrent.sh.xfs 5 sec Failed
  lp-994247.sh.xfs 3 sec Failed
  llseek.sh.xfs 4 sec Failed
  lp-469664.sh.xfs 5 sec Failed
  lp-524919.sh.xfs 5 sec Failed
  lp-509180.sh.xfs 6 sec Failed
  lp-613873.sh.xfs 11 sec Failed
  lp-745836.sh.xfs 10 sec Failed
  lp-870326.sh.xfs 6 sec Failed
  lp-885744.sh.xfs 9 sec Failed
  lp-926292.sh.xfs 9 sec Failed
  inotify.sh.xfs 10 sec Failed
  mmap-bmap.sh.xfs 9 sec Failed
  mmap-close.sh.xfs 10 sec Failed
  mmap-dir.sh.xfs 9 sec Failed
  read-dir.sh.xfs 6 sec Failed
  setattr-flush-dirty.sh.xfs 5 sec Failed
  inode-race-stat.sh.xfs 5 sec Failed
  lp-911507.sh.xfs 5 sec Failed
  lp-561129.sh.xfs 5 sec Failed
  mknod.sh.xfs 7 sec Failed
  link.sh.xfs 3 sec Failed
  xattr.sh.xfs 5 sec Failed
  namelen.sh.xfs 4 sec Failed

Tested with the upstream repo from lp:ecryptfs (revno 894)
There is not much error message to see, these test cases just exit with 1

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.4.0-104-generic 4.4.0-104.127
ProcVersionSignature: Ubuntu 4.4.0-104.127-generic 4.4.98
Uname: Linux 4.4.0-104-generic s390x
NonfreeKernelModules: zfs zunicode zcommon znvpair zavl
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access '/dev/snd/': No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.1-0ubuntu2.14
Architecture: s390x
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
CRDA: Error: command ['iw', 'reg', 'get'] failed with exit code 1: nl80211 not found.
CurrentDmesg:
 [104616.551505] EXT4-fs (loop0): mounting ext3 file system using the ext4 subsystem
 [104616.574291] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,acl,commit=600,barrier=1,data=ordered
Date: Thu Dec 14 05:33:55 2017
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lspci:

Lsusb: Error: command ['lsusb'] failed with exit code 1:
PciMultimedia:

ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=C
 SHELL=/bin/bash
ProcFB: Error: [Errno 2] No such file or directory: '/proc/fb'
ProcKernelCmdLine: root=UUID=44b0b919-a1a4-4849-9425-e71d4ac87d85 crashkernel=196M BOOT_IMAGE=0
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-104-generic N/A
 linux-backports-modules-4.4.0-104-generic N/A
 linux-firmware 1.157.14
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1738178

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Colin Ian King (colin-king)]

I'm having difficulty reproducing this problem with my testing.

4.4.0-109-generic #132-Ubuntu SMP Tue Jan 9 19:58:22 UTC 2018 s390x s390x s390x GNU/Linux

See attached log. Seems OK to me. Can you provide me access to the hosts that are seeing this issue and I'll try and debug this further.

[Colin Ian King (colin-king)]

This does not look like an ecryptfs issue, it's mkfs.xfs breaking with an -EINVAL on a file based file system:

dd if=/dev/zero of=foo bs=1M count=1024
sudo mkfs.xfs -f foo
meta-data=foo isize=512 agcount=4, agsize=64000 blks
         = sectsz=512 attr=2, projid32bit=1
         = crc=1 finobt=1, sparse=0
data = bsize=4096 blocks=256000, imaxpct=25
         = sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=855, version=2
         = sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
existing superblock read failed: Invalid argument
mkfs.xfs: pwrite64 failed: Unknown error -22

[Colin Ian King (colin-king)]

This fails with kernels as old as 4.4.0-101-generic and earlier too, so I don't think this is a kernel regression. I believe it's an issue with xfsprogs. For example, xfsprogs-4.14.0 can create an xfs file system on a file based system where as 4.3.0+nmu1ubuntu1.1 cannot:

buntu@s2lp3:~/xfsprogs-4.14.0$ ./mkfs/mkfs.xfs ~/foo
meta-data=/home/ubuntu/foo isize=512 agcount=2, agsize=4096 blks
         = sectsz=512 attr=2, projid32bit=1
         = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0
data = bsize=4096 blocks=8192, imaxpct=25
         = sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=855, version=2
         = sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0

[Colin Ian King (colin-king)]

Although commit 06ac92fdf8571ea10a3df64c7456eace03cf14c3 fixes this (see https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/commit/?id=06ac92fdf8571ea10a3df64c7456eace03cf14c3 ), a way forward is to specify -d options explicitly such as:

mkfs.xfs -d file=1,name=/home/ubuntu/foo,size=1g

I'll modify the adt tests to force this.

[Colin Ian King (colin-king)]

Fix sent to autotest tools, "[PATCH] UBUNTU: SAUCE: ubuntu_ecryptfs: ensure mkfs.xfs works for all releases (LP:#1738178)". Sam, do you mind testing this for me?

[Colin Ian King (colin-king)]

Fix applied: http://kernel.ubuntu.com/git/ubuntu/autotest-client-tests.git/commit/?id=232247ab488e5d521a9418bb5576cb14572c5078

[Po-Hsu Lin (cypressyew)]

Verified on all 3 Xenial s390x systems.
They can pass this test now. Thanks!

[Po-Hsu Lin (cypressyew)]

Verified with s390x on 4.13 Artful as well. No regression was found.

[Sean Feole (sfeole)]

Hi Sam, looks like this bug was somewhat fixed , still an issue?

very old bug, with no action on it, marking bug as incomplete and will
close in 5 days if no update.

[Po-Hsu Lin (cypressyew)]

Yes this can be closed, thanks.