Regression in overlayfs between 4.4.0-157 and 4.4.0-159

[Logan V]

My xenial gate jobs began failing after the nodes began running 4.4.0-159.

The gist of this bug is that a file sitting in /posix.ini for example, copied there by ucf from /usr/share/php7.2-common/common/posix.ini, cannot be read by non-root users despite its permission bits being 0644. chmodding 0644 on top of the file fixes this, also rebooting the lxc container solves it as well.

After downgrading the same test node to 4.4.0-157 and re-running the same tests, the broken behavior does not occur.

Please see the attachments for the exact commands used to reproduce this.
---
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access '/dev/snd/': No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.20.1-0ubuntu2.19
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
CRDA: N/A
DistroRelease: Ubuntu 16.04
IwConfig: Error: [Errno 2] No such file or directory
Lspci: Error: [Errno 2] No such file or directory
Lsusb: Error: [Errno 2] No such file or directory
MachineType: OpenStack Foundation OpenStack Nova
Package: linux (not installed)
PciMultimedia:

ProcEnviron:
 TERM=screen
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-159-generic root=LABEL=cloudimg-rootfs ro console=tty0 console=ttyS0,115200 no_timer_check nofb nomodeset vga=normal
ProcVersionSignature: Ubuntu 4.4.0-159.187-generic 4.4.185
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-159-generic N/A
 linux-backports-modules-4.4.0-159-generic N/A
 linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory
Tags: xenial xenial
UdevDb: Error: [Errno 2] No such file or directory
Uname: Linux 4.4.0-159-generic x86_64
UnreportableReason: The report belongs to a package that is not installed.
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

WifiSyslog:

_MarkForUpload: False
dmi.bios.date: 04/01/2014
dmi.bios.vendor: SeaBIOS
dmi.bios.version: 1.10.2-1ubuntu1~cloud0
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-i440fx-bionic
dmi.modalias: dmi:bvnSeaBIOS:bvr1.10.2-1ubuntu1~cloud0:bd04/01/2014:svnOpenStackFoundation:pnOpenStackNova:pvr18.2.0:cvnQEMU:ct1:cvrpc-i440fx-bionic:
dmi.product.name: OpenStack Nova
dmi.product.version: 18.2.0
dmi.sys.vendor: OpenStack Foundation

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1840328

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Logan V (loganv)]

apport information

[Logan V (loganv)]

apport information

[Logan V (loganv)]

apport information

[Logan V (loganv)]

apport information

[Logan V (loganv)]

apport information

[Logan V (loganv)]

apport information

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1840328

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Logan V (loganv)]

I also confirmed that using a 'dir' backed (no overlay) LXC container does not exhibit this broken behavior.

[Duncan Overbruck (duncaen)]

I can confirm this bug with overlayfs on top of tmpfs, I think it happens when a program does fsetxattr on a file.

I was able to reproduce it with sed which uses setxattr when it creates a new file, the shell does not do this when creating a file and does not trigger the bug.

`touch /tmp/foo; sed -i '' /tmp/foo; cat /tmp/foo` results in EACCES from open(2) in cat(1).

sed:
fsetxattr(4, "system.posix_acl_access", "\2\0\0\0\1\0\6\0\377\377\377\377\4\0\4\0\377\377\377\377 \0\4\0\377\377\377\377", 28, 0) = 0

cat:
open("/tmp/foo", O_RDONLY) = -1 EACCES (Permission denied)

[Kai-Heng Feng (kaihengfeng)]

Is the issue resolved in latest 16.04 kernel?