[LTCTest][libvpd] Process '/bin/touch /run/run.vpdupdate' failed with exit code 127

[bugproxy]

---Problem Description---
Currently syslog is getting flooded with below log messages ..
Sep 25 03:16:41 ubuntu1710 systemd-udevd[2654]: Process '/bin/touch /run/run.vpdd
update' failed with exit code 127.

This is on UBuntu 17.10 latest build..

---uname output---
Linux ubuntu1710 4.12.0-11-generic #12-Ubuntu SMP Fri Aug 11 12:23:06 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = ZZ-L

---Steps to Reproduce---
 I have seen these messages logged more often when we do any hotplug operations.. ex. cpu hotplug..

Ok. .this is the continuation of LTC #144627 / Launchpad #1682774.

As suggested in that bug, we did change udev script to create temporary file under /run and that patch is available in ubuntu 17.10.

Looks like this is udev script issue. If I modify systemd-udevd like below it works fine.

I've limited udev knowledge. I don't know if you have any other better solution to this issue.

root@ltc-boston123:/lib/systemd/system# diff -Naurp systemd-udevd.service.org systemd-udevd.service
--- systemd-udevd.service.org 2017-09-26 04:37:47.090057318 -0500
+++ systemd-udevd.service 2017-09-26 04:37:55.381739372 -0500
@@ -25,7 +25,7 @@ KillMode=mixed
 WatchdogSec=3min
 TasksMax=infinity
 MountFlags=slave
-MemoryDenyWriteExecute=yes
+#MemoryDenyWriteExecute=yes
 RestrictRealtime=yes
 RestrictAddressFamilies=AF_UNIX AF_NETLINK AF_INET AF_INET6
 SystemCallArchitectures=native

-Vasant

[Steve Langasek (vorlon)]

'MemoryDenyWriteExecute=yes' is defined in systemd.exec(5) as:

If set, attempts to create memory mappings that are writable and executable at the same time, or to change existing memory mappings to become executable, or mapping shared memory segments as executable are prohibited. Specifically, a system call filter is added that rejects mmap(2) system calls with both PROT_EXEC and PROT_WRITE set, mprotect(2) system calls with PROT_EXEC set and shmat(2) system calls with SHM_EXEC set.

It is surprising that /bin/touch should trip this check. This will require investigation.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-09-28 02:13 EDT-------
(In reply to comment #5)
> 'MemoryDenyWriteExecute=yes' is defined in systemd.exec(5) as:
>
> If set, attempts to create memory mappings that are writable and executable
> at the same time, or to change existing memory mappings to become
> executable, or mapping shared memory segments as executable are prohibited.
> Specifically, a system call filter is added that rejects mmap(2) system
> calls with both PROT_EXEC and PROT_WRITE set, mprotect(2) system calls with
> PROT_EXEC set and shmat(2) system calls with SHM_EXEC set.
>
> It is surprising that /bin/touch should trip this check. This will require
> investigation.

Yeah. even I thought its touch issue. I did look into touch source code as well. It uses below flag to create file.

O_WRONLY | O_CREAT | O_NONBLOCK | O_NOCTTY, MODE_RW_UGO);

-Vasant

[Steve Langasek (vorlon)]

On Thu, Sep 28, 2017 at 06:20:03AM -0000, bugproxy wrote:
> Yeah. even I thought its touch issue. I did look into touch source code
> as well. It uses below flag to create file.

> O_WRONLY | O_CREAT | O_NONBLOCK | O_NOCTTY, MODE_RW_UGO);

This is not about file creation flags, it's about memory maps. This is
likely to be either a toolchain regression, or a bug in glibc, since touch
itself should not be calling mmap() for anything that it does.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-09-29 01:44 EDT-------
(In reply to comment #7)
> On Thu, Sep 28, 2017 at 06:20:03AM -0000, bugproxy wrote:
> > Yeah. even I thought its touch issue. I did look into touch source code
> > as well. It uses below flag to create file.
>
> > O_WRONLY | O_CREAT | O_NONBLOCK | O_NOCTTY, MODE_RW_UGO);
>
> This is not about file creation flags, it's about memory maps. This is
> likely to be either a toolchain regression, or a bug in glibc, since touch
> itself should not be calling mmap() for anything that it does.

Looks like this is new flag introduced recently in systemd-udevd.service script.

-Vasant

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-10-10 14:24 EDT-------
(In reply to comment #8)
> (In reply to comment #7)
> > On Thu, Sep 28, 2017 at 06:20:03AM -0000, bugproxy wrote:
> > > Yeah. even I thought its touch issue. I did look into touch source code
> > > as well. It uses below flag to create file.
> >
> > > O_WRONLY | O_CREAT | O_NONBLOCK | O_NOCTTY, MODE_RW_UGO);
> >
> > This is not about file creation flags, it's about memory maps. This is
> > likely to be either a toolchain regression, or a bug in glibc, since touch
> > itself should not be calling mmap() for anything that it does.
>
> Looks like this is new flag introduced recently in systemd-udevd.service
> script.
>

How do we proceed with this bug? any update?

-Vasant

[Michael Hudson-Doyle (mwhudson)]

I tried and failed to reproduce this in an artful vm:

ubuntu@mwhudson-artful:~$ cat /etc/udev/rules.d/my.rules
KERNELS=="*", ACTION=="*", DEVPATH=="/devices/*", RUN+="/bin/touch /run/my-file"
ubuntu@mwhudson-artful:~$ sudo rm -f /run/my-file
ubuntu@mwhudson-artful:~$ sudo udevadm trigger /sys/devices/pci0000:00/0000:00:03.0/virtio2/block/vdb/vdb1
ubuntu@mwhudson-artful:~$ ls -l /run/my-file
-rw-r--r-- 1 root root 0 Nov 8 23:53 /run/my-file

(and there are no interesting messages in syslog when I do this).

Is the bug still affecting you? It's possible it got fixed in systemd or somewhere else in the meantime.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-11-24 07:23 EDT-------
I tested again on 17.10 .. but this time I am not seeing mentioned errors in syslog..
uname -a
Linux lep8b 4.13.0-17-generic #20-Ubuntu SMP Mon Nov 6 10:03:08 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

[Andrew Cloke (andrew-cloke)]

Since this issue cannot be reproduced with 4.13, marking "invalid".