systemd=229-4ubuntu21.8 use of fchownat failes on some systems (openvz)
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| systemd (Ubuntu) |
Undecided
|
Chris Coulson |
Bug Description
The following description is taken from:
https:/
Hello everyone,
I'm running 16.04 LTS on a virtual server which, I think, uses OpenVz. After a recent reboot I found most of my services to be in a failed state. The reason for that, I guess, are these log entries:
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
Nov 17 04:47:42 h2118376 systemd-
To verify I tried this:
/usr/lib/
Reading config file "elasticsearch.
Running create action for entry d /var/run/
Found existing directory "/var/run/
"/run/elasticse
chown "/run/elasticse
fchownat() of /run/elasticsearch failed: Invalid argument
I can manually chown the directories, e.g. "chown elasticsearch:
At this point I don't know what to do.
I'm also confused about the version I have installed, which I thought is systemd-229. Howver, I looked at https:/
Tag v237 (and earlier, including 229):
/.../
}
/.../
Tag v238
/.../
if (fchownat(fd,
return log_error_
/.../
Any help fixing this problem would be highly appreciated.
Many thanks,
Rafael
=== Notes ===
fchownat() was added to Linux in kernel 2.6.16;
library support was added to glibc in version 2.4.
checkinf if it is blocked/
glibc in bionic requires minimum linux 3.2.
CVE References
temporary solution is:
wget http://
dpkg -i libpam-
apt-mark hold systemd libsystemd0 libpam-systemd
Miha Ravšelj (mravselj) wrote : | #3 |
I have the same issue.
My vps is 16.04.5 LTS(openvz)
This bug however has quite an impact. After reboot sshd server fails to start since /var/run/sshd folder is not created since systemd-
I have applied the temporary solution above and can confirm it works ok.
Peter Enns (nelway) wrote : | #4 |
I had the same problem with ssh, after reboot I was not able to SSH to the server.
My solution to the SSH problem was the following (through an emergency console session from the VPS provider's control panel):
sudo nano /etc/ssh/
Find:
UsePrivilegeSep
and change to:
UsePrivilegeSep
sudo service ssh restart
This worked for me, but I am not sure about the security implications of this change.
SV, thank you for the temporary solution, I tried it on my test server and it worked fine.
Stefan Andres (s-andres) wrote : | #5 |
In it's current version (systemd-
Files in /var/run/ will get `Too many levels of symbolic links` files in /run/ will still get `fchownat() of /run/... failed: Invalid argument`
root@sandres-
Reading config file "zabbix-
Running create action for entry d /var/run/zabbix
Failed to validate path /var/run/zabbix: Too many levels of symbolic links
root@sandres-
d /var/run/zabbix 0755 zabbix zabbix - -
root@sandres-
Reading config file "zabbix-
Running create action for entry d /run/zabbix
Found existing directory "/run/zabbix".
"/run/zabbix" has right mode 40755
chown "/run/zabbix" to 108.115
fchownat() of /run/zabbix failed: Invalid argument
root@sandres-
d /run/zabbix 0755 zabbix zabbix - -
root@sandres-
Raimundas (iv123) wrote : | #6 |
I have the same problem with my Ubuntu 16 VPS servers. I had systemd version 229-4ubuntu21.5 on my templates and when it is updated to version 229-4ubuntu21.9 this erroneous behavior occurs. On other client VPS's i have seen that they updated systemd from version 229-4ubuntu21.8 to 229-4ubuntu21.9. Thus it seems like a problem with the November 19 update for systemd - https:/
Here is an alternative "fix" you can use until this problem is properly resolved (just change the "21.5" version, which was working for you before the update):
cd /home/
wget launchpadlibrar
wget launchpadlibrar
wget launchpadlibrar
dpkg -i libpam-
apt-mark hold systemd libsystemd0 libpam-systemd
reboot
Franz Seidl (franz-s) wrote : | #7 |
I've the problem with my VPS at Strato, they use Virtuozzo.
Dimitri John Ledkov (xnox) wrote : | #8 |
Could you please post output of:
uname -a
information type: | Public → Public Security |
tags: | added: regression-update |
description: | updated |
description: | updated |
Seth Arnold (seth-arnold) wrote : | #9 |
I'm also curious what filesystems are showing this issue. If you're affected can you please run this command and include the results here?
mount | grep run
Thanks
Peter Enns (nelway) wrote : | #10 |
Output of uname -a:
Linux newpnc 2.6.32-042stab132.1 #1 SMP Wed Jul 11 13:51:30 MSK 2018 x86_64 x86_64 x86_64 GNU/Linux
Output of mount | grep run:
tmpfs on /run type tmpfs (rw,nosuid,
tmpfs on /run/lock type tmpfs (rw,nosuid,
none on /run/shm type tmpfs (rw,relatime)
Steve Langasek (vorlon) wrote : | #11 |
glibc in Ubuntu 16.04 does have MIN_KERNEL_
So it seems reasonablbe to expect this not to regress as part of a security update of systemd on Ubuntu 16.04.
Peter Enns (nelway) wrote : | #12 |
Ah ok, thank you for the information.
Since it is an OpenVZ container, I assume my only option is to apply the temporary solution above, then start to look for a VPS provider that supports a more current kernel version (or one that would let me upgrade the kernel).
Thanks again.
Peter
Jens Zahner (jens-it-zahner) wrote : | #13 |
I also ran into the problem but solved it by downgrading systemd:
> apt install systemd=
> apt-mark hold systemd
Dimitri John Ledkov (xnox) wrote : | #14 |
@vorlon
fchownat() was added to Linux in kernel 2.6.16, which is prior to 2.6.32 and is a kernel version one of the affected users is reporting. Thus it seems as if, the fchownat() is somehow otherwise broken.
Changed in systemd (Ubuntu): | |
assignee: | nobody → Chris Coulson (chrisccoulson) |
Chris Coulson (chrisccoulson) wrote : | #15 |
We're just going to publish a revert of the CVE-2018-6954 fixes for 16.04 before investigating this further. As far as I can tell, this shouldn't be an issue in bionic where MIN_KERNEL_
Chris Coulson (chrisccoulson) wrote : | #16 |
The issue is that O_PATH doesn't work from these containers:
2025 11:00:08 openat(4, "run", O_RDONLY|
Apparently, O_PATH was added in 2.6.39, so this makes sense now.
Launchpad Janitor (janitor) wrote : | #17 |
This bug was fixed in the package systemd - 229-4ubuntu21.10
---------------
systemd (229-4ubuntu21.10) xenial-security; urgency=medium
[ Chris Coulson ]
* Revert the fixes for CVE-2018-6954 for causing a regression when running
in a container on old kernels (LP: #1804847)
- update debian/
[ Balint Reczey ]
* Fix LP: #1803391 - Don't always trigger systemctl stop of udev service
and sockets
- update debian/
-- Chris Coulson <email address hidden> Tue, 27 Nov 2018 11:10:48 +0000
Changed in systemd (Ubuntu): | |
status: | Confirmed → Fix Released |
Vasily Averin (vvs.at.openvz.org) wrote : | #18 |
Dear All,
my name is Vasily Averin, I'm maintainer of RHEL6-based OpenVz kernel.
We have backported required patches and going to release updated kernel with support for openat(
Fixed kernel 2.6.32-042stab134.7 is under testing now and we're going to publish it in few days.
Thank you,
Vasily Averin
Dimitri John Ledkov (xnox) wrote : | #19 |
@vvs Spasibo!
Vasily Averin (vvs.at.openvz.org) wrote : | #20 |
Updated OpenVz6 kernel was released:
https:/
We are very grateful for Ubuntu team for reverting of patches specially for OpenVz.
For affected hosters: OpenVz6 is great but it is really old,
and similar incidents can happen again and again.
Please think about switch to RHEL7-based OpenVz7.
Thank you,
Vasily Averin
Rene Meier (meier.rene) wrote : | #21 |
Could someone please have a look at https:/
Carlos Garcia (lwrcase) wrote : | #22 |
Just updated 229-4ubuntu21.15 today and this issue is back.
Seth Arnold (seth-arnold) wrote : | #23 |
Hello Carlos, the OpenVZ team was kind enough to backport the necessary kernel feature in November last year. After a month and a half we decided that enough time had elapsed for OpenVZ-based service providers to install new kernels. If your provider has not yet rebooted into a new OpenVZ kernel I suggest you ask them to do so.
We can't reasonably hold off providing this security update to our 16.04 LTS users any longer.
Thanks
Andreas Kar (thexmanxyz) wrote : | #24 |
Hello Seth as I'm also facing a similar issue which is also related to this bug I'm now posting to this bug report as it has the highest heat concerning issues with systemd and systemd-tmpfiles (service do not start correctly). I don't think this is exclusively related to OpenVZ because I'm also affected I don't use OpenVZ.
I have collected a few reports which also relate to the systemd changes which result in services not starting on boot up:
https:/
https:/
https:/
I'm on Armbian on an OrangePI One and I'm also affected by services that won't start anymore. The same was the case with systemd-
Distribution / Kernel
Linux xxx 3.4.113-sun8i #2 SMP PREEMPT Sat Jan 12 15:54:26 CET 2019 armv7l armv7l armv7l GNU/Linux
Distributor ID: Ubuntu
Description: Ubuntu 16.04.5 LTS
Release: 16.04
Codename: xenial
Output of journalctl -b 0 -u systemd-
Jän 14 11:01:51 xxx systemd[1]: Starting Create Volatile Files and Directories...
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd-
Jän 14 11:01:51 xxx systemd[1]: systemd-
Jän 14 11:01:51 xxx systemd[1]: Failed to start Create Volatile Files and Directories.
Jän 14 11:01:51 xxx systemd[1]: systemd-
Jän 14 11:01:51 xxx systemd[1]: systemd-
Affected services:
# dnsmasq.service loaded failed failed dnsmasq - A lightweight DHCP and caching DNS server
# lighttpd.service loaded failed failed Lighttpd Daemon
# <email address hidden> loaded failed failed OpenVPN connection to...
Andreas Kar (thexmanxyz) wrote : | #25 |
Please forgive me if I don't get overall picture correctly because I'm not a professional in kernel development. Moreover I have no insights what is going on with the systemd changes and who is actually affected and why I'm affected. In the Armbian forums is stated that the problem originates in Ubuntu so I decided to share my information here.
I just see right now that I'm limited to either upgrade the Ubuntu revision (reinstall my machine) or hold systemd updates on a revision not facing the issue. I would really appreciate more information if this will ever get fixed again or what options we have in the long run to surpass the issue because I doesn't sound that the problem will ever be fixed again. Sorry again for my stupid question but there might be more people affected with the same problem who probably want to know a resolution.
Andreas Kar (thexmanxyz) wrote : | #26 |
Upgrading to Linux pan 4.19.13-sunxi #5.70 SMP Sat Jan 12 15:43:21 CET 2019 armv7l armv7l armv7l GNU/Linux solved the issue for me.
Status changed to 'Confirmed' because the bug affects multiple users.