du program crashes on /var/lib

[Paul Andre Panon]

We have an Ubuntu 18 installation using xfs file systems
user@hostname:~$ df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 1989640 0 1989640 0% /dev
tmpfs 404008 848 403160 1% /run
/dev/sda1 9754624 3083376 6671248 32% /
tmpfs 2020032 0 2020032 0% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 2020032 0 2020032 0% /sys/fs/cgroup
/dev/sda2 11194368 958336 10236032 9% /var
tmpfs 404004 0 404004 0% /run/user/1001
tmpfs 404004 0 404004 0% /run/user/601005952

du aborts when run in /var and /var/lib, but not in any of the subdirectories of /var/lib
I tried rebooting with linux ... init=/bin/bash to run xfs_repair on the /var/file system and it found no errors.
After the repair command, I mounted /var and ran du again with no errors.
However once I rebooted into the full O/S, du aborted again in /var and /var/lib.
The du and sudo du both abort, but non-privileged du also dumps core.

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: coreutils 8.28-1ubuntu1
ProcVersionSignature: Ubuntu 4.15.0-32.35-generic 4.15.18
Uname: Linux 4.15.0-32-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.2
Architecture: amd64
Date: Mon Aug 20 15:26:49 2018
InstallationDate: Installed on 2018-06-11 (69 days ago)
InstallationMedia: Ubuntu-Server 18.04 LTS "Bionic Beaver" - Release amd64 (20180426)
ProcEnviron:
 TERM=linux
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_CA.UTF-8
 SHELL=/bin/bash
SourcePackage: coreutils
UpgradeStatus: No upgrade log present (probably fresh install)

[C de-Avillez (hggdh2)]

Thank you for opening this bug and helping make Ubuntu better.

You refer to "crash" and "abort", but this bug does not have an apport-generated stacktrace, neither *any* details on how 'du' fails. As it is, there is not much we can do.

On a standard Ubuntu system, if 'du' crashes, it should generate a file under /var/crash/, and then apport should automatically kick in and proposed to report the crash. Did this not happen? If it *does* happen, then please open a new bug when requested by apport, and refer back in the to this one.

[Launchpad Janitor (janitor)]

[Expired for coreutils (Ubuntu) because there has been no activity for 60 days.]

[Bert JW Regeer (ns2-bregeer)]

Coming back to this years later because I am running into this issue as well, and wanted to provide some additional information:

At this time it seems to be crashing when it attempts to traverse from /var/lib into lxcfs:

openat(4, "lxcfs", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_NOFOLLOW|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
fcntl(3, F_GETFL) = 0x38800 (flags O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_NOFOLLOW|O_DIRECTORY)
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
newfstatat(4, "lxcfs", {st_mode=S_IFDIR|0755, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
fstatfs(4, {f_type=XFS_SB_MAGIC, f_bsize=4096, f_blocks=1160704, f_bfree=1014059, f_bavail=1014059, f_files=2326528, f_ffree=2321112, f_fsid={val=[64771, 0]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_RELATIME}) = 0
fcntl(3, F_DUPFD_CLOEXEC, 3) = 5
getdents(3, /* 4 entries */, 32768) = 104
getdents(3, /* 0 entries */, 32768) = 0
close(3) = 0
newfstatat(5, "proc", {st_mode=S_IFDIR|0555, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
openat(5, "proc", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_NOFOLLOW|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
fcntl(3, F_GETFL) = 0x38800 (flags O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_NOFOLLOW|O_DIRECTORY)
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
getpid() = 16911
gettid() = 16911
tgkill(16911, 16911, SIGABRT) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=16911, si_uid=0} ---
+++ killed by SIGABRT (core dumped) +++
Aborted (core dumped)

I've attached the file with the crash from /var/crash.

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=457348k,nr_inodes=114337,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=97968k,mode=755)
/dev/mapper/vg_root-lv_root on / type xfs (rw,relatime,attr2,inode64,noquota)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgr...

[Robert J (roberthurricane)]

Adding on to this to say we're experiencing this issue on all Ubuntu 18.04 systems and we're also running xfs for a filesystem. It seems to be related to /var/lib/lxcfs on all of those systems. When we exclude /var/lib/lxcfs from du, it completes without core dumping.

root@system:/var# du --exclude 'lib/lxcfs/*' -h -d 1 .
3.1G ./log
14M ./backups
3.0G ./cache
480K ./crash
444M ./lib
0 ./local
4.0K ./mail
0 ./opt
1.3M ./spool
4.0K ./tmp
12K ./www
6.5G .

root@system:/var# du -h --max-depth=1 .
873M ./log
14M ./backups
178M ./cache
312K ./crash
Aborted (core dumped)