The fix ran through regression testing with three tools.
No regressions observed between original/patched 5.4.0-21.
- 5.4.0-21-generic #25-Ubuntu SMP Sat Mar 28 13:10:28 UTC 2020
- 5.4.0-21-generic #25+aufs SMP Fri Apr 3 12:09:29 -03 2020
1) stress-ng (on host, and on kube-proxy's aufs mount)
Command: stress-ng --class filesystem --sequential 0 --timeout 5m
(takes about 3 hours to finish.)
The stress-ng logs were normalized for PID/process number
and unique messages then compared. There's no unexpected
new error messages. Also compared the dmesg output.
The runs on the kube-proxy's aufs mountpoint consisted
of finding which /var/lib/docker/aufs/mnt/ directory
is used by kube-proxy (which triggered the problem)
and running stress-ng over there.
2) xfstests-dev, patched to use aufs instead of overlayfs
The xfstests-dev patch is attached ("xfstests-aufs.patch").
It's not yet upstream -- working on v2 for upstream which
also covers fuse-overlayfs.
The set of test failures is identical for original/patched
kernels, seen as one single unique line across the 2 logs:
And the number of failures is (of course, identical.)
"Failed 357 of 648 tests."
Command: "./check -overlay -E /tmp/exclude-tests" with
10 tests excluded, which hang the kernel/blocks tasks.
(steps/details available in the patch message.)
Regression Testing:
---
The fix ran through regression testing with three tools.
No regressions observed between original/patched 5.4.0-21.
- 5.4.0-21-generic #25-Ubuntu SMP Sat Mar 28 13:10:28 UTC 2020
- 5.4.0-21-generic #25+aufs SMP Fri Apr 3 12:09:29 -03 2020
1) stress-ng (on host, and on kube-proxy's aufs mount)
Command: stress-ng --class filesystem --sequential 0 --timeout 5m
(takes about 3 hours to finish.)
The stress-ng logs were normalized for PID/process number
and unique messages then compared. There's no unexpected
new error messages. Also compared the dmesg output.
The runs on the kube-proxy's aufs mountpoint consisted docker/ aufs/mnt/ directory
of finding which /var/lib/
is used by kube-proxy (which triggered the problem)
and running stress-ng over there.
2) xfstests-dev, patched to use aufs instead of overlayfs
The xfstests-dev patch is attached ("xfstests- aufs.patch" ).
It's not yet upstream -- working on v2 for upstream which
also covers fuse-overlayfs.
The set of test failures is identical for original/patched
kernels, seen as one single unique line across the 2 logs:
$ grep -h '^Failures:' xfstests. {orig,patch} .log | sort -u | wc -l
1
And the number of failures is (of course, identical.)
"Failed 357 of 648 tests."
Command: "./check -overlay -E /tmp/exclude-tests" with
10 tests excluded, which hang the kernel/blocks tasks.
(steps/details available in the patch message.)
3) smoke test for aufs (from the kernel team)
The smoke test for aufs from the kernel team is located at: /kernel. ubuntu. com/git/ ubuntu/ autotest- client- tests.git/ tree/ubuntu_ aufs_smoke_ test/ubuntu_ aufs_smoke_ test.sh
https:/
Its output is identical on original/patched kernel.