seccomp tests fail in tumbleweed on 2020-03-05 and 20.04

[Zygmunt Krynicki]

A recent build of snapd in openSUSE tumbleweed uncovered test suite failures.
Those may be related to a kernel or seccomp update and need investigation.

[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:799: snapSeccompSuite.TestCompatArchWorks
[ 354s]
[ 354s] main_test.go:832:
[ 354s] s.runBpf(c, t.seccompWhitelist, t.bpfInput, t.expected)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "read" (failed to run "exit status 127")
[ 354s]
[ 354s]
[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:367: snapSeccompSuite.TestCompile
[ 354s]
[ 354s] main_test.go:476:
[ 354s] s.runBpf(c, t.seccompWhitelist, t.bpfInput, t.expected)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "read" (failed to run "exit status 127")
[ 354s]
[ 354s]
[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:485: snapSeccompSuite.TestCompileSocket
[ 354s]
[ 354s] main_test.go:504:
[ 354s] s.runBpf(c, t.seccompWhitelist, t.bpfInput, t.expected)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "socket AF_UNIX" (failed to run "exit status 127")
[ 354s]
[ 354s]
[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:684: snapSeccompSuite.TestRestrictionsWorkingArgsClone
[ 354s]
[ 354s] main_test.go:705:
[ 354s] s.runBpf(c, t.seccompWhitelist, t.bpfInput, t.expected)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "setns - CLONE_NEWIPC" (failed to run "exit status 127")
[ 354s]
[ 354s]
[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:710: snapSeccompSuite.TestRestrictionsWorkingArgsMknod
[ 354s]
[ 354s] main_test.go:729:
[ 354s] s.runBpf(c, t.seccompWhitelist, t.bpfInput, t.expected)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "mknod - S_IFREG" (failed to run "exit status 127")
[ 354s]
[ 354s]
[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:656: snapSeccompSuite.TestRestrictionsWorkingArgsPrctl
[ 354s]
[ 354s] main_test.go:661:
[ 354s] s.runBpf(c, seccompWhitelist, bpfInputGood, Allow)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "prctl PR_CAP_AMBIENT" (failed to run "exit status 127")
[ 354s]
[ 354s]
[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:734: snapSeccompSuite.TestRestrictionsWorkingArgsPrio
[ 354s]
[ 354s] main_test.go:749:
[ 354s] s.runBpf(c, t.seccompWhitelist, t.bpfInput, t.expected)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "setpriority PRIO_PROCESS" (failed to run "exit status 127")
[ 354s]
[ 354s]
[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:642: snapSeccompSuite.TestRestrictionsWorkingArgsQuotactl
[ 354s]
[ 354s] main_test.go:647:
[ 354s] s.runBpf(c, seccompWhitelist, bpfInputGood, Allow)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "quotactl Q_QUOTAON" (failed to run "exit status 127")
[ 354s]
[ 354s]
[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:607: snapSeccompSuite.TestRestrictionsWorkingArgsSocket
[ 354s]
[ 354s] main_test.go:617:
[ 354s] s.runBpf(c, seccompWhitelist, bpfInputGood, Allow)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "socket AF_UNIX" (failed to run "exit status 127")
[ 354s]
[ 354s]
[ 354s] ----------------------------------------------------------------------
[ 354s] FAIL: main_test.go:754: snapSeccompSuite.TestRestrictionsWorkingArgsTermios
[ 354s]
[ 354s] main_test.go:765:
[ 354s] s.runBpf(c, t.seccompWhitelist, t.bpfInput, t.expected)
[ 354s] main_test.go:334:
[ 354s] c.Fatalf("unexpected error for %q (failed to run %q)", seccompWhitelist, err)
[ 354s] ... Error: unexpected error for "ioctl - TIOCSTI" (failed to run "exit status 127")

[Samuele Pedroni (pedronis)]

this is now happening on 20.04 as well

[Zygmunt Krynicki (zyga)]

This was fixed by https://github.com/snapcore/snapd/pull/8266 but we didn't uncover the root cause.

[Zygmunt Krynicki (zyga)]

Snapd 2.44 has been released, marking as the bug as such.