Running the same thing on zesty to see if the problem is present there too. We get something a bit different but the result ends up being the same, all the test runners crash. ``` buildd07 login: [ 976.607283] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 988.645772] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1004.605673] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1009.642113] INFO: rcu_sched self-detected stall on CPU [ 1009.645498] 3-...: (13599 ticks this GP) idle=769/140000000000001/0 softirq=32564/32569 fqs=6049 [ 1009.649690] INFO: rcu_sched detected stalls on CPUs/tasks: [ 1009.649697] 3-...: (13599 ticks this GP) idle=769/140000000000001/0 softirq=32564/32569 fqs=6049 [ 1009.649699] 11-...: (1 GPs behind) idle=8cd/140000000000000/0 softirq=36685/36686 fqs=6049 [ 1009.649700] (detected by 9, t=15002 jiffies, g=20785, c=20784, q=16519) [ 1009.663598] (t=15005 jiffies g=20785 c=20784 q=16519) [ 1016.645667] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1036.606795] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1044.645665] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1064.605727] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1072.645669] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1092.605690] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1100.645669] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 23s! [lxd:22980] [ 1120.605669] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1128.645698] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 23s! [lxd:22980] [ 1148.605669] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1156.645721] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 23s! [lxd:22980] [ 1176.605670] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1184.645668] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 23s! [lxd:22980] [ 1189.665664] INFO: rcu_sched self-detected stall on CPU [ 1189.669683] INFO: rcu_sched detected stalls on CPUs/tasks: [ 1189.669689] 3-...: (56078 ticks this GP) idle=769/140000000000001/0 softirq=32564/32569 fqs=26269 [ 1189.669695] 11-...: (1 GPs behind) idle=8cd/140000000000000/0 softirq=36685/36686 fqs=26269 [ 1189.669696] (detected by 2, t=60007 jiffies, g=20785, c=20784, q=16775) [ 1189.691113] 3-...: (56078 ticks this GP) idle=769/140000000000001/0 softirq=32564/32569 fqs=26272 [ 1189.692748] (t=60012 jiffies g=20785 c=20784 q=16775) [ 1212.645668] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1216.605666] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [lxd:34563] [ 1240.645876] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1244.606272] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [lxd:34563] [ 1268.645669] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1272.608277] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [lxd:34563] [ 1296.645701] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1300.605699] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [lxd:34563] [ 1324.645670] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1328.605706] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1352.645674] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1356.605673] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1369.685663] INFO: rcu_sched self-detected stall on CPU [ 1369.692335] INFO: rcu_sched detected stalls on CPUs/tasks: [ 1369.692342] 3-...: (98608 ticks this GP) idle=769/140000000000001/0 softirq=32564/32569 fqs=45792 [ 1369.692345] 11-...: (1 GPs behind) idle=8cd/140000000000000/0 softirq=36685/36686 fqs=45792 [ 1369.692345] (detected by 9, t=105012 jiffies, g=20785, c=20784, q=17084) [ 1369.728009] 3-...: (98608 ticks this GP) idle=769/140000000000001/0 softirq=32564/32569 fqs=45797 [ 1369.728591] (t=105021 jiffies g=20785 c=20784 q=17084) [ 1380.645674] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1396.605667] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1408.645668] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1424.605671] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] [ 1436.645773] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [lxd:22980] [ 1452.605666] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [lxd:34563] ``` ``` buildd06 login: [ 1416.276297] BUG: unable to handle kernel paging request at 2717207f [ 1416.279824] IP: mntput_no_expire+0x11/0x170 [ 1416.281522] *pdpt = 000000002afb3001 *pde = 0000000000000000 [ 1416.281525] [ 1416.286449] Oops: 0002 [#1] SMP [ 1416.289702] Modules linked in: ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_filter ip6_tables xt_comment ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs veth ebtable_filter ebtables dm_snapshot dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc zram kvm_intel kvm irqbypass crc32_pclmul pcbc ppdev aesni_intel aes_i586 crypto_simd cryptd joydev input_leds serio_raw i2c_piix4 intel_rapl_perf parport_pc mac_hid lp parport autofs4 btrfs xor raid6_pq psmouse virtio_scsi [ 1416.298979] virtio_net pata_acpi floppy [ 1416.299257] CPU: 3 PID: 20371 Comm: lxd Tainted: G W 4.10.0-8-generic #10-Ubuntu [ 1416.299973] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 [ 1416.300605] task: e5bac980 task.stack: d4b10000 [ 1416.301009] EIP: mntput_no_expire+0x11/0x170 [ 1416.301437] EFLAGS: 00210282 CPU: 3 [ 1416.301780] EAX: e2d3a000 EBX: cfdeb86c ECX: 00000000 EDX: 0100007f [ 1416.302202] ESI: cfdeb870 EDI: e267e000 EBP: d4b11e64 ESP: d4b11e54 [ 1416.302761] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 [ 1416.303149] CR0: 80050033 CR2: 2717207f CR3: 2b0ec660 CR4: 001406f0 [ 1416.303669] Call Trace: [ 1416.303871] mntput+0x20/0x40 [ 1416.304162] simple_release_fs+0x37/0x40 [ 1416.304475] securityfs_remove+0x6a/0x80 [ 1416.304871] __aa_fs_ns_rmdir+0x18a/0x260 [ 1416.305203] destroy_ns+0x90/0xa0 [ 1416.305479] __aa_remove_ns+0x25/0x80 [ 1416.305816] ns_rmdir_op+0x170/0x470 [ 1416.306080] vfs_rmdir+0xa1/0x110 [ 1416.306383] do_rmdir+0x1dd/0x1f0 [ 1416.306642] SyS_unlinkat+0x2d/0x40 [ 1416.306991] do_int80_syscall_32+0x5c/0xc0 [ 1416.307281] entry_INT80_32+0x31/0x31 [ 1416.307609] EIP: 0x82d85ab [ 1416.307844] EFLAGS: 00200246 CPU: 3 [ 1416.308114] EAX: ffffffda EBX: ffffff9c ECX: 972a2a00 EDX: 00000200 [ 1416.308649] ESI: 00000000 EDI: 00000000 EBP: 97177180 ESP: 97453e4c [ 1416.309184] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b [ 1416.309539] Code: 00 3b 05 54 ac c2 cf 7c d9 89 d8 5b 5e 5d c3 8d 74 26 00 8d bc 27 00 00 00 00 55 89 e5 57 56 53 83 ec 04 3e 8d 74 26 00 8b 50 24 <64> ff 0a 8b 50 70 85 d2 74 0d 83 c4 04 5b 5e 5f 5d c3 90 8d 74 [ 1416.310951] EIP: mntput_no_expire+0x11/0x170 SS:ESP: 0068:d4b11e54 [ 1416.311400] CR2: 000000002717207f ``` ``` buildd05 login: [ 1106.249377] BUG: unable to handle kernel paging request at 1d187000 [ 1106.251818] IP: mntget+0xf/0x20 [ 1106.251893] *pdpt = 0000000024273001 *pde = 0000000000000000 [ 1106.251895] [ 1106.252043] Oops: 0002 [#1] SMP [ 1106.252114] Modules linked in: ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_filter ip6_tables xt_comment veth ebtable_filter ebtables binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc zram kvm_intel kvm irqbypass crc32_pclmul pcbc aesni_intel aes_i586 crypto_simd ppdev cryptd i2c_piix4 intel_rapl_perf joydev input_leds parport_pc serio_raw mac_hid lp parport autofs4 btrfs xor raid6_pq psmouse virtio_scsi virtio_net floppy pata_acpi [ 1106.253127] CPU: 4 PID: 28380 Comm: lxd Tainted: G W 4.10.0-8-generic #10-Ubuntu [ 1106.253283] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 [ 1106.253856] task: e4861500 task.stack: ce768000 [ 1106.253999] EIP: mntget+0xf/0x20 [ 1106.254073] EFLAGS: 00210282 CPU: 4 [ 1106.254134] EAX: f4512610 EBX: 00000000 ECX: d8ddcbcc EDX: 00000000 [ 1106.254238] ESI: d8deb86c EDI: d8deb870 EBP: ce769ed4 ESP: ce769ed4 [ 1106.254342] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 [ 1106.254423] CR0: 80050033 CR2: 1d187000 CR3: 29930400 CR4: 001406f0 [ 1106.254529] Call Trace: [ 1106.254585] simple_pin_fs+0x32/0xa0 [ 1106.254661] securityfs_pin_fs+0x1c/0x20 [ 1106.254736] ns_mkdir_op+0xef/0x410 [ 1106.254811] vfs_mkdir+0xf1/0x1d0 [ 1106.254886] ? apparmor_path_mkdir+0x20/0x30 [ 1106.255064] SyS_mkdirat+0xc7/0x110 [ 1106.255141] do_int80_syscall_32+0x5c/0xc0 [ 1106.255225] entry_INT80_32+0x31/0x31 [ 1106.255342] EIP: 0x82b363b [ 1106.255385] EFLAGS: 00200246 CPU: 4 [ 1106.255447] EAX: ffffffda EBX: ffffff9c ECX: 9719ae00 EDX: 000001ed [ 1106.255552] ESI: 00000000 EDI: 00000000 EBP: 9718ea00 ESP: 974058ec [ 1106.255654] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b [ 1106.255738] Code: 10 01 89 d0 5b 5d c3 3b 40 10 74 f2 ba fe ff ff ff eb ef 89 f6 8d bc 27 00 00 00 00 55 89 e5 3e 8d 74 26 00 85 c0 74 06 8b 50 14 <64> ff 02 5d c3 8d b6 00 00 00 00 8d bf 00 00 00 00 55 89 e5 3e [ 1106.256084] EIP: mntget+0xf/0x20 SS:ESP: 0068:ce769ed4 [ 1106.256655] CR2: 000000001d187000 ``` ``` buildd04 login: [ 1479.762405] BUG: unable to handle kernel paging request at 171d901d [ 1479.762628] IP: mntget+0xf/0x20 [ 1479.762745] *pdpt = 0000000024035001 *pde = 0000000000000000 [ 1479.762747] [ 1479.762923] Oops: 0002 [#1] SMP [ 1479.762997] Modules linked in: ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_filter ip6_tables xt_comment veth ebtable_filter ebtables dm_snapshot dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio ufs qnx4 hfsplus hfs minix ntfs msdos jfs xfs binfmt_misc xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c xt_tcpudp iptable_filter ip_tables x_tables bridge stp llc zram kvm_intel kvm irqbypass crc32_pclmul pcbc aesni_intel aes_i586 input_leds joydev crypto_simd ppdev cryptd mac_hid i2c_piix4 intel_rapl_perf parport_pc serio_raw lp parport autofs4 btrfs xor raid6_pq psmouse virtio_scsi [ 1479.764419] virtio_net pata_acpi floppy [ 1479.764508] CPU: 6 PID: 21869 Comm: lxd Tainted: G W 4.10.0-8-generic #10-Ubuntu [ 1479.764690] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 [ 1479.764856] task: ebb33480 task.stack: e2332000 [ 1479.764971] EIP: mntget+0xf/0x20 [ 1479.765051] EFLAGS: 00210286 CPU: 6 [ 1479.765132] EAX: e01dc490 EBX: 00000000 ECX: deddcbcc EDX: 0002801d [ 1479.765275] ESI: dedeb86c EDI: dedeb870 EBP: e2333ed4 ESP: e2333ed4 [ 1479.765444] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 [ 1479.765572] CR0: 80050033 CR2: 171d901d CR3: 28eb5ba0 CR4: 001406f0 [ 1479.765724] Call Trace: [ 1479.765794] simple_pin_fs+0x32/0xa0 [ 1479.765889] securityfs_pin_fs+0x1c/0x20 [ 1479.765972] ns_mkdir_op+0xef/0x410 [ 1479.766073] vfs_mkdir+0xf1/0x1d0 [ 1479.766170] ? apparmor_path_mkdir+0x20/0x30 [ 1479.766293] SyS_mkdirat+0xc7/0x110 [ 1479.766394] do_int80_syscall_32+0x5c/0xc0 [ 1479.766491] entry_INT80_32+0x31/0x31 [ 1479.766581] EIP: 0x82b363b [ 1479.766648] EFLAGS: 00200246 CPU: 6 [ 1479.770851] EAX: ffffffda EBX: ffffff9c ECX: 9720b3b0 EDX: 000001ed [ 1479.771002] ESI: 00000000 EDI: 00000000 EBP: 97007400 ESP: 973338ec [ 1479.771132] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b [ 1479.771243] Code: 10 01 89 d0 5b 5d c3 3b 40 10 74 f2 ba fe ff ff ff eb ef 89 f6 8d bc 27 00 00 00 00 55 89 e5 3e 8d 74 26 00 85 c0 74 06 8b 50 14 <64> ff 02 5d c3 8d b6 00 00 00 00 8d bf 00 00 00 00 55 89 e5 3e [ 1479.771675] EIP: mntget+0xf/0x20 SS:ESP: 0068:e2333ed4 [ 1479.771780] CR2: 00000000171d901d ```