lxd 2.0.11-0ubuntu1~16.04.4 ADT test failure with linux 4.4.0-207.239

[Kelsey Steele]

Testing failed on:
    amd64: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/l/lxd/20210331_002001_322ce@/log.gz
    arm64: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/arm64/l/lxd/20210330_023432_11c32@/log.gz
    i386: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/i386/l/lxd/20210331_001905_1aecd@/log.gz
    ppc64el: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/ppc64el/l/lxd/20210331_001557_b82c6@/log.gz
    s390x: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/s390x/l/lxd/20210331_001215_7b9ae@/log.gz

Starting c1
action=start creation date=2021-03-30T02:42:13+0000 ephemeral=false lvl=eror msg="Failed starting container" name=c1 stateful=false t=2021-03-30T02:42:14+0000
error: Error calling 'lxd forkstart c1 /tmp/autopkgtest.mceqas/build.1QA/src/test/tmp.MBH/dyt/containers /tmp/autopkgtest.mceqas/build.1QA/src/test/tmp.MBH/dyt/logs/c1/lxc.conf': err='Failed to run: /usr/bin/lxd forkstart c1 /tmp/autopkgtest.mceqas/build.1QA/src/test/tmp.MBH/dyt/containers /tmp/autopkgtest.mceqas/build.1QA/src/test/tmp.MBH/dyt/logs/c1/lxc.conf: '
  lxc 20210330024214.649 ERROR lxc_conf - conf.c:run_buffer:286 - Script exited with status 137.
  lxc 20210330024214.649 ERROR lxc_conf - conf.c:lxc_setup:3356 - failed to run mount hooks for container 'c1'.
  lxc 20210330024214.649 ERROR lxc_start - start.c:do_start:1248 - Failed to setup container "c1".
  lxc 20210330024214.649 ERROR lxc_sync - sync.c:__sync_wait:59 - An error occurred in another process (expected sequence number 5)
  lxc 20210330024214.662 ERROR lxc_start - start.c:__lxc_start:1802 - Failed to spawn container "c1".
  lxc 20210330024214.663 ERROR lxc_container - lxccontainer.c:wait_on_daemonized_start:804 - Received container state "ABORTING" instead of "RUNNING"

Try `lxc info --show-log lxd2:c1` for more info
==> Cleaning up
==> Killing LXD at /tmp/autopkgtest.mceqas/build.1QA/src/test/tmp.MBH/zOt
==> Deleting all containers
==> Deleting all images
==> Deleting all profiles
Profile default deleted
Profile docker deleted
==> Checking for locked DB tables
==> Checking for leftover files
==> Checking for leftover DB entries
==> Tearing down directory backend in /tmp/autopkgtest.mceqas/build.1QA/src/test/tmp.MBH/zOt
==> Killing LXD at /tmp/autopkgtest.mceqas/build.1QA/src/test/tmp.MBH/dyt
==> Deleting all containers
==> Deleting all images
==> Deleting all profiles
Profile default deleted
Profile docker deleted
==> Checking for locked DB tables
==> Checking for leftover files
==> Checking for leftover DB entries
==> Tearing down directory backend in /tmp/autopkgtest.mceqas/build.1QA/src/test/tmp.MBH/dyt

==> TEST DONE: remote usage
==> Test result: failure

[Stéphane Graber (stgraber)]

This looks like a kernel regression to me.

[Stéphane Graber (stgraber)]

When a single test fails occasionally, it can be an issue with LXD or with the test, but when a bugfix release of a stable kernel suddenly causes one of the most trivial tests to fail on all architectures, this strongly suggests that the kernel is the issue.

[Stéphane Graber (stgraber)]

Confirmed that on a working system, just updating to the new kernel breaks it.
So that SRU kernel is definitely broken and should not be shipped.

[ 8.996651] BUG: unable to handle kernel NULL pointer dereference at 00000000e12c1a77
[ 8.998738] IP: [<00000000b372dc06>] fuse_do_setattr+0x52/0x640
[ 9.000546] PGD 80000002717c7067 PUD 270e5d067 PMD 0
[ 9.001915] Oops: 0000 [#1] SMP
[ 9.003041] Modules linked in: binfmt_misc veth ip6table_filter ip6_tables xt_CHECKSUM iptable_mangle xt_comment xt_tcpudp iptable_filter ip_tables x_tables kvm_intel kvm irqbypass bridge stp llc joydev input_leds serio_raw lpc_ich 9pnet_virtio 9pnet virtio_rng virtio_input shpchp 8250_fintek mac_hid ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel virtio_gpu ttm aesni_intel drm_kms_helper aes_x86_64 lrw syscopyarea gf128mul glue_helper ablk_helper sysfillrect ahci sysimgblt cryptd fb_sys_fops psmouse drm libahci virtio_scsi
[ 9.019982] CPU: 2 PID: 1929 Comm: mount Not tainted 4.4.0-207-generic #239-Ubuntu
[ 9.021887] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009)/LXD, BIOS 0.0.0 02/06/2015
[ 9.023893] task: 00000000b85f1580 ti: 0000000046f8cfc7 task.ti: 0000000046f8cfc7
[ 9.025775] RIP: 0010:[<ffffffff813307b2>] [<00000000b372dc06>] fuse_do_setattr+0x52/0x640
[ 9.027974] RSP: 0018:ffff880272eb7c20 EFLAGS: 00010246
[ 9.029627] RAX: 0000000000000000 RBX: ffff880272eb7e28 RCX: 000000000000000e
[ 9.031507] RDX: 0000000000000000 RSI: ffff880272eb7e28 RDI: ffff880272eb7cf8
[ 9.033447] RBP: ffff880272eb7d98 R08: 0000000000019580 R09: ffffffff8122c764
[ 9.035159] R10: ffffea0009cd8400 R11: ffff88027203c300 R12: 0000000000000000
[ 9.037004] R13: ffff880272eb7e28 R14: ffff88027203c470 R15: ffff88027203c300
[ 9.038737] FS: 00007f01e82d9840(0000) GS:ffff88027fd00000(0000) knlGS:0000000000000000
[ 9.040811] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9.042470] CR2: 0000000000000458 CR3: 0000000273654000 CR4: 0000000000160670
[ 9.044488] Stack:
[ 9.045578] ffffffff81227c54 ffff880272eb7c84 0000000000000001 000000028186639d
[ 9.047599] 5318e6f4d6b61d94 ffff880272eb7d70 ffff880272eb7d80 ffff880272eb7d70
[ 9.049606] 0000000000000000 ffff880272eb7cf8 ffff880272eb7cf0 ffff880270e50320
[ 9.051576] Call Trace:
[ 9.052746] [<000000004f4fb5e7>] ? lookup_fast+0x184/0x340
[ 9.054366] [<000000004f4fb5e7>] ? lookup_fast+0x184/0x340
[ 9.055970] [<00000000c60c3075>] ? unlazy_walk+0xc1/0x150
[ 9.057542] [<00000000d5cd4375>] ? terminate_walk+0x66/0xd0
[ 9.059307] [<0000000051dc2989>] ? putname+0x54/0x60
[ 9.060934] [<000000005d276838>] fuse_setattr+0xa5/0xf0
[ 9.062454] [<00000000e045b853>] notify_change+0x2dc/0x430
[ 9.064177] [<000000008ae20288>] utimes_common+0xd1/0x1b0
[ 9.065694] [<000000003571704c>] do_utimes+0x125/0x160
[ 9.067102] [<00000000e85b7804>] SyS_utimensat+0x67/0xa0
[ 9.068721] [<00000000fb35cea1>] ent...

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.4.0-208.240

---------------
linux (4.4.0-208.240) xenial; urgency=medium

  * xenial/linux: 4.4.0-208.240 -proposed tracker (LP: #1922069)

  * linux ADT test failure with linux/4.4.0-207.239 -
    ubuntu_qrt_kernel_security.test-kernel-security.py (LP: #1922200) //
    CVE-2018-5953 // CVE-2018-5995 // CVE-2018-7754
    - SAUCE: Revert "printk: hash addresses printed with %p"

  * lxd 2.0.11-0ubuntu1~16.04.4 ADT test failure with linux 4.4.0-207.239
    (LP: #1921969)
    - SAUCE: Fix fuse regression in 4.4.0-207.239

linux (4.4.0-207.239) xenial; urgency=medium

  * xenial/linux: 4.4.0-207.239 -proposed tracker (LP: #1919558)

  * Xenial update: v4.4.262 upstream stable release (LP: #1920221)
    - uapi: nfnetlink_cthelper.h: fix userspace compilation error
    - ath9k: fix transmitting to stations in dynamic SMPS mode
    - net: Fix gro aggregation for udp encaps with zero csum
    - can: skb: can_skb_set_owner(): fix ref counting if socket was closed before
      setting skb ownership
    - can: flexcan: assert FRZ bit in flexcan_chip_freeze()
    - can: flexcan: enable RX FIFO after FRZ/HALT valid
    - netfilter: x_tables: gpf inside xt_find_revision()
    - cifs: return proper error code in statfs(2)
    - floppy: fix lock_fdc() signal handling
    - Revert "mm, slub: consider rest of partial list if acquire_slab() fails"
    - futex: Change locking rules
    - futex: Cure exit race
    - futex: fix dead code in attach_to_pi_owner()
    - net/mlx4_en: update moderation when config reset
    - net: lapbether: Remove netif_start_queue / netif_stop_queue
    - net: davicom: Fix regulator not turned off on failed probe
    - net: davicom: Fix regulator not turned off on driver removal
    - media: usbtv: Fix deadlock on suspend
    - mmc: mxs-mmc: Fix a resource leak in an error handling path in
      'mxs_mmc_probe()'
    - mmc: mediatek: fix race condition between msdc_request_timeout and irq
    - powerpc/perf: Record counter overflow always if SAMPLE_IP is unset
    - PCI: xgene-msi: Fix race in installing chained irq handler
    - s390/smp: __smp_rescan_cpus() - move cpumask away from stack
    - scsi: libiscsi: Fix iscsi_prep_scsi_cmd_pdu() error handling
    - ALSA: hda/hdmi: Cancel pending works before suspend
    - ALSA: hda: Avoid spurious unsol event handling during S3/S4
    - ALSA: usb-audio: Fix "cannot get freq eq" errors on Dell AE515 sound bar
    - s390/dasd: fix hanging DASD driver unbind
    - mmc: core: Fix partition switch time for eMMC
    - scripts/recordmcount.{c,pl}: support -ffunction-sections .text.* section
      names
    - Goodix Fingerprint device is not a modem
    - usb: gadget: f_uac2: always increase endpoint max_packet_size by one audio
      slot
    - usb: renesas_usbhs: Clear PIPECFG for re-enabling pipe with other EPNUM
    - xhci: Improve detection of device initiated wake signal.
    - USB: serial: io_edgeport: fix memory leak in edge_startup
    - USB: serial: ch341: add new Product ID
    - USB: serial: cp210x: add ID for Acuity Brands nLight Air Adapter
    - USB: serial: cp210x: add some more GE USB IDs
    - usbip: fix stub_dev to check for stream ...