linux 4.15.0-38.41 ADT test failure (ubuntu_nbd_smoke_test)

[Kleber Sacilotto de Souza]

Testing failed on:
    i386: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/l/linux/20181019_194651_d42a8@/log.gz

The ubuntu_nbd_smoke_test testcase is hitting the following kernel bug:

16:02:03 DEBUG| [stdout] [ 6587.920070] ------------[ cut here ]------------
16:02:03 DEBUG| [stdout] [ 6587.975274] kernel BUG at /build/linux-aTHL1U/linux-4.15.0/fs/buffer.c:3064!
16:02:03 DEBUG| [stdout] [ 6587.976968] invalid opcode: 0000 [#2] SMP
16:02:03 DEBUG| [stdout] [ 6587.977880] Modules linked in: nbd 842 842_decompress 842_compress lz4hc lz4hc_compress lz4 lz4_compress unix_diag binfmt_misc cuse uhid hid vfio_iommu_type1 vfio hci_vhci bluetooth ecdh_generic vhost_net tap vhost_vsock vmw_vsock_virtio_transport_common vhost vsock snd_seq snd_seq_device snd_timer snd soundcore userio dccp_ipv4 algif_rng salsa20_generic salsa20_i586 camellia_generic cast6_generic cast_common serpent_sse2_i586 serpent_generic glue_helper ablk_helper twofish_generic twofish_i586 twofish_common lrw algif_skcipher tgr192 wp512 rmd320 rmd256 rmd160 rmd128 md4 algif_hash af_alg aufs macsec vxlan ip6_udp_tunnel udp_tunnel vrf 8021q garp mrp bridge stp llc ip_gre ip_tunnel gre cls_u32 sch_htb dummy af_packet_diag sctp_diag sctp libcrc32c dccp_diag dccp tcp_diag udp_diag raw_diag inet_diag
16:02:03 DEBUG| [stdout] [ 6587.994265] xt_TCPMSS xt_tcpudp iptable_mangle kvm_intel kvm irqbypass crc32_pclmul pcbc aesni_intel aes_i586 crypto_simd cryptd joydev input_leds mac_hid qemu_fw_cfg serio_raw sch_fq_codel ip_tables x_tables autofs4 btrfs xor zstd_compress raid6_pq psmouse i2c_piix4 virtio_net virtio_blk pata_acpi floppy [last unloaded: zram]
16:02:03 DEBUG| [stdout] [ 6588.000720] CPU: 3 PID: 3794 Comm: kworker/u8:1 Tainted: G D W OE 4.15.0-38-generic #41-Ubuntu
16:02:03 DEBUG| [stdout] [ 6588.002806] Hardware name: OpenStack Foundation OpenStack Nova, BIOS 1.10.1-1ubuntu1~cloud0 04/01/2014
16:02:03 DEBUG| [stdout] [ 6588.004941] Workqueue: writeback wb_workfn (flush-43:0)
16:02:03 DEBUG| [stdout] [ 6588.006119] EIP: submit_bh_wbc+0x168/0x190
16:02:03 DEBUG| [stdout] [ 6588.007052] EFLAGS: 00010246 CPU: 3
16:02:03 DEBUG| [stdout] [ 6588.007846] EAX: 00620005 EBX: ce002cb0 ECX: ce002cb0 EDX: 00020800
16:02:03 DEBUG| [stdout] [ 6588.009251] ESI: 00020800 EDI: db34e400 EBP: d054bc3c ESP: d054bc2c
16:02:03 DEBUG| [stdout] [ 6588.010656] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
16:02:03 DEBUG| [stdout] [ 6588.011876] CR0: 80050033 CR2: b7f29e44 CR3: 1de14000 CR4: 001406f0
16:02:03 DEBUG| [stdout] [ 6588.013286] Call Trace:
16:02:03 DEBUG| [stdout] [ 6588.013886] __sync_dirty_buffer+0x4d/0xf0
16:02:03 DEBUG| [stdout] [ 6588.014824] ext4_commit_super+0x1e2/0x290
16:02:03 DEBUG| [stdout] [ 6588.015767] __ext4_abort+0x4a/0x120
16:02:03 DEBUG| [stdout] [ 6588.016588] ext4_journal_check_start+0x81/0x90
16:02:03 DEBUG| [stdout] [ 6588.017611] __ext4_journal_start_sb+0x28/0xd0
16:02:03 DEBUG| [stdout] [ 6588.018622] ? ext4_writepages+0x4d4/0x880
16:02:03 DEBUG| [stdout] [ 6588.045547] ext4_writepages+0x4d4/0x880
16:02:03 DEBUG| [stdout] [ 6588.046460] ? blk_mq_dispatch_rq_list+0x74/0x440
16:02:03 DEBUG| [stdout] [ 6588.047552] do_writepages+0x39/0xc0
16:02:03 DEBUG| [stdout] [ 6588.048367] ? blk_mq_sched_dispatch_requests+0x184/0x190
16:02:03 DEBUG| [stdout] [ 6588.049579] __writeback_single_inode+0x3a/0x310
16:02:03 DEBUG| [stdout] [ 6588.050623] writeback_sb_inodes+0x167/0x3a0
16:02:03 DEBUG| [stdout] [ 6588.051601] __writeback_inodes_wb+0x55/0xa0
16:02:03 DEBUG| [stdout] [ 6588.052568] wb_writeback+0x20a/0x2a0
16:02:03 DEBUG| [stdout] [ 6588.053411] ? find_next_bit+0xa/0x10
16:02:03 DEBUG| [stdout] [ 6588.054246] wb_workfn+0x313/0x430
16:02:03 DEBUG| [stdout] [ 6588.055051] ? __switch_to_asm+0x27/0x4c
16:02:03 DEBUG| [stdout] [ 6588.055961] ? __switch_to+0xae/0x360
16:02:03 DEBUG| [stdout] [ 6588.056795] ? __switch_to_asm+0x33/0x4c
16:02:03 DEBUG| [stdout] [ 6588.057685] ? __switch_to_asm+0x27/0x4c
16:02:03 DEBUG| [stdout] [ 6588.058597] process_one_work+0x1b9/0x3d0
16:02:03 DEBUG| [stdout] [ 6588.059514] worker_thread+0x37/0x420
16:02:03 DEBUG| [stdout] [ 6588.060345] kthread+0xf0/0x110
16:02:03 DEBUG| [stdout] [ 6588.061064] ? process_one_work+0x3d0/0x3d0
16:02:03 DEBUG| [stdout] [ 6588.062010] ? kthread_create_worker_on_cpu+0x20/0x20
16:02:03 DEBUG| [stdout] [ 6588.063151] ret_from_fork+0x2e/0x38
16:02:03 DEBUG| [stdout] [ 6588.063963] Code: 08 e8 2d ed 17 00 8d 65 f4 31 c0 5b 5e 5f 5d c3 8d 76 00 f0 80 63 01 f7 e9 06 ff ff ff 8d b6 00 00 00 00 0f 0b 8d b6 00 00 00 00 <0f> 0b 8d b6 00 00 00 00 0f 0b 8d b6 00 00 00 00 0f 0b 8d b6 00
16:02:03 DEBUG| [stdout] [ 6588.068163] EIP: submit_bh_wbc+0x168/0x190 SS:ESP: 0068:d054bc2c
16:02:03 DEBUG| [stdout] [ 6588.069550] ---[ end trace 072e2754d45c92b5 ]---

The testcase were blacklisted until recently, and when not hitting this bug it hits bug 1795408. So this is not a regression given that the testcase never had a successful run.

Note: *not* failing on other architectures.

[Po-Hsu Lin (cypressyew)]

There is a known bug 1793464 for nbd test, but it will only happen after the test was completed.

And for bare-metal testing, I didn't see this on i386 node.

http://10.246.72.4/test-results/4.15.0-38.41-generic/ubuntu_nbd_smoke_test-4.15.0-38.41-i386-index.html

[Kleber Sacilotto de Souza (kleber-souza)]

Testcase still failing with bionic/linux 4.15.0-45.48 (only on i386):
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/i386/l/linux/20190201_035319_9ea9e@/log.gz