Curtin booting after install failed on wily image with a kernel BUG at /build/linux-AxjFAn/linux-4.2.0/drivers/block/virtio_blk.c:172!

[Diogo Matsubara]

As seen here: http://10.245.162.130:8080/job/curtin-vmtest/144/consoleFull, vmtest failed to boot after a successful curtin install with the following call trace:

[ 28.476159] kernel BUG at /build/linux-AxjFAn/linux-4.2.0/drivers/block/virtio_blk.c:172!
[ 28.476159] invalid opcode: 0000 [#1] SMP
[ 28.476159] Modules linked in: xfs libcrc32c dm_crypt drbg ansi_cprng xts gf128mul algif_skcipher af_alg nls_utf8 isofs xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_tcpudp bridge stp llc iptable_filter ip_tables x_tables ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi kvm_intel ppdev kvm joydev input_leds serio_raw parport_pc mac_hid i2c_piix4 parport pvpanic 8250_fintek autofs4 btrfs raid10 multipath linear raid0 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor cirrus psmouse syscopyarea sysfillrect sysimgblt raid6_pq raid1 ttm drm_kms_helper drm pata_acpi floppy
[ 28.476159] CPU: 0 PID: 1454 Comm: dmcrypt_write Not tainted 4.2.0-16-generic #19-Ubuntu
[ 28.476159] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
[ 28.476159] task: ffff880035523e80 ti: ffff8800382e4000 task.ti: ffff8800382e4000
[ 28.476159] RIP: 0010:[<ffffffff815456bf>] [<ffffffff815456bf>] virtio_queue_rq+0x1ef/0x280
[ 28.476159] RSP: 0018:ffff8800382e7b78 EFLAGS: 00010202
[ 28.476159] RAX: 0000000000000082 RBX: ffff88000081ac00 RCX: ffff8800382e7bf8
[ 28.476159] RDX: 0000000000000001 RSI: ffff8800382e7c18 RDI: ffff88000081ac00
[ 28.476159] RBP: ffff8800382e7bc8 R08: ffff880035888e00 R09: ffff8800382e7bf8
[ 28.476159] R10: ffffffffc009387f R11: 0000000000000000 R12: 0000000000000000
[ 28.476159] R13: ffff880035888e00 R14: ffff880035888e00 R15: ffff880037f6acc0
[ 28.476159] FS: 0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
[ 28.476159] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 28.476159] CR2: 00007f489851c000 CR3: 000000003ad80000 CR4: 00000000000006f0
[ 28.476159] Stack:
[ 28.476159] ffff8800382e7bd8 ffffffff813d6e1c ffff8800382e7b98 ffff880035888e00
[ 28.476159] ffff8800382e7bc8 ffff88000081ac00 ffff8800382e7bf8 0000000000000000
[ 28.476159] ffff880035888e00 0000000000000000 ffff8800382e7c68 ffffffff813a229c
[ 28.476159] Call Trace:
[ 28.476159] [<ffffffff813d6e1c>] ? merge+0x5c/0xb0
[ 28.476159] [<ffffffff813a229c>] __blk_mq_run_hw_queue+0x1dc/0x380
[ 28.476159] [<ffffffff813a209e>] blk_mq_run_hw_queue+0x8e/0xb0
[ 28.476159] [<ffffffff813a3543>] blk_mq_insert_requests+0xc3/0x110
[ 28.476159] [<ffffffff813a4159>] blk_mq_flush_plug_list+0xb9/0x150
[ 28.476159] [<ffffffff81399c06>] blk_flush_plug_list+0xd6/0x240
[ 28.476159] [<ffffffff8139a0d4>] blk_finish_plug+0x34/0x50
[ 28.476159] [<ffffffffc04c63af>] dmcrypt_write+0x20f/0x220 [dm_crypt]
[ 28.476159] [<ffffffff810a5b60>] ? wake_up_q+0x70/0x70
[ 28.476159] [<ffffffffc04c61a0>] ? crypt_ctr+0xa80/0xa80 [dm_crypt]
[ 28.476159] [<ffffffff8109a7c8>] kthread+0xd8/0xf0
[ 28.476159] [<ffffffff8109a6f0>] ? kthread_create_on_node+0x1f0/0x1f0
[ 28.476159] [<ffffffff817efe1f>] ret_from_fork+0x3f/0x70
[ 28.476159] [<ffffffff8109a6f0>] ? kthread_create_on_node+0x1f0/0x1f0
[ 28.476159] Code: ff 41 0f b7 85 f4 00 00 00 41 c7 85 78 01 00 00 08 00 00 00 49 c7 85 80 01 00 00 00 00 00 00 41 89 85 7c 01 00 00 e9 ab fe ff ff <0f> 0b 49 8b 87 b0 00 00 00 41 83 e6 ef 4a 8b 3c 20 e8 9b 4c f5
[ 28.476159] RIP [<ffffffff815456bf>] virtio_queue_rq+0x1ef/0x280
[ 28.476159] RSP <ffff8800382e7b78>
[ 29.225842] ---[ end trace 3d5abd69189cb55d ]---

rharper pointed out that this could be https://<email address hidden>/msg42689.html and fixed on kernel 4.3. The image used on that test was

vmtests: DEBUG: b"{'updated': 'Thu, 05 Nov 2015 12:17:51 +0000', 'release': 'wily', 'release_title': '15.10', 'version_name': '20151029', 'size': '394278401', 'item_name': 'root-image.gz', 'version': '15.10', 'subarch': 'hwe-w', 'label': 'daily', 'ftype': 'root-image.gz', 'datatype': 'image-downloads', 'content_id': 'com.ubuntu.maas:daily:v2:download', 'os': 'ubuntu', 'format': 'products:1.0', 'subarches': 'generic,hwe-p,hwe-q,hwe-r,hwe-s,hwe-t,hwe-u,hwe-v,hwe-w', 'arch': 'amd64', 'product_name': 'com.ubuntu.maas.daily:v2:boot:15.10:amd64:hwe-w', 'kflavor': 'generic', 'krel': 'wily', 'release_codename': 'Wily Werewolf', 'item_url': 'file:///srv/images/wily/amd64/20151029/root-image.gz', 'path': 'wily/amd64/20151029/root-image.gz', 'sha256': '4ec483af2ddd587ea06b5728a383a41d2de716de86ebaa9df5947bc63db6db5c'}\n"

which includes kernel 4.2.0-16-generic

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1515687

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Ryan Harper (raharper)]

This does not affect curtin, rather it's a bug in dm_crypt, fixed in upstream linux.

[Diogo Matsubara (matsubara)]

Hi Brad,

I'm unable to run apport-collect in that system as it's an automated CI system which I don't have interactive access while it's running. I'm also unable to reliably reproduce the error. :-(

What else can I try?

(I'm setting this bug back to Confirmed per your instructions)