| 2013-11-18 23:40:07 |
Peter Feiner |
bug |
|
|
added bug |
| 2013-11-18 23:40:07 |
Peter Feiner |
attachment added |
|
dmesg_and_lspci.log https://bugs.launchpad.net/bugs/1252508/+attachment/3912031/+files/dmesg_and_lspci.log |
|
| 2013-11-19 00:00:05 |
Brad Figg |
affects |
linux-meta (Ubuntu) |
linux (Ubuntu) |
|
| 2013-11-19 00:30:17 |
Brad Figg |
linux (Ubuntu): status |
New |
Incomplete |
|
| 2013-11-19 00:30:18 |
Brad Figg |
tags |
|
saucy |
|
| 2013-11-19 14:38:34 |
Peter Feiner |
tags |
saucy |
apport-collected saucy |
|
| 2013-11-19 14:38:36 |
Peter Feiner |
description |
I've hit the same BUG twice in as many working days. I'll give a brief description of what I'm doing, paste the two BUG reports, then paste the requested information.
I've been doing some performance testing on OpenStack. The tests just involve booting and deleting a bunch of virtual machine instances. A couple of things happen during VM creation that seem to cause the bug on. First, tap devices are created in various network namespaces. Second, nbd devices, served locally by qemu-nbd, with ext3 data are connected to /dev/nbd* and mounted and fiddled with.
The ip program is used to operate in the network namespaces (ip netns exec ...). Occasionally, when ip exits, it hits a BUG in some block layer code. Apparently the BUGging ip task had the last reference to a mount.
This BUG is fairly nasty because it happens when the superblock's s_umount semaphore is held, so other tasks that try to do stuff with the mount deadlock.
Here are the processes involved in today's BUG:
15488 sudo ip netns exec qdhcp-1ca153f0-3ece-4574-a0af-317c21cce5b4 ip route list dev tap4fad7e3b-f4
15492 \_ [ip]
27607 /bin/mount /dev/nbd13 /tmp/openstack-vfs-localfsKV3wPV
27592 /usr/bin/qemu-nbd -c /dev/nbd13 /opt/stack/data/nova/instances/4d46d4f9-a812-4e84-8659-b30e8e01dd27/disk
% sudo cat /proc/27607/stack
[<ffffffff811df193>] __blkdev_get+0x63/0x4c0
[<ffffffff811df7b5>] blkdev_get+0x1c5/0x340
[<ffffffff811df9db>] blkdev_open+0x5b/0x80
[<ffffffff811a4cab>] do_dentry_open+0x1eb/0x280
[<ffffffff811a4dc9>] vfs_open+0x49/0x50
[<ffffffff811b4111>] do_last+0x1c1/0xe40
[<ffffffff811b5981>] path_openat+0xc1/0x630
[<ffffffff811b6cfa>] do_filp_open+0x3a/0x90
[<ffffffff811a5fac>] do_sys_open+0x12c/0x270
[<ffffffff811a610e>] SyS_open+0x1e/0x20
[<ffffffff816f521d>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
% sudo cat /proc/27592/task/27595/stack
[<ffffffff81369714>] call_rwsem_down_read_failed+0x14/0x30
[<ffffffff811a99c5>] get_super+0x95/0xe0
[<ffffffff811dee64>] fsync_bdev+0x14/0x60
[<ffffffff81340824>] invalidate_partition+0x24/0x50
[<ffffffff81343a19>] rescan_partitions+0x49/0x2c0
[<ffffffff811df5e2>] __blkdev_get+0x4b2/0x4c0
[<ffffffff811df7b5>] blkdev_get+0x1c5/0x340
[<ffffffff811df9db>] blkdev_open+0x5b/0x80
[<ffffffff811a4cab>] do_dentry_open+0x1eb/0x280
[<ffffffff811a4dc9>] vfs_open+0x49/0x50
[<ffffffff811b4111>] do_last+0x1c1/0xe40
[<ffffffff811b5981>] path_openat+0xc1/0x630
[<ffffffff811b6cfa>] do_filp_open+0x3a/0x90
[<ffffffff811a5fac>] do_sys_open+0x12c/0x270
[<ffffffff811a610e>] SyS_open+0x1e/0x20
[<ffffffff816f521d>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
I'm running on the stock ubuntu kernel; no custom modules have been inserted.
This is the first BUG cut from a few days ago. I don't have the entire dmesg log from this run:
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131201] ------------[ cut here ]------------
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131469] Kernel BUG at ffffffff811d77d4 [verbose debug info unavailable]
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131881] invalid opcode: 0000 [#1] SMP
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.132148] Modules linked in: xt_LOG(F) vhost_net vhost macvtap(F) macvlan(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) ebt_arp(F) ebt_ip(F)
xt_mac(F) xt_physdev(F) veth(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
(F) openvswitch(OF) dm_multipath(F) scsi_dh(F) nbd(F) vesafb(F) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_inte
l(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) gpio_ich microcode(F) joydev(F) sb_edac edac_core mei_me mei lpc_ich ioatdma mac_hid lp(F) p
arport(F) dm_snapshot(F) hid_generic igb isci ixgbe i2c_algo_bit usbhid dca hid libsas(F) ahci(F) ptp(F) libahci(
Nov 15 14:52:44 node-0025904feb5c kernel: F) pps_core(F) mdio scsi_transport_sas(F)
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.138029] CPU: 15 PID: 27942 Comm: ip Tainted: GF O 3.11.0-12-generic #19-Ubuntu
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131881] invalid opcode: 0000 [#1] SMP
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.132148] Modules linked in: xt_LOG(F) vhost_net vhost macvtap(F) macvlan(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) ebt_arp(F) ebt_ip(F)
xt_mac(F) xt_physdev(F) veth(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
(F) openvswitch(OF) dm_multipath(F) scsi_dh(F) nbd(F) vesafb(F) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_inte
l(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) gpio_ich microcode(F) joydev(F) sb_edac edac_core mei_me mei lpc_ich ioatdma mac_hid lp(F) p
arport(F) dm_snapshot(F) hid_generic igb isci ixgbe i2c_algo_bit usbhid dca hid libsas(F) ahci(F) ptp(F) libahci(
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131881] invalid opcode: 0000 [#1] SMP
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.132148] Modules linked in: xt_LOG(F) vhost_net vhost macvtap(F) macvlan(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) ebt_arp(F) ebt_ip(F)
xt_mac(F) xt_physdev(F) veth(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
(F) openvswitch(OF) dm_multipath(F) scsi_dh(F) nbd(F) vesafb(F) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_inte
l(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) gpio_ich microcode(F) joydev(F) sb_edac edac_core mei_me mei lpc_ich ioatdma mac_hid lp(F) p
arport(F) dm_snapshot(F) hid_generic igb isci ixgbe i2c_algo_bit usbhid dca hid libsas(F) ahci(F) ptp(F) libahci(
Nov 15 14:52:44 node-0025904feb5c kernel: F) pps_core(F) mdio scsi_transport_sas(F)
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.138029] CPU: 15 PID: 27942 Comm: ip Tainted: GF O 3.11.0-12-generic #19-Ubuntu
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131881] invalid opcode: 0000 [#1] SMP
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.132148] Modules linked in: xt_LOG(F) vhost_net vhost macvtap(F) macvlan(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) ebt_arp(F) ebt_ip(F)
xt_mac(F) xt_physdev(F) veth(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
(F) openvswitch(OF) dm_multipath(F) scsi_dh(F) nbd(F) vesafb(F) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_inte
l(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) gpio_ich microcode(F) joydev(F) sb_edac edac_core mei_me mei lpc_ich ioatdma mac_hid lp(F) p
arport(F) dm_snapshot(F) hid_generic igb isci ixgbe i2c_algo_bit usbhid dca hid libsas(F) ahci(F) ptp(F) libahci(
Nov 15 14:52:44 node-0025904feb5c kernel: F) pps_core(F) mdio scsi_transport_sas(F)
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.138029] CPU: 15 PID: 27942 Comm: ip Tainted: GF O 3.11.0-12-generic #19-Ubuntu
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.157795] Hardware name: Supermicro X9DRFR/X9DRFR, BIOS 1.0a 09/11/2012
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.167959] task: ffff88053c631770 ti: ffff88060d788000 task.ti: ffff88060d788000
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.189055] RIP: 0010:[<ffffffff811d77d4>] [<ffffffff811d77d4>] _submit_bh+0x184/0x210
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.209728] RSP: 0018:ffff88060d789c98 EFLAGS: 00010246
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.220248] RAX: 0000000000000005 RBX: ffff880772e019c0 RCX: ffff88060d789fd8
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.241884] RDX: 0000000000000000 RSI: ffff880772e019c0 RDI: 0000000000000411
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.263641] RBP: ffff88060d789cb8 R08: 0000000000000000 R09: 0000000000000002
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.286483] R10: ffff880772e01068 R11: ffffffff8128badd R12: 0000000000000411
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.309833] R13: ffff8807a6499400 R14: ffff88076b528000 R15: 00007fff6b62dc00
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.333067] FS: 00007fb0e72c2740(0000) GS:ffff88085fd20000(0000) knlGS:0000000000000000
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.356632] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.368780] CR2: 00007fb1a1025f50 CR3: 00000007be808000 CR4: 00000000000427e0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.384379] nbd9: unknown partition table
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.403462] Stack:
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.414237] ffff880772e019c0 0000000000000411 ffff8807a6499400 ffff88076b528000
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.436155] ffff88060d789cd8 ffffffff811d84a5 ffff880772e019c0 ffff88060155e400
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.457077] kjournald starting. Commit interval 5 seconds
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.480247] EXT3-fs (nbd9): using internal journal
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.480251] EXT3-fs (nbd9): mounted filesystem with ordered data mode
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.469329] ffff88060d789ce8 ffffffff811d8543 ffff88060d789d10 ffffffff812310d0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.513658] Call Trace:
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.524287] [<ffffffff811d84a5>] __sync_dirty_buffer+0x55/0xe0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.534920] [<ffffffff811d8543>] sync_dirty_buffer+0x13/0x20
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.545623] [<ffffffff812310d0>] ext3_commit_super.constprop.89+0x60/0xf0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.556287] [<ffffffff81231b21>] ext3_put_super+0x271/0x2a0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.566461] [<ffffffff811a8eb6>] generic_shutdown_super+0x56/0xe0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.576653] [<ffffffff811a9177>] kill_block_super+0x27/0x70
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.586668] [<ffffffff811a94ed>] deactivate_locked_super+0x3d/0x60
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.596335] [<ffffffff811a9556>] deactivate_super+0x46/0x60
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.606253] [<ffffffff811c549f>] mntput_no_expire+0xef/0x150
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.615477] [<ffffffff811c5526>] mntput+0x26/0x40
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.624622] [<ffffffff811c563d>] namespace_unlock+0xfd/0x110
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.634156] [<ffffffff811c7d4b>] put_mnt_ns+0x4b/0x60
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.643059] [<ffffffff810891ab>] free_nsproxy+0x1b/0x90
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.651554] [<ffffffff810893d5>] switch_task_namespaces+0x45/0x50
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.659768] [<ffffffff810893f0>] exit_task_namespaces+0x10/0x20
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.667855] [<ffffffff81064222>] do_exit+0x2b2/0xa40
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.676050] [<ffffffff81064a2f>] do_group_exit+0x3f/0xa0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.685242] [<ffffffff81064aa4>] SyS_exit_group+0x14/0x20
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.693422] [<ffffffff816f521d>] system_call_fastpath+0x1a/0x1f
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.701737] Code: 52 00 00 89 d8 5b 41 5c 41 5d 41 5e 5d c3 66 0f 1f 84 00 00 00 00 00 40 f6 c7 01 0f 84 e7 fe ff ff f0 80 66 01 f7 e9 dd fe ff ff <0f> 0b 49 8b 54 24 70 c1 e0 09 41 f6 c5 01 41 89 44 24 30 89 42
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.725473] RIP [<ffffffff811d77d4>] _submit_bh+0x184/0x210
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.733137] RSP <ffff88060d789c98>
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.753260] ---[ end trace 79cbd7bd94cca2ea ]---
This is the second BUG cut from today. The full dmesg is attached.
[Mon Nov 18 17:23:02 2013] ------------[ cut here ]------------
[Mon Nov 18 17:23:02 2013] Kernel BUG at ffffffff811d77d4 [verbose debug info unavailable]
[Mon Nov 18 17:23:02 2013] invalid opcode: 0000 [#1] SMP
[Mon Nov 18 17:23:02 2013] Modules linked in: nf_conntrack_ipv6(F) nf_defrag_ipv6(F) xt_mac(F) xt_physdev(F) vhost_net vhost macvtap(F) macvlan(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables(F) ib_iser rdma_cm(F) ib_addr(F) iw_cm(F) ib_cm(F) ib_sa(F) ib_mad(F) ib_core(F) iscsi_tcp(F) libiscsi_tcp(F) libiscsi(F) scsi_transport_iscsi(F) openvswitch(OF) vesafb(F) gpio_ich x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_intel(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) dm_multipath(F) microcode(F) scsi_dh(F) sb_edac edac_core joydev(F) mei_me mei lpc_ich ioatdma mac_hid lp(F) parport(F) dm_snapshot(F) hid_generic igb ixgbe usbhid isci i2c_algo_bit hid dca nbd(F) ahci(F) libsas(F) libahci(F) ptp(F) pps_core(F) mdio scsi_transport_sas(F)
[Mon Nov 18 17:23:02 2013] CPU: 19 PID: 15492 Comm: ip Tainted: GF O 3.11.0-12-generic #19-Ubuntu
[Mon Nov 18 17:23:02 2013] Hardware name: Supermicro X9DRFR/X9DRFR, BIOS 1.0a 09/11/2012
[Mon Nov 18 17:23:02 2013] task: ffff8810516f9770 ti: ffff88103e7ca000 task.ti: ffff88103e7ca000
[Mon Nov 18 17:23:02 2013] RIP: 0010:[<ffffffff811d77d4>] [<ffffffff811d77d4>] _submit_bh+0x184/0x210
[Mon Nov 18 17:23:02 2013] RSP: 0018:ffff88103e7cbc98 EFLAGS: 00010246
[Mon Nov 18 17:23:02 2013] RAX: 0000000000000005 RBX: ffff880701165820 RCX: ffff88103e7cbfd8
[Mon Nov 18 17:23:02 2013] RDX: 0000000000000000 RSI: ffff880701165820 RDI: 0000000000000411
[Mon Nov 18 17:23:02 2013] nbd15: unknown partition table
[Mon Nov 18 17:23:02 2013] RBP: ffff88103e7cbcb8 R08: 0000000000000000 R09: 0000000000000002
[Mon Nov 18 17:23:02 2013] R10: ffff880701165888 R11: ffffffff8128badd R12: 0000000000000411
[Mon Nov 18 17:23:02 2013] kjournald starting. Commit interval 5 seconds
[Mon Nov 18 17:23:02 2013] R13: ffff880802fef400 R14: ffff880eff6b4180 R15: 00007fff345eb930
[Mon Nov 18 17:23:02 2013] EXT3-fs (nbd15): using internal journal
[Mon Nov 18 17:23:02 2013] EXT3-fs (nbd15): mounted filesystem with ordered data mode
[Mon Nov 18 17:23:02 2013] FS: 00007f3a9ddf1740(0000) GS:ffff88107fce0000(0000) knlGS:0000000000000000
[Mon Nov 18 17:23:02 2013] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Mon Nov 18 17:23:02 2013] CR2: 00007fd38a321be0 CR3: 0000000001c0e000 CR4: 00000000000407f0
[Mon Nov 18 17:23:02 2013] Stack:
[Mon Nov 18 17:23:02 2013] ffff880701165820 0000000000000411 ffff880802fef400 ffff880eff6b4180
[Mon Nov 18 17:23:02 2013] ffff88103e7cbcd8 ffffffff811d84a5 ffff880701165820 ffff88085422c000
[Mon Nov 18 17:23:02 2013] ffff88103e7cbce8 ffffffff811d8543 ffff88103e7cbd10 ffffffff812310d0
[Mon Nov 18 17:23:02 2013] Call Trace:
[Mon Nov 18 17:23:02 2013] [<ffffffff811d84a5>] __sync_dirty_buffer+0x55/0xe0
[Mon Nov 18 17:23:02 2013] [<ffffffff811d8543>] sync_dirty_buffer+0x13/0x20
[Mon Nov 18 17:23:02 2013] [<ffffffff812310d0>] ext3_commit_super.constprop.89+0x60/0xf0
[Mon Nov 18 17:23:02 2013] [<ffffffff81231b21>] ext3_put_super+0x271/0x2a0
[Mon Nov 18 17:23:02 2013] [<ffffffff811a8eb6>] generic_shutdown_super+0x56/0xe0
[Mon Nov 18 17:23:02 2013] [<ffffffff811a9177>] kill_block_super+0x27/0x70
[Mon Nov 18 17:23:02 2013] [<ffffffff811a94ed>] deactivate_locked_super+0x3d/0x60
[Mon Nov 18 17:23:02 2013] [<ffffffff811a9556>] deactivate_super+0x46/0x60
[Mon Nov 18 17:23:02 2013] [<ffffffff811c549f>] mntput_no_expire+0xef/0x150
[Mon Nov 18 17:23:02 2013] [<ffffffff811c5526>] mntput+0x26/0x40
[Mon Nov 18 17:23:02 2013] [<ffffffff811c563d>] namespace_unlock+0xfd/0x110
[Mon Nov 18 17:23:02 2013] [<ffffffff811c7d4b>] put_mnt_ns+0x4b/0x60
[Mon Nov 18 17:23:02 2013] [<ffffffff810891ab>] free_nsproxy+0x1b/0x90
[Mon Nov 18 17:23:02 2013] [<ffffffff810893d5>] switch_task_namespaces+0x45/0x50
[Mon Nov 18 17:23:02 2013] [<ffffffff810893f0>] exit_task_namespaces+0x10/0x20
[Mon Nov 18 17:23:02 2013] [<ffffffff81064222>] do_exit+0x2b2/0xa40
[Mon Nov 18 17:23:02 2013] [<ffffffff81064a2f>] do_group_exit+0x3f/0xa0
[Mon Nov 18 17:23:02 2013] [<ffffffff81064aa4>] SyS_exit_group+0x14/0x20
[Mon Nov 18 17:23:02 2013] [<ffffffff816f521d>] system_call_fastpath+0x1a/0x1f
[Mon Nov 18 17:23:02 2013] Code: 52 00 00 89 d8 5b 41 5c 41 5d 41 5e 5d c3 66 0f 1f 84 00 00 00 00 00 40 f6 c7 01 0f 84 e7 fe ff ff f0 80 66 01 f7 e9 dd fe ff ff <0f> 0b 49 8b 54 24 70 c1 e0 09 41 f6 c5 01 41 89 44 24 30 89 42
[Mon Nov 18 17:23:02 2013] RIP [<ffffffff811d77d4>] _submit_bh+0x184/0x210
[Mon Nov 18 17:23:02 2013] RSP <ffff88103e7cbc98>
[Mon Nov 18 17:23:02 2013] ---[ end trace 5fa4e7d1e416ec13 ]---
% uname -a
Linux node-0025904feb5c 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
% cat /proc/version_signature
Ubuntu 3.11.0-12.19-generic 3.11.3 |
I've hit the same BUG twice in as many working days. I'll give a brief description of what I'm doing, paste the two BUG reports, then paste the requested information.
I've been doing some performance testing on OpenStack. The tests just involve booting and deleting a bunch of virtual machine instances. A couple of things happen during VM creation that seem to cause the bug on. First, tap devices are created in various network namespaces. Second, nbd devices, served locally by qemu-nbd, with ext3 data are connected to /dev/nbd* and mounted and fiddled with.
The ip program is used to operate in the network namespaces (ip netns exec ...). Occasionally, when ip exits, it hits a BUG in some block layer code. Apparently the BUGging ip task had the last reference to a mount.
This BUG is fairly nasty because it happens when the superblock's s_umount semaphore is held, so other tasks that try to do stuff with the mount deadlock.
Here are the processes involved in today's BUG:
15488 sudo ip netns exec qdhcp-1ca153f0-3ece-4574-a0af-317c21cce5b4 ip route list dev tap4fad7e3b-f4
15492 \_ [ip]
27607 /bin/mount /dev/nbd13 /tmp/openstack-vfs-localfsKV3wPV
27592 /usr/bin/qemu-nbd -c /dev/nbd13 /opt/stack/data/nova/instances/4d46d4f9-a812-4e84-8659-b30e8e01dd27/disk
% sudo cat /proc/27607/stack
[<ffffffff811df193>] __blkdev_get+0x63/0x4c0
[<ffffffff811df7b5>] blkdev_get+0x1c5/0x340
[<ffffffff811df9db>] blkdev_open+0x5b/0x80
[<ffffffff811a4cab>] do_dentry_open+0x1eb/0x280
[<ffffffff811a4dc9>] vfs_open+0x49/0x50
[<ffffffff811b4111>] do_last+0x1c1/0xe40
[<ffffffff811b5981>] path_openat+0xc1/0x630
[<ffffffff811b6cfa>] do_filp_open+0x3a/0x90
[<ffffffff811a5fac>] do_sys_open+0x12c/0x270
[<ffffffff811a610e>] SyS_open+0x1e/0x20
[<ffffffff816f521d>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
% sudo cat /proc/27592/task/27595/stack
[<ffffffff81369714>] call_rwsem_down_read_failed+0x14/0x30
[<ffffffff811a99c5>] get_super+0x95/0xe0
[<ffffffff811dee64>] fsync_bdev+0x14/0x60
[<ffffffff81340824>] invalidate_partition+0x24/0x50
[<ffffffff81343a19>] rescan_partitions+0x49/0x2c0
[<ffffffff811df5e2>] __blkdev_get+0x4b2/0x4c0
[<ffffffff811df7b5>] blkdev_get+0x1c5/0x340
[<ffffffff811df9db>] blkdev_open+0x5b/0x80
[<ffffffff811a4cab>] do_dentry_open+0x1eb/0x280
[<ffffffff811a4dc9>] vfs_open+0x49/0x50
[<ffffffff811b4111>] do_last+0x1c1/0xe40
[<ffffffff811b5981>] path_openat+0xc1/0x630
[<ffffffff811b6cfa>] do_filp_open+0x3a/0x90
[<ffffffff811a5fac>] do_sys_open+0x12c/0x270
[<ffffffff811a610e>] SyS_open+0x1e/0x20
[<ffffffff816f521d>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
I'm running on the stock ubuntu kernel; no custom modules have been inserted.
This is the first BUG cut from a few days ago. I don't have the entire dmesg log from this run:
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131201] ------------[ cut here ]------------
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131469] Kernel BUG at ffffffff811d77d4 [verbose debug info unavailable]
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131881] invalid opcode: 0000 [#1] SMP
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.132148] Modules linked in: xt_LOG(F) vhost_net vhost macvtap(F) macvlan(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) ebt_arp(F) ebt_ip(F)
xt_mac(F) xt_physdev(F) veth(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
(F) openvswitch(OF) dm_multipath(F) scsi_dh(F) nbd(F) vesafb(F) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_inte
l(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) gpio_ich microcode(F) joydev(F) sb_edac edac_core mei_me mei lpc_ich ioatdma mac_hid lp(F) p
arport(F) dm_snapshot(F) hid_generic igb isci ixgbe i2c_algo_bit usbhid dca hid libsas(F) ahci(F) ptp(F) libahci(
Nov 15 14:52:44 node-0025904feb5c kernel: F) pps_core(F) mdio scsi_transport_sas(F)
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.138029] CPU: 15 PID: 27942 Comm: ip Tainted: GF O 3.11.0-12-generic #19-Ubuntu
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131881] invalid opcode: 0000 [#1] SMP
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.132148] Modules linked in: xt_LOG(F) vhost_net vhost macvtap(F) macvlan(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) ebt_arp(F) ebt_ip(F)
xt_mac(F) xt_physdev(F) veth(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
(F) openvswitch(OF) dm_multipath(F) scsi_dh(F) nbd(F) vesafb(F) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_inte
l(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) gpio_ich microcode(F) joydev(F) sb_edac edac_core mei_me mei lpc_ich ioatdma mac_hid lp(F) p
arport(F) dm_snapshot(F) hid_generic igb isci ixgbe i2c_algo_bit usbhid dca hid libsas(F) ahci(F) ptp(F) libahci(
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131881] invalid opcode: 0000 [#1] SMP
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.132148] Modules linked in: xt_LOG(F) vhost_net vhost macvtap(F) macvlan(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) ebt_arp(F) ebt_ip(F)
xt_mac(F) xt_physdev(F) veth(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
(F) openvswitch(OF) dm_multipath(F) scsi_dh(F) nbd(F) vesafb(F) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_inte
l(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) gpio_ich microcode(F) joydev(F) sb_edac edac_core mei_me mei lpc_ich ioatdma mac_hid lp(F) p
arport(F) dm_snapshot(F) hid_generic igb isci ixgbe i2c_algo_bit usbhid dca hid libsas(F) ahci(F) ptp(F) libahci(
Nov 15 14:52:44 node-0025904feb5c kernel: F) pps_core(F) mdio scsi_transport_sas(F)
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.138029] CPU: 15 PID: 27942 Comm: ip Tainted: GF O 3.11.0-12-generic #19-Ubuntu
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.131881] invalid opcode: 0000 [#1] SMP
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.132148] Modules linked in: xt_LOG(F) vhost_net vhost macvtap(F) macvlan(F) nf_conntrack_ipv6(F) nf_defrag_ipv6(F) ebt_arp(F) ebt_ip(F)
xt_mac(F) xt_physdev(F) veth(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F
) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables
(F) openvswitch(OF) dm_multipath(F) scsi_dh(F) nbd(F) vesafb(F) x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_inte
l(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) gpio_ich microcode(F) joydev(F) sb_edac edac_core mei_me mei lpc_ich ioatdma mac_hid lp(F) p
arport(F) dm_snapshot(F) hid_generic igb isci ixgbe i2c_algo_bit usbhid dca hid libsas(F) ahci(F) ptp(F) libahci(
Nov 15 14:52:44 node-0025904feb5c kernel: F) pps_core(F) mdio scsi_transport_sas(F)
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.138029] CPU: 15 PID: 27942 Comm: ip Tainted: GF O 3.11.0-12-generic #19-Ubuntu
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.157795] Hardware name: Supermicro X9DRFR/X9DRFR, BIOS 1.0a 09/11/2012
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.167959] task: ffff88053c631770 ti: ffff88060d788000 task.ti: ffff88060d788000
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.189055] RIP: 0010:[<ffffffff811d77d4>] [<ffffffff811d77d4>] _submit_bh+0x184/0x210
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.209728] RSP: 0018:ffff88060d789c98 EFLAGS: 00010246
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.220248] RAX: 0000000000000005 RBX: ffff880772e019c0 RCX: ffff88060d789fd8
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.241884] RDX: 0000000000000000 RSI: ffff880772e019c0 RDI: 0000000000000411
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.263641] RBP: ffff88060d789cb8 R08: 0000000000000000 R09: 0000000000000002
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.286483] R10: ffff880772e01068 R11: ffffffff8128badd R12: 0000000000000411
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.309833] R13: ffff8807a6499400 R14: ffff88076b528000 R15: 00007fff6b62dc00
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.333067] FS: 00007fb0e72c2740(0000) GS:ffff88085fd20000(0000) knlGS:0000000000000000
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.356632] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.368780] CR2: 00007fb1a1025f50 CR3: 00000007be808000 CR4: 00000000000427e0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.384379] nbd9: unknown partition table
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.403462] Stack:
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.414237] ffff880772e019c0 0000000000000411 ffff8807a6499400 ffff88076b528000
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.436155] ffff88060d789cd8 ffffffff811d84a5 ffff880772e019c0 ffff88060155e400
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.457077] kjournald starting. Commit interval 5 seconds
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.480247] EXT3-fs (nbd9): using internal journal
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.480251] EXT3-fs (nbd9): mounted filesystem with ordered data mode
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.469329] ffff88060d789ce8 ffffffff811d8543 ffff88060d789d10 ffffffff812310d0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.513658] Call Trace:
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.524287] [<ffffffff811d84a5>] __sync_dirty_buffer+0x55/0xe0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.534920] [<ffffffff811d8543>] sync_dirty_buffer+0x13/0x20
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.545623] [<ffffffff812310d0>] ext3_commit_super.constprop.89+0x60/0xf0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.556287] [<ffffffff81231b21>] ext3_put_super+0x271/0x2a0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.566461] [<ffffffff811a8eb6>] generic_shutdown_super+0x56/0xe0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.576653] [<ffffffff811a9177>] kill_block_super+0x27/0x70
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.586668] [<ffffffff811a94ed>] deactivate_locked_super+0x3d/0x60
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.596335] [<ffffffff811a9556>] deactivate_super+0x46/0x60
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.606253] [<ffffffff811c549f>] mntput_no_expire+0xef/0x150
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.615477] [<ffffffff811c5526>] mntput+0x26/0x40
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.624622] [<ffffffff811c563d>] namespace_unlock+0xfd/0x110
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.634156] [<ffffffff811c7d4b>] put_mnt_ns+0x4b/0x60
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.643059] [<ffffffff810891ab>] free_nsproxy+0x1b/0x90
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.651554] [<ffffffff810893d5>] switch_task_namespaces+0x45/0x50
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.659768] [<ffffffff810893f0>] exit_task_namespaces+0x10/0x20
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.667855] [<ffffffff81064222>] do_exit+0x2b2/0xa40
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.676050] [<ffffffff81064a2f>] do_group_exit+0x3f/0xa0
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.685242] [<ffffffff81064aa4>] SyS_exit_group+0x14/0x20
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.693422] [<ffffffff816f521d>] system_call_fastpath+0x1a/0x1f
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.701737] Code: 52 00 00 89 d8 5b 41 5c 41 5d 41 5e 5d c3 66 0f 1f 84 00 00 00 00 00 40 f6 c7 01 0f 84 e7 fe ff ff f0 80 66 01 f7 e9 dd fe ff ff <0f> 0b 49 8b 54 24 70 c1 e0 09 41 f6 c5 01 41 89 44 24 30 89 42
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.725473] RIP [<ffffffff811d77d4>] _submit_bh+0x184/0x210
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.733137] RSP <ffff88060d789c98>
Nov 15 14:52:44 node-0025904feb5c kernel: [1902861.753260] ---[ end trace 79cbd7bd94cca2ea ]---
This is the second BUG cut from today. The full dmesg is attached.
[Mon Nov 18 17:23:02 2013] ------------[ cut here ]------------
[Mon Nov 18 17:23:02 2013] Kernel BUG at ffffffff811d77d4 [verbose debug info unavailable]
[Mon Nov 18 17:23:02 2013] invalid opcode: 0000 [#1] SMP
[Mon Nov 18 17:23:02 2013] Modules linked in: nf_conntrack_ipv6(F) nf_defrag_ipv6(F) xt_mac(F) xt_physdev(F) vhost_net vhost macvtap(F) macvlan(F) xt_nat(F) xt_REDIRECT(F) ipt_MASQUERADE(F) iptable_nat(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) xt_conntrack(F) nf_conntrack(F) ipt_REJECT(F) xt_CHECKSUM(F) iptable_mangle(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) ebtables(F) x_tables(F) ib_iser rdma_cm(F) ib_addr(F) iw_cm(F) ib_cm(F) ib_sa(F) ib_mad(F) ib_core(F) iscsi_tcp(F) libiscsi_tcp(F) libiscsi(F) scsi_transport_iscsi(F) openvswitch(OF) vesafb(F) gpio_ich x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) ghash_clmulni_intel(F) aesni_intel(F) aes_x86_64(F) lrw(F) gf128mul(F) glue_helper(F) ablk_helper(F) cryptd(F) dm_multipath(F) microcode(F) scsi_dh(F) sb_edac edac_core joydev(F) mei_me mei lpc_ich ioatdma mac_hid lp(F) parport(F) dm_snapshot(F) hid_generic igb ixgbe usbhid isci i2c_algo_bit hid dca nbd(F) ahci(F) libsas(F) libahci(F) ptp(F) pps_core(F) mdio scsi_transport_sas(F)
[Mon Nov 18 17:23:02 2013] CPU: 19 PID: 15492 Comm: ip Tainted: GF O 3.11.0-12-generic #19-Ubuntu
[Mon Nov 18 17:23:02 2013] Hardware name: Supermicro X9DRFR/X9DRFR, BIOS 1.0a 09/11/2012
[Mon Nov 18 17:23:02 2013] task: ffff8810516f9770 ti: ffff88103e7ca000 task.ti: ffff88103e7ca000
[Mon Nov 18 17:23:02 2013] RIP: 0010:[<ffffffff811d77d4>] [<ffffffff811d77d4>] _submit_bh+0x184/0x210
[Mon Nov 18 17:23:02 2013] RSP: 0018:ffff88103e7cbc98 EFLAGS: 00010246
[Mon Nov 18 17:23:02 2013] RAX: 0000000000000005 RBX: ffff880701165820 RCX: ffff88103e7cbfd8
[Mon Nov 18 17:23:02 2013] RDX: 0000000000000000 RSI: ffff880701165820 RDI: 0000000000000411
[Mon Nov 18 17:23:02 2013] nbd15: unknown partition table
[Mon Nov 18 17:23:02 2013] RBP: ffff88103e7cbcb8 R08: 0000000000000000 R09: 0000000000000002
[Mon Nov 18 17:23:02 2013] R10: ffff880701165888 R11: ffffffff8128badd R12: 0000000000000411
[Mon Nov 18 17:23:02 2013] kjournald starting. Commit interval 5 seconds
[Mon Nov 18 17:23:02 2013] R13: ffff880802fef400 R14: ffff880eff6b4180 R15: 00007fff345eb930
[Mon Nov 18 17:23:02 2013] EXT3-fs (nbd15): using internal journal
[Mon Nov 18 17:23:02 2013] EXT3-fs (nbd15): mounted filesystem with ordered data mode
[Mon Nov 18 17:23:02 2013] FS: 00007f3a9ddf1740(0000) GS:ffff88107fce0000(0000) knlGS:0000000000000000
[Mon Nov 18 17:23:02 2013] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Mon Nov 18 17:23:02 2013] CR2: 00007fd38a321be0 CR3: 0000000001c0e000 CR4: 00000000000407f0
[Mon Nov 18 17:23:02 2013] Stack:
[Mon Nov 18 17:23:02 2013] ffff880701165820 0000000000000411 ffff880802fef400 ffff880eff6b4180
[Mon Nov 18 17:23:02 2013] ffff88103e7cbcd8 ffffffff811d84a5 ffff880701165820 ffff88085422c000
[Mon Nov 18 17:23:02 2013] ffff88103e7cbce8 ffffffff811d8543 ffff88103e7cbd10 ffffffff812310d0
[Mon Nov 18 17:23:02 2013] Call Trace:
[Mon Nov 18 17:23:02 2013] [<ffffffff811d84a5>] __sync_dirty_buffer+0x55/0xe0
[Mon Nov 18 17:23:02 2013] [<ffffffff811d8543>] sync_dirty_buffer+0x13/0x20
[Mon Nov 18 17:23:02 2013] [<ffffffff812310d0>] ext3_commit_super.constprop.89+0x60/0xf0
[Mon Nov 18 17:23:02 2013] [<ffffffff81231b21>] ext3_put_super+0x271/0x2a0
[Mon Nov 18 17:23:02 2013] [<ffffffff811a8eb6>] generic_shutdown_super+0x56/0xe0
[Mon Nov 18 17:23:02 2013] [<ffffffff811a9177>] kill_block_super+0x27/0x70
[Mon Nov 18 17:23:02 2013] [<ffffffff811a94ed>] deactivate_locked_super+0x3d/0x60
[Mon Nov 18 17:23:02 2013] [<ffffffff811a9556>] deactivate_super+0x46/0x60
[Mon Nov 18 17:23:02 2013] [<ffffffff811c549f>] mntput_no_expire+0xef/0x150
[Mon Nov 18 17:23:02 2013] [<ffffffff811c5526>] mntput+0x26/0x40
[Mon Nov 18 17:23:02 2013] [<ffffffff811c563d>] namespace_unlock+0xfd/0x110
[Mon Nov 18 17:23:02 2013] [<ffffffff811c7d4b>] put_mnt_ns+0x4b/0x60
[Mon Nov 18 17:23:02 2013] [<ffffffff810891ab>] free_nsproxy+0x1b/0x90
[Mon Nov 18 17:23:02 2013] [<ffffffff810893d5>] switch_task_namespaces+0x45/0x50
[Mon Nov 18 17:23:02 2013] [<ffffffff810893f0>] exit_task_namespaces+0x10/0x20
[Mon Nov 18 17:23:02 2013] [<ffffffff81064222>] do_exit+0x2b2/0xa40
[Mon Nov 18 17:23:02 2013] [<ffffffff81064a2f>] do_group_exit+0x3f/0xa0
[Mon Nov 18 17:23:02 2013] [<ffffffff81064aa4>] SyS_exit_group+0x14/0x20
[Mon Nov 18 17:23:02 2013] [<ffffffff816f521d>] system_call_fastpath+0x1a/0x1f
[Mon Nov 18 17:23:02 2013] Code: 52 00 00 89 d8 5b 41 5c 41 5d 41 5e 5d c3 66 0f 1f 84 00 00 00 00 00 40 f6 c7 01 0f 84 e7 fe ff ff f0 80 66 01 f7 e9 dd fe ff ff <0f> 0b 49 8b 54 24 70 c1 e0 09 41 f6 c5 01 41 89 44 24 30 89 42
[Mon Nov 18 17:23:02 2013] RIP [<ffffffff811d77d4>] _submit_bh+0x184/0x210
[Mon Nov 18 17:23:02 2013] RSP <ffff88103e7cbc98>
[Mon Nov 18 17:23:02 2013] ---[ end trace 5fa4e7d1e416ec13 ]---
% uname -a
Linux node-0025904feb5c 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
% cat /proc/version_signature
Ubuntu 3.11.0-12.19-generic 3.11.3
---
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Nov 15 15:56 seq
crw-rw---- 1 root audio 116, 33 Nov 15 15:56 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.12.5-0ubuntu2.1
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
DistroRelease: Ubuntu 13.10
HibernationDevice: RESUME=UUID=b96acf9b-2873-4ce1-92ca-7f18b74019d2
IwConfig: Error: [Errno 2] No such file or directory
Lsusb:
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0557:2221 ATEN International Co., Ltd Winbond Hermon
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: Supermicro X9DRFR
MarkForUpload: True
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
LANGUAGE=en_CA:en
TERM=xterm-256color
PATH=(custom, no user)
LANG=en_CA.UTF-8
SHELL=/usr/bin/zsh
ProcFB: 0 VESA VGA
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.11.0-12-generic root=UUID=6ebf6389-4dd0-4580-8ebb-b4b4fe8f3a02 ro splash quiet vt.handoff=7
ProcVersionSignature: Ubuntu 3.11.0-12.19-generic 3.11.3
RelatedPackageVersions:
linux-restricted-modules-3.11.0-12-generic N/A
linux-backports-modules-3.11.0-12-generic N/A
linux-firmware 1.116
RfKill: Error: [Errno 2] No such file or directory
Tags: saucy
Uname: Linux 3.11.0-12-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
dmi.bios.date: 09/11/2012
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 1.0a
dmi.board.asset.tag: To be filled by O.E.M.
dmi.board.name: X9DRFR
dmi.board.vendor: Supermicro
dmi.board.version: 0123456789
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 3
dmi.chassis.vendor: Supermicro
dmi.chassis.version: 0123456789
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr1.0a:bd09/11/2012:svnSupermicro:pnX9DRFR:pvr0123456789:rvnSupermicro:rnX9DRFR:rvr0123456789:cvnSupermicro:ct3:cvr0123456789:
dmi.product.name: X9DRFR
dmi.product.version: 0123456789
dmi.sys.vendor: Supermicro |
|
| 2013-11-19 14:38:37 |
Peter Feiner |
attachment added |
|
BootDmesg.txt https://bugs.launchpad.net/bugs/1252508/+attachment/3912650/+files/BootDmesg.txt |
|
| 2013-11-19 14:38:39 |
Peter Feiner |
attachment added |
|
CurrentDmesg.txt https://bugs.launchpad.net/bugs/1252508/+attachment/3912651/+files/CurrentDmesg.txt |
|
| 2013-11-19 14:38:40 |
Peter Feiner |
attachment added |
|
Lspci.txt https://bugs.launchpad.net/bugs/1252508/+attachment/3912652/+files/Lspci.txt |
|
| 2013-11-19 14:38:42 |
Peter Feiner |
attachment added |
|
ProcCpuinfo.txt https://bugs.launchpad.net/bugs/1252508/+attachment/3912653/+files/ProcCpuinfo.txt |
|
| 2013-11-19 14:38:44 |
Peter Feiner |
attachment added |
|
ProcInterrupts.txt https://bugs.launchpad.net/bugs/1252508/+attachment/3912654/+files/ProcInterrupts.txt |
|
| 2013-11-19 14:38:46 |
Peter Feiner |
attachment added |
|
ProcModules.txt https://bugs.launchpad.net/bugs/1252508/+attachment/3912655/+files/ProcModules.txt |
|
| 2013-11-19 14:38:48 |
Peter Feiner |
attachment added |
|
UdevDb.txt https://bugs.launchpad.net/bugs/1252508/+attachment/3912656/+files/UdevDb.txt |
|
| 2013-11-19 14:38:50 |
Peter Feiner |
attachment added |
|
UdevLog.txt https://bugs.launchpad.net/bugs/1252508/+attachment/3912657/+files/UdevLog.txt |
|
| 2013-11-19 14:38:52 |
Peter Feiner |
attachment added |
|
WifiSyslog.txt https://bugs.launchpad.net/bugs/1252508/+attachment/3912658/+files/WifiSyslog.txt |
|
| 2013-11-19 14:41:14 |
Peter Feiner |
linux (Ubuntu): status |
Incomplete |
Confirmed |
|
| 2013-11-19 19:56:01 |
Joseph Salisbury |
linux (Ubuntu): importance |
Undecided |
High |
|
| 2013-11-19 19:57:00 |
Joseph Salisbury |
tags |
apport-collected saucy |
apport-collected kernel-da-key saucy |
|
| 2013-11-21 00:10:56 |
Peter Feiner |
attachment added |
|
dmesg.log https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1252508/+attachment/3913632/+files/dmesg.log |
|
| 2013-11-21 00:11:22 |
Peter Feiner |
tags |
apport-collected kernel-da-key saucy |
apport-collected kernel-bug-exists-upstream kernel-da-key saucy |
|
