Oct 26 06:43:55 stack01 kernel: [ 9329.215317] init: tty1 main process ended, respawning
Oct 26 06:43:55 stack01 kernel: [ 9329.222202] init: tty2 main process ended, respawning
Oct 26 08:16:22 stack01 kernel: [14876.125055] general protection fault: 0000 [#1] SMP 
Oct 26 08:16:22 stack01 kernel: [14876.125362] CPU 2 
Oct 26 08:16:22 stack01 kernel: [14876.125489] Modules linked in: ebt_arp ebt_ip 8021q garp ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iscsi_trgt ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_amd kvm vesafb ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nbd amd64_edac_mod edac_core psmouse sp5100_tco serio_raw dm_multipath ghes joydev hed i2c_piix4 k10temp edac_mce_amd lp parport raid10 raid456 async_pq async_xor usbhid hid xor async_memcpy async_raid6_recov pata_atiixp ahci igb libahci dca raid6_pq async_tx raid1 raid0 multipath linear
Oct 26 08:16:22 stack01 kernel: [14876.128706] 
Oct 26 08:16:22 stack01 kernel: [14876.128706] Pid: 23247, comm: bash Not tainted 3.0.0-12-server #20-Ubuntu HP ProLiant DL165 G7
Oct 26 08:16:22 stack01 kernel: [14876.128706] RIP: 0010:[<ffffffff81183b30>]  [<ffffffff81183b30>] dup_fd+0x150/0x2f0
Oct 26 08:16:22 stack01 kernel: [14876.128706] RSP: 0018:ffff8801367b9d80  EFLAGS: 00010206
Oct 26 08:16:22 stack01 kernel: [14876.128706] RAX: 00000000000000ff RBX: ffff88031de83000 RCX: bfffffffffffffff
Oct 26 08:16:22 stack01 kernel: [14876.128706] RDX: 00008802b808f800 RSI: 0000000000000100 RDI: ffff88027a14b000
Oct 26 08:16:22 stack01 kernel: [14876.128706] RBP: ffff8801367b9de0 R08: 0000000000000003 R09: 0000000000000001
Oct 26 08:16:22 stack01 kernel: [14876.128706] R10: 4000000000000000 R11: 0000000000015cf0 R12: ffff880135840840
Oct 26 08:16:22 stack01 kernel: [14876.128706] R13: ffff88027a14b6c0 R14: ffff88027a94c000 R15: 0000000000000100
Oct 26 08:16:22 stack01 kernel: [14876.128706] FS:  00007ffde7eb3720(0000) GS:ffff88013fc40000(0000) knlGS:0000000000000000
Oct 26 08:16:22 stack01 kernel: [14876.128706] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 26 08:16:22 stack01 kernel: [14876.128706] CR2: 0000000000a460a8 CR3: 000000031e265000 CR4: 00000000000006e0
Oct 26 08:16:22 stack01 kernel: [14876.128706] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 26 08:16:22 stack01 kernel: [14876.128706] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 26 08:16:22 stack01 kernel: [14876.128706] Process bash (pid: 23247, threadinfo ffff8801367b8000, task ffff880135b49720)
Oct 26 08:16:22 stack01 kernel: [14876.128706] Stack:
Oct 26 08:16:22 stack01 kernel: [14876.128706]  ffff8801367b9d90 00000001375f4560 ffff88027a14b180 0000000000000020
Oct 26 08:16:22 stack01 kernel: [14876.128706]  ffff88027a14b6c0 ffff880138bbac80 0000000000000000 ffff8801375f4560
Oct 26 08:16:22 stack01 kernel: [14876.128706]  0000000001200011 00007ffde7eb39f0 0000000000000000 ffff8801367b9f58
Oct 26 08:16:22 stack01 kernel: [14876.128706] Call Trace:
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff8105d0d6>] copy_process.part.23+0x656/0xe00
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff8115352c>] ? kmem_cache_alloc_trace+0xfc/0x120
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff8105d8f7>] copy_process+0x77/0x80
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff8105da0e>] do_fork+0xbe/0x330
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff81183dc7>] ? alloc_fd+0xf7/0x150
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff815fe82e>] ? _raw_spin_lock+0xe/0x20
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff811658e1>] ? fd_install+0x61/0x80
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff81171353>] ? do_pipe_flags+0xc3/0x120
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff81012b58>] sys_clone+0x28/0x30
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff81606f23>] stub_clone+0x13/0x20
Oct 26 08:16:22 stack01 kernel: [14876.128706]  [<ffffffff81606c02>] ? system_call_fastpath+0x16/0x1b
Oct 26 08:16:22 stack01 kernel: [14876.128706] Code: 8b 45 b8 49 8b 7d 10 48 8b 71 10 4c 89 c2 e8 c8 ef 16 00 45 85 ff 74 6c 41 8d 77 ff 31 c0 41 b9 01 00 00 00 48 83 c6 01 eb 13 90 <f0> 48 ff 42 30 48 89 14 c3 48 83 c0 01 48 39 f0 74 3e 49 8b 14 
Oct 26 08:16:22 stack01 kernel: [14876.128706] RIP  [<ffffffff81183b30>] dup_fd+0x150/0x2f0
Oct 26 08:16:22 stack01 kernel: [14876.128706]  RSP <ffff8801367b9d80>
Oct 26 08:16:22 stack01 kernel: [14876.179023] ---[ end trace 3db970a3e7847d7a ]---
Oct 26 08:16:23 stack01 kernel: [14876.180538] general protection fault: 0000 [#2] SMP 
Oct 26 08:16:23 stack01 kernel: [14876.184008] CPU 2 
Oct 26 08:16:23 stack01 kernel: [14876.184008] Modules linked in: ebt_arp ebt_ip 8021q garp ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iscsi_trgt ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_amd kvm vesafb ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nbd amd64_edac_mod edac_core psmouse sp5100_tco serio_raw dm_multipath ghes joydev hed i2c_piix4 k10temp edac_mce_amd lp parport raid10 raid456 async_pq async_xor usbhid hid xor async_memcpy async_raid6_recov pata_atiixp ahci igb libahci dca raid6_pq async_tx raid1 raid0 multipath linear
Oct 26 08:16:23 stack01 kernel: [14876.184008] 
Oct 26 08:16:23 stack01 kernel: [14876.184008] Pid: 23247, comm: bash Tainted: G      D     3.0.0-12-server #20-Ubuntu HP ProLiant DL165 G7
Oct 26 08:16:23 stack01 kernel: [14876.184008] RIP: 0010:[<ffffffff811656e9>]  [<ffffffff811656e9>] filp_close+0x19/0x90
Oct 26 08:16:23 stack01 kernel: [14876.184008] RSP: 0018:ffff8801367b9b28  EFLAGS: 00010286
Oct 26 08:16:23 stack01 kernel: [14876.184008] RAX: ffff88027a94c7f8 RBX: 00000000000000c0 RCX: 0000000000000000
Oct 26 08:16:23 stack01 kernel: [14876.184008] RDX: ffff880138bbac00 RSI: ffff880138bbac00 RDI: 00008802b808f800
Oct 26 08:16:23 stack01 kernel: [14876.184008] RBP: ffff8801367b9b48 R08: ffffffff81132ce5 R09: 0000000000000000
Oct 26 08:16:23 stack01 kernel: [14876.184008] R10: ffff8802b80429d0 R11: 0000000000000006 R12: 0000000000000004
Oct 26 08:16:23 stack01 kernel: [14876.184008] R13: 0000000000000001 R14: ffff88027a14b180 R15: 00000000000000ff
Oct 26 08:16:23 stack01 kernel: [14876.184008] FS:  00007ffde7eb3720(0000) GS:ffff88013fc40000(0000) knlGS:0000000000000000
Oct 26 08:16:23 stack01 kernel: [14876.184008] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 26 08:16:23 stack01 kernel: [14876.184008] CR2: 0000000000a460a8 CR3: 0000000001c03000 CR4: 00000000000006e0
Oct 26 08:16:23 stack01 kernel: [14876.184008] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 26 08:16:23 stack01 kernel: [14876.184008] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 26 08:16:23 stack01 kernel: [14876.184008] Process bash (pid: 23247, threadinfo ffff8801367b8000, task ffff880135b49720)
Oct 26 08:16:23 stack01 kernel: [14876.184008] Stack:
Oct 26 08:16:23 stack01 kernel: [14876.184008]  ffffea00043349e8 00000000000000c0 0000000000000004 0000000000000001
Oct 26 08:16:23 stack01 kernel: [14876.184008]  ffff8801367b9b98 ffffffff81060f4a 0000000000000000 ffff880138bbac00
Oct 26 08:16:23 stack01 kernel: [14876.184008]  ffff880135b49d20 ffff880138bbac00 ffff880135b49d20 ffff880135b49720
Oct 26 08:16:23 stack01 kernel: [14876.184008] Call Trace:
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff81060f4a>] put_files_struct.part.14+0x7a/0xe0
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff810629a8>] put_files_struct+0x18/0x20
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff81062a74>] exit_files+0x54/0x70
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff81062f8d>] do_exit+0x19d/0x440
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff815ffcd0>] oops_end+0xb0/0xf0
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff8100d8c8>] die+0x58/0x90
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff815ff7c2>] do_general_protection+0x162/0x170
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff815fefe5>] general_protection+0x25/0x30
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff81183b30>] ? dup_fd+0x150/0x2f0
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff8105d0d6>] copy_process.part.23+0x656/0xe00
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff8115352c>] ? kmem_cache_alloc_trace+0xfc/0x120
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff8105d8f7>] copy_process+0x77/0x80
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff8105da0e>] do_fork+0xbe/0x330
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff81183dc7>] ? alloc_fd+0xf7/0x150
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff815fe82e>] ? _raw_spin_lock+0xe/0x20
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff811658e1>] ? fd_install+0x61/0x80
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff81171353>] ? do_pipe_flags+0xc3/0x120
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff81012b58>] sys_clone+0x28/0x30
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff81606f23>] stub_clone+0x13/0x20
Oct 26 08:16:23 stack01 kernel: [14876.184008]  [<ffffffff81606c02>] ? system_call_fastpath+0x16/0x1b
Oct 26 08:16:23 stack01 kernel: [14876.184008] Code: 00 00 00 00 e8 f9 e2 f8 ff 48 83 c4 18 5b 5d c3 66 90 55 48 89 e5 48 83 ec 20 48 89 5d e8 4c 89 65 f0 4c 89 6d f8 66 66 66 66 90 
Oct 26 08:16:23 stack01 kernel: [14876.184008] RIP  [<ffffffff811656e9>] filp_close+0x19/0x90
Oct 26 08:16:23 stack01 kernel: [14876.184008]  RSP <ffff8801367b9b28>
Oct 26 08:16:23 stack01 kernel: [14877.268472] ---[ end trace 3db970a3e7847d7b ]---
Oct 26 08:16:23 stack01 kernel: [14877.282526] Fixing recursive fault but reboot is needed!
Oct 26 11:31:32 stack01 kernel: [26585.854781] init: nova-api main process (20543) terminated with status 143
Oct 26 11:47:25 stack01 kernel: [27537.802098] general protection fault: 0000 [#3] SMP 
Oct 26 11:47:25 stack01 kernel: [27537.805708] CPU 1 
Oct 26 11:47:25 stack01 kernel: [27537.805708] Modules linked in: speedstep_lib ebt_arp ebt_ip 8021q garp ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iscsi_trgt ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_amd kvm vesafb ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nbd amd64_edac_mod edac_core psmouse sp5100_tco serio_raw dm_multipath ghes joydev hed i2c_piix4 k10temp edac_mce_amd lp parport raid10 raid456 async_pq async_xor usbhid hid xor async_memcpy async_raid6_recov pata_atiixp ahci igb libahci dca raid6_pq async_tx raid1 raid0 multipath linear
Oct 26 11:47:25 stack01 kernel: [27537.805708] 
Oct 26 11:47:25 stack01 kernel: [27537.805708] Pid: 11900, comm: bash Tainted: G      D     3.0.0-12-server #20-Ubuntu HP ProLiant DL165 G7
Oct 26 11:47:25 stack01 kernel: [27537.805708] RIP: 0010:[<ffffffff81183b30>]  [<ffffffff81183b30>] dup_fd+0x150/0x2f0
Oct 26 11:47:25 stack01 kernel: [27537.805708] RSP: 0018:ffff88024b385d80  EFLAGS: 00010206
Oct 26 11:47:25 stack01 kernel: [27537.805708] RAX: 00000000000000ff RBX: ffff880332672800 RCX: bfffffffffffffff
Oct 26 11:47:25 stack01 kernel: [27537.805708] RDX: 0000880327f455c0 RSI: 0000000000000100 RDI: ffff88027a3457c0
Oct 26 11:47:25 stack01 kernel: [27537.805708] RBP: ffff88024b385de0 R08: 0000000000000003 R09: 0000000000000001
Oct 26 11:47:25 stack01 kernel: [27537.805708] R10: 4000000000000000 R11: 0000000000015d58 R12: ffff880100013440
Oct 26 11:47:25 stack01 kernel: [27537.805708] R13: ffff88027a345d80 R14: ffff88024b1ad800 R15: 0000000000000100
Oct 26 11:47:25 stack01 kernel: [27537.805708] FS:  00007f58bea79720(0000) GS:ffff88013fc20000(0000) knlGS:0000000000000000
Oct 26 11:47:25 stack01 kernel: [27537.805708] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 26 11:47:25 stack01 kernel: [27537.805708] CR2: 0000000000d58108 CR3: 000000027a6fb000 CR4: 00000000000006e0
Oct 26 11:47:25 stack01 kernel: [27537.805708] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 26 11:47:25 stack01 kernel: [27537.805708] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 26 11:47:25 stack01 kernel: [27537.805708] Process bash (pid: 11900, threadinfo ffff88024b384000, task ffff880134924560)
Oct 26 11:47:25 stack01 kernel: [27537.805708] Stack:
Oct 26 11:47:25 stack01 kernel: [27537.805708]  ffff88024b385d90 0000000136e58000 ffff880279f47bc0 0000000000000020
Oct 26 11:47:25 stack01 kernel: [27537.805708]  ffff88027a345d80 ffff880135983200 0000000000000000 ffff880136e58000
Oct 26 11:47:25 stack01 kernel: [27537.805708]  0000000001200011 00007f58bea799f0 0000000000000000 ffff88024b385f58
Oct 26 11:47:25 stack01 kernel: [27537.805708] Call Trace:
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff8105d0d6>] copy_process.part.23+0x656/0xe00
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff8115352c>] ? kmem_cache_alloc_trace+0xfc/0x120
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff8105d8f7>] copy_process+0x77/0x80
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff8105da0e>] do_fork+0xbe/0x330
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff81183dc7>] ? alloc_fd+0xf7/0x150
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff815fe82e>] ? _raw_spin_lock+0xe/0x20
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff811658e1>] ? fd_install+0x61/0x80
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff81171353>] ? do_pipe_flags+0xc3/0x120
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff81012b58>] sys_clone+0x28/0x30
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff81606f23>] stub_clone+0x13/0x20
Oct 26 11:47:25 stack01 kernel: [27537.805708]  [<ffffffff81606c02>] ? system_call_fastpath+0x16/0x1b
Oct 26 11:47:25 stack01 kernel: [27537.805708] Code: 8b 45 b8 49 8b 7d 10 48 8b 71 10 4c 89 c2 e8 c8 ef 16 00 45 85 ff 74 6c 41 8d 77 ff 31 c0 41 b9 01 00 00 00 48 83 c6 01 eb 13 90 <f0> 48 ff 42 30 48 89 14 c3 48 83 c0 01 48 39 f0 74 3e 49 8b 14 
Oct 26 11:47:25 stack01 kernel: [27537.805708] RIP  [<ffffffff81183b30>] dup_fd+0x150/0x2f0
Oct 26 11:47:25 stack01 kernel: [27537.805708]  RSP <ffff88024b385d80>
Oct 26 11:47:25 stack01 kernel: [27538.574115] ---[ end trace 3db970a3e7847d7c ]---
Oct 26 11:47:26 stack01 kernel: [27538.593964] general protection fault: 0000 [#4] SMP 
Oct 26 11:47:26 stack01 kernel: [27538.596254] CPU 1 
Oct 26 11:47:26 stack01 kernel: [27538.596254] Modules linked in: speedstep_lib ebt_arp ebt_ip 8021q garp ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iscsi_trgt ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_amd kvm vesafb ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nbd amd64_edac_mod edac_core psmouse sp5100_tco serio_raw dm_multipath ghes joydev hed i2c_piix4 k10temp edac_mce_amd lp parport raid10 raid456 async_pq async_xor usbhid hid xor async_memcpy async_raid6_recov pata_atiixp ahci igb libahci dca raid6_pq async_tx raid1 raid0 multipath linear
Oct 26 11:47:26 stack01 kernel: [27538.596254] 
Oct 26 11:47:26 stack01 kernel: [27538.596254] Pid: 11900, comm: bash Tainted: G      D     3.0.0-12-server #20-Ubuntu HP ProLiant DL165 G7
Oct 26 11:47:26 stack01 kernel: [27538.596254] RIP: 0010:[<ffffffff811656e9>]  [<ffffffff811656e9>] filp_close+0x19/0x90
Oct 26 11:47:26 stack01 kernel: [27538.596254] RSP: 0018:ffff88024b385b28  EFLAGS: 00010286
Oct 26 11:47:26 stack01 kernel: [27538.596254] RAX: ffff88024b1adff8 RBX: 00000000000000c0 RCX: 0000000000000000
Oct 26 11:47:26 stack01 kernel: [27538.596254] RDX: ffff880135983180 RSI: ffff880135983180 RDI: 0000880327f455c0
Oct 26 11:47:26 stack01 kernel: [27538.596254] RBP: ffff88024b385b48 R08: ffffffff81132ce5 R09: 0000000000000000
Oct 26 11:47:26 stack01 kernel: [27538.596254] R10: ffff88033215acd0 R11: 0000000000000006 R12: 0000000000000004
Oct 26 11:47:26 stack01 kernel: [27538.596254] R13: 0000000000000001 R14: ffff880279f47bc0 R15: 00000000000000ff
Oct 26 11:47:26 stack01 kernel: [27538.596254] FS:  00007f58bea79720(0000) GS:ffff88013fc20000(0000) knlGS:0000000000000000
Oct 26 11:47:26 stack01 kernel: [27538.596254] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 26 11:47:26 stack01 kernel: [27538.596254] CR2: 0000000000d58108 CR3: 0000000001c03000 CR4: 00000000000006e0
Oct 26 11:47:26 stack01 kernel: [27538.596254] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 26 11:47:26 stack01 kernel: [27538.596254] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 26 11:47:26 stack01 kernel: [27538.596254] Process bash (pid: 11900, threadinfo ffff88024b384000, task ffff880134924560)
Oct 26 11:47:26 stack01 kernel: [27538.596254] Stack:
Oct 26 11:47:26 stack01 kernel: [27538.596254]  ffffea00043349e8 00000000000000c0 0000000000000004 0000000000000001
Oct 26 11:47:26 stack01 kernel: [27538.596254]  ffff88024b385b98 ffffffff81060f4a 0000000000000000 ffff880135983180
Oct 26 11:47:26 stack01 kernel: [27538.596254]  ffff880134924b60 ffff880135983180 ffff880134924b60 ffff880134924560
Oct 26 11:47:26 stack01 kernel: [27538.596254] Call Trace:
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff81060f4a>] put_files_struct.part.14+0x7a/0xe0
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff810629a8>] put_files_struct+0x18/0x20
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff81062a74>] exit_files+0x54/0x70
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff81062f8d>] do_exit+0x19d/0x440
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff815ffcd0>] oops_end+0xb0/0xf0
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff8100d8c8>] die+0x58/0x90
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff815ff7c2>] do_general_protection+0x162/0x170
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff815fefe5>] general_protection+0x25/0x30
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff81183b30>] ? dup_fd+0x150/0x2f0
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff8105d0d6>] copy_process.part.23+0x656/0xe00
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff8115352c>] ? kmem_cache_alloc_trace+0xfc/0x120
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff8105d8f7>] copy_process+0x77/0x80
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff8105da0e>] do_fork+0xbe/0x330
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff81183dc7>] ? alloc_fd+0xf7/0x150
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff815fe82e>] ? _raw_spin_lock+0xe/0x20
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff811658e1>] ? fd_install+0x61/0x80
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff81171353>] ? do_pipe_flags+0xc3/0x120
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff81012b58>] sys_clone+0x28/0x30
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff81606f23>] stub_clone+0x13/0x20
Oct 26 11:47:26 stack01 kernel: [27538.596254]  [<ffffffff81606c02>] ? system_call_fastpath+0x16/0x1b
Oct 26 11:47:26 stack01 kernel: [27538.596254] Code: 00 00 00 00 e8 f9 e2 f8 ff 48 83 c4 18 5b 5d c3 66 90 55 48 89 e5 48 83 ec 20 48 89 5d e8 4c 89 65 f0 4c 89 6d f8 66 66 66 66 90 
Oct 26 11:47:26 stack01 kernel: [27538.596254] RIP  [<ffffffff811656e9>] filp_close+0x19/0x90
Oct 26 11:47:26 stack01 kernel: [27538.596254]  RSP <ffff88024b385b28>
Oct 26 11:47:26 stack01 kernel: [27539.529870] ---[ end trace 3db970a3e7847d7d ]---
Oct 26 11:47:26 stack01 kernel: [27539.544119] Fixing recursive fault but reboot is needed!
Oct 26 11:49:38 stack01 kernel: [27672.040067] init: nova-api main process (14622) terminated with status 143
Oct 26 12:23:06 stack01 kernel: [29680.122851]  nbd15: p1
Oct 26 12:23:09 stack01 kernel: [29682.984224] EXT4-fs (dm-3): mounted filesystem with ordered data mode. Opts: (null)
Oct 26 12:23:11 stack01 kernel: [29685.344660] nbd15: NBD_DISCONNECT
Oct 26 12:23:11 stack01 kernel: [29685.358514] nbd15: Receive control failed (result -32)
Oct 26 12:23:12 stack01 kernel: [29685.372369] nbd15: queue cleared
Oct 26 12:23:13 stack01 kernel: [29686.599048] type=1400 audit(1319628193.211:33): apparmor="DENIED" operation="open" parent=1558 profile="/usr/lib/libvirt/virt-aa-helper" name="/var/lib/nova/instances/_base/77de68daecd823babbb58edb1c8e14d7106e83bb_sm" pid=28042 comm="virt-aa-helper" requested_mask="r" denied_mask="r" fsuid=0 ouid=107
Oct 26 12:23:13 stack01 kernel: [29687.122770] type=1400 audit(1319628193.735:34): apparmor="STATUS" operation="profile_load" name="libvirt-a1a6cd80-cadf-c9a7-069b-8c982119b50a" pid=28043 comm="apparmor_parser"
Oct 26 12:23:14 stack01 kernel: [29687.629456] device vnet3 entered promiscuous mode
Oct 26 12:23:14 stack01 kernel: [29687.647966] br101: port 4(vnet3) entering forwarding state
Oct 26 12:23:14 stack01 kernel: [29687.664064] br101: port 4(vnet3) entering forwarding state
Oct 26 12:23:22 stack01 kernel: [29695.939096] kvm: 28195: cpu0 unhandled rdmsr: 0xc0010001
Oct 26 12:23:24 stack01 kernel: [29698.032027] vnet3: no IPv6 routers present
Oct 26 12:28:59 stack01 kernel: [30032.473283] general protection fault: 0000 [#5] SMP 
Oct 26 12:28:59 stack01 kernel: [30032.476951] CPU 2 
Oct 26 12:28:59 stack01 kernel: [30032.476951] Modules linked in: speedstep_lib ebt_arp ebt_ip 8021q garp ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iscsi_trgt ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_amd kvm vesafb ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nbd amd64_edac_mod edac_core psmouse sp5100_tco serio_raw dm_multipath ghes joydev hed i2c_piix4 k10temp edac_mce_amd lp parport raid10 raid456 async_pq async_xor usbhid hid xor async_memcpy async_raid6_recov pata_atiixp ahci igb libahci dca raid6_pq async_tx raid1 raid0 multipath linear
Oct 26 12:28:59 stack01 kernel: [30032.476951] 
Oct 26 12:28:59 stack01 kernel: [30032.476951] Pid: 18608, comm: bash Tainted: G      D     3.0.0-12-server #20-Ubuntu HP ProLiant DL165 G7
Oct 26 12:28:59 stack01 kernel: [30032.476951] RIP: 0010:[<ffffffff81183b30>]  [<ffffffff81183b30>] dup_fd+0x150/0x2f0
Oct 26 12:28:59 stack01 kernel: [30032.476951] RSP: 0018:ffff88028554bd80  EFLAGS: 00010206
Oct 26 12:28:59 stack01 kernel: [30032.476951] RAX: 00000000000000ff RBX: ffff88024b277800 RCX: bfffffffffffffff
Oct 26 12:28:59 stack01 kernel: [30032.476951] RDX: 00008802b8df6300 RSI: 0000000000000100 RDI: ffff88027a02ffc0
Oct 26 12:28:59 stack01 kernel: [30032.476951] RBP: ffff88028554bde0 R08: 0000000000000003 R09: 0000000000000001
Oct 26 12:28:59 stack01 kernel: [30032.476951] R10: 4000000000000000 R11: 0000000000015d58 R12: ffff880135843180
Oct 26 12:28:59 stack01 kernel: [30032.476951] R13: ffff88027a02f6c0 R14: ffff88024b1ae800 R15: 0000000000000100
Oct 26 12:28:59 stack01 kernel: [30032.476951] FS:  00007f8035982720(0000) GS:ffff88013fc40000(0000) knlGS:0000000000000000
Oct 26 12:28:59 stack01 kernel: [30032.476951] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 26 12:28:59 stack01 kernel: [30032.476951] CR2: 0000000000c9d108 CR3: 000000032e0fc000 CR4: 00000000000006e0
Oct 26 12:28:59 stack01 kernel: [30032.476951] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 26 12:28:59 stack01 kernel: [30032.476951] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 26 12:28:59 stack01 kernel: [30032.476951] Process bash (pid: 18608, threadinfo ffff88028554a000, task ffff880138800000)
Oct 26 12:28:59 stack01 kernel: [30032.476951] Stack:
Oct 26 12:28:59 stack01 kernel: [30032.476951]  ffff88028554bd90 0000000138caae40 ffff8802b8359d40 0000000000000020
Oct 26 12:28:59 stack01 kernel: [30032.476951]  ffff88027a02f6c0 ffff8801370f4800 0000000000000000 ffff880138caae40
Oct 26 12:28:59 stack01 kernel: [30032.476951]  0000000001200011 00007f80359829f0 0000000000000000 ffff88028554bf58
Oct 26 12:28:59 stack01 kernel: [30032.476951] Call Trace:
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff8105d0d6>] copy_process.part.23+0x656/0xe00
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff8115352c>] ? kmem_cache_alloc_trace+0xfc/0x120
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff8105d8f7>] copy_process+0x77/0x80
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff8105da0e>] do_fork+0xbe/0x330
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff81183dc7>] ? alloc_fd+0xf7/0x150
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff815fe82e>] ? _raw_spin_lock+0xe/0x20
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff811658e1>] ? fd_install+0x61/0x80
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff81171353>] ? do_pipe_flags+0xc3/0x120
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff81012b58>] sys_clone+0x28/0x30
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff81606f23>] stub_clone+0x13/0x20
Oct 26 12:28:59 stack01 kernel: [30032.476951]  [<ffffffff81606c02>] ? system_call_fastpath+0x16/0x1b
Oct 26 12:28:59 stack01 kernel: [30032.476951] Code: 8b 45 b8 49 8b 7d 10 48 8b 71 10 4c 89 c2 e8 c8 ef 16 00 45 85 ff 74 6c 41 8d 77 ff 31 c0 41 b9 01 00 00 00 48 83 c6 01 eb 13 90 <f0> 48 ff 42 30 48 89 14 c3 48 83 c0 01 48 39 f0 74 3e 49 8b 14 
Oct 26 12:28:59 stack01 kernel: [30032.476951] RIP  [<ffffffff81183b30>] dup_fd+0x150/0x2f0
Oct 26 12:28:59 stack01 kernel: [30032.476951]  RSP <ffff88028554bd80>
Oct 26 12:28:59 stack01 kernel: [30033.350657] ---[ end trace 3db970a3e7847d7e ]---
Oct 26 12:29:00 stack01 kernel: [30033.369199] general protection fault: 0000 [#6] SMP 
Oct 26 12:29:00 stack01 kernel: [30033.372159] CPU 2 
Oct 26 12:29:00 stack01 kernel: [30033.372159] Modules linked in: speedstep_lib ebt_arp ebt_ip 8021q garp ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iscsi_trgt ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_amd kvm vesafb ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nbd amd64_edac_mod edac_core psmouse sp5100_tco serio_raw dm_multipath ghes joydev hed i2c_piix4 k10temp edac_mce_amd lp parport raid10 raid456 async_pq async_xor usbhid hid xor async_memcpy async_raid6_recov pata_atiixp ahci igb libahci dca raid6_pq async_tx raid1 raid0 multipath linear
Oct 26 12:29:00 stack01 kernel: [30033.372159] 
Oct 26 12:29:00 stack01 kernel: [30033.372159] Pid: 18608, comm: bash Tainted: G      D     3.0.0-12-server #20-Ubuntu HP ProLiant DL165 G7
Oct 26 12:29:00 stack01 kernel: [30033.372159] RIP: 0010:[<ffffffff811656e9>]  [<ffffffff811656e9>] filp_close+0x19/0x90
Oct 26 12:29:00 stack01 kernel: [30033.372159] RSP: 0018:ffff88028554bb28  EFLAGS: 00010286
Oct 26 12:29:00 stack01 kernel: [30033.372159] RAX: ffff88024b1aeff8 RBX: 00000000000000c0 RCX: 0000000000000000
Oct 26 12:29:00 stack01 kernel: [30033.372159] RDX: ffff8801370f4780 RSI: ffff8801370f4780 RDI: 00008802b8df6300
Oct 26 12:29:00 stack01 kernel: [30033.372159] RBP: ffff88028554bb48 R08: ffffffff81132ce5 R09: 0000000000000000
Oct 26 12:29:00 stack01 kernel: [30033.372159] R10: ffff8802b838fb10 R11: 0000000000000006 R12: 0000000000000004
Oct 26 12:29:00 stack01 kernel: [30033.372159] R13: 0000000000000001 R14: ffff8802b8359d40 R15: 00000000000000ff
Oct 26 12:29:00 stack01 kernel: [30033.372159] FS:  00007f8035982720(0000) GS:ffff88013fc40000(0000) knlGS:0000000000000000
Oct 26 12:29:00 stack01 kernel: [30033.372159] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 26 12:29:00 stack01 kernel: [30033.372159] CR2: 0000000000c9d108 CR3: 0000000001c03000 CR4: 00000000000006e0
Oct 26 12:29:00 stack01 kernel: [30033.372159] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 26 12:29:00 stack01 kernel: [30033.372159] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 26 12:29:00 stack01 kernel: [30033.372159] Process bash (pid: 18608, threadinfo ffff88028554a000, task ffff880138800000)
Oct 26 12:29:00 stack01 kernel: [30033.372159] Stack:
Oct 26 12:29:00 stack01 kernel: [30033.372159]  ffffea00043349e8 00000000000000c0 0000000000000004 0000000000000001
Oct 26 12:29:00 stack01 kernel: [30033.372159]  ffff88028554bb98 ffffffff81060f4a 0000000000000000 ffff8801370f4780
Oct 26 12:29:00 stack01 kernel: [30033.372159]  ffff880138800600 ffff8801370f4780 ffff880138800600 ffff880138800000
Oct 26 12:29:00 stack01 kernel: [30033.372159] Call Trace:
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff81060f4a>] put_files_struct.part.14+0x7a/0xe0
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff810629a8>] put_files_struct+0x18/0x20
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff81062a74>] exit_files+0x54/0x70
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff81062f8d>] do_exit+0x19d/0x440
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff815ffcd0>] oops_end+0xb0/0xf0
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff8100d8c8>] die+0x58/0x90
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff815ff7c2>] do_general_protection+0x162/0x170
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff815fefe5>] general_protection+0x25/0x30
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff81183b30>] ? dup_fd+0x150/0x2f0
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff8105d0d6>] copy_process.part.23+0x656/0xe00
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff8115352c>] ? kmem_cache_alloc_trace+0xfc/0x120
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff8105d8f7>] copy_process+0x77/0x80
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff8105da0e>] do_fork+0xbe/0x330
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff81183dc7>] ? alloc_fd+0xf7/0x150
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff815fe82e>] ? _raw_spin_lock+0xe/0x20
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff811658e1>] ? fd_install+0x61/0x80
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff81171353>] ? do_pipe_flags+0xc3/0x120
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff81012b58>] sys_clone+0x28/0x30
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff81606f23>] stub_clone+0x13/0x20
Oct 26 12:29:00 stack01 kernel: [30033.372159]  [<ffffffff81606c02>] ? system_call_fastpath+0x16/0x1b
Oct 26 12:29:00 stack01 kernel: [30033.372159] Code: 00 00 00 00 e8 f9 e2 f8 ff 48 83 c4 18 5b 5d c3 66 90 55 48 89 e5 48 83 ec 20 48 89 5d e8 4c 89 65 f0 4c 89 6d f8 66 66 66 66 90 
Oct 26 12:29:00 stack01 kernel: [30033.372159] RIP  [<ffffffff811656e9>] filp_close+0x19/0x90
Oct 26 12:29:00 stack01 kernel: [30033.372159]  RSP <ffff88028554bb28>
Oct 26 12:29:00 stack01 kernel: [30034.302296] ---[ end trace 3db970a3e7847d7f ]---
Oct 26 12:29:00 stack01 kernel: [30034.316447] Fixing recursive fault but reboot is needed!
Oct 26 12:36:54 stack01 kernel: [30508.142422] init: apport pre-start process (556) terminated with status 1
Oct 26 12:36:54 stack01 kernel: [30508.162826] init: apport post-stop process (558) terminated with status 1