general protection fault, Fixing recursive fault but reboot is needed!

[Kiall Mac Innes]

This seems to be happening occasionally, twice today. From what I can tell, its always a bash process which is attemping to open a small text file with `nano`.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: linux-image-3.0.0-12-server 3.0.0-12.20
ProcVersionSignature: Ubuntu 3.0.0-12.20-server 3.0.4
Uname: Linux 3.0.0-12-server x86_64
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 2011-10-26 04:08 seq
 crw-rw---- 1 root audio 116, 33 2011-10-26 04:08 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 1.23-0ubuntu3
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
Date: Wed Oct 26 12:37:25 2011
HibernationDevice: RESUME=UUID=ad4d4e75-77c4-4d7b-a07b-f413b07fab38
InstallationMedia: Ubuntu-Server 11.10 "Oneiric Ocelot" - Beta amd64 (20110921)
IwConfig: Error: [Errno 2] No such file or directory
MachineType: HP ProLiant DL165 G7
PciMultimedia:

ProcEnviron:
 LANGUAGE=en_IE:en
 LANG=en_IE.UTF-8
 SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.0.0-12-server root=/dev/mapper/hostname-root ro
RelatedPackageVersions:
 linux-restricted-modules-3.0.0-12-server N/A
 linux-backports-modules-3.0.0-12-server N/A
 linux-firmware 1.60
RfKill: Error: [Errno 2] No such file or directory
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 09/06/2010
dmi.bios.vendor: HP
dmi.bios.version: O37
dmi.chassis.type: 23
dmi.chassis.vendor: HP
dmi.modalias: dmi:bvnHP:bvrO37:bd09/06/2010:svnHP:pnProLiantDL165G7:pvr:cvnHP:ct23:cvr:
dmi.product.name: ProLiant DL165 G7
dmi.sys.vendor: HP

[Kiall Mac Innes (kiall)]

Humm - Looks like the relevant syslog output wasn't attached:

[27537.802098] general protection fault: 0000 [#3] SMP
[27537.805708] CPU 1
[27537.805708] Modules linked in: speedstep_lib ebt_arp ebt_ip 8021q garp ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iscsi_trgt ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_amd kvm vesafb ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nbd amd64_edac_mod edac_core psmouse sp5100_tco serio_raw dm_multipath ghes joydev hed i2c_piix4 k10temp edac_mce_amd lp parport raid10 raid456 async_pq async_xor usbhid hid xor async_memcpy async_raid6_recov pata_atiixp ahci igb libahci dca raid6_pq async_tx raid1 raid0 multipath linear
[27537.805708]
[27537.805708] Pid: 11900, comm: bash Tainted: G D 3.0.0-12-server #20-Ubuntu HP ProLiant DL165 G7
[27537.805708] RIP: 0010:[<ffffffff81183b30>] [<ffffffff81183b30>] dup_fd+0x150/0x2f0
[27537.805708] RSP: 0018:ffff88024b385d80 EFLAGS: 00010206
[27537.805708] RAX: 00000000000000ff RBX: ffff880332672800 RCX: bfffffffffffffff
[27537.805708] RDX: 0000880327f455c0 RSI: 0000000000000100 RDI: ffff88027a3457c0
[27537.805708] RBP: ffff88024b385de0 R08: 0000000000000003 R09: 0000000000000001
[27537.805708] R10: 4000000000000000 R11: 0000000000015d58 R12: ffff880100013440
[27537.805708] R13: ffff88027a345d80 R14: ffff88024b1ad800 R15: 0000000000000100
[27537.805708] FS: 00007f58bea79720(0000) GS:ffff88013fc20000(0000) knlGS:0000000000000000
[27537.805708] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[27537.805708] CR2: 0000000000d58108 CR3: 000000027a6fb000 CR4: 00000000000006e0
[27537.805708] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[27537.805708] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[27537.805708] Process bash (pid: 11900, threadinfo ffff88024b384000, task ffff880134924560)
[27537.805708] Stack:
[27537.805708] ffff88024b385d90 0000000136e58000 ffff880279f47bc0 0000000000000020
[27537.805708] ffff88027a345d80 ffff880135983200 0000000000000000 ffff880136e58000
[27537.805708] 0000000001200011 00007f58bea799f0 0000000000000000 ffff88024b385f58
[27537.805708] Call Trace:
[27537.805708] [<ffffffff8105d0d6>] copy_process.part.23+0x656/0xe00
[27537.805708] [<ffffffff8115352c>] ? kmem_cache_alloc_trace+0xfc/0x120
[27537.805708] [<ffffffff8105d8f7>] copy_process+0x77/0x80
[27537.805708] [<ffffffff8105da0e>] do_fork+0xbe/0x330
[27537.805708] [<ffffffff81183dc7>] ? alloc_fd+0xf7/0x150
[27537.805708] [<ffffffff815fe82e>] ? _raw_spin_lock+0xe/0x20
[27537.805708] [<ffffffff811658e1>] ? fd_install+0x61/0x80
[27537.805708] [<ffffffff81171353>] ? do_pipe_flags+0xc3/0x120
[27537.805708] [<ffffffff81012b58>] sys_clone+0x28/0x30
[27537.805708] [<ffffffff81606f23>] stub_clone+0x13/0x20
[27537.805708] [<ffffffff81606c02>] ? system_call_fastpath+0x16/0x1b
[27537.805708] Code: 8b 45 b8 49 8b 7d 10 48 8b 71 10 4c 89 c2 e8 c8 ef 16 00 45 85 ff 74 6c 41 8d 77 ff 31 c0 41 b9 01 00 00 00 48 83 c6 01 eb 13 90 <f0> 48 ff 42 30...

[Kiall Mac Innes (kiall)]

I've

# grep -r 'general protection fault' /var/log/syslog.1
Oct 26 02:21:53 stack01 kernel: [87049.200979] general protection fault: 0000 [#1] SMP
Oct 26 02:21:53 stack01 kernel: [87049.469110] general protection fault: 0000 [#2] SMP

# grep -r 'general protection fault' /var/log/syslog
Oct 26 08:16:22 stack01 kernel: [14876.125055] general protection fault: 0000 [#1] SMP
Oct 26 08:16:23 stack01 kernel: [14876.180538] general protection fault: 0000 [#2] SMP
Oct 26 11:47:25 stack01 kernel: [27537.802098] general protection fault: 0000 [#3] SMP
Oct 26 11:47:26 stack01 kernel: [27538.593964] general protection fault: 0000 [#4] SMP
Oct 26 12:28:59 stack01 kernel: [30032.473283] general protection fault: 0000 [#5] SMP
Oct 26 12:29:00 stack01 kernel: [30033.369199] general protection fault: 0000 [#6] SMP
Oct 26 13:47:59 stack01 kernel: [34771.996207] general protection fault: 0000 [#7] SMP
Oct 26 17:41:31 stack01 kernel: [48784.076457] general protection fault: 0000 [#8] SMP
Oct 26 17:41:32 stack01 kernel: [48784.781867] general protection fault: 0000 [#9] SMP
Oct 26 18:13:29 stack01 kernel: [50701.797683] general protection fault: 0000 [#10] SMP
Oct 26 18:13:29 stack01 kernel: [50702.525257] general protection fault: 0000 [#11] SMP

[Kiall Mac Innes (kiall)]

I've just noticed this has happened 14 times so far today, full syslogs attached...

[Joseph Salisbury (jsalisbury)]

Have you run previous versions of Ubuntu on that machine? If so, did the issue happen with the previous versions, or just with Oneiric?

Would it be possible for you to test the latest upstream kernel? It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . If possible, please test the latest kernel (Not a kernel in the daily directory). Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag(Only that one tag, please leave the others). This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text.

If this bug is fixed by the mainline kernel, please add the following tag 'kernel-fixed-upstream-KERNEL-VERSION'. For example, if kernel version 3.1-rc9 fixed and issue, the tag would be: 'kernel-fixed-upstream-v3.1-rc9'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

Thanks in advance.

[Carson Longhorn (gadjet)]

To answer Joseph's question, I upgraded from Natty to Oneiric, the problem was not present in Natty, and is now happening frequently in Oneiric.

I am experiencing basically the same problem.

[Kiall Mac Innes (kiall)]

This appears to be solved in the mainline v3.2-precise kernel[1]. Things have been much more stable since installing this kernel.. Including some other kernel issues I've had on those servers.

[1] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.2-precise/