Bad page map in openjdk-7

[stevenschlansker]

[ 2808.161850] BUG: Bad page map in process java pte:00000320 pmd:e8048067
[ 2808.161860] addr:00007f968a63a000 vm_flags:08000070 anon_vma: (null) mapping: (null) index:7f968a63a
[ 2808.161866] CPU: 0 PID: 18543 Comm: java Tainted: G B 3.13.0-24-generic #46-Ubuntu
[ 2808.161868] ffff8800e7f623c0 ffff8800e8277a98 ffffffff81715a64 00007f968a63a000
[ 2808.161871] ffff8800e8277ae0 ffffffff81174183 0000000000000320 00000007f968a63a
[ 2808.161872] ffff8800e80481d0 0000000000000320 00007f968a63a000 00007f968a63b000
[ 2808.161874] Call Trace:
[ 2808.161881] [<ffffffff81715a64>] dump_stack+0x45/0x56
[ 2808.161884] [<ffffffff81174183>] print_bad_pte+0x1a3/0x250
[ 2808.161886] [<ffffffff81175439>] vm_normal_page+0x69/0x80
[ 2808.161888] [<ffffffff8117580b>] unmap_page_range+0x3bb/0x7f0
[ 2808.161890] [<ffffffff81175cc1>] unmap_single_vma+0x81/0xf0
[ 2808.161892] [<ffffffff81176d39>] unmap_vmas+0x49/0x90
[ 2808.161894] [<ffffffff8117feec>] exit_mmap+0x9c/0x170
[ 2808.161898] [<ffffffff8110fcf3>] ? __delayacct_add_tsk+0x153/0x170
[ 2808.161901] [<ffffffff8106482c>] mmput+0x5c/0x120
[ 2808.161903] [<ffffffff81069bbc>] do_exit+0x26c/0xa50
[ 2808.161906] [<ffffffff810d7591>] ? __unqueue_futex+0x31/0x60
[ 2808.161907] [<ffffffff810d83b6>] ? futex_wait+0x126/0x290
[ 2808.161909] [<ffffffff8106a41f>] do_group_exit+0x3f/0xa0
[ 2808.161912] [<ffffffff8107a050>] get_signal_to_deliver+0x1d0/0x6f0
[ 2808.161916] [<ffffffff81013448>] do_signal+0x48/0x960
[ 2808.161919] [<ffffffff8101b7d9>] ? sched_clock+0x9/0x10
[ 2808.161921] [<ffffffff8109d13d>] ? sched_clock_local+0x1d/0x80
[ 2808.161924] [<ffffffff811112fc>] ? acct_account_cputime+0x1c/0x20
[ 2808.161925] [<ffffffff8109d76b>] ? account_user_time+0x8b/0xa0
[ 2808.161927] [<ffffffff8109dd84>] ? vtime_account_user+0x54/0x60
[ 2808.161929] [<ffffffff81013dc9>] do_notify_resume+0x69/0xb0
[ 2808.161932] [<ffffffff817266ea>] int_signal+0x12/0x17

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: linux-image-3.13.0-24-generic 3.13.0-24.46
ProcVersionSignature: Ubuntu 3.13.0-24.46-generic 3.13.9
Uname: Linux 3.13.0-24-generic x86_64
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Apr 16 22:18 seq
 crw-rw---- 1 root audio 116, 33 Apr 16 22:18 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.14.1-0ubuntu3
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory: 'iw'
Date: Thu Apr 17 00:06:35 2014
Ec2AMI: ami-2ef19b1e
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-west-2a
Ec2InstanceType: m3.medium
Ec2Kernel: aki-fc8f11cc
Ec2Ramdisk: unavailable
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lspci: Error: [Errno 2] No such file or directory: 'lspci'
Lsusb: Error: [Errno 2] No such file or directory: 'lsusb'
PciMultimedia:

ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: root=UUID=ecdfe459-e35e-4416-a9c1-9ee71f9f93e7 ro
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-24-generic N/A
 linux-backports-modules-3.13.0-24-generic N/A
 linux-firmware N/A
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
---
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Apr 17 17:56 seq
 crw-rw---- 1 root audio 116, 33 Apr 17 17:56 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.14.1-0ubuntu3
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
CurrentDmesg: Error: command ['sh', '-c', 'dmesg | comm -13 --nocheck-order /var/log/dmesg -'] failed with exit code 1: comm: /var/log/dmesg: Permission denied
DistroRelease: Ubuntu 14.04
Ec2AMI: ami-2ef19b1e
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-west-2a
Ec2InstanceType: m3.medium
Ec2Kernel: aki-fc8f11cc
Ec2Ramdisk: unavailable
IwConfig: Error: [Errno 2] No such file or directory
Lspci: Error: [Errno 2] No such file or directory
Lsusb: Error: [Errno 2] No such file or directory
Package: linux (not installed)
PciMultimedia:

ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/zsh
ProcFB:

ProcKernelCmdLine: root=UUID=ecdfe459-e35e-4416-a9c1-9ee71f9f93e7 ro
ProcVersionSignature: Ubuntu 3.13.0-24.46-generic 3.13.9
RfKill: Error: [Errno 2] No such file or directory
Tags: trusty ec2-images
Uname: Linux 3.13.0-24-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

WifiSyslog:

_MarkForUpload: True
---
ApportVersion: 2.14.1-0ubuntu3
Architecture: amd64
DistroRelease: Ubuntu 14.04
Ec2AMI: ami-2ef19b1e
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-west-2a
Ec2InstanceType: m3.medium
Ec2Kernel: aki-fc8f11cc
Ec2Ramdisk: unavailable
Package: linux (not installed)
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
Tags: trusty ec2-images
Uname: Linux 3.15.0-031500rc1-generic x86_64
UnreportableReason: The running kernel is not an Ubuntu kernel
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm audio cdrom dialout dip floppy netdev plugdev sudo video
_MarkForUpload: True

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1308796

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.15 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.15-rc1-trusty/

[stevenschlansker (stevenschlansker)]

Sorry, the above were accidentally collected as duplicates.

I am now running Linux ip-10-70-6-145 3.15.0-031500rc1-generic #201404131835 SMP Sun Apr 13 22:36:23 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux as requested. I can still trigger this, easiest way to reproduce is just running 'jps':

[ 503.124559] BUG: Bad page map in process jps pte:00000320 pmd:e77af067
[ 503.124571] addr:00007fa42ebc1000 vm_flags:08000070 anon_vma: (null) mapping: (null) index:7fa42ebc1
[ 503.124577] CPU: 0 PID: 4240 Comm: jps Not tainted 3.15.0-031500rc1-generic #201404131835
[ 503.124579] 00007fa42ebc1000 ffff8800e77fd9c8 ffffffff8175ba7d 0000000000000007
[ 503.124581] ffff880004cfa8a0 ffff8800e77fda18 ffffffff8118311f ffffffff810056e1
[ 503.124583] dead000000100100 ffffea00039dfc80 0000000000000000 00007fa42ebc1000
[ 503.124584] Call Trace:
[ 503.124592] [<ffffffff8175ba7d>] dump_stack+0x46/0x58
[ 503.124596] [<ffffffff8118311f>] print_bad_pte+0x1bf/0x290
[ 503.124600] [<ffffffff810056e1>] ? __raw_callee_save_xen_pte_val+0x11/0x1e
[ 503.124602] [<ffffffff8118451e>] vm_normal_page+0x6e/0x80
[ 503.124604] [<ffffffff8118558f>] zap_pte_range+0x13f/0x450
[ 503.124606] [<ffffffff81185a74>] unmap_page_range+0x1d4/0x310
[ 503.124608] [<ffffffff81006a47>] ? xen_unpin_page+0x27/0x30
[ 503.124610] [<ffffffff81185c37>] unmap_single_vma+0x87/0x100
[ 503.124612] [<ffffffff81186514>] unmap_vmas+0x54/0xa0
[ 503.124614] [<ffffffff8118f64c>] exit_mmap+0x9c/0x170
[ 503.124617] [<ffffffff810667fa>] mmput.part.25+0x4a/0x120
[ 503.124619] [<ffffffff81066900>] mmput+0x30/0x40
[ 503.124621] [<ffffffff8106b49a>] exit_mm+0x13a/0x190
[ 503.124623] [<ffffffff81118e3b>] ? taskstats_exit+0x1cb/0x270
[ 503.124625] [<ffffffff8106b63b>] do_exit+0x14b/0x470
[ 503.124627] [<ffffffff8107871b>] ? __dequeue_signal+0x6b/0xb0
[ 503.124629] [<ffffffff8106b9f4>] do_group_exit+0x44/0xa0
[ 503.124631] [<ffffffff8107ba30>] get_signal_to_deliver+0x230/0x580
[ 503.124634] [<ffffffff81013ca7>] do_signal+0x47/0x140
[ 503.124637] [<ffffffff810a1259>] ? account_user_time+0x99/0xb0
[ 503.124639] [<ffffffff810a188d>] ? vtime_account_user+0x5d/0x70
[ 503.124642] [<ffffffff81159a8f>] ? context_tracking_user_exit+0x6f/0xf0
[ 503.124644] [<ffffffff81013e42>] do_notify_resume+0xa2/0xd0
[ 503.124647] [<ffffffff8177212a>] int_signal+0x12/0x17

[stevenschlansker (stevenschlansker)]

Apport does not seem to want to collect another report -- I chose to "send report" but nothing is appearing in this bug :(

If you would like additional files let me know and I can collect them by hand.

[Viktor Szathmáry (phraktle)]

I'm also experiencing this problem on an EC2 paravirtualized instance.

Based on the kernel changelog, it seems this bug was fixed in 3.13.5:

See https://www.kernel.org/pub/linux/kernel/v3.0/ChangeLog-3.13.5 (search for "Bad page map")

(I have not tested it though)

[Ernestas (ernetas)]

Upgraded to 3.15, fixed the bug for me.

[Thiago Martins (martinx)]

Ubuntu 12.04.4 with Linux 3.13 on Amazon EC2, dumps this error a lot!

I'll try it with Linux 3.11 / 3.8 instead....

[Brian Moyles (bmoyles)]

@Viktor, I believe that patch was later reverted and a new one was issued to fix

https://lkml.org/lkml/2014/4/10/174

https://lkml.org/lkml/2014/4/8/173

[Jonathan Matthews (jonathan-matthews-t)]

This is still an issue with linux-image-3.13.0-27-generic version 3.13.0-27.50.

I'm unsure why this crashing bug is not receiving more attention for the current LTS release; it renders certain workloads on Xen (i.e. AWS) almost impossible to carry out. Launching java any more than quite-infrequently seems to be untenable.

Any word if the fix for this bug from 3.15 or, as @e-brian suggests, later, can be used? Running a kernel from http://kernel.ubuntu.com/~kernel-ppa/mainline/ seems like /massive/ overkill for something relatively trivially fixable. AIUI.

[Tim Gardner (timg-tpi)]

Jonathan - test kernel at http://kernel.ubuntu.com/~rtg/3.13.0-29_3-lp1308796

wget http://kernel.ubuntu.com/~rtg/3.13.0-29_3-lp1308796/linux-image-3.13.0-29-generic_3.13.0-29.52_amd64.deb
wget http://kernel.ubuntu.com/~rtg/3.13.0-29_3-lp1308796/linux-image-extra-3.13.0-29-generic_3.13.0-29.52_amd64.deb
sudo dpkg -i linux-image*.deb
sudo reboot

[Jonathan Matthews (jonathan-matthews-t)]

Tim - this looks to have resolved the issue; many thanks.

Previously, roughly 1 in 3 invocations of this litmus test on my CI server failed: "java -jar /opt/jenkins/jenkins-cli.jar -s http://localhost:8080 list-plugins". After installing the test kernel, 0 from ~100 invocations failed.

Attached is the AWS instance's dmesg output from the test kernel, in case it's helpful.
This is using AMI ubuntu-trusty-14.04-amd64-server-20140416.1 (ami-018c9568).

[stevenschlansker (stevenschlansker)]

Seems to fix it for me too, at least from a first check.

[Tim Gardner (timg-tpi)]

As it turns out, this commit is already in the stable pipe. Dupe'ing this bug against 1313450.

[Michael Schurter (schmichael)]

I ran into this bug on AWS and installed Tim's debs as instructed -- except that I didn't install -extra as the server did not have -extra packages installed before.

However on reboot I received the following kernel errors:

[ 1.274049] VFS: Cannot open root device "LABEL=cloudimg-rootfs" or unknown-block(0,0): error -6
[ 1.274054] Please append a correct "root=" boot option; here are the available partitions:
[ 1.274061] ca01 52428800 xvda1 driver: vbd
[ 1.274065] ca10 83874816 xvdb driver: vbd
[ 1.274068] ca20 83874816 xvdc driver: vbd
[ 1.274071] ca30 209715200 xvdd driver: vbd
[ 1.274076] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 1.274082] CPU: 4 PID: 1 Comm: swapper/0 Not tainted 3.13.0-29-generic #52
[ 1.274086] 0000000000008001 ffff8803a6d7bdf0 ffffffff8171a194 ffffffff81a3b888
[ 1.274095] ffff8803a6d7be68 ffffffff81713395 3033616300000010 ffff8803a6d7be78
[ 1.274100] ffff8803a6d7be18 ffff000064647678 ffff8803a6d7be88 0000000000000074
[ 1.274104] Call Trace:
[ 1.274113] [<ffffffff8171a194>] dump_stack+0x45/0x56
[ 1.274117] [<ffffffff81713395>] panic+0xc8/0x1d7
[ 1.274122] [<ffffffff81d3646a>] mount_block_root+0x225/0x2b0
[ 1.274126] [<ffffffff81d36692>] mount_root+0x53/0x56
[ 1.274129] [<ffffffff81d36801>] prepare_namespace+0x16c/0x1a4
[ 1.274132] [<ffffffff81d3616e>] kernel_init_freeable+0x1f3/0x200
[ 1.274136] [<ffffffff81d358e5>] ? do_early_param+0x88/0x88
[ 1.274140] [<ffffffff81708670>] ? rest_init+0x80/0x80
[ 1.274144] [<ffffffff8170867e>] kernel_init+0xe/0x130
[ 1.274148] [<ffffffff8172aabc>] ret_from_fork+0x7c/0xb0
[ 1.274152] [<ffffffff81708670>] ? rest_init+0x80/0x80

I'm now unable to boot this machine and have since terminated it. Am I doing something wrong?

[Michael Schurter (schmichael)]

FWIW, we hit the bug with Oracle Java 7u55, but as #1313450 has illustrated this doesn't appear to be a Java specific bug.

[Michael Schurter (schmichael)]

It appears that no /boot/initrd.img-3.13.0-29-generic was generated, but on a fresh system it was generated. I'm not sure why it wasn't generated on the initial machine.