Upgrade to Intrepid: kernel crash because of locales

[Holger]

Binary package hint: locales

Ubuntu 8.10
locales (2.7.9-5)

While I was trying an update from 8.04 to 8.10 using the update script from the alternate CD (8.10 final), I faced a kernel panic within gnome while the update script was installing/configuring the package locales.

After some work (uncommenting all despite one locale in /var/lib/locales/supported.d/de en and local) I was finally able to go on with the package installation - till I totally had to wipe locales package to finish the package configuration process and have a basic running system.

And still, a new install of locales results in the same basic problem, despite that not the whole system crashes, but for instance tty2 or synaptic.

But finally I was able to get the important kernel log:

Oct 31 02:32:51 amd64 kernel: [ 395.934791] __ratelimit: 12 callbacks suppressed
Oct 31 02:32:51 amd64 kernel: [ 395.934804] localedef[7998]: segfault at 0 ip 000000000043a20e sp 00007fffce59a718 error 4 in localedef[400000+49000]
Oct 31 02:32:51 amd64 kernel: [ 395.945398] Eeek! page_mapcount(page) went negative! (-16777216)
Oct 31 02:32:51 amd64 kernel: [ 395.945409] page pfn = ce9e
Oct 31 02:32:51 amd64 kernel: [ 395.945410] page->flags = 100000000000068
Oct 31 02:32:51 amd64 kernel: [ 395.945412] page->count = 1
Oct 31 02:32:51 amd64 kernel: [ 395.945413] page->mapping = 00ff88002989fe01
Oct 31 02:32:51 amd64 kernel: [ 395.945434] vma->vm_ops = 0x0
Oct 31 02:32:51 amd64 kernel: [ 395.945451] ------------[ cut here ]------------
Oct 31 02:32:51 amd64 kernel: [ 395.945453] kernel BUG at /build/buildd/linux-2.6.27/mm/rmap.c:662!
Oct 31 02:32:51 amd64 kernel: [ 395.945455] invalid opcode: 0000 [1] SMP
Oct 31 02:32:51 amd64 kernel: [ 395.945457] CPU 0
Oct 31 02:32:51 amd64 kernel: [ 395.945459] Modules linked in: binfmt_misc prism2_usb p80211 ppdev cpufreq_ondemand cpufreq_powersave cpufreq_conservative cpufreq_stats freq_table cpufreq_userspace pci_slot container video output sbs wmi sbshc battery ipv6 iptable_filter ip_tables x_tables dm_crypt crypto_blkcipher dm_mod ac it87 hwmon_vid sbp2 parport_pc lp parport pcspkr snd_hda_intel k8temp joydev usblp evdev snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore i2c_nforce2 button snd_page_alloc i2c_core shpchp pci_hotplug ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif sg pata_acpi ata_generic usbhid hid ohci1394 ieee1394 pata_amd sata_nv skge forcedeth libata scsi_mod dock ehci_hcd ohci_hcd usbcore thermal processor fan fbcon tileblit font bitblit softcursor fuse
Oct 31 02:32:51 amd64 kernel: [ 395.945508] Pid: 7998, comm: localedef Not tainted 2.6.27-7-generic #1
Oct 31 02:32:51 amd64 kernel: [ 395.945510] RIP: 0010:[<ffffffff802ca891>] [<ffffffff802ca891>] page_remove_rmap+0x121/0x130
Oct 31 02:32:51 amd64 kernel: [ 395.945518] RSP: 0018:ffff88003783daf8 EFLAGS: 00010246
Oct 31 02:32:51 amd64 kernel: [ 395.945519] RAX: 0000000000000000 RBX: ffffe2000033a780 RCX: 0000000000000000
Oct 31 02:32:51 amd64 kernel: [ 395.945521] RDX: 00000000ffffffff RSI: 0000000000000046 RDI: 0000000000000246
Oct 31 02:32:51 amd64 kernel: [ 395.945523] RBP: ffff88003783db08 R08: 0000000000000000 R09: 0000000000000006
Oct 31 02:32:51 amd64 kernel: [ 395.945525] R10: ffff88003783d768 R11: ffff8800b783d877 R12: ffff88003d921c60
Oct 31 02:32:51 amd64 kernel: [ 395.945527] R13: 00007fffc4670000 R14: ffff880032400380 R15: ffff88003783dca8
Oct 31 02:32:51 amd64 kernel: [ 395.945530] FS: 00007fffc65766e0(0000) GS:ffffffff806e1a80(0000) knlGS:00000000f7338920
Oct 31 02:32:51 amd64 kernel: [ 395.945532] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 31 02:32:51 amd64 kernel: [ 395.945534] CR2: 0000000000000000 CR3: 0000000016ddd000 CR4: 00000000000006e0
Oct 31 02:32:51 amd64 kernel: [ 395.945536] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 31 02:32:51 amd64 kernel: [ 395.945538] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 31 02:32:51 amd64 kernel: [ 395.945541] Process localedef (pid: 7998, threadinfo ffff88003783c000, task ffff88003d9d1670)
Oct 31 02:32:51 amd64 kernel: [ 395.945543] Stack: ffffe2000033a780 ffff88000100f8a0 ffff88003783db98 ffffffff802beef2
Oct 31 02:32:51 amd64 kernel: [ 395.945547] 0000000000000001 000000000000000c ffffe20000c90000 800000000ce9e067
Oct 31 02:32:51 amd64 kernel: [ 395.945551] ffffe20000c90010 00007fffc4800000 ffff88003d921c60 ffff88003c9416c0
Oct 31 02:32:51 amd64 kernel: [ 395.945554] Call Trace:
Oct 31 02:32:51 amd64 kernel: [ 395.945558] [<ffffffff802beef2>] zap_pte_range+0x1f2/0x400
Oct 31 02:32:51 amd64 kernel: [ 395.945561] [<ffffffff802c015a>] unmap_page_range+0x2da/0x360
Oct 31 02:32:51 amd64 kernel: [ 395.945565] [<ffffffff802c09df>] unmap_vmas+0x17f/0x2b0
Oct 31 02:32:51 amd64 kernel: [ 395.945568] [<ffffffff802c4f65>] exit_mmap+0x95/0x120
Oct 31 02:32:51 amd64 kernel: [ 395.945571] [<ffffffff8024ca20>] mmput+0x40/0xe0
Oct 31 02:32:51 amd64 kernel: [ 395.945574] [<ffffffff8025114e>] exit_mm+0x11e/0x160
Oct 31 02:32:51 amd64 kernel: [ 395.945580] [<ffffffff80502446>] ? _spin_lock_irq+0x16/0x20
Oct 31 02:32:51 amd64 kernel: [ 395.945582] [<ffffffff8025319a>] do_exit+0x19a/0x410
Oct 31 02:32:51 amd64 kernel: [ 395.945585] [<ffffffff80253457>] do_group_exit+0x47/0xc0
Oct 31 02:32:51 amd64 kernel: [ 395.945590] [<ffffffff8025eaec>] get_signal_to_deliver+0x1ac/0x3a0
Oct 31 02:32:51 amd64 kernel: [ 395.945593] [<ffffffff80253e79>] ? set_normalized_timespec+0x9/0x90
Oct 31 02:32:51 amd64 kernel: [ 395.945597] [<ffffffff802124a5>] do_signal+0x75/0x200
Oct 31 02:32:51 amd64 kernel: [ 395.945602] [<ffffffff802461f3>] ? hrtick_start_fair+0x173/0x190
Oct 31 02:32:51 amd64 kernel: [ 395.945605] [<ffffffff805005b4>] ? thread_return+0x37/0x3c3
Oct 31 02:32:51 amd64 kernel: [ 395.945608] [<ffffffff802441ab>] ? finish_task_switch+0x2b/0xf0
Oct 31 02:32:51 amd64 kernel: [ 395.945611] [<ffffffff805005b4>] ? thread_return+0x37/0x3c3
Oct 31 02:32:51 amd64 kernel: [ 395.945614] [<ffffffff8021265d>] do_notify_resume+0x2d/0x50
Oct 31 02:32:51 amd64 kernel: [ 395.945617] [<ffffffff802130c4>] retint_signal+0x60/0xbc
Oct 31 02:32:51 amd64 kernel: [ 395.945619]
Oct 31 02:32:51 amd64 kernel: [ 395.945620]
Oct 31 02:32:51 amd64 kernel: [ 395.945621] Code: e8 15 f9 ff ff 49 8b 84 24 90 00 00 00 48 85 c0 74 19 48 8b 40 20 48 85 c0 74 10 48 8b 70 58 48 c7 c7 00 50 5f 80 e8 ef f8 ff ff <0f> 0b eb fe 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57
Oct 31 02:32:51 amd64 kernel: [ 395.945645] RIP [<ffffffff802ca891>] page_remove_rmap+0x121/0x130
Oct 31 02:32:51 amd64 kernel: [ 395.945647] RSP <ffff88003783daf8>
Oct 31 02:32:51 amd64 kernel: [ 395.945650] ---[ end trace 357ac21820ad3e45 ]---
Oct 31 02:32:51 amd64 kernel: [ 395.945652] Fixing recursive fault but reboot is needed!

[Holger (dev-nul-nul)]

I have additionally created a traceback for Generating the locales - maybe this is helpful.
The Generation process ends up locked, but in this case, it doesn't affect the whole system in a total crash.

[Martin Pitt (pitti)]

The localedef hang is a duplicate of bug 25660. Thus let's use this report for the kernel crash; it is very likely that the mmap() crash is the real reason for localedef hanging.

[kernel-janitor (kernel-janitor)]

Hi Holger,

This bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? Can you try with the latest development release of Ubuntu? ISO CD images are available from http://cdimage.ubuntu.com/releases/karmic .

If it remains an issue, could you run the following command from a Terminal (Applications->Accessories->Terminal). It will automatically gather and attach updated debug information to this report.

apport-collect -p linux 291359

Also, if you could test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

[Jeremy Foshee (jeremyfoshee)]

This bug report was marked as Incomplete and has not had any updated comments for quite some time. As a result this bug is being closed. Please reopen if this is still an issue in the current Ubuntu release http://www.ubuntu.com/getubuntu/download . Also, please be sure to provide any requested information that may have been missing. To reopen the bug, click on the current status under the Status column and change the status back to "New". Thanks.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

[Holger (dev-nul-nul)]

It has been a long time ago I faced this bug and was unable to give the asked information. Recently I tried to do a fresh install of mythbuntu 10.04 and faced a problem which looks exactly like this one.
During installation, when it comes to locale installation, the computer just crashes. I don´t really know if this is a coincident or it might be hardware related, but it is not very likely that it crashes always at the exact same position if there is a hardware related problem.

If you tell me how I should provide you with information during installation process ending in a hardlock, I try to give you needed information to point out, if this is really the same bug.

[Moritz (msmaster2000)]

I faced the same bug on my AMD64 machine while installing lucid. What is astonishing, that on random occasions the installing process can be finished. After installing and updating todays language pack, this happend and crashed my system (NO switching, NO typing possible)

Syslog excerpt:

May 7 12:12:27 xxx [ 5140.569573] localedef[3909]: segfault at 7f9b7f5effc9 ip 00007f9b82e218fd sp 00007fff0f7ca118 error 4 in libc-2.11.1.so[7f9b82d9c000+178000]
May 7 12:12:29 xxx: [ 5142.634978] BUG: Bad page state in process localedef pfn:0f164
May 7 12:12:29 xxx: [ 5142.634985] page:ffffea000034cde0 flags:0100000000100008 count:0 mapcount:255 mapping:(null) index:7f780d15c
May 7 12:12:29 xxx: [ 5142.634991] Pid: 3944, comm: localedef Tainted: P 2.6.32-22-generic #33-Ubuntu
May 7 12:12:29 xxx: [ 5142.634994] Call Trace:
May 7 12:12:29 xxx: [ 5142.635004] [<ffffffff810f81a5>] bad_page+0xf5/0x150
May 7 12:12:29 xxx: [ 5142.635009] [<ffffffff810fb277>] free_hot_cold_page+0x1e7/0x240
May 7 12:12:29 xxx: [ 5142.635015] [<ffffffff8113c9bb>] ? mem_cgroup_update_tree+0x1cb/0x1d0
May 7 12:12:29 xxx: [ 5142.635019] [<ffffffff810fb3ad>] free_hot_page+0x2d/0x60
May 7 12:12:29 xxx: [ 5142.635024] [<ffffffff810fe00d>] put_page+0x4d/0x180
May 7 12:12:29 xxx: [ 5142.635028] [<ffffffff811223fa>] free_page_and_swap_cache+0x2a/0x60
May 7 12:12:29 xxx: [ 5142.635032] [<ffffffff81111148>] zap_pte_range+0x388/0x400
May 7 12:12:29 xxx: [ 5142.635038] [<ffffffff81058620>] ? finish_task_switch+0x50/0xe0
May 7 12:12:29 xxx: [ 5142.635042] [<ffffffff811120b5>] unmap_page_range+0x1c5/0x2a0
May 7 12:12:29 xxx: [ 5142.635045] [<ffffffff811128c8>] unmap_vmas+0x178/0x310
May 7 12:12:29 xxx: [ 5142.635050] [<ffffffff811172fb>] exit_mmap+0xbb/0x190
May 7 12:12:29 xxx: [ 5142.635054] [<ffffffff81064b3d>] mmput+0x4d/0x110
May 7 12:12:29 xxx: [ 5142.635058] [<ffffffff8106b112>] exit_mm+0x112/0x150
May 7 12:12:29 xxx: [ 5142.635063] [<ffffffff81540c15>] ? _spin_lock_irq+0x15/0x20
May 7 12:12:29 xxx: [ 5142.635067] [<ffffffff8106b385>] do_exit+0x125/0x380
May 7 12:12:29 xxx: [ 5142.635071] [<ffffffff8106b635>] do_group_exit+0x55/0xd0
May 7 12:12:29 xxx: [ 5142.635075] [<ffffffff8106b6c7>] sys_exit_group+0x17/0x20
May 7 12:12:29 xxx: [ 5142.635080] [<ffffffff810131b2>] system_call_fastpath+0x16/0x1b

This bug seems to be present since Ubuntu 8.10 when i first encountered it.
Gentoo users observed a similar bug here: http://bugs.gentoo.org/show_bug.cgi?id=214687

[Moritz (msmaster2000)]

To be able to use updates etc. or installing mainline kernels i have edited locale-gen script to prevent
dpkg --configure -a to hardlock the system by calling buggy localedef.
I know that this can cause some problems but i cannot do anything else to get the system usuable again.

[Moritz (msmaster2000)]

AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.21.
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC1: moritz 1483 F.... pulseaudio
CRDA: Error: [Errno 2] No such file or directory
Card0.Amixer.info:
 Card hw:0 'HDMI'/'HDA ATI HDMI at 0xfe210000 irq 19'
   Mixer name : 'ATI R6xx HDMI'
   Components : 'HDA:1002aa01,00aa0100,00100100'
   Controls : 4
   Simple ctrls : 1
Card0.Amixer.values:
 Simple mixer control 'IEC958',0
   Capabilities: pswitch pswitch-joined penum
   Playback channels: Mono
   Mono: Playback [off]
Card1.Amixer.info:
 Card hw:1 'CK804'/'NVidia CK804 with ALC850 at irq 22'
   Mixer name : 'Realtek ALC850 rev 0'
   Components : 'AC97a:414c4790'
   Controls : 42
   Simple ctrls : 27
DistroRelease: Ubuntu 10.04
HibernationDevice: RESUME=UUID=0a952ef8-2b44-4f2e-852b-e48ebdbf0797
InstallationMedia: Ubuntu 10.04 LTS "Lucid Lynx" - Release amd64 (20100429)
Lsusb:
 Bus 002 Device 003: ID 0c0b:001c Dura Micro, Inc. (Acomdata)
 Bus 002 Device 002: ID 046d:c03e Logitech, Inc. Premium Optical Wheel Mouse
 Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
NonfreeKernelModules: fglrx
Package: linux (not installed)
ProcCmdLine: BOOT_IMAGE=/boot/vmlinuz-2.6.32-22-generic root=UUID=005ab1fb-d47f-4b73-b952-023480764e04 ro quiet splash
ProcEnviron:
 LANG=en_US.utf8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.32-22.33-generic 2.6.32.11+drm33.2
Regression: No
RelatedPackageVersions: linux-firmware 1.34
Reproducible: Yes
RfKill:
 0: phy0: Wireless LAN
  Soft blocked: no
  Hard blocked: no
Tags: lucid kconfig needs-upstream-testing
Uname: Linux 2.6.32-22-generic x86_64
UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare
dmi.bios.date: 01/25/2005
dmi.bios.vendor: Phoenix Technologies, LTD
dmi.bios.version: 6.00 PG
dmi.board.name: LP NF4 Series
dmi.board.vendor: DFI Corp,LTD
dmi.board.version: 1.0
dmi.chassis.type: 3
dmi.modalias: dmi:bvnPhoenixTechnologies,LTD:bvr6.00PG:bd01/25/2005:svn:pn:pvr:rvnDFICorp,LTD:rnLPNF4Series:rvr1.0:cvn:ct3:cvr:

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

apport information

[Moritz (msmaster2000)]

I have executed the apport-collect command because Holger cannot start his system.
Hopefully this will trigger something :)

[Moritz (msmaster2000)]

Arrrghhh, too fast :/ please delete unuseful informations ....

[Jeremy Foshee (jeremyfoshee)]

Moritz,
   I'd actually prefer you report a new bug. No matter how similar a bug may look, I'd like to treat them as separate.

Thanks!

~JFo

[Holger (dev-nul-nul)]

We just pointed out, that we both have exactly the same mainboard - DFI Lanparty UT with nforce 4 chipset, me using a Athlon 64X2 and he is using a single core. So it seems to be the same bug.

[Jeremy Foshee (jeremyfoshee)]

Holger,
     While I understand that completely, the Kernel Team is moving to a model whereby all reporters need to file separate bugs for their issues. This includes exact hardware and error matches. The reason for this is that we want to take a close look at all possible items that could affect a particular bug. Additionally, should the fix that resolves your issue not resolve someone else with the same issue, we'd then have all debug information at hand to investigate.

Thanks!

~JFo

[Moritz (msmaster2000)]

Ok, I have some news:

1. I finally checked my BIOS revision and -- sigh! --- updated it to the latest version (N4D623-3)
    This is best done with WinFlash (awdflash under syslinux and freedos did not work properly)

2. Afterwards I cleared CMOS and loaded optimized defaults.

3. I have gathered the recent update of libglibc.

The command: sudo strace locale-gen worked as you can see below:

rt_sigprocmask(SIG_BLOCK, [CHLD], [CHLD], 8) = 0
rt_sigprocmask(SIG_SETMASK, [CHLD], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [CHLD], 8) = 0
rt_sigaction(SIGINT, {0x442510, [], SA_RESTORER, 0x7efff92b6af0}, {SIG_DFL, [], SA_RESTORER, 0x7efff92b6af0}, 8) = 0
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 1803
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 1804
wait4(-1, en_AG.UTF-8... done
  en_AU.UTF-8... done
  en_BW.UTF-8... done
  en_CA.UTF-8... done
  en_DK.UTF-8... done
  en_GB.UTF-8... done
  en_HK.UTF-8... done
  en_IE.UTF-8... done
  en_IN.UTF-8... done
  en_NG.UTF-8... done
  en_NZ.UTF-8... done
  en_PH.UTF-8... done
  en_SG.UTF-8... done
  en_US.UTF-8... done
  en_ZA.UTF-8... done
  en_ZW.UTF-8... done
[{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 1805
rt_sigprocmask(SIG_SETMASK, [CHLD], NULL, 8) = 0
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7efff92b6af0}, {0x442510, [], SA_RESTORER, 0x7efff92b6af0}, 8) = 0
close(4) = -1 EBADF (Bad file descriptor)
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
wait4(-1, 0x7fff2ae21d1c, WNOHANG, NULL) = -1 ECHILD (No child processes)
rt_sigreturn(0xffffffffffffffff) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
read(255, "echo \"Generation complete.\"\n", 6425) = 28
write(1, "Generation complete.\n", 21Generation complete.
) = 21

[Holger (dev-nul-nul)]

Seems like I have found a workaround:
There are 2 options in BIOS, which deal with Erratas.
Errata 94 and 123 Enhancements. I disabled them and tried a dpkg-reconfigure locales.

No error yet.

[Brad Figg (brad-figg)]

This bug was filed against a series that is no longer supported and so is being marked as Won't Fix. If this issue still exists in supported series, ple
ase file a new bug.