localedef assert failure: malloc(): memory corruption

[Pedro Linha]

When running locale-gen after a vanilla Ubuntu 18.04 installation this error appears. Any locale can not be created or generated.

This was the first occurrence of this bug:

root@knlf1:~# apt install language-pack-br
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  language-pack-br-base
The following NEW packages will be installed:
  language-pack-br language-pack-br-base
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 81.8 kB of archives.
After this operation, 321 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://br.archive.ubuntu.com/ubuntu bionic/main amd64 language-pack-br-base all 1:18.04+20180423 [79.9 kB]
Get:2 http://br.archive.ubuntu.com/ubuntu bionic/main amd64 language-pack-br all 1:18.04+20180423 [1892 B]
Fetched 81.8 kB in 0s (928 kB/s)
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
 LANGUAGE = (unset),
 LC_ALL = "pt_BR.UTF-8",
 LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Selecting previously unselected package language-pack-br-base.
(Reading database ... 58623 files and directories currently installed.)
Preparing to unpack .../language-pack-br-base_1%3a18.04+20180423_all.deb ...
Unpacking language-pack-br-base (1:18.04+20180423) ...
Selecting previously unselected package language-pack-br.
Preparing to unpack .../language-pack-br_1%3a18.04+20180423_all.deb ...
Unpacking language-pack-br (1:18.04+20180423) ...
Setting up language-pack-br (1:18.04+20180423) ...
Setting up language-pack-br-base (1:18.04+20180423) ...
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (pt_BR.UTF-8)
Generating locales (this might take a while)...
  br_FR.UTF-8...malloc(): memory corruption
/usr/sbin/locale-gen: line 177: 3257 Aborted localedef $no_archive -i $input -c -f $charset $locale_alias $locale
 done
Generation complete.

-------------------
This is another one:

root@knlf1:~# apt install language-pack-pt
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  language-pack-pt-base
The following NEW packages will be installed:
  language-pack-pt language-pack-pt-base
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 2570 kB of archives.
After this operation, 12.5 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://br.archive.ubuntu.com/ubuntu bionic/main amd64 language-pack-pt-base all 1:18.04+20180423 [2568 kB]
Get:2 http://br.archive.ubuntu.com/ubuntu bionic/main amd64 language-pack-pt all 1:18.04+20180423 [1896 B]
Fetched 2570 kB in 0s (9185 kB/s)
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
 LANGUAGE = (unset),
 LC_ALL = "pt_BR.UTF-8",
 LANG = "pt_BR.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Selecting previously unselected package language-pack-pt-base.
(Reading database ... 62304 files and directories currently installed.)
Preparing to unpack .../language-pack-pt-base_1%3a18.04+20180423_all.deb ...
Unpacking language-pack-pt-base (1:18.04+20180423) ...
Selecting previously unselected package language-pack-pt.
Preparing to unpack .../language-pack-pt_1%3a18.04+20180423_all.deb ...
Unpacking language-pack-pt (1:18.04+20180423) ...
Setting up language-pack-pt (1:18.04+20180423) ...
Setting up language-pack-pt-base (1:18.04+20180423) ...
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (pt_BR.UTF-8)
Generating locales (this might take a while)...
  pt_BR.UTF-8...malloc(): memory corruption
/usr/sbin/locale-gen: line 177: 6523 Aborted (core dumped) localedef $no_archive -i $input -c -f $charset $locale_alias $locale
 done
  pt_PT.UTF-8...malloc(): memory corruption
/usr/sbin/locale-gen: line 177: 6540 Aborted (core dumped) localedef $no_archive -i $input -c -f $charset $locale_alias $locale
 done
Generation complete.

-----------------------

The first CPU on /proc/cpuinfo

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 87
model name : Intel(R) Xeon Phi(TM) CPU 7250F @ 1.40GHz
stepping : 1
microcode : 0xffff0190
cpu MHz : 999.724
cache size : 1024 KB
physical id : 0
siblings : 272
core id : 0
cpu cores : 68
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch ring3mwait cpuid_fault epb pti tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms avx512f rdseed adx avx512pf avx512er avx512cd xsaveopt dtherm ida arat pln pts
bugs : cpu_meltdown spectre_v1 spectre_v2
bogomips : 2793.34
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
power management:

(the processor count goes down to 271).

root@knlf1:~# free
              total used free shared buff/cache available
Mem: 115357040 994456 113797016 2644 565568 113522356
Swap: 97654780 0 97654780
root@knlf1:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 57651028 0 57651028 0% /dev
tmpfs 11535704 2644 11533060 1% /run
/dev/sda1 95789468 1629624 89250944 2% /
tmpfs 57678520 0 57678520 0% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 57678520 0 57678520 0% /sys/fs/cgroup
/dev/sda6 575657040 73756 546271748 1% /scr-knlf1
tmpfs 11535704 0 11535704 0% /run/user/1000
root@knlf1:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04 LTS
Release: 18.04
Codename: bionic
root@knlf1:~#

Thanks in advance!

ProblemType: Crash
DistroRelease: Ubuntu 18.04
Package: libc-bin 2.27-3ubuntu1
ProcVersionSignature: Ubuntu 4.15.0-20.21-generic 4.15.17
Uname: Linux 4.15.0-20-generic x86_64
ApportVersion: 2.20.9-0ubuntu7
Architecture: amd64
AssertionMessage: malloc(): memory corruption
Date: Thu May 3 12:19:41 2018
Dependencies:
 gcc-8-base 8-20180414-1ubuntu2
 libc6 2.27-3ubuntu1
 libgcc1 1:8-20180414-1ubuntu2
ExecutablePath: /usr/bin/localedef
ExecutableTimestamp: 1523909660
ProcCmdline: localedef -i br_FR -c -f UTF-8 -A /usr/share/locale/locale.alias br_FR.UTF-8
ProcCwd: /root
ProcEnviron:
 LANG=pt_BR.UTF-8
 TERM=xterm-256color
 SHELL=/bin/bash
 PATH=(custom, no user)
Signal: 6
SourcePackage: glibc
StacktraceTop:
 __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7f246674cb9a "%s\n") at ../sysdeps/posix/libc_fatal.c:181
 malloc_printerr (str=str@entry=0x7f246674ae0e "malloc(): memory corruption") at malloc.c:5350
 _int_malloc (av=av@entry=0x7f2466981c40 <main_arena>, bytes=bytes@entry=4064) at malloc.c:3738
 __GI___libc_malloc (bytes=4064) at malloc.c:3057
 ?? ()
Title: localedef assert failure: malloc(): memory corruption
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

[Apport retracing service (apport)]

StacktraceTop:
 __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7f246674cb9a "%s\n") at ../sysdeps/posix/libc_fatal.c:181
 malloc_printerr (str=str@entry=0x7f246674ae0e "malloc(): memory corruption") at malloc.c:5350
 _int_malloc (av=av@entry=0x7f2466981c40 <main_arena>, bytes=bytes@entry=4064) at malloc.c:3738
 __GI___libc_malloc (bytes=4064) at malloc.c:3057
 ?? ()

[Seth Arnold (seth-arnold)]

I'm able to add and remove this langpack from my (much more pedestrian) i7 laptop; do you notice other problems on this system? e.g. can you do a kernel build on it without trouble? (I'm surprised something that feels as basic as locales generation fails, when it manages to get to a prompt..)

Thanks

[Gunnar Hjalmarsson (gunnarhj)]

It looks like you set pt_BR.UTF-8 in /etc/default/locale before that locale had been generated. Does it make a difference if you set a locale which exists (such as en_US.UTF-8), reboot, and then run locale-gen?

[Pedro Linha (pedropl)]

No, it does not make any difference.

Just to be sure I've changed the default locale to C (because en_US.UTF-8 was not compiled neither), reboot and tried locale-gen again. Same error.

Seth, I've installed a bunch of packages after this error appears. It was surprising to me too that a deep malloc error doesn't stopped the machine to install and configure packages.

Anyway I'll do a new fresh ubuntu 18.04 installation ASAP. The problem is that this machine is a headless computing node, so the installation procedure is by netboot packages only (PXE, tfpboot etc).

And, if this error continues, I'll do another try with ubuntu 17.10.

Thanks in advance!

[Pedro Linha (pedropl)]

I've reinstalled ubuntu 18.04 on the same machine. Choose "C" as default locale. The only package that I've choose on installation was OpenSSH server.

Here are the commands on the first login (using a serial-over-lan terminal):

------------------------
root@knlf1:~# locale
LANG=C
LANGUAGE=
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=
root@knlf1:~# locale-gen --lang en_US
Generating locales (this might take a while)...
  en_US.UTF-8...corrupted size vs. prev_size
/usr/sbin/locale-gen: line 177: 2922 Aborted localedef $no_archive -i $input -c -f $charset $locale_alias $locale
 done
Generation complete.
root@knlf1:~#
----------------------

Debbuging a little bit I've found the specific line on locale-gen script that produces the error (localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8).

localedef has a "verbose" output. So...

----------------------

root@knlf1:~# localedef -v -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
/usr/share/i18n/locales/iso14651_t1_common:7120: LC_COLLATE: symbol `pure-ta-zh' not known
[warning] LC_NAME: field `name_gen' not defined
[verbose] LC_CTYPE: table for class "upper": 94715077278256 bytes
[verbose] LC_CTYPE: table for class "lower": 94715077281472 bytes
[verbose] LC_CTYPE: table for class "alpha": 94715077284640 bytes
[verbose] LC_CTYPE: table for class "digit": 94715077287856 bytes
[verbose] LC_CTYPE: table for class "xdigit": 94715077291072 bytes
[verbose] LC_CTYPE: table for class "space": 94715077294288 bytes
[verbose] LC_CTYPE: table for class "print": 94715077297504 bytes
[verbose] LC_CTYPE: table for class "graph": 94715077300672 bytes
[verbose] LC_CTYPE: table for class "blank": 94715077303888 bytes
[verbose] LC_CTYPE: table for class "cntrl": 94715077307104 bytes
[verbose] LC_CTYPE: table for class "punct": 94715077310320 bytes
[verbose] LC_CTYPE: table for class "alnum": 94715077313488 bytes
[verbose] LC_CTYPE: table for class "combining": 94715077316704 bytes
[verbose] LC_CTYPE: table for class "combining_level3": 94715077319920 bytes
[verbose] LC_CTYPE: table for map "toupper": 0 bytes
[verbose] LC_CTYPE: table for map "tolower": 140258418968639 bytes
[verbose] LC_CTYPE: table for map "totitle": 50331644 bytes
[verbose] LC_CTYPE: table for width: 0 bytes
root@knlf1:~#

-----------------------

Where those huge numbers came from (around 140 terabytes)???

Round 2...

Installed 17.10. Same procedure, same commands:

-----------------------

Ubuntu 17.10 knlf1 ttyS0

root@knlf1:/home/dummy# locale
LANG=C
LANGUAGE=
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=
root@knlf1:/home/dummy# locale-gen --lang en_US
Generating locales (this might take a while)...
  en_US.UTF-8...*** Error in `localedef': corrupted size vs. prev_size: 0x00000000058c2750 ***
/usr/sbin/locale-gen: line 177: 2892 Aborted localedef $no_archive -i $input -c -...