mawk memory corruption on recent tzdb data

[Paul Eggert]

mawk corrupts memory and dumps core when processing recent tzdb releases. Although Ubuntu users can work around the problem by using 'make AWK=gawk', it would be better if ordinary 'make' worked (where AWK defaults to awk, and awk on Ubuntu defaults to mawk.

Since this is memory corruption there may well be a security vulnerability in mawk. I have not checked for this, though.

A simple fix would be to upgrade mawk to the current upstream release. I see that there's already a request to do that; see Bug#1332114. I don't know why Debian and Ubuntu are wedged on an ancient upstream version.

To reproduce the problem, download the most recent tzdb release and run 'make AWK=mawk vanguard.zi'. A shell transcript follows. I ran this on Ubuntu 16.04.4 LTS x86-64; 'dpkg -s mawk' reports 'Version: 1.3.3-17ubuntu2'. The shell commands I ran were:

wget https://www.iana.org/time-zones/repository/releases/tzdb-2018e.tar.lz
tar xf tzdb-2018e.tar.lz
cd tzdb-2018e
make AWK=mawk vanguard.zi

Here's the behavior I observed:

$ wget https://www.iana.org/time-zones/repository/releases/tzdb-2018e.tar.lz
--2018-07-18 04:09:59-- https://www.iana.org/time-zones/repository/releases/tzdb-2018e.tar.lz
Resolving www.iana.org (www.iana.org)... 192.0.32.8, 2620:0:2d0:200::8
Connecting to www.iana.org (www.iana.org)|192.0.32.8|:443... connected.
HTTP request sent, awaiting response... 302 FOUND
Location: https://data.iana.org/time-zones/releases/tzdb-2018e.tar.lz [following]
--2018-07-18 04:10:00-- https://data.iana.org/time-zones/releases/tzdb-2018e.tar.lz
Resolving data.iana.org (data.iana.org)... 72.21.81.189, 2606:2800:11f:bb5:f27:227f:1bbf:a0e
Connecting to data.iana.org (data.iana.org)|72.21.81.189|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 437679 (427K) [application/x-tar]
Saving to: ‘tzdb-2018e.tar.lz’

tzdb-2018e.tar.lz 100%[===================>] 427.42K --.-KB/s in 0.06s

2018-07-18 04:10:00 (6.49 MB/s) - ‘tzdb-2018e.tar.lz’ saved [437679/437679]

$ tar xf tzdb-2018e.tar.lz
$ cd tzdb-2018e
$ make AWK=mawk vanguard.zi
mawk -v DATAFORM=`expr vanguard.zi : '\(.*\).zi'` -f ziguard.awk \
   africa antarctica asia australasia europe northamerica southamerica etcetera systemv factory backward >vanguard.zi.out
*** Error in `mawk': malloc(): memory corruption: 0x0000000001ebc4f0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fb09870f7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8213e)[0x7fb09871a13e]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7fb09871c184]
mawk[0x40ff0f]
mawk[0x405dff]
mawk[0x40e1e0]
mawk[0x406b6e]
mawk[0x40185d]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fb0986b8830]
mawk[0x40188d]
======= Memory map: ========
00400000-0041b000 r-xp 00000000 08:01 2622228 /usr/bin/mawk
0061a000-0061b000 r--p 0001a000 08:01 2622228 /usr/bin/mawk
0061b000-0061d000 rw-p 0001b000 08:01 2622228 /usr/bin/mawk
0061d000-00621000 rw-p 00000000 00:00 0
01ea0000-01ec1000 rw-p 00000000 00:00 0 [heap]
7fb094000000-7fb094021000 rw-p 00000000 00:00 0
7fb094021000-7fb098000000 ---p 00000000 00:00 0
7fb098482000-7fb098498000 r-xp 00000000 08:01 3019293 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb098498000-7fb098697000 ---p 00016000 08:01 3019293 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb098697000-7fb098698000 rw-p 00015000 08:01 3019293 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb098698000-7fb098858000 r-xp 00000000 08:01 3018864 /lib/x86_64-linux-gnu/libc-2.23.so
7fb098858000-7fb098a58000 ---p 001c0000 08:01 3018864 /lib/x86_64-linux-gnu/libc-2.23.so
7fb098a58000-7fb098a5c000 r--p 001c0000 08:01 3018864 /lib/x86_64-linux-gnu/libc-2.23.so
7fb098a5c000-7fb098a5e000 rw-p 001c4000 08:01 3018864 /lib/x86_64-linux-gnu/libc-2.23.so
7fb098a5e000-7fb098a62000 rw-p 00000000 00:00 0
7fb098a62000-7fb098b6a000 r-xp 00000000 08:01 3018856 /lib/x86_64-linux-gnu/libm-2.23.so
7fb098b6a000-7fb098d69000 ---p 00108000 08:01 3018856 /lib/x86_64-linux-gnu/libm-2.23.so
7fb098d69000-7fb098d6a000 r--p 00107000 08:01 3018856 /lib/x86_64-linux-gnu/libm-2.23.so
7fb098d6a000-7fb098d6b000 rw-p 00108000 08:01 3018856 /lib/x86_64-linux-gnu/libm-2.23.so
7fb098d6b000-7fb098d91000 r-xp 00000000 08:01 3018860 /lib/x86_64-linux-gnu/ld-2.23.so
7fb098f69000-7fb098f6d000 rw-p 00000000 00:00 0
7fb098f8f000-7fb098f90000 rw-p 00000000 00:00 0
7fb098f90000-7fb098f91000 r--p 00025000 08:01 3018860 /lib/x86_64-linux-gnu/ld-2.23.so
7fb098f91000-7fb098f92000 rw-p 00026000 08:01 3018860 /lib/x86_64-linux-gnu/ld-2.23.so
7fb098f92000-7fb098f93000 rw-p 00000000 00:00 0
7ffc1066f000-7ffc10690000 rw-p 00000000 00:00 0 [stack]
7ffc106a1000-7ffc106a4000 r--p 00000000 00:00 0 [vvar]
7ffc106a4000-7ffc106a6000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Aborted (core dumped)
Makefile:565: recipe for target 'vanguard.zi' failed
make: *** [vanguard.zi] Error 134
$

[Robie Basak (racb)]

Thank you for taking the time to report this bug and helping to make Ubuntu better.

Reproduced on Xenial. I had to install "make" and "lzip".

[Paul Eggert (eggert-cs)]

This bug is fixed in Ubuntu 20.10 (and probably earlier versions), as Ubuntu 20.10 uses mawk 1.3.4 20200120. So you can close the bug report. (I'd close the bug report myself but don't seem to have the bits to do that.)