UML kernel built in Intrepid SEGFAULTs immediately with buffer overflow (UML bug)

[Roman Yepishev]

Distributor ID: Ubuntu
Description: Ubuntu intrepid (development branch)
Release: 8.10
Codename: intrepid

GCC Version: 4:4.3.1-1ubuntu2

First time noticed in alpha6 in KVM, now the real system has the same problem (both x86)

UML kernel built using Intrepid tools does not boot:

rtg$ Downloads/Linux/linux-2.6.27.1/linux mem=128M ubda=vm/UML/hardy.img
Locating the bottom of the address space ... 0x0
Locating the top of the address space ... 0xc0000000
Core dump limits :
 soft - 0
 hard - NONE
Checking that ptrace can change system call numbers...OK
Checking syscall emulation patch for ptrace...OK
Checking advanced syscall emulation patch for ptrace...OK
Checking for tmpfs mount on /dev/shm...OK
Checking PROT_EXEC mmap in /dev/shm/...OK
Checking for the skas3 patch in the host:
  - /proc/mm...not found: No such file or directory
  - PTRACE_FAULTINFO...not found
  - PTRACE_LDT...not found
UML running in SKAS0 mode
Adding 15499264 bytes to physical memory to account for exec-shield gap
*** buffer overflow detected ***: Downloads/Linux/linux-2.6.27.1/linux terminated
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(__fortify_fail+0x48)[0xb7f4d558]
/lib/tls/i686/cmov/libc.so.6[0xb7f4b680]
/lib/tls/i686/cmov/libc.so.6[0xb7f4af87]
/lib/tls/i686/cmov/libc.so.6(__snprintf_chk+0x34)[0xb7f4ae74]
Downloads/Linux/linux-2.6.27.1/linux[0x8063d11]
======= Memory map: ========
00000000-00001000 rwxp 00000000 00:00 0
08048000-08278000 rwxp 00000000 fe:04 4604241 /home/rtg/Downloads/Linux/linux-2.6.27.1/linux
08278000-0828d000 rwxp 08278000 00:00 0
09134000-09155000 rwxp 09134000 00:00 0 [heap]
09155000-10f10000 rwxs 0110d000 00:14 135092 /dev/shm/vm_file-bCxFhj (deleted)
b7e3f000-b7e4c000 r-xp 00000000 fe:00 40978 /lib/libgcc_s.so.1
b7e4c000-b7e4d000 r-xp 0000c000 fe:00 40978 /lib/libgcc_s.so.1
b7e4d000-b7e4e000 rwxp 0000d000 fe:00 40978 /lib/libgcc_s.so.1
b7e4e000-b7e53000 rwxp b7e4e000 00:00 0
b7e53000-b7fab000 r-xp 00000000 fe:00 41590 /lib/tls/i686/cmov/libc-2.8.90.so
b7fab000-b7fad000 r-xp 00158000 fe:00 41590 /lib/tls/i686/cmov/libc-2.8.90.so
b7fad000-b7fae000 rwxp 0015a000 fe:00 41590 /lib/tls/i686/cmov/libc-2.8.90.so
b7fae000-b7fb1000 rwxp b7fae000 00:00 0
b7fb1000-b7fb3000 r-xp 00000000 fe:00 41610 /lib/tls/i686/cmov/libutil-2.8.90.so
b7fb3000-b7fb4000 r-xp 00001000 fe:00 41610 /lib/tls/i686/cmov/libutil-2.8.90.so
b7fb4000-b7fb5000 rwxp 00002000 fe:00 41610 /lib/tls/i686/cmov/libutil-2.8.90.so
b7fb5000-b7fb7000 rwxp b7fb5000 00:00 0
b7fb7000-b7fd1000 r-xp 00000000 fe:00 41116 /lib/ld-2.8.90.so
b7fd1000-b7fd2000 r-xp b7fd1000 00:00 0 [vdso]
b7fd2000-b7fd3000 r-xp 0001a000 fe:00 41116 /lib/ld-2.8.90.so
b7fd3000-b7fd4000 rwxp 0001b000 fe:00 41116 /lib/ld-2.8.90.so
bfabf000-bfad4000 rwxp bffeb000 00:00 0 [stack]
Segmentation fault

The same kernel built on Hardy boots properly in Hardy and Intrepid.

The reason of such crash is invalid size of array holding the socket name. The structure sockaddr_un contains sun_path of 108 chars long while os_create_unix_socket (called by mconsole_init) passes a file argument which is 256 chars long. Buffer overflow protection fires and abort()s the execution.

[Roman Yepishev (rye)]

Built a debug version:

rtg$ gdb ./linux
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
(gdb) set args ubda=/home/rtg/test.img
(gdb) run
Starting program: /home/rtg/Downloads/Linux/linux-2.6.27.1/linux ubda=/home/rtg/test.img
Locating the bottom of the address space ...
Program received signal SIGSEGV, Segmentation fault.
0x08087451 in page_ok (page=0) at arch/um/os-Linux/sys-i386/task_size.c:31
31 n = *address;
(gdb) l
26 * still in the kernel area. As a sanity check, we'll fail if
27 * the mmap succeeds, but gives us an address different from
28 * what we wanted.
29 */
30 if (setjmp(buf) == 0)
31 n = *address;
32 else {
33 mapped = mmap(address, UM_KERN_PAGE_SIZE,
34 PROT_READ | PROT_WRITE,
35 MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
(gdb) bt
#0 0x08087451 in page_ok (page=0) at arch/um/os-Linux/sys-i386/task_size.c:31
#1 0x08087691 in os_get_top_address () at arch/um/os-Linux/sys-i386/task_size.c:100
#2 0x0804b7f1 in linux_main (argc=2, argv=0xbfa2d734) at arch/um/kernel/um_arch.c:277
#3 0x0804cdf0 in main (argc=2, argv=0xbfa2d734, envp=0xbfa2d740) at arch/um/os-Linux/main.c:150
(gdb)

The crash happens upon n=*address assignment when address equals to 0x0. Actually running both uml kernels in gdb produce this result while hardy one operates fine when running outside of debugger.

[Roman Yepishev (rye)]

Ok, null pointer dereferencing has nothing to do with the original bug, it was UML way to probe for address space setting up a custom segv handler.

So further investigation led me to the following:
522 addr.sun_family = AF_UNIX;
(gdb) bt
#0 os_create_unix_socket (file=0xb45be30 "/home/rtg/.uml/fYodC2/mconsole", len=256, close_on_exec=1) at arch/um/os-Linux/file.c:522
#1 0x0804c0b8 in mconsole_init () at arch/um/drivers/mconsole_kern.c:794
#2 0x08064505 in do_one_initcall (fn=0x804c02a <mconsole_init>) at init/main.c:715
#3 0x080492a9 in kernel_init (unused=0x0) at init/main.c:754
#4 0x0808184c in run_kernel_thread (fn=0x80491b0 <kernel_init>, arg=0x0, jmp_ptr=0xb45ae80) at arch/um/os-Linux/process.c:267
#5 0x080670cb in new_thread_handler () at arch/um/kernel/process.c:151
#6 0x00000000 in ?? ()
(gdb) s
65 return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
(gdb) s
*** buffer overflow detected ***: /home/rtg/Downloads/Linux/linux-2.6.27.1/linux-intrepid terminated

__builtin___snprintf_chk is not called on kernel built with Hardy toolchain, it goes directly to vsnprintf.

I guess this bug should go to glibc package now

[Roman Yepishev (rye)]

Moving to glibc as check failure happens with __snprintf_chk on (looks like) legitimate input.

[Roman Yepishev (rye)]

Fixed.

It is even not a glibc problem, it is UML one.

For the history: The actual crash happens inside os_create_unix_socket. Length is char[256] while in sockaddr_un sun_path which receives the file argument it is only 108 bytes long. __snprintf_chk catches this condition as we have a potential buffer overflow.

The short term fix is to tweak arch/um/drivers/mconsole_kern.c as follows:

...
 static int __init mconsole_init(void)
 {
      /* long to avoid size mismatch warnings from gcc */
      long sock;
      int err;
- char file[256];
+ char file[106];

      if (umid_file_name("mconsole", file, sizeof(file)))
          return -1;
...

This should be patched in kernel. Any build of UML in Intrepid or any other distro which uses _FORTIFY_SOURCE by default will fail with the cryptic messages as above.

[Roman Yepishev (rye)]

Moved to linux as it is a UML kernel bug.

[Leann Ogasawara (leannogasawara)]

I believe this should be resolved in Karmic which contains the following patch. I'm marking this Fix Released for now. Thanks.

ogasawara@yoji:~/ubuntu-karmic$ git show 361371201b60ffd686a694c848c1d5ad6061725f
commit 361371201b60ffd686a694c848c1d5ad6061725f
Author: Balbir Singh <email address hidden>
Date: Tue Dec 9 13:14:07 2008 -0800

    uml: boot broken due to buffer overrun

    mconsole_init() passed 256 bytes as length in os_create_unix_socket, while
    the sizeof UNIX_PATH_MAX is 108. This patch fixes that problem and avoids
    a big overrun bug reported on UML bootup.