Randomly segfault on BYT with 4.4 kernel

[Ying-Chun Liu]

We have a randomly crash bug when using 14.04 (trusty) upstart (1.12.1-0ubuntu4.2).

We've tried the workarounds of LP:1447756 but it doesn't work.
Also the test case doesn't crash. So we think this is a new bug.

The bug is also happened on very early stages. And inside the installer, it crashes more often then the system. We see it core dumped in the installer, but not about to get the core file because the squashfs only keeps file in the memory.

Also when run on system, the filesystem seems to be read-only then so there's also no logs.

We are using 4.4 kernel (linux-image-4.4.0-22-generic 4.4.0-22.40~14.04.1)

I'll paste some images and logs after.

[Ying-Chun Liu (paulliu)]

I tried to remove exit(signum) in crash_handler. And I got a core dump (might not be the first time crash).

Here are the backtrace:
#0 0x00007fa5cef53f90 in sigprocmask () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1 0x0000563682722e92 in ?? ()
No symbol table info available.
#2 <signal handler called>
No symbol table info available.
#3 0x00007fa5cfd6dc9c in nih_list_destroy ()
   from /lib/x86_64-linux-gnu/libnih.so.1
No symbol table info available.
#4 0x00007fa5cfd6c3a9 in nih_discard () from /lib/x86_64-linux-gnu/libnih.so.1
No symbol table info available.
#5 0x0000563682731895 in ?? ()
No symbol table info available.
#6 0x0000563682730ccc in ?? ()
No symbol table info available.
#7 0x0000563682731d0a in ?? ()
No symbol table info available.
#8 0x00005636827369bd in ?? ()
No symbol table info available.
#9 0x00007fa5cfd742ba in nih_main_loop ()
   from /lib/x86_64-linux-gnu/libnih.so.1
No symbol table info available.
#10 0x00005636827225f9 in ?? ()
#11 0x00007fa5cef3eec5 in __libc_start_main ()
   from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#12 0x0000563682722712 in ?? ()

[Ying-Chun Liu (paulliu)]

The bug is happened randomly. In the installer, it is about 90% rate. On system, it is lesser. So the kernel panic is also random in place.

[Ying-Chun Liu (paulliu)]

We've tried upstart 1.13.2 from xenial. I download 1.13.2-0ubuntu16 from Xenial. And re-built it in Trusty chroot (also build all of the reverse dependencies). However the package doesn't provide /sbin/init. Thus we cannot use it.

[Yuan-Chen Cheng (ycheng-twn)]

too old.