apache2 crashed with SIGSEGV in ap_run_mpm_query()

[Stephan Kern]

Freshly installed on a VM. All updates applied. Installed apache mysql and php. Got this error after installing.

Description: Ubuntu Zesty Zapus (development branch)
Release: 17.04

Apache version
apache2:
  Installed: 2.4.25-3ubuntu2
  Candidate: 2.4.25-3ubuntu2
  Version table:
 *** 2.4.25-3ubuntu2 500
        500 http://us.archive.ubuntu.com/ubuntu zesty/main amd64 Packages
        100 /var/lib/dpkg/status

php version

php:
  Installed: 1:7.0+49
  Candidate: 1:7.0+49
  Version table:
 *** 1:7.0+49 500
        500 http://us.archive.ubuntu.com/ubuntu zesty/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu zesty/main i386 Packages
        100 /var/lib/dpkg/status

What i expected. Normal apache startup with no crash.
What actually happened. It crashed.

ProblemType: Crash
DistroRelease: Ubuntu 17.04
Package: apache2-bin 2.4.25-3ubuntu2
ProcVersionSignature: Ubuntu 4.10.0-8.10-generic 4.10.0-rc8
Uname: Linux 4.10.0-8-generic x86_64
ApportVersion: 2.20.4-0ubuntu2
Architecture: amd64
Date: Fri Feb 24 23:06:38 2017
ExecutablePath: /usr/sbin/apache2
InstallationDate: Installed on 2017-02-22 (3 days ago)
InstallationMedia: Xubuntu 17.04 "Zesty Zapus" - Alpha amd64 (20170220)
ProcCmdline: /usr/sbin/apache2 -k start
ProcEnviron:
 PATH=(custom, no user)
 LANG=C
SegvAnalysis:
 Segfault happened at: 0x562d6e249ef4 <ap_run_mpm_query+20>: mov 0x18(%rax),%rbx
 PC (0x562d6e249ef4) ok
 source "0x18(%rax)" (0x7ff6705b0fb0) not located in a known VMA region (needed readable region)!
 destination "%rbx" ok
SegvReason: reading unknown VMA
Signal: 11
SourcePackage: apache2
StacktraceTop:
 ?? ()
 <signal handler called>
 __do_global_dtors_aux () from /lib/x86_64-linux-gnu/libnss_nis.so.2
 _dl_fini () at dl-fini.c:235
 __run_exit_handlers (status=0, listp=0x7ff66fb545d8 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:83
Title: apache2 crashed with SIGSEGV in <signal handler called>()
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

[Apport retracing service (apport)]

StacktraceTop:
 ?? ()
 <signal handler called>
 __do_global_dtors_aux () from /tmp/apport_sandbox_RntTSy/lib/x86_64-linux-gnu/libnss_nis.so.2
 ?? ()
 ?? ()

[Christian Ehrhardt  (paelzer)]

Hi Stephan,
I was trying to reproduce today.

On Zesty:
$ apt-get install apache2 php7 mysql-server
$ service mysql status
$ service apache2 status

mysql and apache running fine, no crash in my retry.

checking versions:
apache2 2.4.25-3ubuntu2
mysql-server 5.7.17-0ubuntu1
php7.0 7.0.15-1ubuntu
php 1:7.0+49

It seems that does not trigger for me.
Is that for you 100% reproducable e.g. still on restarting services or was that something that happened only once?

Also you said you "just installed", so did your steps towards the issue match the ones I described above?

[Stephan Kern (swmmrman)]

Happened one time right after the install. Your steps matched. I tried to recreate this today. And it was not reproducable. I was able to recreate it before reinstalling the VM. However my host windows install had some driver corruption that may have impacted the VM install. This can be closed as no longer reproducable.

[Christian Ehrhardt  (paelzer)]

Ok, thanks Stephan.
If you'd be ok please mark it non-private to allow others to search for it - just in case we will get more hits by others to the same issue.

[oldrocker99 (fhwri99)]

This happened just after bootup.

[Brian Murray (brian-murray)]

StacktraceTop:
 ?? ()
 <signal handler called>
 __do_global_dtors_aux () from /mnt/sec-machines/apport-sandbox-dir/Ubuntu 17.04/amd64/report-sandbox/lib/x86_64-linux-gnu/libnss_nis.so.2
 ?? () from /mnt/sec-machines/apport-sandbox-dir/Ubuntu 17.04/amd64/report-sandbox/lib64/ld-linux-x86-64.so.2
 __run_exit_handlers (status=0, listp=0x7ff66fb545d8 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:83

[Brian Murray (brian-murray)]

I got a better stacktrace, so I'm flipping this back to New.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christian Ehrhardt  (paelzer)]

Thanks Brian for the extra stacktrace,
I found two more reports that seem to be the same to me - I dupped them here to focus efforts.

All reports seem to be 17.4 (x2) and 17.10, so rather new apache versions.

Is there any way to reliably reproduce known yet?

[Andreas Hasenack (ahasenack)]

Hmm, I'm getting crash reports, let's see what they are about:
ubuntu@zesty-apache-segfault:~$ l /var/crash/
total 2.0K
drwxrwxrwt 2 root root 2 Aug 11 01:18 .
drwxr-xr-x 14 root root 16 Aug 25 20:55 ..
ubuntu@zesty-apache-segfault:~$ sudo su -
root@zesty-apache-segfault:~# service apache2 restart
root@zesty-apache-segfault:~# l /var/crash/
total 704K
drwxrwxrwt 2 root root 4 Aug 25 20:59 .
drwxr-xr-x 14 root root 16 Aug 25 20:55 ..
-rwxr-xr-x 1 root root 0 Aug 25 20:59 .lock
-rw-r----- 1 root root 699K Aug 25 20:59 _usr_sbin_apache2.0.crash

[Andreas Hasenack (ahasenack)]

I tried apport-unpack on this file, but it crashed ;)

[Brian Murray (brian-murray)]

Andreas - could you open a bug report of apport-unpack failing to work with the .crash file you provided? Thanks in advance.

[Andreas Hasenack (ahasenack)]

It already existed: https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1427600

$ apport-unpack _usr_sbin_apache2.0.crash output-dir
Traceback (most recent call last):
  File "/usr/bin/apport-unpack", line 73, in <module>
    pr.extract_keys(f, bin_keys, dir)
  File "/usr/lib/python3/dist-packages/problem_report.py", line 270, in extract_keys
    [item for item, element in b64_block.items() if element is False])
ValueError: ['UserGroups'] has no binary content

The crash file has:
(...)
Uname: Linux 4.10.0-32-generic x86_64
UserGroups:
CoreDump: base64
 H4sICAA(...)