segfault in server/mpm/event/event.c:process_socket

Bug #1630413 reported by Ian Wienand on 2016-10-05
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
apache2 (Ubuntu)
Undecided
Unassigned

Bug Description

We have seen consistent but infrequent segfaults of apache on a trusty production server with 2.4.7-1ubuntu4.13 (for more examples, see [1])

---
Oct 2 19:01:03 static kernel: [8029151.932468] apache2[10642]: segfault at 7fac797803a8 ip 00007fac90b345e0 sp 00007fac84ff8e20 error 6 in mod_mpm_event.so[7fac90b2e000+d000]
---

Taking the ip - base seems to put us at a consistent offset

---
(gdb) p/x 0x7fac90b345e0 - 0x7fac90b2e000
$1 = 0x65e0

$ addr2line -e ./mod_mpm_event.so 0x65e0
/build/apache2-Rau9Dr/apache2-2.4.7/server/mpm/event/event.c:1064
---

which is at the bottom of process_socket(), which looks like

---
  1058 /*
  1059 * Prevent this connection from writing to our connection state after it
  1060 * is no longer associated with this thread. This would happen if the EOR
  1061 * bucket is destroyed from the listener thread due to a connection abort
  1062 * or timeout.
  1063 */
  1064 c->sbh = NULL;
  1065 return;
  1066 }
---

1064 seems at least plausible as a faulting location...

Some digging through httpd history reveals that this assignment was removed on the 2.4 branch with commit [2], which seems to be largely based on [3]. Things have been shuffled around so much it's hard to tell exactly what might have avoided us going down this path. Even so I'm honestly not sure how to reproduce it -- on a fairly busy server it's seen at most a few times a day.

[1] http://paste.openstack.org/show/584330/
[2] https://github.com/apache/httpd/commit/043eba1a0a190829c073d9ef084358f6693dbbd2
[3] https://github.com/apache/httpd/commit/285e67883e396f97dc3aad50d9dc345f15220827

Brian Morton (rokclimb15) wrote :

Hi Ian, can you raise ulimit, add CoreDumpDirectory, and install apache2-dbg (will restart to make prior two changes effective)? If you make CoreDumpDirectory /tmp, make sure to move your core dump out of the way before you reboot.

https://httpd.apache.org/dev/debugging.html#crashes

Then you'll get a core dump for analysis. If you post it here I can analyze further.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers