foxtrotgps crashed with SIGSEGV in ___vfprintf_chk()

[Linus Hoppe]

Binary package hint: foxtrotgps

Crash when trying to get a route via openrouteservice

ProblemType: Crash
DistroRelease: Ubuntu 11.04
Package: foxtrotgps 0.99.4+debian3-3
ProcVersionSignature: Ubuntu 2.6.38-8.42-generic-pae 2.6.38.2
Uname: Linux 2.6.38-8-generic-pae i686
NonfreeKernelModules: wl fglrx
Architecture: i386
Date: Wed May 25 09:23:35 2011
ExecutablePath: /usr/bin/foxtrotgps
ProcCmdline: foxtrotgps
ProcEnviron:
 LANGUAGE=de_DE:en
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
SegvAnalysis:
 Segfault happened at: 0xb6b749d0 <___vfprintf_chk+32>: mov (%esi),%eax
 PC (0xb6b749d0) ok
 source "(%esi)" (0x00000000) not located in a known VMA region (needed readable region)!
 destination "%eax" ok
 Stack memory exhausted (SP below stack segment)
SegvReason: reading NULL VMA
Signal: 11
SourcePackage: foxtrotgps
StacktraceTop:
 ___vfprintf_chk (fp=0x0, flag=1, format=0xb6d7836a "%s:%d: ", ap=0xb03a2d98 "\320W \255\001") at vfprintf_chk.c:31
 xmlGenericErrorDefaultFunc () from /usr/lib/libxml2.so.2
 ?? () from /usr/lib/libxml2.so.2
 __xmlRaiseError () from /usr/lib/libxml2.so.2
 ?? () from /usr/lib/libxml2.so.2
Title: foxtrotgps crashed with SIGSEGV in ___vfprintf_chk()
UpgradeStatus: Upgraded to natty on 2011-05-24 (0 days ago)
UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare vboxusers

[Timo Juhani Lindfors (timo-lindfors)]

Thanks for the report. I work on Debian but here's some information:

If I select

Start 60.121075,24.460361
End 60.121075,24.460361
Service openrouteservice.org

I get

do_pickpoint():
close(14) in netlib_connectsock()
close(14) in netlib_connectsock()
close(14) in netlib_connectsock()
close(14) in netlib_connectsock()
fetch_track(): 60.121075,24.460361, 60.116558,24.448656
URL ROUTE www.tangogps.org/friends/navtrack.php?service=1&start=60.121075,24.460361&end=60.116558,24.448656
close(15) in netlib_connectsock()
HTTP-GET: size: 24, statuscode 203
noname.xml:1: parser error : Start tag expected, '<' not found
temporarily out of order
^
Failed to parse document
close(14) in netlib_connectsock()
close(14) in netlib_connectsock()

but no crash (this is foxtrotgps 0.99.4+debian3-3).

We have no control over www.tangogps.org. Can you reproduce the crash at will? Could you capture the network traffic between you and www.tangogps.org when that happens so that I could replay it?

You can record it without root privileges if you do

"strace -o foxtrotgps.strace -s4096 -f foxtrotgps"

and attach the foxtrotgps.strace here.

[Timo Juhani Lindfors (timo-lindfors)]

The CoreDump.gz that you attached shows at least one occurence of a HTTP reply that has just "temporarily out of order" in it.

I looked at the coredump with gdb. If I go to frame 7 I see that gpx_string contains just "temporarily out of order".

[Timo Juhani Lindfors (timo-lindfors)]

Hmm, www.tangogps.org seems to be down completely now. Can't test the bug anymore :-/

[Linus Hoppe (linus-hoppe-deactivatedaccount)]

Can reproduce. The first time I try there was an error: "temporarily out of order". when clicking a second time on "OK", foxtrotgps crashed. Output for "strace -o foxtrotgps.strace -s4096 -f foxtrotgps":

(foxtrotgps:2603): Gtk-CRITICAL **: IA__gtk_toolbar_set_icon_size: assertion `icon_size != GTK_ICON_SIZE_INVALID' failed
(foxtrotgps:2603): Gtk-CRITICAL **: IA__gtk_toolbar_set_icon_size: assertion `icon_size != GTK_ICON_SIZE_INVALID' failed
GCONF:
 -- name: OSM
 -- uri: http://tile.openstreetmap.org/%d/%d/%d.png
 -- dir: /home/linus/.maps/OSM
GCONF:
 -- name: Maps-for-free.com
 -- uri: maps-for-free
 -- dir: /home/linus/Maps/maps4free
GCONF:
 -- name: Opencyclemap
 -- uri: http://a.andy.sandbox.cloudmade.com/tiles/cycle/%d/%d/%d.png
 -- dir: /home/linus/Maps/opencyclemap
GCONF:
 -- name: Google Maps (testing only)
 -- uri: http://mt0.google.com/vt/hl=en&x=%d&y=%d&z=%d
 -- dir: /home/linus/Maps/googlemaps
GCONF:
 -- name: Google Sat (testing only)
 -- uri: http://khm.google.com/kh/v=53&x=%d&y=%d&z=%d
 -- dir: /home/linus/Maps/googlesat
gconf GPSD address not set
gconf GPSD port not set
*** on_drawingarea1_configure_event():
pixmap created
close(20) in netlib_connectsock()
[...]
close(20) in netlib_connectsock()
*** on_drawingarea1_configure_event():
pixmap created
close(20) in netlib_connectsock()
[...]
close(20) in netlib_connectsock()
fetch_track(): 60.121075,24.460361, 60.121075,24.460361
URL ROUTE www.tangogps.org/friends/navtrack.php?service=1&start=60.121075,24.460361&end=60.121075,24.460361
HTTP-GET: size: 24, statuscode 203
noname.xml:1: parser error : Start tag expected, '<' not found
temporarily out of order
^
Failed to parse document
close(21) in netlib_connectsock()
fetch_track(): 60.121075,24.460361, 60.121075,24.460361
URL ROUTE www.tangogps.org/friends/navtrack.php?service=1&start=60.121075,24.460361&end=60.121075,24.460361
close(22) in netlib_connectsock()
HTTP-GET: size: 24, statuscode 203
Speicherzugriffsfehler (Speicherabzug geschrieben)

[Linus Hoppe (linus-hoppe-deactivatedaccount)]

Running gdb says:

(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /usr/bin/foxtrotgps
[Thread debugging using libthread_db enabled]

(foxtrotgps:2813): Gtk-CRITICAL **: IA__gtk_toolbar_set_icon_size: assertion `icon_size != GTK_ICON_SIZE_INVALID' failed

(foxtrotgps:2813): Gtk-CRITICAL **: IA__gtk_toolbar_set_icon_size: assertion `icon_size != GTK_ICON_SIZE_INVALID' failed
[New Thread 0xb6518b70 (LWP 2814)]
[New Thread 0xb5bffb70 (LWP 2815)]
GCONF:
 -- name: OSM
 -- uri: http://tile.openstreetmap.org/%d/%d/%d.png
 -- dir: /home/linus/.maps/OSM
GCONF:
 -- name: Maps-for-free.com
 -- uri: maps-for-free
 -- dir: /home/linus/Maps/maps4free
GCONF:
 -- name: Opencyclemap
 -- uri: http://a.andy.sandbox.cloudmade.com/tiles/cycle/%d/%d/%d.png
 -- dir: /home/linus/Maps/opencyclemap
GCONF:
 -- name: Google Maps (testing only)
 -- uri: http://mt0.google.com/vt/hl=en&x=%d&y=%d&z=%d
 -- dir: /home/linus/Maps/googlemaps
GCONF:
 -- name: Google Sat (testing only)
 -- uri: http://khm.google.com/kh/v=53&x=%d&y=%d&z=%d
 -- dir: /home/linus/Maps/googlesat
gconf GPSD address not set
gconf GPSD port not set
*** on_drawingarea1_configure_event():
pixmap created
[New Thread 0xb231bb70 (LWP 2816)]
close(22) in netlib_connectsock()
[...]
close(22) in netlib_connectsock()
[Thread 0xb6518b70 (LWP 2849) exited]
fetch_track(): 60.121075,24.460361, 60.121075,24.460361
[New Thread 0xb6518b70 (LWP 2850)]
URL ROUTE www.tangogps.org/friends/navtrack.php?service=1&start=60.121075,24.460361&end=60.121075,24.460361
HTTP-GET: size: 24, statuscode 203
noname.xml:1: parser error : Start tag expected, '<' not found
temporarily out of order
^
Failed to parse document
[Thread 0xb6518b70 (LWP 2850) exited]
[New Thread 0xb6518b70 (LWP 2851)]
close(23) in netlib_connectsock()
[...]
[New Thread 0xb6518b70 (LWP 2853)]
close(23) in netlib_connectsock()
[Thread 0xb6518b70 (LWP 2853) exited]
fetch_track(): 60.121075,24.460361, 60.121075,24.460361
[New Thread 0xb6518b70 (LWP 2854)]
URL ROUTE www.tangogps.org/friends/navtrack.php?service=1&start=60.121075,24.460361&end=60.121075,24.460361
HTTP-GET: size: 24, statuscode 203

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb6518b70 (LWP 2854)]
___vfprintf_chk (fp=0x0, flag=1, format=0xb756036a "%s:%d: ", ap=0xb6517d98 "\200\303\t\b\001") at vfprintf_chk.c:31
31 vfprintf_chk.c: Datei oder Verzeichnis nicht gefunden.
 in vfprintf_chk.c

[Linus Hoppe (linus-hoppe-deactivatedaccount)]

Maybe the "temporarily out of order"-error is the reason for the crash. I don't know how foxtrotgps handles such errors, but when clicking the second time on "OK" when tangogps is down, foxtrotgps crashed.

[Timo Juhani Lindfors (timo-lindfors)]

Aha, if I click "Ok" two times I can also crash foxtrotgps, even on Debian.

[Linus Hoppe (linus-hoppe-deactivatedaccount)]

Setting to confirmed because the crash was reproduced several times on different machines

[Timo Juhani Lindfors (timo-lindfors)]

I looked at the disassembly of

  void XMLCDECL
  xmlGenericErrorDefaultFunc(void *ctx ATTRIBUTE_UNUSED, const char *msg, ...) {
      va_list args;

     if (xmlGenericErrorContext == NULL)
          xmlGenericErrorContext = (void *) stderr;

      va_start(args, msg);
      vfprintf((FILE *)xmlGenericErrorContext, msg, args);
      va_end(args);
  }

on amd64. It seems that xmlGenericErrorContext is (*__xmlGenericErrorContext()) and seems to evaluate differently in the three calls that are made here. The first two return the same value but the third returns a different value. This causes vfprintf to get NULL as the first argument even though the code appears to guard against that.

The implementation has

void * *
__xmlGenericErrorContext(void) {
    if (IS_MAIN_THREAD)
        return (&xmlGenericErrorContext);
    else
        return (&xmlGetGlobalState()->xmlGenericErrorContext);
}

and it seems that IS_MAIN_THREAD evaluates to zero on all three cases. xmlGetGlobalState(), however, returns first 0x7fffe4019170 and then 0x7fffe4019540. Both have sane data but of course only the first one is updated to refer to stderr, the second one has xmlGenericErrorContext set to zero.

xmlGetGlobalState looks very complicated, I can't immediately see why it would return different value for the same thread during the same function call.

Anyways, if I apply

diff -u libxml2-2.7.8.dfsg/debian/changelog libxml2-2.7.8.dfsg/debian/changelog
--- libxml2-2.7.8.dfsg/debian/changelog
+++ libxml2-2.7.8.dfsg/debian/changelog
@@ -1,3 +1,9 @@
+libxml2 (2.7.8.dfsg-2lindi0) unstable; urgency=low
+
+ * Try to workaround https://bugs.launchpad.net/ubuntu/+source/foxtrotgps/+bug/787953
+
+ -- Timo Lindfors <email address hidden> Wed, 01 Jun 2011 00:55:10 +0300
+
 libxml2 (2.7.8.dfsg-2) unstable; urgency=low

   * xpath.c: Fix a double-freeing error in XPath processing code.
only in patch2:
unchanged:
--- libxml2-2.7.8.dfsg.orig/error.c
+++ libxml2-2.7.8.dfsg/error.c
@@ -70,12 +70,13 @@
 void XMLCDECL
 xmlGenericErrorDefaultFunc(void *ctx ATTRIBUTE_UNUSED, const char *msg, ...) {
     va_list args;
+ void *errorContext = xmlGenericErrorContext;

- if (xmlGenericErrorContext == NULL)
- xmlGenericErrorContext = (void *) stderr;
+ if (errorContext == NULL)
+ errorContext = (void *) stderr;

     va_start(args, msg);
- vfprintf((FILE *)xmlGenericErrorContext, msg, args);
+ vfprintf((FILE *)errorContext, msg, args);
     va_end(args);
 }

to libxml2 I do not see the crash anymore.

I doubt the bug is in libxml2 itself but this information might help in any case. I suspect some threading bug in foxtrotgps.

[Joshua Judson Rosen (rozzin)]

I don't quite understand the code-path that leads to vfprintf() being passed a NULL file-handle, but I see that we're calling xmlCleanupParser() in both load_gpx_string_into_list() and load_gpx_file_into_list(), and we should *never* be calling that, because it basically unloads libxml2.

cf.:

    http://0pointer.de/blog/projects/beware-of-xmlCleanupParser
    http://lists.fedoraproject.org/pipermail/devel/2010-January/129143.html

And, sure enough--if I remove the calls to xmlCleanupParser(), the crash appears to be fixed.

How did this ever work? :)

[Joshua Judson Rosen (rozzin)]

Er, that second URL was supposed to be this (from the libxml2 manual):

    http://xmlsoft.org/html/libxml-parser.html#xmlCleanupParser

[Linus Hoppe (linus-hoppe-deactivatedaccount)]

Timo Juhani Lindfors, should we report this bug upstream (libxml2)?

[Timo Juhani Lindfors (timo-lindfors)]

Linus Hoppe, no. It's a bug in foxtrotgps. As Joshua pointed out, we shouldn't be calling xmlCleanupParser like that.

[Timo Juhani Lindfors (timo-lindfors)]

Linus, what do you mean by Status:Invalid?