crash accessing font info with xfs in fontpath

[Ashley Yakeley]

This occurs even AFTER the fix released in https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/650539 . So please do not mark this bug "duplicate" without first reopening 650539.

Symptoms are exactly the same, when I try to launch Google Earth, Skype or certain other apps my X session crashes.

I am running on a 64bit machine with Ubuntu 10.10, nvidia-current 260.19.06-0ubuntu1 and xserver-xorg-core 2:1.9.0-0ubuntu7.3. Also happens with xserver-xorg-core 2:1.9.0-0ubuntu7.1.

[Jon TURNEY (jon-turney)]

Doing some investigation of this reported crash whilst using xserver 1.9.2 with xfs: http://cygwin.com/ml/cygwin-xfree/2010-11/msg00008.html, I'm able to reproduce this crash with the following sequence of actions:

X &
xterm &
xset fp+ tcp/myfontserver:7100
xlsfonts

Program received signal SIGSEGV, Segmentation fault.
[Switching to thread 4712.0x1220]
0x005b2e29 in doListFontsAndAliases (client=0x125fdc0, c=0x120fce8) at dixfonts.c:611
611 fpe = c->fpe_list[c->current.current_fpe];
(gdb) p c
$1 = (LFclosurePtr) 0x120fce8
(gdb) p *c
$2 = {client = 0x120fce0, num_fpes = 18939104, fpe_list = 0x0, names = 0x0, current = {
    pattern = "▒W a\002\000\000\000ed-medium-r-normal--24-230-75-75-c-240-jisx0208.1983-0\va▒\207\va\000\000\000\000▒▒ \0019\017\000\000\b\000\000\000\000\000\000\000|▒%a\000\000\000\000\001\000\000\000\001\000\000\000▒\207\va\220▒\va@▒\va\000\000\000\000\030▒ \001\t\017\000\000\t\000\000\000\024\004\000\000▒▒%a\000\000\000\000\000\000\000\000\001\000\000\000\020\231\va▒\227\va▒\207\va\000\000\000\000H▒ \001▒\016\000\000\v\000\000\000\024\004\000\000▒▒%a\000\000\000\000\000\000\000\000\001\000\000\000"..., patlen = 62, current_fpe = 5, max_names = 3, list_started = 0, private = 0x12df550},
  saved = {
    pattern = "*\000\000\000\001\000\000\000\020\231\va▒\227\va▒\207\va\000\000\000\000▒&\001i\017\000\000\005\000\000\000\024\004\000\000|▒#a\000\000\000\000\000\000\000\000\001\000\000\000\020\231\va▒\227\va▒\207\va\000\000\000\000▒▒ \0019\017\000\000\b\000\000\000\000\000\000\000|▒%a\000\000\000\000\001\000\000\000\001\000\000\000▒\207\va\220▒\va@▒\va\000\000\000\000\030▒ \001\t\017\000\000\t\000\000\000\024\004\000\000▒▒%a\000\000\000\000\000\000\000\000\001\000\000\000\020\231\va▒\227\va▒\207\va\000\000\000\000H▒ \001▒\016\000\000\v\000\000\000\024\004\000\000▒▒%a\000\000\000\000\000\000\000\000\001\000\000\000"..., patlen = 1, current_fpe = 0, max_names = 65535, list_started = 1, private = 0x125fb10},
  haveSaved = 1, savedName = 0x12df338 "-jis-fixed-medium-r-normal--24-230-75-75-c-240-jisx0208.1983-0",
  savedNameLen = 64}
(gdb)

On further investigation, the reason for the closure c being corrupt seems to be related to the changes added in commit 3ab6cd31 to fix bug #3040.

In doListFontsAndAliases(), if we get a Suspended result when the client is already sleeping with the same closure (I don't really understand enough what the code is doing to know if that's expected or not), then the xinerama_sleep code frees() the closure c, so when it next wakes, the closure c is being used after being freed.

Being a use after free bug possibly explains why the original reporter is able to avoid the crash by changing the sequence of actions slightly.

The crash is not observed after reverting the noted commit, or with xserver 1.8.2

[Jon TURNEY (jon-turney)]

See also https://bugzilla.redhat.com/show_bug.cgi?id=658587

[James Jones (cubanismo)]

Created attachment 40973
Possible fix

Haven't tested this thoroughly, as I'm not convinced early-out is really the right way to handle Xinerama here, but I think this should fix the crashes at least. Feel free to try it out.

[Colin-harrison (colin-harrison)]

Hi,

It's not pretty, but James' patch fixes the crash for me.

Thanks,
Colin Harrison

[Julien Cristau (jcristau)]

*** Bug 31675 has been marked as a duplicate of this bug. ***

[Ashley Yakeley (ashley-semantic)]

Xorg.0.log.old attached.

[Bryce Harrington (bryce)]

That log file just shows a generic lockup, nothing actually usable for diagnosis. Please collect a full backtrace - see http://wiki.ubuntu.com/X/Backtracing for directions.

Offhand, maybe it's an ABI incompatibility - those proprietary applications might need rebuilt against current X or something.

[Ashley Yakeley (ashley-semantic)]

Occurred on opening keepassx, which is not a proprietary app. Apport did not create a crash report.

[Ashley Yakeley (ashley-semantic)]

Occurred while running calibre.

[Ashley Yakeley (ashley-semantic)]

Attached GDB session.

[Ashley Yakeley (ashley-semantic)]

From GDB session above:
0x00007f8df99b5826 in FontFileListNextFontWithInfo ()
   from /usr/lib/libXfont.so.1

This was while running calibre.

[Ashley Yakeley (ashley-semantic)]

OK, this crash happened when I launched Google Earth. I've installed all the relevant debug symbols.

[Ashley Yakeley (ashley-semantic)]

This one happened while running calibre.

[Ashley Yakeley (ashley-semantic)]

This one from KeePassX.

[Ashley Yakeley (ashley-semantic)]

Possibly the same bug, involving NVIDIA and X crashing in doListFontsWithInfo:

https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/581071
https://bugs.launchpad.net/ubuntu/+source/gnome-session/+bug/583525
https://bugs.launchpad.net/ubuntu/+source/nvidia-graphics-drivers/+bug/659527
https://bugs.launchpad.net/ubuntu/+source/nvidia-graphics-drivers/+bug/683016

[Ashley Yakeley (ashley-semantic)]

Similar bug in Fedora, apparently also with an NVIDIA card. https://bugzilla.redhat.com/show_bug.cgi?id=658587

[Ashley Yakeley (ashley-semantic)]

Uninstalling the xfs package fixed the problem.

[Jordan Bradley (jordan-w-bradley)]

I'm experiencing an X crashing issue with Thunderbird, the computer it occurs on has an NVidia chip. Could this be related?

[Bryce Harrington (bryce)]

Thanks for collecting the gdb backtraces; in this case having several is very informative - the code crashes in different places each time, however it's the same routine, so presumably it's the same root cause. Since uninstalling xfs made it go away that makes me think xfs is corrupting the font structure at some point.

Also thanks for listing possible dupe bugs. I've reviewed them and duped a couple, and commented on a couple others that look like they might be dupes but it's unclear.

[Bryce Harrington (bryce)]

Ashely, if you're interested in debugging this particular issue further, here are some actions that could be taken:

1. Reproduce the issue and using gdb with a break set on doListFontsWithInfo and 'print c ' after the segfault to see what the contents of that structure is. This is to see if like in bug #683016 the c->num_fpes value is undefined.

2. Add a check in doListFontsWithInfo() to bail out early if c->fpe_list is NULL. I don't think that would be a proper fix, just paper over the issue, but it might be a good workaround for natty if it works. However if it does work it may simply move the bad behavior to somewhere else...

3. Test the current git version of xfs. There's been a couple changes to how font paths are done in xfs to match what xserver expects; perhaps that better avoids this situation. If you can't reproduce the crash when using this newer xfs, perhaps we simply need to update to newer xfs (or at least backport the font path patches).

[Timo Aaltonen (tjaalton)]

This is known upstream, the workaround is to not use xfs.

[Jeremy Sequoia (jeremyhu)]

James, have you sent this to xorg-devel for review / comment? I didn't see it in a quick check of emails from you in December.

[Julien Cristau (jcristau)]

*** Bug 36060 has been marked as a duplicate of this bug. ***

[Acelists (acelists)]

*** Bug 34007 has been marked as a duplicate of this bug. ***

[Cyril Brulebois (kibi)]

Here's another ping for this bad crasher. :)

[Jeremy Sequoia (jeremyhu)]

I've sent it to xorg-devel. Please review and comment in that thread

[James Jones (cubanismo)]

Discussion regarding why my patch ("Possible fix"), and a proposed patch from Adam Jackson, aren't functionally correct from a long time ago:

< jamesjones> ajax: Did that patch I attached to the font loading crash bug look at all correct?
< ajax> oh man, i seriously just finished writing a patch for that
< ajax> if you beat me to it i'm going to feel very lame
< jamesjones> sorry
< jamesjones> I'm not at all sure it's correct.
< ajax> oh wow, i see what you did.
< jamesjones> It seems like the whole idea of bailing out if you need to sleep ruins the point of Xinerama looping
< ajax> yeah, i don't think that's right
< ajax> one second...
< jamesjones> but I was too lazy to dig deeper since that patch was enough to unblock my testing.
< ajax> jamesjones: give this one a try? http://fpaste.org/BqJf/raw/
< ajax> i'm a bit ill at how that whole thing works, honestly.
< jamesjones> Give me a second, rebuilding
< jamesjones> But yeah, it doesn't make sense to me.
< ajax> at any rate it fixes my testcase of xset fp+ unix/:7100 && xlsfonts
< jturney> heh, it removes code, so it must be right :)
< jamesjones> Hehe
< ajax> nothing like having a library where your app has to implement functions with particular names and signatures to work
< jamesjones> but why the looping, if it's OK to return Success when the operation isn't done on screen[n] where n>0?
< ajax> jamesjones: it's not quite that.
< ajax> most of the font ops don't have any screen state, so you only have to call them once.
< ajax> PolyText and ImageTexto sleep do have to hit every screen, so you want to make sure that they only bump the client's sleep count once if they d
< ajax> so the minimal thing would have been to only change the closures for {Poly,Image}Text to not keep a sleep count
< ajax> but if you're doing that you may as well do them all
< jamesjones> right, I figured it was something like that, but then 1) Why loop the others 2) Why is it OK to early-out in PolyText/ImageText? Does the looping get restarted somewhere I'm not seeing when the client wakes up?
< jamesjones> I suppose I should just load up xinerama and step through it
< ajax> the others aren't actually looped by xinerama
< ajax> (which i had missed the first time around, i just assumed they were)
< jamesjones> ah, k
< jamesjones> that makes more sense.
< ajax> the text rendering routines push themselves onto the work queue if they don't have a font they need because the font server hasn't gotten back to them
< ajax> check out the context around ClientSleep in doImageText, it's quite horrific
< ajax> the way they wake up is that the font server's fd selects readable, and the handler for that notices which request tat reply was for and pokes the client back into the work queue
< ajax> i think that's right anyway? i kind of want to not know.
< jamesjones> it sounds right if there's only one screen
< ajax> well, if there's multiple screens, you might run through the sleep pattern multiple times
< jamesjones> Yeah, but it looks like each screen would loop in and block, only sleeping once
< jamesjones> then when it woke up, only the first screen would be serviced.
< jamesjones> *loop in and detect the need to sleep ra...

[Bryce Harrington (bryce)]

@Timo, there have been a couple other bugs reported like this, where the workaround is to remove xfs. Should we just drop xfs from the archive? We don't need it for anything afaik?

[Daniel Schlieper (schlieper-r)]

Bug #885813 is probably a dupe.

[Julien Cristau (jcristau)]

*** Bug 48900 has been marked as a duplicate of this bug. ***

[vandyswa (ajv-cauriumbin)]

> @Timo, there have been a couple other bugs reported like this, where the workaround is to remove xfs.
> Should we just drop xfs from the archive? We don't need it for anything afaik?

Please note that xfs can run over TCP. So this is roughly like having a crash in ssh, and asking if the fix would be to remove sshd.
We run N (where N >> 1) Ubuntu instances here, and it makes life much, much easier to add fonts at a central server.

[Ajv-741-876-6683 (ajv-741-876-6683)]

Please note that those of us who live with Ubuntu's LTS world will be arriving at 12.04 over the coming months. This bug will jump right out at you if you run an environment with many stations and non-standard collections of fonts. It sure blew *me* right out of the water...

[christopherbalz (christophermbalz)]

Uninstalling xfs, for example by simply using Synaptic (search on 'xfs', and do 'Remove' (but not 'Remove Completely'), worked for me on Ubuntu 12.04 LTS all-up-to-date as of now.

[V (vcgamesii)]

According to the link:
http://ubuntuforums.org/showthread.php?t=2183029

I commented out from /etc/X11/xorg.conf :
#Section "Files"
# FontPath "unix/:7100"
#EndSection

to achieve a similar effect of uninstalling xfs
This seems to be another successful workaround. (Unless xfs is manually called or something).

[Jon TURNEY (jon-turney)]

See also http://patchwork.freedesktop.org/patch/14507/

[penalvch (penalvch)]

Ashley Yakeley, this bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? If so, could you please test for this with the latest development release of Ubuntu? ISO images are available from http://cdimage.ubuntu.com/daily-live/current/ .

If it remains an issue, could you please run the following command in the development release from a Terminal (Applications->Accessories->Terminal), as it will automatically gather and attach updated debug information to this report:

apport-collect -p xorg-server REPLACE-WITH-BUG-NUMBER

Please note, given you already have done this in a prior release, performing this on a release prior to Trusty would not be helpful.

Thank you for your understanding.

Helpful bug reporting tips:
https://wiki.ubuntu.com/ReportingBugs

[Ivan Frederiks (idfred)]

@Christopher M. Penalver
Is xfs package available in trusty?

[Keith Packard (keithp)]

Created attachment 118203
Allow font library to return Suspended more than once per request

fonts: Continue when font calls return Suspended more than once

    Patch 3ab6cd31cbdf8095b2948034fce5fb645422d8da fixed Xinerama
    interactions with font servers by not putting clients to sleep
    multiple times. However, it introduced additional changes dealing with
    libXfont routine returning Suspended more than once for the same
    request. This additional change was to abandon processing of the
    current request and free the closure data by jumping to
    'xinerama_sleep' in each of the functions.

    Font library functions shouldn't return Suspended more than once,
    except for ListFontsWithInfo, which produces multiple replies, and
    thus ends up returning Suspended many times during processing.

    With the jump to xinerama_sleep occurring after the first reply was
    processed, the closure for the request was freed and future calls into
    the ListFontsWithInfo callback resulted in dereferencing freed
    memory.

    This patch removes the added branches, reverting the code to its
    previous behaviour, which permitted multiple Suspended returns and
    simply waited for the client to be signaled again so that the callback
    could continue processing the request.

    Signed-off-by: Keith Packard <email address hidden>
    Cc: Adam Jackson <email address hidden>

[Jon TURNEY (jon-turney)]

Thanks for taking a look at this.

This appears to fix my testcase above, so Tested-by:

I don't know if this is likely to regress bug #3040, or how to test that.

[Paul White (paulw2u)]

Both upstream bug reports were fixed along time ago

An 'Incomplete' bug report raised against Ubuntu 10.10 is
of little use now as the last manual comment was made almost
ten years ago.

Will close as being fixed due to any comments suggesting that
the bug is still extant in a currently supported release.