gnome-keyring-daemon crashed with SIGSEGV in location_manager_hal_init()

[Matt LaPaglia]

Binary package hint: gnome-keyring

I fired up Ubuntu Hardy (with all updates) and I am getting segfaults after I log into GDM. The default "loading background" appears, and I can move my mouse and click CTRL + ALT + F1 to get to a shell.

Issuing "sudo /etc/init.d/gdm stop" takes about 20 seconds for the [OK] to appear.

ProblemType: Crash
Architecture: i386
CrashCounter: 1
Date: Wed Apr 16 16:54:29 2008
DistroRelease: Ubuntu 8.04
ExecutablePath: /usr/bin/gnome-keyring-daemon
NonfreeKernelModules: nvidia
Package: gnome-keyring 2.22.1-1
PackageArchitecture: i386
ProcCmdline: /usr/bin/gnome-keyring-daemon -d --login
ProcCwd: /var/lib/gdm
ProcEnviron:
 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
 LANG=en_US.UTF-8
Signal: 11
SourcePackage: gnome-keyring
StacktraceTop:
 ?? ()
 ?? ()
 ?? ()
 ?? ()
 ?? ()
Title: gnome-keyring-daemon crashed with SIGSEGV
Uname: Linux 2.6.24-16-generic i686
UserGroups: adm admin audio cdrom dialout dip floppy fuse lpadmin plugdev video

[Steve Anelay (steve-anelay)]

I can confirm this bug. Just installed Hardy on SDHC card of Asus eee pc. Have to launch virtual terminal to get to desktop. /var/log/messages shows:

[ 685.989640] gnome-keyring-d[10185]: segfault at 00000014 eip 080759c7 esp bfa29ab0 error 6
[ 686.809061] gnome-keyring-d[10250]: segfault at 00000014 eip 080759c7 esp b7a40dc0 error 6
[ 712.699273] gnome-keyring-d[10371]: segfault at 00000014 eip 080759c7 esp b7a06dc0 error 6

Reinstalling gnome-keyring packages and gdm makes no difference. I've also tried changing the gdmgreeter to the old gdmlogin in gdm.conf, but again no difference. Same thing when reinstalling Hardy. At the first launch I can login through gdm, but after getting to the desktop I get a message about gnome-keyring crashing. After that no more logins are possible.

[webjames (james-olney)]

I confirm this on my desktop, however not my laptop running the same version, both with all updates.

i have been logging in fine, however now i look at dmesg and i see gnome-keyring-d segfault...

no graphical login is possible, however i can login via Ctrl+Alt+F1

[Paul Albrecht (albrecht-rdi1)]

Same problem with all updates applied to hardy heron 8.04rc:

[ 79.911924] [drm] Initialized i915 1.6.0 20060119 on minor 0
[ 79.912922] PCI: Enabling device 0000:00:02.1 (0000 -> 0002)
[ 79.912936] PCI: Setting latency timer of device 0000:00:02.1 to 64
[ 79.915050] [drm] Initialized i915 1.6.0 20060119 on minor 1
[ 84.589104] NET: Registered protocol family 10
[ 84.590723] lo: Disabled Privacy Extensions
[ 84.592772] ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 95.506786] eth1: no IPv6 routers present
[ 98.898495] gnome-keyring-d[5310]: segfault at 00000014 eip 080759c7 esp bfcad8d0 error 6
[ 108.792250] gnome-keyring-d[5433]: segfault at 00000014 eip 080759c7 esp b7a11dc0 error 6

[Paul Albrecht (albrecht-rdi1)]

I don't know if this helps, but here's my .xsession-errors:

/etc/gdm/Xsession: Beginning session setup...
Setting IM through im-switch for locale=en_US.
Start IM through /etc/X11/xinit/xinput.d/all_ALL linked to /etc/X11/xinit/xinput.d/default.
SESSION_MANAGER=local/ubuntu:/tmp/.ICE-unix/5313

** (x-session-manager:5313): WARNING **: couldn't connect to daemon at $GNOME_KEYRING_SOCKET: /tmp/keyring-Gngyx6/socket: Connection refused
ALSA lib control.c:909:(snd_ctl_open_noupdate) Invalid CTL front:0
ALSA lib control.c:909:(snd_ctl_open_noupdate) Invalid CTL front:0
Shutdown failed or nothing to shut down.
xrdb: "*Label.background" on line 220 overrides entry on line 150
xrdb: "*Text.background" on line 226 overrides entry on line 191
xrdb: "*Label.foreground" on line 232 overrides entry on line 151
xrdb: "*Text.foreground" on line 238 overrides entry on line 192

[zehel (3-launchpad-zehel-com)]

I'm getting the same thing.
Here is the tail of my /var/log/messages:
Apr 24 11:23:09 zehel-desktop dhcdbd: message_handler: message handler not found
 under /com/redhat/dhcp/eth0 for sub-path eth0.dbus.get.nis_servers
Apr 24 11:23:09 zehel-desktop dhcdbd: message_handler: message handler not found
 under /com/redhat/dhcp/eth0 for sub-path eth0.dbus.get.interface_mtu
Apr 24 11:23:10 zehel-desktop kernel: [ 86.662554] NET: Registered protocol fa
mily 10
Apr 24 11:23:10 zehel-desktop kernel: [ 86.662731] lo: Disabled Privacy Extens
ions
Apr 24 11:23:43 zehel-desktop kernel: [ 118.591626] gnome-keyring-d[6650]: segf
ault at 00000014 eip 080759c7 esp bf8adb90 error 6
Apr 24 11:23:44 zehel-desktop kernel: [ 119.964952] gnome-keyring-d[6715]: segf
ault at 00000014 eip 080759c7 esp b7a3edc0 error 6
Apr 24 11:24:37 zehel-desktop kernel: [ 172.787138] mtrr: type mismatch for c00
00000,10000000 old: write-back new: write-combining
Apr 24 11:24:39 zehel-desktop kernel: [ 175.152422] gnome-keyring-d[6849]: segf
ault at 00000014 eip 080759c7 esp b7a5edc0 error 6
Apr 24 11:24:43 zehel-desktop kernel: [ 178.526000] UDF-fs: Partition marked re
adonly; forcing readonly mount
Apr 24 11:24:43 zehel-desktop kernel: [ 178.559386] UDF-fs INFO UDF 0.9.8.1 (20
04/29/09) Mounting volume 'DVD7_G1', timestamp 2007/08/10 11:08 (1f10)
zehel@zehel-desktop:~$

[Grugnog (grugnog)]

Ditto, exact same situation as the above - login to gdm causes it to halt permanently on beige screen (and I can still Ctrl-Alt-F1 and log in fine on the terminal). Log messages are also identical, so I won't bother reposting.

I can add however that this bug reproduces on a completely new hardy install (including a new user profile) both before and after all updates are applied. I have 2 other machines also with clean hardy installs that this doesn't occur on however, so this suggests perhaps some kind of hardware related issue.

Interestingly I was able to log on twice successfully on the clean install (although the keyring still crashed shortly after logging in) before this problem started preventing gdm logon at all, even though no changes or updates were applied between it working and starting to fail. This suggests perhaps some kind of race condition.

[Grugnog (grugnog)]

I checked out the trunk for gnome-keyring-daemon from svn.gnome.org, installed gnome-devel and recompiled (installing with --prefix=/usr - ugly, but not likely to break things worse than they are now!). I restarted gdm and could now log in fine (and the daemon did not even crash after login).

The libraries that autogen.sh detected are as follows:
PAM: no
DBus: 1.1.20
HAL: no
SSH Agent: yes
Root certs: no

Assuming that the Ubuntu build is incorporating these dependencies that suggests that the problem is with the PAM or HAL integration. I am not much of a gnome hacker (can you tell!), but I am happy to try things if people have any suggestions.

[Karl Dane (karl-rince)]

Me too! Exactly the same...

[ 112.009494] gnome-keyring-d[5672]: segfault at 00000014 eip 080759c7 esp bfc629a0 error 6
[ 113.411076] gnome-keyring-d[5786]: segfault at 00000014 eip 080759c7 esp b79d6dc0 error 6
[ 319.909153] gnome-keyring-d[5977]: segfault at 00000014 eip 080759c7 esp bfad3b60 error 6
[ 320.928020] gnome-keyring-d[6094]: segfault at 00000014 eip 080759c7 esp b79abdc0 error 6

My hardy is currently unusable.

Thanks,

KD

[jeremyh (jeremy-highley)]

Same exact problem here. 64-bit version on a dell poweredge server. Problem started on upgrade to 8.04 this morning.

[jmar71n (jmar71n)]

I'm getting the exact same problem, I've been using the release candidate which worked perfectly until i did the last set of upgrades about 4 days ago, so the problem is some where in the upgrades from the release candidate to the final release.

The login screen comes up --> login --> Blank orange screen with a mouse curser

Ctrl+Alt+F1-F6 works and runs as it should.

I now have a fresh install of 8.04 from the final release CD and getting the same problems.

[Sebastien Bacher (seb128)]

Thanks for your bug report. Please try to obtain a backtrace http://wiki.ubuntu.com/DebuggingProgramCrash and attach the file to the bug report. This will greatly help us in tracking down your problem.

[Sebastien Bacher (seb128)]

you can also enable apport in /etc/defaults/apport and use it to send a new bug

[Matt LaPaglia (mlapaglia)]

I also have this now on a fresh install on a custom build desktop with hardy, I can't install backtrace software because I have to log into the network at school first.

[Matt LaPaglia (mlapaglia)]

I got the internet working on the custom build, but my backtrace isn't capturing the segfaults.

I can run gnome-keyring-daemon inside of gdb, but it doesn't report any problems, and it also exits normally after outputting the folder it created and the PID of the task. I can't install valgrind, because the site is bogged down with ubuntu hardy down loaders and it is going very slow. I looked at the PID of the keyring-daemon that segfaults, there are two of them for me, but the specific PID is only created immediately after you log in.

I'm new to gdb, is there someway to get a backtrace?

[Matt LaPaglia (mlapaglia)]

"apport-cli -p gnome-keyring-daemon" doesn't find any bugs to be reported either.. I'm still downloading valgrind.

[Steve Anelay (steve-anelay)]

I tried the insructions to do a backtrace but the hardy repositories just give a 404 error. So I enabled apport. Not sure what you need but I've attached the log messages and the /etc/crash messages

[jmar71n (jmar71n)]

I'm getting the same results. After login 2 instances of gnome-keyring-daemon open and close almost instantly, so i haven't been able to get gdb to report any problems. the errors from dmesg are;

[ 451.243188] gnome-keyring-d[6769]: segfault at 00000014 eip 080759c7 esp bf91c9a0 error 6
[ 451.874928] gnome-keyring-d[6935]: segfault at 00000014 eip 080759c7 esp b7a52dc0 error 6

Matt, are you you using a hardware raid controller?
I tried removing my raid card ( adaptec 2410sa, raid 5) and using the motherboard sata ports on one HDD and everything worked fine. But the card worked fine with the release candidate.

[Steve Anelay (steve-anelay)]

Aso attached the /var/log/apport.log.1

[Matt LaPaglia (mlapaglia)]

OK here's the valgrind output, not sure if it's going to help.

[Matt LaPaglia (mlapaglia)]

jmar71n, sorry I didn't see you post before I uploaded the valgrind. No I'm not using a RAID, I'm actually booting off of a USB drive. The live cd worked fine as well.

[Jonathan Heard (jon-launchpad-jeh)]

I've also hit the same bug after live upgrading from Ubuntu Gutsy to Hardy. The Live CD also works fine for me but GDM hangs on login for my disk environment.

I removed the following line from /etc/pam.d/gdm :
session optional pam_gnome_keyring.so auto_start

But obviously now, although my session starts, I have to manually enter things like my WiFi Key as the keyring manager isn't available.

This is a pretty critical bug in my opinion as it cripples the graphical environment in Ubuntu - How did this get through beta??

N.B. I've just had to kill nm-applet as it's been using >90% of my CPU for most of the evening (didn't notice at first) - Not sure if my little PAM hack might have caused a different problem!!

[Arnaldo Candido Junior (arnaldocan)]

Same error here booting ubuntu from an USB-Disk. I found another way to initiliaze with gnome-keyring:

1 - Login in text mode (ctrl+alt+f1)
2 - Run:
$ xinit -- :1
3 - Run on the new window (ctrl+alt+f9 in my case):
$ gnome-session

[zehel (3-launchpad-zehel-com)]

Can we rename Hardy Herron to Heartless Hyena?

[Kristian Rosenvold (kristian-rosenvold)]

I'm booting 64-bit 8.04 off a USB stick and I have the same problem. I have no physical sata/ide disk attached.

[doobydave (dooby-dave)]

I can confirm the usb-boot and nm-applet at 100%. I will try to install on ide hd to see if any different.

[doobydave (dooby-dave)]

No issues when installed to internal drive.

[jmar71n (jmar71n)]

Does anyone know how to get a backtrace from gdb?

I've been following the guide from https://wiki.ubuntu.com/DebuggingProgramCrash & https://wiki.ubuntu.com/Backtrace

As far as i know gdb only works if it is used on a application which is already running or you start a application inside of gdb to capture the the debugging information.

I've tried starting gnome-keyring-daemon inside gdb and get no errors

So my question is how do you get gdb capture debugging info of gonme-keyring-daemon if starts and closes/crashes in 1-2 seconds after the login screen?

[Jonathan Heard (jon-launchpad-jeh)]

I'm possibly noticing a trend here.... I've seen someone say that this bug happens on an Asus EePC which has a flash-based hard disk. Some of the above comments suggest that the bug occurs on a USB Flash drive booted with Hard but not on a real internal disk.
And in my case, my laptop has some thermal issues and kills disks so I'm booting off an 8gb compact flash drive using a CF to IDE convertor.

It seems too wacky to be true, but anyone confirm that they are seeing this bug whilst booted from a real disk?

With reference to getting a proper backtrace, the only place I could find 'gnome-keyring-daemon' mentioned was in a binary file /etc/alternatives/x-session-manager so I can't find a script which I can hijack. I also tried writing an interposing script a /usr/bin/gnome-keyring-daemon but this never got loaded, so I'm clueless about how to get this to work.

Given that another user has successfully downloaded gnome-dev and compiled a working keyring daemon into /usr I'm not hopeful that this problem will be very easy to recreate if we compiled debug binaries.
However if someone from Ubuntu could supply an intrumented/debug binary which is built in the exact same environment as the failing version we might stand a chance??

Whichever way this goes, the bug has to be fixed at it's a show-stopper for Hopeless Hippo :-)

[jmar71n (jmar71n)]

bbauto has a temporary workaround - Bug #221850

""Alt-Ctrl-F2" , "Login" by Textmode. "sudo rm /tmp/.X0-lock" , "startx" then it starts up OK, after that i "Enabled Automatic Login", under "System-->Administration-->Login Window"."

[Grugnog (grugnog)]

@Ovation1357 - I think you are correct that the USB/flash drive is a correlation, however I don't think it is an absolute dependence, because I was using a fresh Ubuntu install on a totally normal HP desktop machine hard disk with no USB/flash in sight. The only factor I can think of is perhaps the hardware RAID controller.

In terms of backtraces etc, I had a similar problem in that I had no obvious way to run it through any of the usual tools, because it is automatically started by the X session management code (without checking if a suitable instance is already running). You can start an instance yourself and then restart gdm (etc) but a new keyring instance is automatically started (and crashes) and the one you started is completely ignored and doesn't crash.

I guess apport is one option. The other options would be to hack and recompile the x-session-manager (or whatever is executing keyring) or else to figure out a functional wrapper script or alias of some kind.

Thanks all for the workarounds, they are certainly easier than recompiling from SVN ;)

[bbauto (berglund-bengt)]

I checked my syslogs and can confirm "gnome-keyring-d [...] segfault..." When booting i also get "usb1-2, device not accepting address 2, error -71, assuming drive cash.." probably not relevant but those are the only err-messages i can find.
Workaround (works for me cause i'm single user on a USB-stick):
"Alt-Ctrl-F2" , "Login" by Textmode. "sudo rm /tmp/.X0-lock" , "startx" then it starts up OK, after that i "Enabled Automatic Login", under "System-->Administration-->Login Window-->Safety..."

[webjames (james-olney)]

I am getting the same problem. I do not use any flash based storage. I have installed the hardy release the live cd boots up fine, i install, restart log in then I'm confronted with the same problem of the original bug post.

I did a clean install with a burned ISO of final release of hardy, not beta or rc. This bug is still present in the release version.

I use two hardware raid arrays. one to store /, swap and one to store /home. My controller is: Adaptec 2610sa

I have been using this card since feisty, perhaps before.
A solution would be nice as i'm having to use the live cd at the moment.

[sunbird (sunbird)]

I can confirm the same issue. I am not using any flash-based storage, just a regular hdd.

I upgraded 7.10 x64 --> 8.04 x64 on my Macbook Pro 3,1. Upgrade appeared to go okay, and the machine starts, but login hangs. I can access tty. Without gdm, 8.04 is a no-go for me. I've reverted back to 7.10.

Any idea when this will be fixed? I use ubuntu as my primary os so this is a big bummer. Why is this only a medium priority???

[jmar71n (jmar71n)]

I'm using an Adaptec 2410sa controller and getting the same problems.

This problem only seems to affect USB drives, solid state discs and raid controllers which i think are all are all SCSI devices.

[webjames (james-olney)]

SUMMARY

Hardware Affected
It seems to me, some people using eee pc's, adaptec raid cards models 2410sa and 2610sa, usb drives.

IDE drives seem unaffected, thanks doobydave.

Two possible solutions have been found, perhaps someone could test them:
1. Grugnog suggest https://bugs.launchpad.net/ubuntu/+source/gnome-keyring/+bug/218434/comments/8
2. bbauto suggests this https://bugs.launchpad.net/ubuntu/+source/gnome-keyring/+bug/218434/comments/32 this is also found in bug: https://bugs.launchpad.net/bugs/221850

perhaps if users are tracking this bug, they could report what hardware they are running hardy on. esp. if it similar to what is mentioned above.

[sunbird (sunbird)]

I *was* running hardy on my macbook pro 3,1. Without a working gdm, I was forced to revert to Gutsy because this is my primary OS. So, I can't test these patches. Here's my hardware info:

description: IDE interface
product: 82801HBM/HEM (ICH8M/ICH8M-E) SATA IDE Controller
vendor: Intel Corporation
physical id:
1f.2
bus info:
pci@0000:00:1f.2
logical name:
scsi2
version: 03
width: 32 bits
clock: 66MHz
capabilities: ide pm bus_master cap_list emulated
configuration:
driver = ata_piix
latency = 0
module = ata_piix

---

id:
disk
description: SCSI Disk
product: FUJITSU MHW2120B
vendor: ATA
physical id:
0.0.0
bus info:
scsi@2:0.0.0
logical name:
/dev/sda
version: 0081
serial: NZ0ST772P2FD
size: 111GB
capabilities: partitioned partitioned:dos
configuration:
ansiversion = 5

I'm sorry, I'm not a coder so I'm not sure if this info will be helpful at all. If there's some other info that is needed, please let me know.

[Sebastien Bacher (seb128)]

could people stop spaming this bug now, what is required is a debug stracktrace or a valgrind log, not random comment about user getting the issue, the bug tracker is not an user forum to add random comments it's a tool to get bugs worked

[Steve Anelay (steve-anelay)]

My valgrind log

[Matt LaPaglia (mlapaglia)]

Sebastien -

How can we get the program to crash in debug mode?

[Jonathan Heard (jon-launchpad-jeh)]

I've been doing a bit of investigation into this where I can spare the time. The gnome-keyring-daemon is getting started by the gdm PAM module:
gnome-keyring-2.22.1/pam/gkr-pam-module.c
In function setup_child()
Line
274 char *args[] = { GNOME_KEYRING_DAEMON, "-d", "--login", NULL};
then
Lines
329 /* Now actually execute the process */
330 execve (args[0], args, pam_getenvlist (ph));

I booted up and logged in on TTY1 then made and ran the following script:
#!/bin/bash
export PID=""
while [[ $PID == "" ]]; do
PID=`pgrep keyring`
done
gdb -p $PID

Then I switched back to X on TTY7 and logged in. The big bonus of latching gdb to the PID is that it actually halts execution. If we've been lucky with the timing, the login screen freezes with the password hash greyed out.
Switching back to TTY1 we should find gdb waiting for instructions.

The problem is, that gnome-keyring-manager is a stripped binary and so the stack-backtraces at any point have no symbol names (I hope I'm not mis-understanding the meaning of a stripped binary!!!). I've tried including the path to the source for GDM but I still can't get it to add any symbols names.
I presume the lack of symbols is because it handles security. It may be necessary to build a binary which is not stripped in order to get a meaningful backtrace.

If I use 'cont' to continue execution, gdb stops twice with a Broken Pipe signal and then another 'cont' returns that the program exited normally. So I don't actually see a SEGV at all!

Perhaps someone who has more experience than me might be able to get further with gdb?

[Sebastien Bacher (seb128)]

Thanks for your work on that, https://wiki.ubuntu.com/DebuggingProgramCrash has details on how to install debug variants

[Jonathan Heard (jon-launchpad-jeh)]

I've done a lot more work on this tonight and have the following information to submit:

Workaround
---------------
This isn't really a workaround as you won't have any keyring functionality but at least you can log in.
1) Enable login by disabling the gnome keyring PAM Module:
# cp /etc/pam.d/gdm /etc/pam.d/gdm.old
* Edit /etc/pam.d/gdm and delete all lines which reference 'pam_gnome_keyring.so' (Expect to find 2 lines)
2) Restart GDM or Reboot
3) Log in and you should get in, but obviously without the Keyring features.

Reproduction
-----------------
The occurrence of this bug seems most prevalent on USB, Compact Flash (attached via IDE) and SCSI based boot devices.
1) Use the Workaround above to be able to log in as a user.
2) Open a terminal
3) Run:
# echo "<password>" | /usr/bin/gnome-keyring-manager -f --login
N.B. <password> would normally be the user's password it crashes regardless of what you give it.
4) The keyring daemon should now list it's environment variables, a few messasges, a warning and then will visibly die with a SEGV.
NOTE: Allowing gnome-keyring-daemon to daemonize itself with the '-d' option (as it gets called from the PAM module) will surpress all useful messages including the SEGV notification!!

Debug
--------
I have attached gdb to gnome-keyring-manager and captured a full backtrace, register info and tread backtrace at the point of SEGV. As I ran gdb pointed at the sources for keyring-daemon I have also captured 'list'.

On the console I get the following output:
>** Message: adding removable location: volume_label_Ubuntu_8_04_i386 at /media/cdrom0
>** Message: adding removable location: volume_uuid_87cfbf2f_6fcb_42cb_95ef_92e3aec6d4f1 at /
>
>** (gnome-keyring-daemon:6249): WARNING **: location device 'FILE' already registered at: /
>
>Program received signal SIGSEGV, Segmentation fault.

It seems that gnome-keyring-daemon is being screwed up while it tries to probe the HAL about storage - This might explain the apparent correlation between boot disk type and whether one sees the bug.

We die in hal_device_property() at gkr-location.c:324

 323 locvol = g_hash_table_lookup (pv->volumes_by_name, name);
 324 locvol->hal_volume = TRUE;

It seems that we might benefit from some kind of bounds check in this code as we seem to be taking it as gospel that 'locvol' will always return a valid address.

The SEGV happens while executing the instruction @ 0x080759c7 - This has been consistent throughout my old /var/log/messages files:

0x080759c2 <hal_device_property+834>: call 0x804f8e0 <g_hash_table_lookup@plt>
0x080759c7 <hal_device_property+839>: movl $0x1,0x14(%eax)
0x080759ce <hal_device_property+846>: jmp 0x80758a5 <hal_device_property+549>

So in order to set locvol->hal_volume=TRUE we take $eax + 0x14, dereference it and write a gboolean there. This is fine for the first few volumes and $eax always = 0x80ca828 which I trust is the valid address of a GkrLocationVolume structure. But when I get a SEGV $eax = 0 (which helps me understand the fact the segfault is reported at '00000014' in /var/log/messages) - of course, 0x14 is not a vaild address.

The reason we SEGV'd is bec...

[Karl Dane (karl-rince)]

It's worth noting that shortly after experiencing this problem, I
followed Grugnog's advice on this thread and checked out and compiled
the latest version of gnome-keyring from subversion. Since then, I've
had no issues, and gnome-keyring has worked like a charm. So it seems
that, whether by accident or by design, this particular bug has been
fixed. (I had a quick browse through its Changelog, but couldn't see
anything relevant.)

Ovation1357 wrote:
> I've done a lot more work on this tonight and have the following
> information to submit:
>
> Workaround
> ---------------
> This isn't really a workaround as you won't have any keyring functionality but at least you can log in.
> 1) Enable login by disabling the gnome keyring PAM Module:
> # cp /etc/pam.d/gdm /etc/pam.d/gdm.old
> * Edit /etc/pam.d/gdm and delete all lines which reference 'pam_gnome_keyring.so' (Expect to find 2 lines)
> 2) Restart GDM or Reboot
> 3) Log in and you should get in, but obviously without the Keyring features.
>
> Reproduction
> -----------------
> The occurrence of this bug seems most prevalent on USB, Compact Flash (attached via IDE) and SCSI based boot devices.
> 1) Use the Workaround above to be able to log in as a user.
> 2) Open a terminal
> 3) Run:
> # echo "<password>" | /usr/bin/gnome-keyring-manager -f --login
> N.B. <password> would normally be the user's password it crashes regardless of what you give it.
> 4) The keyring daemon should now list it's environment variables, a few messasges, a warning and then will visibly die with a SEGV.
> NOTE: Allowing gnome-keyring-daemon to daemonize itself with the '-d' option (as it gets called from the PAM module) will surpress all useful messages including the SEGV notification!!
>
> Debug
> --------
> I have attached gdb to gnome-keyring-manager and captured a full backtrace, register info and tread backtrace at the point of SEGV. As I ran gdb pointed at the sources for keyring-daemon I have also captured 'list'.
>
> On the console I get the following output:
>> ** Message: adding removable location: volume_label_Ubuntu_8_04_i386 at /media/cdrom0
>> ** Message: adding removable location: volume_uuid_87cfbf2f_6fcb_42cb_95ef_92e3aec6d4f1 at /
>>
>> ** (gnome-keyring-daemon:6249): WARNING **: location device 'FILE' already registered at: /
>>
>> Program received signal SIGSEGV, Segmentation fault.
>
> It seems that gnome-keyring-daemon is being screwed up while it tries to
> probe the HAL about storage - This might explain the apparent
> correlation between boot disk type and whether one sees the bug.
>
> We die in hal_device_property() at gkr-location.c:324
>
> 323 locvol = g_hash_table_lookup (pv->volumes_by_name, name);
> 324 locvol->hal_volume = TRUE;
>
> It seems that we might benefit from some kind of bounds check in this
> code as we seem to be taking it as gospel that 'locvol' will always
> return a valid address.
>
> The SEGV happens while executing the instruction @ 0x080759c7 - This has
> been consistent throughout my old /var/log/messages files:
>
> 0x080759c2 <hal_device_property+834>: call 0x804f8e0 <g_hash_table_lookup@plt>
> 0x080759c7 <hal_dev...

[Matt LaPaglia (mlapaglia)]

Sebastian we've already tried both of those methods, they don't work because the keyring only seems to fail when gdm is using it. Neither backtracing or using valgrind produces the segfault when running from terminal. Is there a different method I can try to help?

[Matt LaPaglia (mlapaglia)]

Nevermind, it seems that Ovation is onto the answer...

Since it's been fixed upstream, should we attempt to solve it anyway?

[Sebastien Bacher (seb128)]

the coredump attached to the bug gives this stacktrace

"#0 hal_device_property (hal_ctx=0x80af768,
    udi=0x80b0138 "/org/freedesktop/Hal/devices/volume_uuid_744a680e_31d7_4045_bc59_111f6592138f", key=0x808faf4 "volume.is_mounted", is_removed=0, is_added=1)
    at gkr-location.c:324
324 gkr-location.c: No such file or directory.
 in gkr-location.c
(gdb) bt
#0 hal_device_property (hal_ctx=0x80af768,
    udi=0x80b0138 "/org/freedesktop/Hal/devices/volume_uuid_744a680e_31d7_4045_bc59_111f6592138f", key=0x808faf4 "volume.is_mounted", is_removed=0, is_added=1)
    at gkr-location.c:324
#1 0x08075e59 in location_manager_hal_init (locmgr=<value optimized out>)
    at gkr-location.c:360
#2 0xb7ec2f46 in IA__g_type_create_instance (type=134928648)
    at /build/buildd/glib2.0-2.16.3/gobject/gtype.c:1575
#3 0xb7ea8242 in g_object_constructor (type=134928648,
    n_construct_properties=0, construct_params=0x0)
    at /build/buildd/glib2.0-2.16.3/gobject/gobject.c:1046
#4 0xb7ea8a08 in IA__g_object_newv (object_type=134928648, n_parameters=0,
    parameters=0x0) at /build/buildd/glib2.0-2.16.3/gobject/gobject.c:937
#5 0xb7ea9551 in IA__g_object_new_valist (object_type=134928648,
    first_property_name=0x0,
    var_args=0xbfe8f878 "�����$��9B\a\b\204\222\t\b��迾B\a\b\001")
    at /build/buildd/glib2.0-2.16.3/gobject/gobject.c:986
#6 0xb7ea96c0 in IA__g_object_new (object_type=134928648,
    first_property_name=0x0)
    at /build/buildd/glib2.0-2.16.3/gobject/gobject.c:795
#7 0x0807426d in gkr_location_manager_get () at gkr-location.c:624
#8 0x080742be in location_to_volume (loc=0, remainder=0x80ae0c0)
    at gkr-location.c:905
---Type <return> to continue, or q <return> to quit---
#9 0x0807444d in gkr_location_to_path (loc=0) at gkr-location.c:972
#10 0x080848ec in keyrings_init () at gkr-keyrings.c:184
#11 0x08084d69 in gkr_keyrings_find (name=0x8088e3a "login")
    at gkr-keyrings.c:325
#12 0x08084e00 in gkr_keyrings_get_login () at gkr-keyrings.c:266
#13 0x08085492 in gkr_keyring_login_unlock (
    password=0xb7ab9140 "userpassword") at gkr-keyring-login.c:141"

[Sebastien Bacher (seb128)]

Thank you for your work on the issue. i've opened the bug upstream on http://bugzilla.gnome.org/show_bug.cgi?id=530316 and I think it has enough informations to be debugged

[Sebastien Bacher (seb128)]

There is no code change upstream which seem likely to do a change there, maybe the people building from svn didn't build the pam integration code though?

[Karl Dane (karl-rince)]

Sebastien - that could well be it.

I don't have a log of precisely what I did, but it boils down to roughly:

- install gnome-devel
- download gnome-keyring from svn
- configure with prefix = /usr
- install

I doubt gnome-devel installs any PAM headers, so gnome-keyring probably
didn't attempt to build any pam integration code.

An ldd of my /usr/bin/gnome-keyring-daemon gives:

 linux-gate.so.1 => (0xb7f93000)
 libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0 (0xb7f7d000)
 librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0xb7f74000)
 libgconf-2.so.4 => /usr/lib/libgconf-2.so.4 (0xb7f43000)
 libdbus-1.so.3 => /usr/lib/libdbus-1.so.3 (0xb7f0d000)
 libgcrypt.so.11 => /lib/libgcrypt.so.11 (0xb7ec0000)
 libtasn1.so.3 => /usr/lib/libtasn1.so.3 (0xb7eb0000)
 libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0xb7e74000)
 libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0xb7dc3000)
 libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7daa000)
 libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7c5b000)
 libselinux.so.1 => /lib/libselinux.so.1 (0xb7c42000)
 /lib/ld-linux.so.2 (0xb7f94000)
 libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0xb7c3e000)
 libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7c3a000)
 libORBit-2.so.0 => /usr/lib/libORBit-2.so.0 (0xb7be7000)
 libgpg-error.so.0 => /lib/libgpg-error.so.0 (0xb7be3000)
 libpcre.so.3 => /usr/lib/libpcre.so.3 (0xb7bbc000)

I don't see anything mentioning PAM. But I'm a perl hacker, not a C
coder, and I have no dev experience of gnome, so this may not prove a thing!

Please shout if there are any tests on my setup that anybody needs me to
perform.

Ta,

Karl

Sebastien Bacher wrote:
> There is no code change upstream which seem likely to do a change there,
> maybe the people building from svn didn't build the pam integration code
> though?
>
> ** Changed in: gnome-keyring (Ubuntu)
> Importance: Medium => High
>

[Sebastien Bacher (seb128)]

the configure summary shows you if the pam module is built or not

[Jonathan Heard (jon-launchpad-jeh)]

Hi Karl,
     If you still have the stuff below installed, please can you rerun your configure command and post back whether your build enables or disables HAL support (you could just attach the configure output):

- install gnome-devel
- download gnome-keyring from svn
- configure with prefix = /usr
- install

I believe PAM has nothing to do with this problem, except it is the gnome-keyring PAM module which executes 'gnome-keyring-daemon -d --login' . It is 'gnome-keyring-daemon' itself which crashes and unfortunately the -d option means it deamonizes and returns exit 0, before the crash occurrs causing PAM (or anything else which cares like gdb) to think it exited normally. Then the sneaky little bugger goes and dies and the only thing which seems to notice it is the kernel.

The actual problem lies within the gnome-keyring-daemon code while it tries to manage removeable storage and you'll only hit this part of the code if gnome-keyring-daemon was built with HAL Support for Removable Devices enabled.

The reason that the work-around using 'auto-login' works is that GDM uses a different pam config for automatic login (/etc/pam.d/gdm-autologin) which for some reason does not include the gnome-keyring module. Hence it's the same as deleting the gnome-keyring references from /etc/pam.d/gdm

Having discussed this bug with a few knowledgable colleagues the concensus is that regardless of why g_hash_table_lookup() returns null or whether it ever should, we should always check the return code:

at hal_device_property() in gkr-location.c
 323 locvol = g_hash_table_lookup (pv->volumes_by_name, name);
 324 locvol->hal_volume = TRUE;

Between 323 and 324 we need a check that 'locvol' is not null before using it as an address.
If g_hash_table_lookup should never return 'null' then perhaps an 'assert(locvol != null)' should be added, but if 'null' is a valid retun then we need extra code to check it and handle it within this function, the simplest being 'if (locvol != null ) { locvol->hal_volume=TRUE; }'

[Karl Dane (karl-rince)]

> - install gnome-devel
done

> - download gnome-keyring from svn
actually used the copy I had already downloaded; my copy is at revision
1137.

> - configure with prefix = /usr
done - full output of configure is attached, but the summary is:

OPTIONAL DEPENDENCIES
   PAM: no
   DBus: 1.1.20
   HAL: no

CONFIGURATION
   SSH Agent: yes
   Root Certificates: none

BUILD
   Debug Build: no
   Unit Tests: no

> - install
done - restarted gnome, and everything still works great. e.g. my ssh
keys were automatically unlocked as you'd expect. No crashes reported.

Hope this helps. Let me know if you need me to build and install with
PAM and / or HAL supported.

Cheerio,

Karl

>
> I believe PAM has nothing to do with this problem, except it is the
> gnome-keyring PAM module which executes 'gnome-keyring-daemon -d
> --login' . It is 'gnome-keyring-daemon' itself which crashes and
> unfortunately the -d option means it deamonizes and returns exit 0,
> before the crash occurrs causing PAM (or anything else which cares like
> gdb) to think it exited normally. Then the sneaky little bugger goes and
> dies and the only thing which seems to notice it is the kernel.
>
> The actual problem lies within the gnome-keyring-daemon code while it
> tries to manage removeable storage and you'll only hit this part of the
> code if gnome-keyring-daemon was built with HAL Support for Removable
> Devices enabled.
>
> The reason that the work-around using 'auto-login' works is that GDM
> uses a different pam config for automatic login (/etc/pam.d/gdm-
> autologin) which for some reason does not include the gnome-keyring
> module. Hence it's the same as deleting the gnome-keyring references
> from /etc/pam.d/gdm
>
> Having discussed this bug with a few knowledgable colleagues the
> concensus is that regardless of why g_hash_table_lookup() returns null
> or whether it ever should, we should always check the return code:
>
> at hal_device_property() in gkr-location.c
> 323 locvol = g_hash_table_lookup (pv->volumes_by_name, name);
> 324 locvol->hal_volume = TRUE;
>
> Between 323 and 324 we need a check that 'locvol' is not null before using it as an address.
> If g_hash_table_lookup should never return 'null' then perhaps an 'assert(locvol != null)' should be added, but if 'null' is a valid retun then we need extra code to check it and handle it within this function, the simplest being 'if (locvol != null ) { locvol->hal_volume=TRUE; }'
>

[Jonathan Heard (jon-launchpad-jeh)]

Thanks for that Karl - this confirms my suspicions that you're not compiling with HAL support so should not see this bug (but won't be able to store keyrings on removeable media).

The code where the bug is is:
 324 locvol->hal_volume = TRUE;

and earlier in the same file, the GkrLocationVolume struct is defined:
  44 typedef struct _GkrLocationVolume {
  45 GQuark volume_loc;
  46 gchar *name;
  47 gchar *prefix;
  48 gchar *friendly;
  49 gboolean hidden;
  50 #ifdef WITH_HAL
  51 gboolean hal_volume;
  52 #endif
  53 } GkrLocationVolume;

'hal_volume' is only included in the structure if we're compiling with HAL support, so whatever code branches are called should never reach line 324 in your configuration ( and if it did it would probably crash with an undefined symbol error of some kind).

If I get some more free time I'll see if I can diagnose further - I assume that I now need to submit proposed fixes or root-cause analysis to the upstream bug at http://bugzilla.gnome.org/show_bug.cgi?id=530316
(This is my first involvement in any linux bugs so forgive me if I don't know the process yet..)

[Bigm8 (mfulk2)]

Same problem compact flash drive on a cf/ide adapter in a Thinkpad R40. After reading comments above, I have a temporary workaround for this is, stop hal. Ctrl-Alt-F1, sudo /etc/init.d/hal stop, sudo /etc/init.d/gdm restart. This allowed me to login and do some stuff. Network
Manager would come up and wireless worked. Several things not working because hal missing but it will enable one to get on a wireless network to do and update when ever the fix becomes available, for those with wireless only.

[bbauto (berglund-bengt)]

If you --> "Ctrl-Alt-F2" --> "login" --> "rm /tmp/X0-lock" --> "startx"
Then everything seems to work, even "Gnome-keyring...". You can also
Click: System-->Administration-->Login window-->Saftey--> Click
"Automatic Login" --> "user x", This works great until you need to
change user, then you will have to make the same manover again.

tor 2008-05-01 klockan 14:15 +0000 skrev Bigm8:
> Same problem compact flash drive on a cf/ide adapter in a Thinkpad R40. After reading comments above, I have a temporary workaround for this is, stop hal. Ctrl-Alt-F1, sudo /etc/init.d/hal stop, sudo /etc/init.d/gdm restart. This allowed me to login and do some stuff. Network
> Manager would come up and wireless worked. Several things not working because hal missing but it will enable one to get on a wireless network to do and update when ever the fix becomes available, for those with wireless only.
>

[nyamap (nyamap)]

Hi, All. I encountered this problem on my asus eee pc with 8G SDHC card.
My quick and dirty workaround is make root filesystem not removable.

1. Login with Ctrl-Alt-F2 or Xterm session.
2. run 'hal-device' to find device which contains root file system. (device means not media but controller device)
3. mark that device to unremovable using 'hal-set-priority --udi "/org/freedesktop/Hal/devices/storage_USB2_0_CardReader..." --key storage.removable --bool false'

I temporary added this line to /etc/rc.local. I could get login to my laptop.

[bbauto (berglund-bengt)]

Did not find any man pages for "hal-device" is it possible to write to
file, edit the file and write it back again --> "hal-device > Hal.txt"
"edit the file Hal.txt" and write back "hal-device < Hal.txt"

Best regards Bengt Berglund

fre 2008-05-02 klockan 03:03 +0000 skrev nyamap:
> Hi, All. I encountered this problem on my asus eee pc with 8G SDHC card.
> My quick and dirty workaround is make root filesystem not removable.
>
> 1. Login with Ctrl-Alt-F2 or Xterm session.
> 2. run 'hal-device' to find device which contains root file system. (device means not media but controller device)
> 3. mark that device to unremovable using 'hal-set-priority --udi "/org/freedesktop/Hal/devices/storage_USB2_0_CardReader..." --key storage.removable --bool false'
>
> I temporary added this line to /etc/rc.local. I could get login to my
> laptop.
>

[Stewart Midwinter (midtoad)]

@nyamap, thanks for your tip which has allowed me (with a mod) to log in to Gnome desktop on EeePC.

You have two errors in your step 3 (you must use hal-set-property, not hal-set-priority; also, you need to use the UUID string, not the info.udi string), so here are the corrected steps to follow:

1. Login with Ctrl-Alt-F2 or Xterm session.
2. run 'hal-device' to find device which contains root file system. (device means not media but controller device)

on my EeePC, it is device 2. Look for the device that has volume.mount_point = '/'. Make a note of the udi string.

3. mark that device to unremovable using 'hal-set-property --udi "/org/freedesktop/Hal/devices/volume_uuid_6662900c...." --key storage.removable --bool false'

Your own device will have a different string beginning with "volume_uuid_"

Run this in the shell and if it works, put in rc.local, then restart gdm with 'sudo /etc/init.d/gdm restart'

[MattPie (piechota)]

@Stewart, nyamap: This seems to be the solution, thanks a ton! GDM must get confused when the root device is classified by hal to be removable. Perhaps the solution is to add some logic to HAL to force the root device to be non-removable. It doesn't make much sense to be able to remove / anyways.

In my case, the odd thing is my USB external hard disk, OK; my USB external flash disk, not OK. HAL must classify them differently for some reason.

In any case, for my configuration I had to use a line closer to nyamap's, using the storage device instead of the volume like Stewart's. On the USB Flash, the volume_uuid_... entry didn't have storage.removable defined, and setting it to false had no effect. The USB Flash Device (storage_serial_PNY...) had storage.removable set to true. Setting that to false allows me to log in properly. There's probably slight difference based on CF cards, odd SCSI controllers, USB, etc.

Oh, and for people testing, a one-liner that helps extract the udi:
hal-device | grep ^8: | awk '{print $4;};' | sed s/\'//g

In this case, 8 is the device number for my device. You have to figure that part by looking at the hal-device output first. :)

[nyamap (nyamap)]

Stewart, MattPie, bbauto:
I am sorry. hal-set-property is right.
In my case, it is same as MattPie. The volume entry does not have storage.removable key. How about this:

hal-get-property --key block.storage_device --udi `hal-find-by-property --key volume.mount_point --string /` | xargs -l1 hal-set-property --key storage.removable --bool false --udi

I hope this will help until it fixed in upstream.

[Steve Anelay (steve-anelay)]

Thanks nyamap, that one worked for me.

[Robin (robingape)]

Likewise, adding that line in to /etc/rc.local on an Asus PC701 (eee) allowed me to boot from a 4G USB stick, and then in to Gnome. Thanks to all.

[bbauto (berglund-bengt)]

Thanks! It worked for me as well...

sön 2008-05-04 klockan 01:19 +0000 skrev Robin:
> Likewise, adding that line in to /etc/rc.local on an Asus PC701 (eee)
> allowed me to boot from a 4G USB stick, and then in to Gnome. Thanks to
> all.
>

[Luigi Rosa (lrosa)]

Confirmed the bug on a fresh install on AMD x86_64 SMP.
Confirmed that the nyamap's workaround allows me to log in as expected.

[ramesh (ramesh-info)]

Thanks nyamap. Your fix worked well for my 4GB hitachi microdrive attached to a IDE-CFII adapter.

[Glen Walpert (gwalpert)]

The method of setting the boot device as not removable described by Stewart and nyamap above worked well for my system with an Adaptec 5405 RAID controller, with the exception that I had to do this for the parent of the device with volume.mount_point = '/'. In my case both the root file system (with volume.mount_point = '/') on device 12, and swap on device 13, had a line in their hal-device listing:

info.parent ='/org/freedesktop/Hal/devices/storage_serial_SAdapter_mylabel_2E2B5983'

where the string after the = was the uid of device 14, which had the storage.removable property true until changed.

[zehel (3-launchpad-zehel-com)]

Thank you for finding a fix. When will 2.22.2 be out?

[Martin Pitt (pitti)]

Accepted into -proposed, please test and give feedback here

[MattPie (piechota)]

The -proposed package, 2.22.1-1ubuntu1, appears to have fixed the issue for me. My config is a Hardy 8.04-release installed on a PNY USB Flash Drive with a swap and root partition.

As a matter of correctness (but not really in the scope of this big), should the device that holds the root partition be allowed to be flagged as removable at all? Maybe there should be a HAL update to force the root device to be non-removable.

[Martin Pitt (pitti)]

Hi,

MattPie [2008-05-20 17:16 -0000]:
> As a matter of correctness (but not really in the scope of this big),
> should the device that holds the root partition be allowed to be flagged
> as removable at all? Maybe there should be a HAL update to force the
> root device to be non-removable.

If it is actually removable (such as a CD, or an SD-Card in a card
reader), why should we poke false information into Hal? It should
reflect the physical reality, not what we'd like it to be. :-)

[Wenzhuo Zhang (wenzhuo)]

The proposed update seems to have fixed the problem for me. I haven't seen gnome-keyring-daemon segfaults ever since yesterday afternoon when I enabled the hardy-proposed channel and applied all the updates therein.

Hardy is installed on an IDE disk, which is attached to the following IDE controller:

00:1f.1 IDE interface: Intel Corporation 82801DBM (ICH4-M) IDE Controller (rev 01)

Prior to the proposed update, gnome-keyring-daemon often segfaulted upon gnome login. I was able to login gnome though.

$ zgrep keyring /var/log/messages.1.gz
May 14 06:59:28 thinkpad kernel: [ 81.939465] gnome-keyring-d[5345]: segfault at ba7ab81a eip 080ab018 esp bfac48ac error 4
May 15 21:07:37 thinkpad kernel: [ 195.836481] gnome-keyring-d[5286]: segfault at 4394db4d eip 080ad140 esp 4394db51 error 6
May 15 21:15:52 thinkpad kernel: [ 258.633868] gnome-keyring-d[5754]: segfault at ba7ab81a eip 080ab018 esp bf9d7c0c error 4
May 16 14:08:54 thinkpad kernel: [ 111.294674] gnome-keyring-d[5207]: segfault at ba7ab81a eip 080ab018 esp bfd1b63c error 4
May 16 22:21:15 thinkpad kernel: [ 77.757725] gnome-keyring-d[5245]: segfault at ba7ab81a eip 080ab018 esp bfdbb13c error 4
May 16 22:28:28 thinkpad kernel: [ 336.826516] gnome-keyring-d[7783]: segfault at ba7ab81a eip 080ab018 esp bf86e4fc error 4
May 17 07:19:22 thinkpad kernel: [ 95.575549] gnome-keyring-d[5265]: segfault at ba7ab81a eip 080ab018 esp bfb0322c error 4

[Martin Pitt (pitti)]

Copied to hardy-updates.

[Martin Pitt (pitti)]

Copied hardy-proposed to intrepid.

[talowery (talowery-us)]

Note that this bug also occurs in Gutsy (7.10) with current updates. The proposed work-around (marking the root device as non-removable) seems to solve the problem here as well. Thanks!