Unable to debug under 2.6.22 on i386: Failed to read a valid object file image from memory

[Matt Zimmerman]

Binary package hint: gdb

I'm unable to use gdb under the 2.6.19 kernel in Feisty. Works fine on a stock Edgy system, doesn't work on an Edgy system with the kernel upgraded.

mizar:[/tmp] gdb sleep
GNU gdb 6.5-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...(no debugging symbols found)
Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1".

(gdb) run 300
Starting program: /bin/sleep 300
Failed to read a valid object file image from memory.
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)

Program received signal SIGINT, Interrupt.
0xb7f32410 in ?? ()
(gdb) bt
#0 0xb7f32410 in ?? ()
#1 0xbfef2458 in ?? ()
#2 0xb7ddb68c in ?? ()
#3 0x00000000 in ?? ()
(gdb)

[Ben Collins (ben-collins)]

Confirmed. From what I can tell so far, bfd is getting unexpected info from proc mem files under 2.6.19.

[Ben Collins (ben-collins)]

This appears fixed in 2.6.20 git. So should be ok with next upload.

[Sebastien Bacher (seb128)]

doing the same with 2.6.20-1-generic

[Ben Collins (ben-collins)]

Please retest against 2.6.20-2 when it is available in the feisty archive.

[Ben Collins (ben-collins)]

Seems to be working now under 2.6.20.

[Anderson (amg1127)]

I get the same bad bactktrace under 2.6.20-5:

$ gdb sleep
GNU gdb 6.6-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
(no debugging symbols found)
Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1".
(gdb) run 300
Starting program: /bin/sleep 300
(no debugging symbols found)
Failed to read a valid object file image from memory.
(no debugging symbols found)

Program received signal SIGINT, Interrupt.
0xb7f83410 in ?? ()
(gdb) bt
#0 0xb7f83410 in ?? ()
#1 0xbfd75668 in ?? ()
#2 0xb7e1e68c in ?? ()
#3 0x00000000 in ?? ()
(gdb) quit
The program is running. Exit anyway? (y or n) y

$ uname -r
2.6.20-5-generic

[edschofield (schofield)]

I'm also seeing this in 2.6.20-5-generic.

[Anderson (amg1127)]

I think Linux 2.6.20-6-generic fixes the issue.

[Anderson (amg1127)]

I think Linux 2.6.20-6-generic fixes the issue.

[Martin Pitt (pitti)]

This issue seems to be back on i386 kernels in gutsy. It works fine on amd64 and powerpc for me, but Seb can reproduce it on i386.

[Shirish Agarwal (shirishag75)]

Ok here's mine :-

shirish@ubuntu:~$ gdb sleep
GNU gdb 6.6-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1".
(gdb) run 500
Starting program: /bin/sleep 500
Failed to read a valid object file image from memory.

Program received signal SIGINT, Interrupt.
0xffffe410 in ?? ()
(gdb) bt
#0 0xffffe410 in ?? ()
#1 0xbfaecae8 in ?? ()
#2 0xb7dca68c in ?? ()
#3 0x00000000 in ?? ()
(gdb) quit
The program is running. Exit anyway? (y or n) y
shirish@ubuntu:~$ uname -r
2.6.22-7-generic

I have both gdb-dbgsym as well as coreutils-dbgsym installed . Did CRTL^C while run 500 was happening.

[Martin Pitt (pitti)]

For comparison on amd64: with coreutils-dbgsym, I get a good backtrace and no 'Failed to read a valid object file image from memory.' error.

(gdb) bt
#0 0x00002accdc287750 in nanosleep () from /lib/libc.so.6
#1 0x00000000004026b4 in xnanosleep (seconds=<value optimized out>) at xnanosleep.c:101
#2 0x00000000004012ca in main (argc=2, argv=0x1) at sleep.c:148
#3 0x00002accdc20e8e4 in __libc_start_main () from /lib/libc.so.6
#4 0x0000000000400f99 in _start ()

[Martin Pitt (pitti)]

I finally downloaded a i386 desktop CD, and confirm Seb's and Shirish's finding.

[Ben Collins (ben-collins)]

Matthias is investigating if this needs to be fixed in bfd or not.

[Sebastien Bacher (seb128)]

https://bugzilla.novell.com/show_bug.cgi?id=258433 mentions 'echo 0 > /proc/sys/vm/vdso_enabled' as a workaround

[Jonathan Anderson (jonathan-anderson)]

I tried 'echo 0 > /proc/sys/vm/vdso_enabled', and my computer locked up hard. What is a VDSO? Could this lockup be related to FGLRX (it seems that most lockups are)?

[Jonathan Anderson (jonathan-anderson)]

When I execute 'echo 0 > /proc/sys/vm/vdso_enabled' with no X server running and fglrx unloaded, I can no longer input anything via keyboard, and I can't connect over SSH, either. Not even the power button works.

What's funny is that I *can* switch VTs... other than that, nothing.

[Anderson (amg1127)]

By adding "vdso=0" to kernel command line options and rebooting, my system works well and I can get the following backtrace:

[amg1127@amg1127:~]$ gdb sleep
GNU gdb 6.6-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
(no debugging symbols found)
Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1".
(gdb) run 5000
Starting program: /bin/sleep 5000
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)

Program received signal SIGINT, Interrupt.
0xb7fa97f2 in 9�t����t��z�:t�1��댍�& () from /lib/ld-linux.so.2
(gdb) bt
#0 0xb7fa97f2 in 9�t����t��z�:t�1��댍�& () from /lib/ld-linux.so.2
#1 0xb7ebee50 in nanosleep () from /lib/tls/i686/cmov/libc.so.6
#2 0x0804a20a in ?? ()
#3 0xbf92c914 in ?? ()
#4 0x00000000 in ?? ()

The #0 position on stack seems really strange...

After installing coreutils-dbgsym package, I get:

GNU gdb 6.6-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1".
(gdb) run 5000
Starting program: /bin/sleep 5000

Program received signal SIGINT, Interrupt.
0xb7f077f2 in 9�t����t��z�:t�1��댍�& () from /lib/ld-linux.so.2
(gdb) bt
#0 0xb7f077f2 in 9�t����t��z�:t�1��댍�& () from /lib/ld-linux.so.2
#1 0xb7e1ce50 in nanosleep () from /lib/tls/i686/cmov/libc.so.6
#2 0x0804a20a in xnanosleep (seconds=5000) at xnanosleep.c:101
#3 0x08048f04 in main (argc=2, argv=Cannot access memory at address 0x4
) at sleep.c:148
#4 0xb7da0040 in __libc_start_main () from /lib/tls/i686/cmov/libc.so.6
#5 0x08048b71 in _start ()

[Martin Pitt (pitti)]

Too late for a full kernel upload now, unless this is an unintrusive one and does not break ABI.

[Jonathan Anderson (jonathan-anderson)]

I respectfully submit that on OS which doesn't let developers debug is not a very useful OS.

If the choice is between breaking ABI (while two months away from 7.10) and breaking the most basic of functionality for an entire Ubuntu release, well, I should hope that the choice would be obvious.

If the ABI is deemed more important, perhaps the default kernel arguments in GRUB could at least be modified to include "vdso=0" (while recognizing that this is a workaround, not a fix)?

[Matt Zimmerman (mdz)]

On Tue, Aug 07, 2007 at 01:47:06PM -0000, Jon Anderson wrote:
> I respectfully submit that on OS which doesn't let developers debug is
> not a very useful OS.
>
> If the choice is between breaking ABI (while two months away from 7.10)
> and breaking the most basic of functionality for an entire Ubuntu
> release, well, I should hope that the choice would be obvious.

We are in a temporary freeze to prepare a milestone for release and testing,
which means this is an inconvenient time to change the kernel ABI, because
this requires a lengthy process of rebuilds to get everything back into
sync. Besides, as far as I see in this bug report, we don't actually have a
fix to apply, so there's no need to hypothesize about whether it might
change the ABI.

This bug remains open and is targeted for the next milestone. In other
words, don't panic.

--
 - mdz

[Jonathan Anderson (jonathan-anderson)]

Ok, so the freeze isn't for Gutsy, but just the next Tribe? That sounds reasonable.

[David Tomaschik (matir)]

Definitely still present on 2.6.22-9-generic. Exact same results as above, x86.

[Andre Klapper (a9016009)]

i see the same problems in bugzilla.gnome.org, especially for reports from opensuse and debian 2.18, but fedora 7 also is not really useful currently, though not that bad as the other ones.
also see https://bugzilla.novell.com/show_bug.cgi?id=258433 , loïc also commented on that one.

[Sebastien Bacher (seb128)]

opensuse has fixed the bug with this patch apparently, https://bugzilla.novell.com/attachment.cgi?id=159330

[Martin Pitt (pitti)]

It is not invalid at all, it still happens under Gutsy, and there is a kernel patch for it.

[Matthias Klose (doko)]

ouch, there was no gdb task ...

[Jonathan Kyle Mitchell (kmitch87-deactivatedaccount)]

It seems this bug also affects the Debian kernel packages as well. I added the appropriate Debian BTS bug number. According to that report, this problem persists even with Debian's latest 2.6.22 kernels in Sid.

[Henrik Nilsen Omma (henrik)]

No need for a Gutsy task on the Feisty kernel (in fact should 2.6.20 simply be a WontFix?).

[Matt Zimmerman (mdz)]

Still happening here, on current Gutsy

[Jonathan Kyle Mitchell (kmitch87-deactivatedaccount)]

I think I may have stumbled on to something interesting playing around with gdb's settings.

When running inside the gdb command line, but the target program has not yet run, type `set debug target n' where `n' is any whole integer greater than or equal to one. Then proceed to run the program inside gdb with the `run' command.

For me, at least, this method does not produce the `Failed to load valid object from memory' error and instead produces screenfulls of debug information (like one would expect in the first place).

My machine I tried this on is running an up-to-date Debian etch install.

Cheers.

[Shirish Agarwal (shirishag75)]

Dunno if its due to J Kyle's thing but now it seems to be ready for debugging, can somebody confirm this?

Adding output as seen from gdb, if somebody wants to do some more tests or something, please lemme know.

[Juan Pablo Salazar Bertín (snifer)]

I haven't made any change, and I'm not getting the "failed to read..." message.
I think it has been fixed in 2.6.22.11.

[Matti Lindell (mlind)]

This should be fixed in gutsy, since 2.6.22-11.32:
  [Upstream Kernel Changes]
  .
  * i386: allow debuggers to access the vsyscall page with compat vDSO

[Leann Ogasawara (leannogasawara)]

Just cleaning up old kernel tasks. I'm going to go ahead and close this against 2.6.20. Thanks.