Ubuntu

libc6-xen not used by dynamic linker

Reported by Jan Schneider on 2008-07-08
72
This bug affects 7 people
Affects Status Importance Assigned to Milestone
glibc (Debian)
Fix Released
Unknown
glibc (Ubuntu)
High
Ubuntu Toolchain Hackers
Hardy
High
Ubuntu Toolchain Hackers
Intrepid
High
Ubuntu Toolchain Hackers
Jaunty
High
Ubuntu Toolchain Hackers
linux (Ubuntu)
Undecided
Unassigned
Hardy
Undecided
Unassigned
Intrepid
Undecided
Unassigned
Jaunty
Undecided
Unassigned
xen-3.3 (Ubuntu)
Undecided
Unassigned
Hardy
Undecided
Unassigned
Intrepid
Undecided
Unassigned
Jaunty
Undecided
Unassigned

Bug Description

Binary package hint: libc6-xen

The xen friendly libraries are no longer used with Ubuntu 8.04.
At this moment the only solution is to completely remove /lib/tls in the xen host and guests.
I don't know how big the performance impact by this is.
I can't narrow down the problem cause i am missing some technical information on how dynamic linking is controlled bei hardware flags.
My best guess is tha5t the linux-image-xen needs to be patched.

I've found some information here but i can't adapt it to ubuntu.
http://lists.xensource.com/archives/html/xen-devel/2007-12/msg00260.html

Tristan Hill (stan) wrote :

/etc/ld.so.conf.d/xen.conf containing "hwcap 0 nosegneg" appears to be missing from the libc6-xen package

No "/etc/ld.so.conf.d/xen.conf" containing "hwcap 0 nosegneg" isn't missing.
The output of "ldconfig -p" even lists the xen libraries including there
hwcap flags(?).
What seems to fail is the identification of the "nosegneq hwcap", but i
have no idea how to verify this.

Tristan Hill wrote:
> /etc/ld.so.conf.d/xen.conf containing "hwcap 0 nosegneg" appears to be missing from the libc6-xen package
>

Alex Tomlins (alex-tomlins) wrote :

On my hardy install, the xen.conf file was missing.

Also, according to /arch/x86/xen/vdso.h in the kernel source, the hwcaps bit should be 1, not 0.

I've tried it all 3 ways (without the config, with it set to 0, and 1), and none of them work. Output of "ldd /bin/true" is always:

        linux-gate.so.1 => (0xf57fe000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7db0000)
        /lib/ld-linux.so.2 (0xb7f05000)

I did notice something in the output from ldconfig -p though. For the cmov version, the 4th nibble is set to 8, whereas it's 0 on the nosegneg version.

with hwcap 1 nosegneg:
# ldconfig -p | grep ld-linux.so.2
        ld-linux.so.2 (ELF, hwcap: 0x8028000000000000) => /lib/tls/i686/nosegneg/ld-linux.so.2
        ld-linux.so.2 (ELF, hwcap: 0x8008000000008000) => /lib/tls/i686/cmov/ld-linux.so.2
        ld-linux.so.2 (ELF) => /lib/ld-linux.so.2

with hwcap 0 nosegneg:
# ldconfig -p | grep ld-linux.so.2
        ld-linux.so.2 (ELF, hwcap: 0x8018000000000000) => /lib/tls/i686/nosegneg/ld-linux.so.2
        ld-linux.so.2 (ELF, hwcap: 0x8008000000008000) => /lib/tls/i686/cmov/ld-linux.so.2
        ld-linux.so.2 (ELF) => /lib/ld-linux.so.2

with no xen.conf:
# ldconfig -p | grep ld-linux.so.2
        ld-linux.so.2 (ELF, hwcap: 0x8008000000008000) => /lib/tls/i686/cmov/ld-linux.so.2
        ld-linux.so.2 (ELF) => /lib/ld-linux.so.2

I don't know if that's significant. I don't know how to find out what hwcaps the kernel is advertising.

thanks,
Alex

Adam Spain (adamspain) wrote :

On my Hardy server I find the output of "ldd /bin/bash" is different depending on which kernel I use, so I think it might be an issue with linux-image-xen. Booting the kernel provided by linux-image-xen I get:

ldd /bin/bash
        linux-gate.so.1 => (0xf57fe000)
        libncurses.so.5 => /lib/libncurses.so.5 (0xb7f5a000)
        libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7f56000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7e06000)
        /lib/ld-linux.so.2 (0xb7f90000)

However if I run the Xen kernel from Debain Etch (linux-image-2.6.18-6-xen-686) I get:

ldd /bin/bash
 linux-gate.so.1 => (0xb7ef8000)
 libncurses.so.5 => /lib/libncurses.so.5 (0xb7ec2000)
 libdl.so.2 => /lib/tls/i686/nosegneg/libdl.so.2 (0xb7ebe000)
 libc.so.6 => /lib/tls/i686/nosegneg/libc.so.6 (0xb7d6b000)
 /lib/ld-linux.so.2 (0xb7ef9000)

I don't really understand what's going on, but maybe this will help to figure it out.

agent 8131 (agent-8131) wrote :

I can confirm this bug. Besides there already being another report I also encountered this today when updating an i386 domu from gutsy to hardy. Most of my servers are 64-bit so I haven't dealt with this until today when updating one of my few development machines for testing i386. I've gone and ahead and moves /lib/tls out of the way to get work done but I'd be interested in actually figuring out how to fix this bug.

Changed in glibc:
status: New → Confirmed
Alex Tomlins (alex-tomlins) wrote :

I've now compiled up kernel 2.6.18.8 supplied with the xen source, and it works correctly with hwcap 0 nosegneg, so It does look like an issue with the xen kernel not setting the hwcap correctly, as opposed to an issue with glibc.

Adam Spain (adamspain) wrote :

Has anyone tried Intrepid to see if it handles this correctly?

Sergio Tosti (zeno979) wrote :

The only woarkaroud that works for hardy is to remove /lib/tls directory, but for me remains the warning in guest domains.
This is an important bug that affects an LTS distribution, so need urgently a fix. Debian seems to be fixed in newer version of glibc.

Oliver Rompcik (oliver-rompcik) wrote :

I can confirm this bug. It does only apply to the i386 platform, as amd64 doesn't use libc6-xen at all.
Seems hwcap isn't exported the right way.

Removing /lib/tls is not a preferable solution to this problem, this has been discussed several times. It leads to a very poor performance for threaded applications. The correct way is to find the bug, which may hide somewhere in the vdso-code...(?)

Seems this bug has already been detected during hardy-beta and it is not fixed until now. You should think about renaming LTS to LTB (Long Term Bugs)...

Loïc Minier (lool) wrote :

There are two different bugs; one is that hardy's libc6-xen isn't picked up anymore; that's a combination of a bunch of reasons described in bug #330039 which I'll merge with this one:
"""
First, it's not picked up automatically; ldconfig -p reports no nosegneg dirs, so it's useless; if I create a /etc/ld.so.conf.d/xen.conf with "hwcap 1 nosegneg", these dirs appear in the "ldconfig -p" output.

Second, if I check with "ldd" on random binaries, the cmov version is preferred over the xen version when both are installed; this also renders the libc6-xen useless because libc6-i686 is always installed (dep of ubuntu-minimal) right now...

Finally, AFAICT, all i386 glibc passes are built with:
libc_extra_cflags = -mno-tls-direct-seg-refs
so if you compare i686 and xen passes:
i686_extra_cflags = -march=i686 -mtune=i686 -O3
xen_extra_cflags = -march=i686 -mtune=i686 -O3 -mno-tls-direct-seg-refs

it seems to me the xen pass is entirely useless as a result.
"""

However this shouldn't ever cause any problem since the regular libc6-i686 in hardy is built with no-tls-direct-seg-refs, and hence is supposed to work in Xen; do you have libc6-i686 installed?

Another bug, is that it seems some people say the hardy kernel doesn't the nosegneg extra hwcap via vdso; I'm not sure about that, perhaps it's an issue if you don't have libc6-i686 installed indeed. In this case, the extra hwcap mapping is probably required to locate alternate libc6 binaries; that is, ldconfig needs to be told that "/nosegneg" dirs should be scanned and correspond to the first extra hwcap:
hwcap 0 nosegneg

Thomas Morgan (thomasmorgan) wrote :

A few notes from my observations and recent tests:

My tests have mostly been on intrepid so far. I believe everything I mentioned below is the same on hardy, as the compile config for glibc is the same in intrepid as noted in the previous comment. Testing hardy is still on my todo list.

At least on intrepid, libc6-i686 does /not/ appear to actually be compiled with no-tls-direct-seg-refs, only the i386 and xen variants are compiled as such. Because ubuntu-minimal depends on libc6-i686, the i386 version will always be ignored. This suggests to me that libc6-xen is still needed.

The file /etc/ld.so.conf.d/xen.conf with "hwcap 1 nosegneg" is required. In the absence of that file, ldconfig continues to use the i686 libraries.

"hwcap 1 nosegneg" is a reasonable default. The linux kernel used to use bit 0 (hwcap 0 nosegneg) and changed to bit 1 somewhere around 2.6.23.x. It might be worthwhile to include a comment in that file mentioning the need to change it to 0 for older kernels (especially with 2.6.18.8 being a favorite for use with xen). Unfortunately, ldconfig complains if both lines are in there.

To address a comment in one of the bugs marked as a duplicate to this one, both intrepid and debian lenny will choose the nosegneg libraries over another variant when /etc/ld.so.conf.d/xen.conf is configured properly.

Loïc Minier (lool) wrote :

Oh indeed, the i686 pass is the issue; the regular pass and the xen pass are built with nosegneg, but not the 686 one which will always be installed and selected by default, even if you install xen because we don't have the ld.so.conf snippet.

Changed in glibc (Ubuntu):
importance: Undecided → High
Changed in xen-3.3 (Ubuntu):
status: New → Invalid
Changed in linux (Ubuntu):
status: New → Invalid
Changed in linux (Ubuntu Hardy):
assignee: nobody → ubuntu-toolchain
importance: Undecided → High
milestone: none → ubuntu-8.04.3
status: New → Triaged
Loïc Minier (lool) on 2009-04-19
Changed in glibc (Ubuntu Hardy):
assignee: nobody → ubuntu-toolchain
importance: Undecided → High
milestone: none → ubuntu-8.04.3
status: New → Triaged
Changed in linux (Ubuntu Hardy):
assignee: ubuntu-toolchain → nobody
importance: High → Undecided
milestone: ubuntu-8.04.3 → none
status: Triaged → Invalid
Changed in linux (Ubuntu Intrepid):
status: New → Invalid
Changed in xen-3.3 (Ubuntu Intrepid):
status: New → Invalid
Changed in xen-3.3 (Ubuntu Hardy):
status: New → Invalid
Changed in glibc (Ubuntu Intrepid):
assignee: nobody → ubuntu-toolchain
importance: Undecided → High
status: New → Triaged
Changed in glibc (Ubuntu Jaunty):
assignee: nobody → ubuntu-toolchain
milestone: none → jaunty-updates
status: Confirmed → Triaged
Changed in glibc (Ubuntu Intrepid):
milestone: none → intrepid-updates

Is this bug actually merged with bug #260825 and how is the status?

Alex Valavanis (valavanisalex) wrote :

Intrepid Ibex reached end-of-life on 30 April 2010 so I am closing the
report. The bug is still marked as confirmed in later versions of Ubuntu.

Changed in glibc (Ubuntu Intrepid):
status: Triaged → Invalid
Clint Byrum (clint-fewbar) wrote :

Jaunty is well past its EOL, closing Jaunty task as Won't Fix.

Changed in glibc (Ubuntu Jaunty):
status: Triaged → Won't Fix
Changed in glibc (Debian):
status: Unknown → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.