Ubuntu
linux package

Xen - nosegneg broken on hardy

Bug #260825 reported by Alexander Wetzel on 2008-08-24

This bug report is a duplicate of: Bug #246625: libc6-xen not used by dynamic linker. Edit Remove

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	linux (Ubuntu)	New	Undecided	Unassigned

Bug Description

Binary package hint: linux-image-2.6.24-19-xen

Error occurs on:
Ubuntu 8.0.4.1 (Server installation CD with online updates)

After freshly installing Hardy and switching to the xen kernel (install ubuntu-xen-server) the wrong tls libraries are used. The following warning is displayed during boot:
***************************************************************
***************************************************************
** WARNING: Currently emulating unsupported memory accesses **
** in /lib/tls glibc libraries. The emulation is **
** slow. To ensure full performance you should **
** install a 'xen-friendly' (nosegneg) version of **
** the library, or disable tls support by executing **
** the following as root: **
** mv /lib/tls /lib/tls.disabled **
** Offending process: init (pid=2710) **
***************************************************************
***************************************************************

ldd /bin/bash also show that the cmov variant of the libc is used, not the nosegneg.

Moving /lib/tls to /lib/tls.disabled fixes the warning, but this is prone to break again by installing updates.
The correct solution would be to use the nosegneg version of libc.
libc6-xen is installed and the neccessary libraries are in /lib/tls/i686/nosegneg but are not used, since /etc/ld.so.conf.d/xen.conf is missing.
Creating the missing file with the content "hwcap 1 nosegneg" and running ldconfig doesn't help. (Note: On Ubunto 7.10 the file had to read "hwcap 0 nosegneg" The flag has been changed to 1. This is documented in the kernel sources: linux-2.6.24/arch/x86/kernel/vsyscall-note_32-S. Nevertheless I also tested "0" with the Hardy kernel to no avail).

Installing linux-image-2.6.22-15-xen from Ubuntu 7.10 and setting "hwcap 0 nosegneg" fixes the problem.
ldd /bin/bash prove that the nosegneg libraries are now prefered.
Therfore I assume the linker itself is ok and the hwcap flag must be missing in the kernel. Unfortunatelly I'm unable to find an way to display the hwcap flags from the running kernel.

# uname -a
Linux Xen 2.6.24-19-xen #1 SMP Sat Jul 12 03:55:08 UTC 2008 i686 GNU/Linux

# cat /proc/version_signature
Ubuntu 2.6.24-4.6-generic

See original description

Tags:

Revision history for this message

Alexander Wetzel (alexander-wetzel) wrote on 2008-08-24:

dmesg output Edit (28.7 KiB, text/plain)

description:	updated
description:	updated
description:	updated

Revision history for this message

Leann Ogasawara (leannogasawara) wrote on 2008-08-28:

The Ubuntu Kernel Team is planning to move to the 2.6.27 kernel for the upcoming Intrepid Ibex 8.10 release. As a result, the kernel team would appreciate it if you could please test this newer 2.6.27 Ubuntu kernel. There are one of two ways you should be able to test:

1) If you are comfortable installing packages on your own, the linux-image-2.6.27-* package is currently available for you to install and test.

--or--

2) The upcoming Alpha5 for Intrepid Ibex 8.10 will contain this newer 2.6.27 Ubuntu kernel. Alpha5 is set to be released Thursday Sept 4. Please watch http://www.ubuntu.com/testing for Alpha5 to be announced. You should then be able to test via a LiveCD.

Please let us know immediately if this newer 2.6.27 kernel resolves the bug reported here or if the issue remains. More importantly, please open a new bug report for each new bug/regression introduced by the 2.6.27 kernel and tag the bug report with 'linux-2.6.27'. Also, please specifically note if the issue does or does not appear in the 2.6.26 kernel. Thanks again, we really appreicate your help and feedback.

Revision history for this message

Alexander Wetzel (alexander-wetzel) wrote on 2008-08-30:

dso-xen Edit (4.0 KiB, application/octet-stream)

Download full text (5.2 KiB)

Cannot test any linux-image-2.6.27-* kernel, since there isn't any for xen...

That seems to be an known issue:
https://answers.launchpad.net/ubuntu/+source/linux/+question/42448

Nevertheless I also tried to recompile the hardy kernel from git.
It doesn't work with any xen kernel running. In all my tests the system kind of crashes with the following error message. Somtimes the system is still able to reboot, somtimes not.

Aug 24 20:20:41 Xen kernel: [ 896.629486] invalid opcode: 0000 [#1] SMP
Aug 24 20:20:41 Xen kernel: [ 896.629534] Modules linked in: bridge iptable_filter ip_tables x_tables ipv6 parport_pc lp parport loop af_packet button evdev i82875p_edac edac_core 8250_pnp 8250 serial_c
Aug 24 20:20:41 Xen kernel: [ 896.630264]
Aug 24 20:20:41 Xen kernel: [ 896.630286] Pid: 31232, comm: cp Not tainted (2.6.24-19-xen #1)
Aug 24 20:20:41 Xen kernel: [ 896.630311] EIP: 0061:[<c1df1a49>] EFLAGS: 00010a86 CPU: 1
Aug 24 20:20:41 Xen kernel: [ 896.630338] EIP is at 0xc1df1a49
Aug 24 20:20:41 Xen kernel: [ 896.630360] EAX: c1ee4220 EBX: c1ee8420 ECX: 00000000 EDX: 00000000
Aug 24 20:20:41 Xen kernel: [ 896.630386] ESI: 00000001 EDI: 40040000 EBP: 00000000 ESP: e4d93e94
Aug 24 20:20:41 Xen kernel: [ 896.630410] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
Aug 24 20:20:41 Xen kernel: [ 896.630435] Process cp (pid: 31232, ti=e4d92000 task=ed504cb0 task.ti=e4d92000)
Aug 24 20:20:41 Xen kernel: [ 896.630461] Stack: c01623a5 e6a2ed40 c03fd800 e4d93f00 00000001 0000000b e4d93ed0 c0162456
Aug 24 20:20:41 Xen kernel: [ 896.630563] c1ee4220 c177d410 c03fd800 0000000e c0165997 0000000e 00000000 0000000b
Aug 24 20:20:41 Xen kernel: [ 896.630664] 00000000 c1abeb40 c1abeb20 c1abeb80 c1abeba0 c1abeb60 c1abfc00 c1abfc40
Aug 24 20:20:41 Xen kernel: [ 896.630765] Call Trace:
Aug 24 20:20:41 Xen kernel: [ 896.630803] [free_hot_cold_page+0x195/0x220] free_hot_cold_page+0x195/0x220
Aug 24 20:20:41 Xen kernel: [ 896.630852] [__pagevec_free+0x26/0x30] __pagevec_free+0x26/0x30
Aug 24 20:20:42 Xen kernel: [ 896.630893] [release_pages+0x137/0x160] release_pages+0x137/0x160
Aug 24 20:20:42 Xen kernel: [ 896.630966] [free_pages_and_swap_cache+0x74/0xa0] free_pages_and_swap_cache+0x74/0xa0
Aug 24 20:20:42 Xen kernel: [ 896.631012] [exit_mmap+0xe7/0x100] exit_mmap+0xe7/0x100
Aug 24 20:20:42 Xen kernel: [ 896.631060] [mmput+0x23/0x80] mmput+0x23/0x80
Aug 24 20:20:42 Xen kernel: [ 896.631096] [do_exit+0x165/0x8b0] do_exit+0x165/0x8b0
Aug 24 20:20:42 Xen kernel: [ 896.631146] [filp_close+0x49/0x80] filp_close+0x49/0x80
Aug 24 20:20:42 Xen kernel: [ 896.631188] [do_group_exit+0x2a/0xa0] do_group_exit+0x2a/0xa0
Aug 24 20:20:42 Xen kernel: [ 896.631227] [syscall_call+0x7/0x0b] syscall_call+0x7/0xb
Aug 24 20:20:42 Xen kernel: [ 896.631292] =======================
Aug 24 20:20:42 Xen kernel: [ 896.631314] Code: 15 e5 80 00 00 40 01 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 c0 3c 40 ed 00 10 15 e5 00 00 00 40 00 00 00 00 ff <ff> ff ff 34 42 ee c1 00
Aug 24 20:20:42 Xen kernel: [ 896.631837] EIP: [<c1df1a49>] 0xc1df1a49 SS:ESP 0069:e4d93e94
Aug 24 20:20:42 Xen kernel: [ 896.632703] ---[ end trace 93c63cdaf03f11e8 ...

Cannot test any linux-image-2.6.27-* kernel, since there isn't any for xen...

That seems to be an known issue: 
https://answers.launchpad.net/ubuntu/+source/linux/+question/42448

Aug 24 20:20:41 Xen kernel: [  896.629486] invalid opcode: 0000 [#1] SMP
Aug 24 20:20:41 Xen kernel: [  896.629534] Modules linked in: bridge iptable_filter ip_tables x_tables ipv6 parport_pc lp parport loop af_packet button evdev i82875p_edac edac_core 8250_pnp 8250 serial_c
Aug 24 20:20:41 Xen kernel: [  896.630264]
Aug 24 20:20:41 Xen kernel: [  896.630286] Pid: 31232, comm: cp Not tainted (2.6.24-19-xen #1)
Aug 24 20:20:41 Xen kernel: [  896.630311] EIP: 0061:[<c1df1a49>] EFLAGS: 00010a86 CPU: 1
Aug 24 20:20:41 Xen kernel: [  896.630338] EIP is at 0xc1df1a49
Aug 24 20:20:41 Xen kernel: [  896.630360] EAX: c1ee4220 EBX: c1ee8420 ECX: 00000000 EDX: 00000000
Aug 24 20:20:41 Xen kernel: [  896.630386] ESI: 00000001 EDI: 40040000 EBP: 00000000 ESP: e4d93e94
Aug 24 20:20:41 Xen kernel: [  896.630410]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
Aug 24 20:20:41 Xen kernel: [  896.630435] Process cp (pid: 31232, ti=e4d92000 task=ed504cb0 task.ti=e4d92000)
Aug 24 20:20:41 Xen kernel: [  896.630461] Stack: c01623a5 e6a2ed40 c03fd800 e4d93f00 00000001 0000000b e4d93ed0 c0162456
Aug 24 20:20:41 Xen kernel: [  896.630563]        c1ee4220 c177d410 c03fd800 0000000e c0165997 0000000e 00000000 0000000b
Aug 24 20:20:41 Xen kernel: [  896.630664]        00000000 c1abeb40 c1abeb20 c1abeb80 c1abeba0 c1abeb60 c1abfc00 c1abfc40
Aug 24 20:20:41 Xen kernel: [  896.630765] Call Trace:
Aug 24 20:20:41 Xen kernel: [  896.630803]  [free_hot_cold_page+0x195/0x220] free_hot_cold_page+0x195/0x220
Aug 24 20:20:41 Xen kernel: [  896.630852]  [__pagevec_free+0x26/0x30] __pagevec_free+0x26/0x30
Aug 24 20:20:42 Xen kernel: [  896.630893]  [release_pages+0x137/0x160] release_pages+0x137/0x160
Aug 24 20:20:42 Xen kernel: [  896.630966]  [free_pages_and_swap_cache+0x74/0xa0] free_pages_and_swap_cache+0x74/0xa0
Aug 24 20:20:42 Xen kernel: [  896.631012]  [exit_mmap+0xe7/0x100] exit_mmap+0xe7/0x100
Aug 24 20:20:42 Xen kernel: [  896.631060]  [mmput+0x23/0x80] mmput+0x23/0x80
Aug 24 20:20:42 Xen kernel: [  896.631096]  [do_exit+0x165/0x8b0] do_exit+0x165/0x8b0
Aug 24 20:20:42 Xen kernel: [  896.631146]  [filp_close+0x49/0x80] filp_close+0x49/0x80
Aug 24 20:20:42 Xen kernel: [  896.631188]  [do_group_exit+0x2a/0xa0] do_group_exit+0x2a/0xa0
Aug 24 20:20:42 Xen kernel: [  896.631227]  [syscall_call+0x7/0x0b] syscall_call+0x7/0xb
Aug 24 20:20:42 Xen kernel: [  896.631292]  =======================
Aug 24 20:20:42 Xen kernel: [  896.631314] Code: 15 e5 80 00 00 40 01 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 c0 3c 40 ed 00 10 15 e5 00 00 00 40 00 00 00 00 ff <ff> ff ff 34 42 ee c1 00
Aug 24 20:20:42 Xen kernel: [  896.631837] EIP: [<c1df1a49>] 0xc1df1a49 SS:ESP 0069:e4d93e94
Aug 24 20:20:42 Xen kernel: [  896.632703] ---[ end trace 93c63cdaf03f11e8 ]---
Aug 24 20:20:42 Xen kernel: [  896.632902] Fixing recursive fault but reboot is needed!

If I'm using a non xen kernel it works fine. (Of course I'm compiling the kernel with /lib/tls out of the way.)

On another track I tried to read the vDSO and look for the nosegneg flag. (I've attached the vDSO from  running 2.6.24-21-xen)
Unfortunatelly I was unable to locate the flag in the file. But It looks like the dom0 isn't using virtual system calls at all:

user@Xen:~$ objdump -T dso-xen

dso-xen:     file format elf32-i386

DYNAMIC SYMBOL TABLE:
f57fe400 g    DF .text  00000003  LINUX_2.5   __kernel_vsyscall
00000000 g    DO *ABS*  00000000  LINUX_2.5   LINUX_2.5
f57fe440 g    DF .text  00000007  LINUX_2.5   __kernel_rt_sigreturn
f57fe420 g    DF .text  00000008  LINUX_2.5   __kernel_sigreturn

user@Xen:~$ objdump -d --start-address=0xf57fe400 --stop-address=0xf57fe414 dso-xen

dso-xen:     file format elf32-i386

Disassembly of section .text:

f57fe400 <__kernel_vsyscall>:
f57fe400:       cd 80                   int    $0x80
f57fe402:       c3                      ret
f57fe403:       90                      nop
f57fe404:       90                      nop
f57fe405:       90                      nop
f57fe406:       90                      nop
f57fe407:       90                      nop
f57fe408:       90                      nop
f57fe409:       90                      nop
f57fe40a:       90                      nop
f57fe40b:       90                      nop
f57fe40c:       90                      nop
f57fe40d:       90                      nop
f57fe40e:       90                      nop
f57fe40f:       90                      nop
f57fe410:       90                      nop
f57fe411:       90                      nop
f57fe412:       90                      nop
f57fe413:       90                      nop

This is according to http://www.trilithium.com/johan/2005/08/linux-gate/ the old and slow interrupt method.
Nevertheless that doesn't seems to be related to the nosegneg flag problem, its the same on my working Ubuntu Gutsy System running 2.6.22.

I hope that someone with insight into the linux kernel can sort things out.

Revision history for this message

Alexander Wetzel (alexander-wetzel) wrote on 2008-09-03:

I've verified the Bug on a Thinkpad z61p with VT enabled. (KVM was able to run Windows XP before installing xen.) So this problem cannot be related to the older P4 CPU used in my testing System.

Didn't tried to compile an kernel, but the same error message with the hint to move /lib/tls out of the way.
This looks fundamentally broken. I assume most of the xen users just rename /lib/tls and do this again when it breaks after an update...

Revision history for this message

Adam Spain (adamspain) wrote on 2008-10-23:

Is this the same problem/related to bug #246625?

Revision history for this message

Alexander Wetzel (alexander-wetzel) wrote on 2008-10-23:

Yes, looks like the same problem.
The bugs should be merged.

Revision history for this message

petervanbussel (info-petervanbussel) wrote on 2009-12-08:

Is this bug actually merged with bug #246625 and how is the status?

Report a bug

This report contains Public information

Everyone can see this information.

Duplicate of bug #246625 Remove

You are

Subscribing...

Edit bug mail

Other bug subscribers

Bug attachments

Add attachment

Remote bug watches

Bug watches keep track of this bug in other bug trackers.

Ubuntulinux package

Xen - nosegneg broken on hardy

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches

Ubuntu
linux package