jupp: destroys large files on ^K/sort

[Thorsten Glaser]

The attached file gets destroyed on ^K/sort but not when piped through sort, on Debian wheezy/amd64 and sid/i386.

[Thorsten Glaser (mirabilos)]

The error only occurs when *not* running under Valgrind, even with the same binary.

WTF?

[Thorsten Glaser (mirabilos)]

This happens during the write part of ^K/ → probably bsavefd in bw.c (WTF?!?!?!)

[Thorsten Glaser (mirabilos)]

  'jupp x' enter ^K / 'diff -u - x' enter

results in a diff…

[Thorsten Glaser (mirabilos)]

3.1.24-1~wtf50+1 on squeeze works; scp’ing it to a sid system makes it fail; scp’ing jupp CVS HEAD to a squeeze system makes it work.

[Thorsten Glaser (mirabilos)]

It’s almost certainly eglibc’s libc.so.6 that makes a difference, between squeeze and wheezy; it’s definitely not the SSE3 memcpy issue.

[Thorsten Glaser (mirabilos)]

Okay, to recap:

This happens on Debian wheezy/amd64, wheezy/i386, sid/i386 (with the amd64 kernel), but not lenny or squeeze, and not in Valgrind.

But it also doesn’t happen on Jupp’s (my friend, not the editor) box: Linux deimos 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux

It also doesn’t happen on another wheezy/amd64 box I have access to at work:

root@showcase:~ # uname -a; cat /proc/cpuinfo
Linux showcase.lan.tarent.de 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2 x86_64 GNU/Linux
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 13
model name : QEMU Virtual CPU version (cpu64-rhel6)
stepping : 3
microcode : 0x1
cpu MHz : 2999.554
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm up nopl pni cx16 hypervisor lahf_lm
bogomips : 5999.10
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:

But it does happen on this one (i386 even):

root@evolvis:~ # uname -a; cat /proc/cpuinfo
Linux evolvis.org 3.2.0-4-686-pae #1 SMP Debian 3.2.39-2 i686 GNU/Linux
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 1.1.2
stepping : 3
microcode : 0x1
cpu MHz : 2666.760
cache size : 4096 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm pni cx16 hypervisor lahf_lm
bogomips : 5333.52
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

I’d say it’s a virtualisation issue, were it not for my bare-metal desktop workstation (sid/i386 running an amd64 kernel) which does have Linux-kvm installed and in use, but where the issue occurs outside of any VM:

tglase@tglase:~ $ uname -a; sed '/^$/q' /proc/cpuinfo
Linux tglase.lan.tarent.de 3.9-1-amd64 #1 SMP Debian 3.9.6-1 i686 GNU/Linux
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz
stepping : 5
microcode : 0x11
cpu MHz : 3060.000
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips : 6128.71
clflush size : 6...

[Thorsten Glaser (mirabilos)]

It’s not an amd64 thing either… the binary I scp’d also to an i386 wheezy box where it works.

root@evolvis-ci:~ # uname -a; cat /proc/cpuinfo
Linux evolvis-ci.lan.tarent.de 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 3
model name : QEMU Virtual CPU version 1.1.2
stepping : 3
microcode : 0x1
cpu MHz : 3103.574
cache size : 4096 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep pge cmov mmx fxsr sse sse2 up pni popcnt hypervisor
bogomips : 6207.14
clflush size : 32
cache_alignment : 32
address sizes : 36 bits physical, 32 bits virtual
power management:

[Thorsten Glaser (mirabilos)]

Confirmed as a data corruption bug, only affecting the ^K/ (pipe) command, masked by a timing issue (only apparent on the faster multi-CPU machines, hidden from debuggers).

Fixed in jupp25.