[Karmic] pipes are somewhat brocken

[Andreas Schultz]

Binary package hint: bash

some kind of pipes seem to be broken.

Running a simple chain like:

tar -xzOf zlib_1.2.3-3_i386.ipk ./data.tar.gz | tar tzf - | sed -e"s/^\\.//"

works when run from the shell but failes when run from a skript. This used to work in Jaunty.

I'll attach a simple test skript that triggers the error every time.

ProblemType: Bug
Architecture: amd64
Date: Fri Oct 16 19:40:59 2009
DistroRelease: Ubuntu 9.10
NonfreeKernelModules: nvidia
Package: bash 4.0-5ubuntu2
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-14.46-generic
SourcePackage: bash
Uname: Linux 2.6.31-14-generic x86_64

[Andreas Schultz (aschultz)]

expected result of running demo.sh (tried on Jaunty):

$ ./demo.sh
before
after

result on karmic:

$ ./demo.sh
before

[Andreas Schultz (aschultz)]

it's even simpler to reproduce, with the files from my demo archive:

$ tar tvzf zlib_1.2.3-3_i386.ipk

$ tar tvzf data.tar.gz ; echo $?
drwxr-xr-x root/root 0 2009-10-16 17:49 ./
drwxr-xr-x root/root 0 2009-10-16 17:49 ./usr/
drwxr-xr-x root/root 0 2009-10-16 18:43 ./usr/lib/
-rwxr-xr-x root/root 57488 2009-10-16 18:43 ./usr/lib/libz.so.1.2.3
lrwxrwxrwx root/root 0 2009-10-16 18:43 ./usr/lib/libz.so.1 -> libz.so.1.2.3
lrwxrwxrwx root/root 0 2009-10-16 18:43 ./usr/lib/libz.so -> libz.so.1.2.3
0

$ tar tvzf - < data.tar.gz ; echo $?
drwxr-xr-x root/root 0 2009-10-16 17:49 ./
drwxr-xr-x root/root 0 2009-10-16 17:49 ./usr/
drwxr-xr-x root/root 0 2009-10-16 18:43 ./usr/lib/
-rwxr-xr-x root/root 57488 2009-10-16 18:43 ./usr/lib/libz.so.1.2.3
lrwxrwxrwx root/root 0 2009-10-16 18:43 ./usr/lib/libz.so.1 -> libz.so.1.2.3
lrwxrwxrwx root/root 0 2009-10-16 18:43 ./usr/lib/libz.so -> libz.so.1.2.3
141

So having tar read the file directly works (exit code is 0), but piping it into tar is broken (exit code indicates an error).

The wired thing is, when stracing tar, it always exits with a exit code of 0

[C de-Avillez (hggdh2)]

Andreas and I had a chat on #ubuntu-bugs. Ends up I cannot reproduce, although Andreas has the same error on two different machines (both of us running x86_64). I suggested using '--to-command', and it worked.

Andreas will build a third box, and try again.

I am not an expert on 'tar', but the fact that there is an option to pipe output makes me wonder...

Marking Incomplete/Medium, waiting on tests.

[Andreas Schultz (aschultz)]

Took some time, but i have now been able to setup a new karmic instance on a VM and it exhibits the very same problem.

So at the moment i have three 64bit karmic instances (two VM's and one physical box) that show this problem.

The two VM are installed with the -server kernel, the physical is a workstation with the -generic kernel

[Ralf Doering (rdoering)]

Seems a little bit not-deterministic: if run multiple times it works correctly in a few cases, but fails in most runs.. It is not specific to bash, I see the same behaviour with ksh and /bin/sh (dash). It happens in both gnome-terminal and xterm, so it is not terminal dependent. I will get strace logs for both cases (working/nonworking).

[Ralf Doering (rdoering)]

I have run this on the same machine and got strace logs for both cases: where it works as aspected (printing booth "before" and "after") and where it fails (printing only before).

[Andreas Schultz (aschultz)]

Seems to work in about 5% of the tries for me now. I'll attach strace logs for both cases

[Andreas Schultz (aschultz)]

strace of nonworking run

[Ralf Doering (rdoering)]

As discussed on #debian-bugs, this is tar's fault.

[Ralf Doering (rdoering)]

This seems to be fixed in latest lucid packages, but does persist in Karmic.

[Tony Whelan (tony-whelan)]

Having rebuilt my Karmic AMD64 machine three times in the last week because of this problem, I was on the verge of going back to Jaunty.

I find some DEBs install OK but others don't, but maybe its just chance as to which work and when.
When the DEB DPKG fails it tends to hang and I can't close the window, so logout needed. Very tedious.

The error I get in DPKG is
gzip: stdin: invalid compressed data--format violated
tar: Unexpected EOF in archive
tar: Error is not recoverable, exiting now.

[Tony Whelan (tony-whelan)]

An update to my post earlier today, my problem was not caused by tar. It appears that downloading DEBs to a network drive was causing them to corrupt in some way. Downloading them to the desktop and running them from there resolved the matter.

[Jort (jort)]

I have had this problem untarring, too - using "-f - " and "-f /dev/stdin"; "-f data.tar.gz" works fine.

See files attached to #584769 (which is now marked as duplicate of this bug). Same $?=141.

[Alex Thurgood (alex-thurgood)]

I have a tgz file that weighs in at 4Gb because it is a backup of my system produced with the SimpleBackup utility, stored to an Iomega 500Gb USB drive. When I try to untar this file, whether on Windows XP with 7zip, or on my Ubuntu Lucid Aspire One netbook, I systematically get the EOF error message, for example, when using the Archive Mounter or the "Extract here" command from Nautilus. The version of tar installed on my system is 1.22-2.

My questions are : What is the point of making backups if one can't then replay them back when needed (as is my case at the moment) because the utility is faulty (under the most commonly used circumstances) ?

Has this bug really been fixed ? In my case at least, it would appear not to have been.

Alex