tar reports 'short read' and fails to untar archive

[Andrew McDermott]

Steps to reproduce:

root@genericarmv8:~# curl -s -O ftp://ftp.gnu.org/gnu/autoconf/autoconf-2.10.tar.gz
root@genericarmv8:~# tar zxf autoconf-2.10.tar.gz
tar: short read
root@genericarmv8:~# echo $?
1

root@genericarmv8:~# tar --version
tar: unrecognized option '--version'
BusyBox v1.21.1 (2013-07-17 13:47:15 BST) multi-call binary.

Usage: tar -[cxtZzjahvO] [-X FILE] [-T FILE] [-f TARFILE] [-C DIR] [FILE]...

I see similar problems with other archives.

[Riku Voipio (riku-voipio)]

I think the root cause is probable the same as:

https://bugs.launchpad.net/linaro-oe/+bug/1204384

Have you run strace on either of the failing commands?

[Andrew McDermott (frobware)]

No, haven't run strace.

Given this is all busybox related I'm inclined to agree that there is something common to these failures.

[Riku Voipio (riku-voipio)]

Andrew,

it seems files get corrupted on download already. can you check if file size and md5sum are correct or broken in your usecase?

using curl -s -O ftp://ftp.gnu.org/gnu/autoconf/autoconf-2.10.tar.gz on both pc and simulator:

root@genericarmv8:~# md5sum autoconf-2.10.tar.gz
25182c4a1370c83ffd4a9505f0ecfebf autoconf-2.10.tar.gz
root@genericarmv8:~# ls -l autoconf-2.10.tar.gz
-rw-r--r-- 1 root root 349596 Jul 26 13:20 autoconf-2.10.tar.gz

vs on pc:

fe332d45a554c81bd5a1a758ea2c53be autoconf-2.10.tar.gz
-rw-r--r-- 1 voipio voipio 380539 Jul 26 16:28 autoconf-2.10.tar.gz

[Andrew McDermott (frobware)]

On my desktop:

$ curl -L -s -O ftp://ftp.gnu.org/gnu/autoconf/autoconf-2.10.tar.gz
$ md5sum autoconf-2.10.tar.gz
fe332d45a554c81bd5a1a758ea2c53be autoconf-2.10.tar.gz

On the fastmodel:

root@genericarmv8:~# curl -L -s -O ftp://ftp.gnu.org/gnu/autoconf/autoconf-2.10.tar.gz
root@genericarmv8:~# md5sum autoconf-2.10.tar.gz
8cd26ac64002be1199281cd0d8ac0eeb autoconf-2.10.tar.gz

# and running it again gets a different value.

root@genericarmv8:~# rm autoconf-2.10.tar.gz
root@genericarmv8:~# curl -L -s -O ftp://ftp.gnu.org/gnu/autoconf/autoconf-2.10.tar.gz
root@genericarmv8:~# md5sum autoconf-2.10.tar.gz
e57a98f2dc75ea2069f8f58b9f1a9606 autoconf-2.10.tar.gz

[Andrew McDermott (frobware)]

I guess this is similar to the wget problem. The reason I thought it was tar related is because I switched to using curl to getting stuff on to the target was because of the issues I had with wget. And I have been been doing this (curl + tar) to get the mauve tests on to the fastmodel to avoid a reboot cycle quite a lot recently.

[Andrew McDermott (frobware)]

One thing I find a bit odd is the write size patterns.

On my desktop I see a regular 4K write:

: strace -e write curl -L -s -O ftp://ftp.gnu.org/gnu/autoconf/autoconf-2.10.tar.gz
write(5, "\37\213\10\0\261\207\2171\0\3\354<kW\333H\226\375\225:\363#\252\35\237\23\314\332\2\203\301\333"..., 4096) = 4096
write(5, "\311,\340\7\365\261\322\235xD\252\300\310\236@\r\256ai\223\343cbY\253gK\231:\21\200\244"..., 4096) = 4096
write(5, "\243\354W3\345\0\22\316\351\263\21\320#W\240\340&BD\3078\325\7D\35\250\304\276$\247\332\241"..., 4096) = 4096
write(5, "*\345\367\314\314\274{}\334\370a\306\253d,o\\m\31\326\3659)R\257\210K\0\357\314\253\370"..., 4096) = 4096
write(5, "R7\247\3020\0\244\3747\177\t\236.\373q=\372\344\351\362t\213\225\321\36\257HN\340\220\34\304"..., 4096) = 4096
write(5, "\202\224K\277\226]\217!\330\223\333;\310\212\336i\350\316\234\335)\251\241\211\214\316\230\23\322\237\223\305"..., 4096) = 4096
write(5, "\212\322\212\250h\210)\212\201\1ZT\205`1\2334\341\2\r_-\352>\307V\0 \374\371\316\326"..., 4096) = 4096
write(5, "\n\37)\3518\1c\367\223\4\257C\204^\244\273 v\223O\237\177\21\306\33547\226u&(\270"..., 4096) = 4096
write(5, "\341\v\307I\232\27I3\217\21628'@u\342U5\323\302_G\220\334\20Yp\217T\332\256\260"..., 4096) = 4096
write(5, "ir\342\335[nV\7\221\343\227\321i;\16\277\342\256(\375~o\0\234\2204\224$d\331\245\226"..., 4096) = 4096
write(5, ";\370\2\222=\203\tz\4&R\21\341&\275\215\201H1r\216\206\322\201`\227=\207-\340p\t"..., 4096) = 4096
write(5, "\3\367\243\266\302\352\267\266\325`[\177\274\205\316\251\v\347k\363Ze\22\275\223\0051\r\5\250\361\265"..., 4096) = 4096
write(5, "B\5:\"ncP\5\10k\266B\314\\\243k\275\233\343@Y\317\345dH\223r\215wx\355\216"..., 4096) = 4096
write(5, "\323\347\251\0\342B\224\250\25\231\0333\344\n\340@\3324p\311\213w\226S>\314\334\211\354M\355\364"..., 4096) = 4096
write(5, "\206&\32\316\261\221\256m\312\34\243)>+\27\352\354r\321\231a?\231!\312\245\356\270-\251\203R"..., 4096) = 4096
write(5, "\247\306j\314\374E\366\206\371\33\200\267\205\322\301\344\337\3213\313\333?A\376\336:\335f\372\va\252"..., 4096) = 4096
write(5, "<\251\7\371\353\265\220\222J\3\211\320.\233\"\254g\363\250\237t\223(i\250ED\232\353t'\216"..., 4096) = 4096
....

On the fastmodel I see the 4K, 12288, and the odd 8K.

write(6, "\37\213\10\0\261\207\2171\0\3\354<kW\333H\226\375\225:\363#\252\35\237\23\314\332\2\203\301\333"..., 4096) = 4096
write(6, "\311,\340\7\365\261\322\235xD\252\300\310\236@\r\256ai\223\343cbY\253gK\231:\21\200\244"..., 4096) = 4096
write(6, "\243\354W3\345\0\22\316\351\263\21\320#W\240\340&BD\3078\325\7D\35\250\304\276$\247\332\241"..., 4096) = 4096
write(6, "*\345\367\314\314\274{}\334\370a\306\253d,o\\m\31\326\3659)R\257\210K\0\357\314\253\370"..., 12288) = 12288
write(6, "\212\322\212\250h\210)\212\201\1ZT\205`1\2334\341\2\r_-\352>\307V\0 \374\371\316\326"..., 4096) = 4096
write(6, "\n\37)\3518\1c\367\223\4\257C\204^\244\273 v\223O\237\177\21\306\33547\226u&(\270"..., 12288) = 12288
write(6, ";\370\2\222=\203\tz\4&R\21\341&\275\215\201H1r\216\206\322\201`\227=\207-\340p\t"..., 4096) = 4096
write(6, "\3\367\243\266\302\352\267\266\325`[\177\274\205\316\251\v\347k\363Ze\22\275\223\0051\r\5\250\361\265"..., 8192) = 819...

[Andrew McDermott (frobware)]

scp works:

root@genericarmv8:/tmp# scp aim@spicy:/tmp/autoconf-2.10.tar.gz .
The authenticity of host 'spicy (192.168.1.2)' can't be established.
ECDSA key fingerprint is 69:5a:85:c2:47:9f:74:ac:bc:0c:03:a2:4b:f6:d1:f7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'spicy,192.168.1.2' (ECDSA) to the list of known hosts.
aim@spicy's password:
autoconf-2.10.tar.gz 100% 372KB 371.6KB/s 00:01
root@genericarmv8:/tmp# md5sum autoconf-2.10.tar.gz
fe332d45a554c81bd5a1a758ea2c53be autoconf-2.10.tar.gz

which matches the checksums from comments #3 and #4.

And tar seems fine too:

root@genericarmv8:/tmp# tar ztf autoconf-2.10.tar.gz
[ ... tar output elided ... ]
root@genericarmv8:/tmp# echo $?
0

This bug should now be renamed.

[Riku Voipio (riku-voipio)]

I'm merging this with the wget bug, since I'm quite sure the background issue is the same.