[Dapper, Hardy] zip crashes when building large large archives - no large file support?

[JP Vossen]

Binary package hint: zip

zip (and presumably unzip though I didn't check) should have "large file support" compiled in for Dapper. If this was not "LTS" I'd say forget it and upgrade to Edgy (which I did not test), but for "LTS" that's not a good answer. Tested on a fully up-to-date Dapper install (broken) and more or less up-to-date RHEL/CentOS-4.4 (older (Red Hat) zip worked).

To reproduce, create 5 0.5G files and try to zip them, which will fail, then try to tar them, which will work, thus showing that the file system is OK and that at least some programs work right.

~~~~~~~ Ubuntu 6.06
$ dpkg -l zip
[…]
ii zip 2.31-3 Archiver for .zip files

# Create some nice big uncompressible files, just for fun (kinda slow)
$ dd if=/dev/urandom of=test_file bs=1024 count=512500
$ for i in {1..4}; do cp test_file test_file_$i; done

$ ll -h
total 2.5G
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:03 test_file
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:06 test_file_1
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:07 test_file_2
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:08 test_file_3
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:09 test_file_4

# We have lots of space, so that's not the problem
$ df -hl
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 144G 44G 94G 32% /
[…]

$ zip -0 gonna_fail.zip test_file*
  adding: test_file (stored 0%)
  adding: test_file_1 (stored 0%)
  adding: test_file_2 (stored 0%)
  adding: test_file_3 (stored 0%)
  adding: test_file_4File size limit exceeded

# bummer!
$ ll -h
total 4.5G
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:03 test_file
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:06 test_file_1
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:07 test_file_2
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:08 test_file_3
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:09 test_file_4
-rw-r--r-- 1 jp jp 2.0G 2006-11-08 00:14 ziSjqr5G

$ tar -cvf gonna_work.tgz test_file*
test_file
test_file_1
test_file_2
test_file_3
test_file_4

# Hey, 2.5G > 2.0G, tar works, zip doesn't.
$ ll -h
total 6.9G
-rw-r--r-- 1 jp jp 2.5G 2006-11-08 00:20 gonna_work.tgz
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:03 test_file
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:06 test_file_1
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:07 test_file_2
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:08 test_file_3
-rw-r--r-- 1 jp jp 501M 2006-11-08 00:09 test_file_4
-rw-r--r-- 1 jp jp 2.0G 2006-11-08 00:14 ziSjqr5G

$ zip -v
Copyright (C) 1990-2005 Info-ZIP
Type 'zip "-L"' for software license.
This is Zip 2.31 (March 8th 2005), by Info-ZIP.
Currently maintained by Onno van der Linden. Please send bug reports to
the authors using http://www.info-zip.org/zip-bug.html; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip, as of
above date; see http://www.info-zip.org for other sites.

Compiled with gcc 4.0.1 20050522 (prerelease) (Debian 4.0.0-7ubuntu7) for Unix (Linux ELF) on May 26 2005.

Zip special compilation options:
        ASM_CRC
        ASMV
        USE_EF_UT_TIME
        [encryption, version 2.9 of 05 May 2000]
Encryption notice:
        The encryption code of this program is not copyrighted and is
        put in the public domain. It was originally written in Europe
        and, to the best of our knowledge, can be freely distributed
        in both source and object forms from any country, including
        the USA under License Exception TSU of the U.S. Export
        Administration Regulations (section 740.13(e)) of 6 June 2002.

Zip environment options:
             ZIP: [none]
          ZIPOPT: [none]

~~~~~~~ RHEL/CentOS-4.4

$ cat /etc/redhat-release
CentOS release 4.4 (Final)

$ rpm -qa | grep '^zip'
zip-2.3-27

$ dd if=/dev/urandom of=test_file bs=1024 count=512500
$ for i in {1..4}; do cp test_file test_file_$i; done

$ ll -h
total 2.5G
-rw-r--r-- 1 jp jp 501M Nov 8 00:48 test_file
-rw-r--r-- 1 jp jp 501M Nov 8 00:51 test_file_1
-rw-r--r-- 1 jp jp 501M Nov 8 00:51 test_file_2
-rw-r--r-- 1 jp jp 501M Nov 8 00:52 test_file_3
-rw-r--r-- 1 jp jp 501M Nov 8 00:52 test_file_4

$ df -hl
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      218G 139G 68G 68% /data

$ zip -0 gonna_fail.zip test_file*
  adding: test_file (stored 0%)
  adding: test_file_1 (stored 0%)
  adding: test_file_2 (stored 0%)
  adding: test_file_3 (stored 0%)
  adding: test_file_4 (stored 0%)

# It worked!
$ ll -h
total 4.9G
-rw-r--r-- 1 jp jp 2.5G Nov 8 01:17 gonna_fail.zip
-rw-r--r-- 1 jp jp 501M Nov 8 00:48 test_file
-rw-r--r-- 1 jp jp 501M Nov 8 00:51 test_file_1
-rw-r--r-- 1 jp jp 501M Nov 8 00:51 test_file_2
-rw-r--r-- 1 jp jp 501M Nov 8 00:52 test_file_3
-rw-r--r-- 1 jp jp 501M Nov 8 00:52 test_file_4

$ zip -v
Copyright (C) 1990-1999 Info-ZIP
Type 'zip "-L"' for software license.
This is Zip 2.3 (November 29th 1999), by Info-ZIP.
Currently maintained by Onno van der Linden. Please send bug reports to
the authors at <email address hidden>; see README for details.

Latest sources and executables are at ftp://ftp.cdrom.com/pub/infozip, as of
above date; see http://www.cdrom.com/pub/infozip/Zip.html for other sites.

Compiled with gcc 3.4.3 20041212 (Red Hat 3.4.3-9.EL4) for Unix (Linux ELF) on Feb 21 2005.

Zip special compilation options:
        ASM_CRC
        ASMV
        USE_EF_UT_TIME
        [encryption, version 2.9 of 22 April 2000]

Zip environment options:
             ZIP: [none]
          ZIPOPT: [none]

[didier (did447-deactivatedaccount)]

But is it a good idea? A lot of softwares won't be able to uncompress them.

[JP Vossen (jp-jpsdomain)]

What software would not be able to uncompress it? Anything without large file support, and anything on a file system with a 2G limit, I guess. But if Dapper zip/unzip both have large file support, what's the problem?

Also, note that Dapper's tar and RHEL's zip worked, so would your objection apply to them as well?

Am I missing something?

[oss_test_launchpad (oss-test-launchpad)]

Is there any reason why Ubuntu would have to restrict zip file sizes, or is this just a programming error?

I confirm this bug. Testing with a USB HD and Ubuntu 8.04.1 32-Bit, trying to put about 15GB into a zip archive, the nautilus built-in archiver stops at 1.5 GB, reporting: "zip I/O error: File too large".

[oss_test_launchpad (oss-test-launchpad)]

PS Strangely this bug cannot be assigned to file-roller, too. AFAIR the archiver that ships with nautilus is called file-roller, but launchpad knows of no such program.

[oss_test_launchpad (oss-test-launchpad)]

Just noticed that this also occurs when you try to use .jar instead of .zip, similar error message. Maybe this is related (I read in wikipedia that there is some connection between zip and jar).

[Jan (jan23)]

I can not open or decompress a 6 GB zip file created on Windows (both with WinZip or Windows' own tool). The command line tool zip tells me "zip error: Nothing to do!". Also unzip and the Gnome archiver failed. Is this problem related to this thread/bug?

[oss_test_launchpad (oss-test-launchpad)]

1) - Has there been any progress on this issue?
2) - Are discussions here automatically being delivered to the developers?

[oss_test_launchpad (oss-test-launchpad)]

1) - Has there been any progress on this issue?
2) - Is anyone going to deliver this to the developers?