software-center chokes on packages larger than 2GB

[Francois Gouget]

software-center chokes on packages larger than 2GB:
$ ls -lg testpkg_1.0-1_all.deb
-rw-r--r-- 1 weaver 2517352042 2011-12-29 11:52 testpkg_1.0-1_all.deb
$ software-center testpkg_1.0-1_all.deb
Gives me the following error message:

    Internal Error
    The file "/home/weaver/testpkg_1.0-1_all.deb" could not be opened.

I did not see anything interesting in the --debug output.

I'm attaching a small tar.gz file that contains a script to build such a package. To create the package simply do (creating the uncompressible dummy payload will take a bit initially):
    cd testpkg
    fakseroot ./build

Note that I have also reproduced this issue on Ubuntu 11.04 64bit.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: software-center 5.0.1.4
ProcVersionSignature: Ubuntu 3.0.0-12.20-generic 3.0.4
Uname: Linux 3.0.0-12-generic i686
ApportVersion: 1.23-0ubuntu3
Architecture: i386
Date: Thu Dec 29 12:23:00 2011
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release i386 (20111012)
PackageArchitecture: all
SourcePackage: software-center
UpgradeStatus: No upgrade log present (probably fresh install)

[Francois Gouget (fgouget)]

I have tracked this down a bit and here's what I believe is happening:
  software-center creates a DebPackage object (in /usr/share/software-center/softwarecenter/db/debfile.py)
    DebPackage.__init__() (from /usr/share/pyshared/apt/debfile.py) calls
      DebPackage.open() which calls
        apt_inst.DebFile which is implemented in libapt-inst.so by apt-inst/deb/debfile.cc from the apt package
          which creates an AR object implemented in apt-inst/contrib/arfile.cc
            which calls ARArchive::LoadHeaders()
              which calls StrToNum(Head.Size,Memb->Size,sizeof(Head.Size))

I have confirmed with traces that we get a Python exception during the execution of the apt_inst.DebFile constructor. The full text of the exception is:

E:Could not open file /home/weaver/testpkg_1.0-1_all.deb - open (75: Value too large for defined data type), E:Unable to determine the file size - fstat (9: Bad file descriptor), E:Read error - read (9: Bad file descriptor)

I have not confirmed where the problem occurs in the lower levels though, they may happen in the File() constructor.

StrToNum() returns an unsigned long so on 32bit platforms it definitely cannot deal with package sizes between 4GiB and 10GB (which the ar file format can deal with). We find a similar issue with ARArchive::Member.Size. So there's apparently no support for large files on 32bit platforms.

However we get an error with 2GiB already on 32bit platforms. The first thing ARArchive::LoadHeaders() does is:

   signed long Left = File.Size();

On 32bit platforms that's a signed 32bit quantity and will thus end up being negative for a 2.4GiB package, then shunting 'while (Left > 0)' so we won't find the control and data members of the archive. This in turn will cause the DebFile() constructor to return an error when CheckMember("control.tar.gz") returns False.

Unfortunately that's likely just the tip of the iceberg.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Dan Kegel (dank)]

I see this also on ubunu 11.04 64 bit. 'dpkg -i' is a workaround, at least for users not afraid of the commandline.

[dino99 (9d9)]

oneiric have reached EOL now ; closing that report. Feel free to report against a newer version if needed, with ubuntu-bug

[Francois Gouget (fgouget)]

This bug is still present in Ubuntu 13.04 with 5.6.0-0ubuntu2. I'm attaching an updated trace generated with 'software-center --debug --disable-buy $HOME/testpkg_1.0-1_all.deb'.

[dino99 (9d9)]

That version is no more maintained

[Francois Gouget (fgouget)]

The bug is still present in Ubuntu 15.04's software-center 13.10-0ubuntu6, as well as in the 14.04 LTS version 13.10-0ubuntu4.

[Francois Gouget (fgouget)]

I retested this with "Ubuntu Software" (/snap/bin/snap-store) which seems be the replacement for the software-center and it installed the test package without trouble.

So if the software-center is indeed deprecated and not in use anymore this bug can be closed.