APT fails on packages >= 2 GB

Bug #845716 reported by Per Kristian Hove
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
apt (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

Ubuntu 10.04 apt-0.7.25.3ubuntu9.6

"apt-get install <package>" fails when trying to install packages larger than approximately 2 GB.

The reason is that APT uses incorrect data types - it counts bytes in floating point(!) as well as signed 32-bit variables, and uses atoi() to convert string representations of sizes to numbers.

The attached patch makes APT use ssize_t and atol().

Symptoms when installing from a private repository containing large packages:

# apt-get install matlab2011b
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  matlab2011b-bin
The following NEW packages will be installed:
  matlab2011b matlab2011b-bin
0 upgraded, 2 newly installed, 0 to remove and 126 not upgraded.
2.2856e+09,-2.00936e+09
How odd.. The sizes didn't match, email <email address hidden>
Need to get 2,286MB/2,286MB of archives.
After this operation, 0B of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://XXXXXXXX/ common/ matlab2011b-bin 1 [2,286MB]
Fetched 1B in 24s (0B/s)
Failed to fetch http://XXXXXXXX/common/matlab2011b-bin_1_amd64.deb Size mismatch
W: Bizarre Error - File size is not what the server reported 2285595598 18446744071700179918
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

Tags: patch lucid
Revision history for this message
Per Kristian Hove (perhov) wrote :
description: updated
tags: added: lucid patch
Revision history for this message
David Kalnischkies (donkult) wrote :

Using ssize_t doesn't help 32bit systems as ssize_t is only on 64bit systems larger than an int.

Either way, the issue should be fixed in a newer release of APT, see LP: #250909 and debians APT version 0.7.26~exp6 for details. It might be possible that this is not completely fixed for everyone through as debugs #632271 suggests which is only fixed in debian experimental for now (as these kind of changes make an abi break nessary and are therefore out of question for a sane backport).

It would be cool if you could test these versions and report back. I am closing this bug in the meantime. If it is really not fixed in newer versions please reopen!

Changed in apt (Ubuntu):
status: New → Invalid
Revision history for this message
Per Kristian Hove (perhov) wrote :

I've downloaded 0.7.26~exp12ubuntu1.tar.gz from https://launchpad.net/ubuntu/+source/apt/0.7.26~exp12ubuntu1 and compiled it on 10.04. I assume this is newer than 0.7.26~exp6 and sufficient to test what you suggest. Please let me know if this isn't what you asked me to test.

0.7.26~exp12ubuntu1 still fails:

root@XXXXXXXX:/var/tmp/apt-0.7.26~exp12ubuntu1/bin# LD_LIBRARY_PATH=. ./apt-get install matlab2011b
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  matlab2011b-bin
The following NEW packages will be installed:
  matlab2011b matlab2011b-bin
0 upgraded, 2 newly installed, 0 to remove and 20 not upgraded.
Need to get 2,286MB of archives.
After this operation, 0B of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://XXXXXXXX/ common/ matlab2011b-bin 1 [2,286MB]
Get:2 http://XXXXXXXX/ common/ matlab2011b 1 [7,676B]
Fetched 18.4EB in 29s (616PB/s)
Failed to fetch http://XXXXXXXX/common/matlab2011b-bin_1_amd64.deb Size mismatch
W: Bizarre Error - File size is not what the server reported 2285595598 18446744071700179918
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

Revision history for this message
Per Kristian Hove (perhov) wrote :

0.8.16~exp5ubuntu6 succeeds, but with a warning:

root@XXXXXXXX:/var/tmp/apt.pkg/apt-0.8.16~exp5ubuntu6/bin# LD_LIBRARY_PATH=. ./apt-get install matlab2011b
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  matlab2011b-bin
The following NEW packages will be installed:
  matlab2011b matlab2011b-bin
0 upgraded, 2 newly installed, 0 to remove and 20 not upgraded.
Need to get 2,286 MB of archives.
After this operation, 0 B of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://XXXXXXXX/ common/ matlab2011b-bin 1 [2,286 MB]
Get:2 http://XXXXXXXX/ common/ matlab2011b 1 [7,676 B]
Fetched 2,286 MB in 30s (75.7 MB/s)
Selecting previously deselected package matlab2011b-bin.
(Reading database ... 644888 files and directories currently installed.)
Unpacking matlab2011b-bin (from .../matlab2011b-bin_1_amd64.deb) ...
Selecting previously deselected package matlab2011b.
Unpacking matlab2011b (from .../matlab2011b_1_amd64.deb) ...
Processing triggers for desktop-file-utils ...
Processing triggers for python-gmenu ...
Rebuilding /usr/share/applications/desktop.en_US.utf8.cache...
Processing triggers for python-support ...
Setting up matlab2011b-bin (1) ...
Setting up matlab2011b (1) ...
update-alternatives: using /usr/bin/matlab2011b to provide /usr/bin/matlab (matlab) in auto mode.

W: Size of file /var/cache/apt/archives/partial/matlab2011b-bin_1_amd64.deb is not what the server reported 2285595598 18446744071700179918

It also shows the wrong file size while downloading:

51% [1 matlab2011b-bin 1,173 MB/18.4 EB 0%] 81.2 MB/s 13s

This seems to be only cosmetic.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.