Bazaar crashes when adding a files larger than 4GB

[David Coles]

Wanted to see if how well Bazaar supported large (multi-gigabyte) files. Looks like the state file doesn't support sizes that can't be represented in 32 bits.

Arguably storing files larger than 4GB in a distributed version control system is probably not the best idea, but there are occasionally cases like VM images or other large binary assets that might hit this limit. At very least a "Sorry. Too big" error would be nice.

Steps to reproduce:
    mkdir /tmp/large
    cd /tmp/large
    bzr init .
    dd if=/dev/zero of=large.dat bs=1000 count=5000000
    bzr commit -m "Large file"

ProblemType: Crash
DistroRelease: Ubuntu 11.04
Package: bzr 2.3.4-0ubuntu1
ProcVersionSignature: Ubuntu 2.6.38-11.48-generic 2.6.38.8
Uname: Linux 2.6.38-11-generic x86_64
NonfreeKernelModules: nvidia
Architecture: amd64
BzrDebugFlags: set()
BzrVersion: 2.3.4
CommandLine: ['/usr/bin/bzr', 'commit', '-m', 'Large']
CrashDb: bzr
Date: Thu Sep 1 18:09:53 2011
ExecutablePath: /usr/bin/bzr
FileSystemEncoding: UTF-8
InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Beta amd64 (20110413)
InterpreterPath: /usr/bin/python2.7
Locale: en_US.UTF-8
PackageArchitecture: all
Platform: Linux-2.6.38-11-generic-x86_64-with-Ubuntu-11.04-natty
ProcCmdline: /usr/bin/python /usr/bin/bzr commit -m Large
ProcEnviron:
 SHELL=/bin/bash
 PATH=(custom, user)
 LANG=en_US.UTF-8
 LANGUAGE=en_US:en
PythonVersion: 2.7.1
SourcePackage: bzr
Title: bzr crashed with error in pack_stat(): 'L' format requires 0 <= number <= 4294967295
UpgradeStatus: No upgrade log present (probably fresh install)
UserEncoding: UTF-8
UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare

[Jelmer Vernooij (jelmer)]

Newer versions of bzr will skip files larger than 20Mb and warn.

[Martin Packman (gz)]

The recent fix for bug 54624 should help here by preventing people accidentally committing huge files, but the bug is in the dirstate code (both python and pyrex versions) trying to fit the filesize in a 32 bit value.