cp crashes when copying large files

[David Mabe]

I was trying to copy a large tar file (7.4 GB) from my internal hard-drive to an external one connected over USB 2. The "Copy" and "Paste" inside Nautilus crashed twice so I decided to try cp from the "Terminal". This crashed also, producing the message, "File size limit exceeded (core dumped)". I was fairly surprised at this. It seems that it can only copy 4.0 GB before crashing.

"Report" attached (I hope).

[Kai Kasurinen (kai-kasurinen)]

What filesystem do you have in destination (usb harddrive)?

[David Mabe (davemicc)]

I didn't consider that. It's FAT32 which has a file size limit of 4GB - so that explains this.

I guess this bug is bogus then...

[edschofield (schofield)]

No, this bug is not bogus! cp should check this before it starts copying a > 4GB file and give a helpful error message. It should not waste the user's time copying only the first 4GB of the file and then dump core.

This bug is simple to reproduce: create a large (>4GB) tar file and copy it to a VFAT partition (e.g. an iPod).

At least one bug is a duplicate of this bug: #75574. Bug #63900 is also related.

[Albert Cardona (cardona)]

I am running into a similar problem: when copying the folder of a large postgresql database (152 Gb), cp crashes. I tried rsync, and got farther (to 85% or so) and then crashed as well. Of course I can just reboot, the ext3 journal re-runs and fixes itself, and continue until all is copied. But it's not the expected, to say the least, way to do it.

The folder is composed of numerous files, among which 150 files of about 1Gb each.

The file system consists of a a small boot partition, a root ext3, a swap, and 2 lvm, each using 2 hard drives of 500 Gb each (so, 2 LVM of 1 terabyte each), using a single ext3 partition inside each lvm.

What happens is this: at some point, the entire operating system freezes, and the drives start making a periodic ticking noise every 3 or 4 seconds.
The computer and drives are new, not even one week old. I can send all hardware info if needed.

Running edgy-amd64 at it's latest as of today.

[hronir (hronir)]

I saw the same behaviour: I didn't know of the 4GB limit, still the (s)cp started. With scp i got no error message at all, while cp exited (after copying the first 4GB part of my file) with the following error message: File size limit exceeded (core dumped)

[kvaibhav (vaibhav-kaware)]

It seems, the problem is recognized and 'popular' enough!
But is there any workaround?
If yes, where?
If no, where could one find at least something to do.

[Micah Cowan (micahcowan)]

A patch has been submitted against the kernel. See bug 69807, of which this bug is a duplicate. Further communications should possibly go to that bug instead.
Workarounds would include writing a wrapper program that sets the signal handler for SIGXFSZ to SIG_IGN.

Note that the kernel patch only eliminates the core-dumping signal: the write() command will still fail in the same circumstances: the difference is that it will return EFBIG instead of catching SIGXFSZ and dumping core. :) This allows things like multi-file copies to continue copying the rest of the files.

[Micah Cowan (micahcowan)]

Sorry; I meant bug 75574, of course.