Copying large amount of files to USB stick makes system laggy or even freeze/hang (temporary)

[Vladimir Rutsky]

WORKAROUND: With disabled swap all works smoothly even with large vm sysctl settings and low free memory:
vm.dirty_ratio = 60
vm.dirty_background_ratio = 40

1. In order to reproduce this, create large enough temporary file:
dd if=/dev/zero of=/tmp/zeroes bs=8196 count=1000000
1000000+0 records in
1000000+0 records out
8196000000 bytes (8.2 GB) copied, 31.5395 s, 260 MB/s

2. Mount USB stick/external USB hard drive.
3. Start copy to external device:
cp /tmp/zeroes /media/bob/16\ GB\ USB\ 3.0/

Expected behavior: file being copying in background for a quite a lot of time (since USB sticks not so fast) not interrupting other work on system.
Observed behavior: after some time of copying system starts to lag --- mouse cursor jerking, alt+tab lags, windows became dimmed, music lags.
If I try to copy around 20 GB of files (e.g. photoes collections) system became almost completely unresponsive and I can observe in syslog following messages:

Jan 9 02:36:19 bob-laptop kernel: [120872.200909] INFO: task kworker/u16:1:15378 blocked for more than 120 seconds.
Jan 9 02:36:19 bob-laptop kernel: [120872.200955] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 9 02:36:19 bob-laptop kernel: [120872.200997] kworker/u16:1 D ffff88025f254580 0 15378 2 0x00000000
Jan 9 02:36:19 bob-laptop kernel: [120872.201046] ffff880159bebc10 0000000000000046 ffff880159bebfd8 0000000000014580
Jan 9 02:36:19 bob-laptop kernel: [120872.201090] ffff880159bebfd8 0000000000014580 ffff880218c6aee0 7fffffffffffffff
Jan 9 02:36:19 bob-laptop kernel: [120872.201134] ffff880159bebd68 ffff880159bebd70 ffff880218c6aee0 ffff880159bebde0
Jan 9 02:36:19 bob-laptop kernel: [120872.201178] Call Trace:
Jan 9 02:36:19 bob-laptop kernel: [120872.201200] [<ffffffff816ec5b9>] schedule+0x29/0x70
Jan 9 02:36:19 bob-laptop kernel: [120872.201229] [<ffffffff816ea499>] schedule_timeout+0x239/0x2d0
Jan 9 02:36:19 bob-laptop kernel: [120872.201263] [<ffffffff81091ac8>] ? resched_task+0x28/0x80
Jan 9 02:36:19 bob-laptop kernel: [120872.201294] [<ffffffff81092555>] ? check_preempt_curr+0x75/0xa0
Jan 9 02:36:19 bob-laptop kernel: [120872.201327] [<ffffffff81092599>] ? ttwu_do_wakeup+0x19/0xd0
Jan 9 02:36:19 bob-laptop kernel: [120872.201362] [<ffffffff816eca4d>] wait_for_completion+0xad/0x120
Jan 9 02:36:19 bob-laptop kernel: [120872.201396] [<ffffffff81094520>] ? wake_up_state+0x20/0x20
Jan 9 02:36:19 bob-laptop kernel: [120872.201429] [<ffffffff81084606>] kthread_create_on_node+0xa6/0x120
Jan 9 02:36:19 bob-laptop kernel: [120872.201467] [<ffffffff8107de90>] ? manage_workers.isra.25+0x2a0/0x2a0
Jan 9 02:36:19 bob-laptop kernel: [120872.201503] [<ffffffff8107da5b>] create_worker+0xeb/0x200
Jan 9 02:36:19 bob-laptop kernel: [120872.201533] [<ffffffff8107027c>] ? mod_timer+0x13c/0x210
Jan 9 02:36:19 bob-laptop kernel: [120872.201563] [<ffffffff8107dd42>] manage_workers.isra.25+0x152/0x2a0
Jan 9 02:36:19 bob-laptop kernel: [120872.201598] [<ffffffff8107e20a>] worker_thread+0x37a/0x3c0
Jan 9 02:36:19 bob-laptop kernel: [120872.201632] [<ffffffff8107de90>] ? manage_workers.isra.25+0x2a0/0x2a0
Jan 9 02:36:19 bob-laptop kernel: [120872.201668] [<ffffffff81084740>] kthread+0xc0/0xd0
Jan 9 02:36:19 bob-laptop kernel: [120872.201697] [<ffffffff81084680>] ? kthread_create_on_node+0x120/0x120
Jan 9 02:36:19 bob-laptop kernel: [120872.201734] [<ffffffff816f716c>] ret_from_fork+0x7c/0xb0
Jan 9 02:36:19 bob-laptop kernel: [120872.201765] [<ffffffff81084680>] ? kthread_create_on_node+0x120/0x120

Lagging stops when copying finishes. This issue observed with all tested USB storage devices (two USB sticks and one external USB-HDD), in both USB 2 and USB 3 ports, with destination filesystem ext4 and NTFS.

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: linux-image-3.11.0-15-generic 3.11.0-15.23
ProcVersionSignature: Ubuntu 3.11.0-15.23-generic 3.11.10
Uname: Linux 3.11.0-15-generic x86_64
ApportVersion: 2.12.5-0ubuntu2.2
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: bob 20835 F.... pulseaudio
                      bob 27269 F.... pulseaudio
 /dev/snd/pcmC0D0p: bob 20835 F...m pulseaudio
CheckboxSubmission: 3d16077c4fdd6a017d47f6e3dc4f3c54
CheckboxSystem: b633b4f40868d491c2ae5b50030ce6f3
Date: Fri Jan 10 04:05:31 2014
HibernationDevice: RESUME=UUID=e4b8cb86-0729-4f43-896f-89d9216cbc0f
InstallationDate: Installed on 2014-01-01 (8 days ago)
InstallationMedia: Ubuntu 13.10 "Saucy Salamander" - Release amd64 (20131016.1)
MachineType: Sony Corporation SVS1311S9RB
MarkForUpload: True
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.11.0-15-generic.efi.signed root=/dev/mapper/ubuntu--vg-root ro quiet splash
RelatedPackageVersions:
 linux-restricted-modules-3.11.0-15-generic N/A
 linux-backports-modules-3.11.0-15-generic N/A
 linux-firmware 1.116
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 05/17/2012
dmi.bios.vendor: Insyde Corp.
dmi.bios.version: R0142C5
dmi.board.asset.tag: N/A
dmi.board.name: VAIO
dmi.board.vendor: Sony Corporation
dmi.board.version: N/A
dmi.chassis.asset.tag: N/A
dmi.chassis.type: 10
dmi.chassis.vendor: Sony Corporation
dmi.chassis.version: N/A
dmi.modalias: dmi:bvnInsydeCorp.:bvrR0142C5:bd05/17/2012:svnSonyCorporation:pnSVS1311S9RB:pvrC60ARA3P:rvnSonyCorporation:rnVAIO:rvrN/A:cvnSonyCorporation:ct10:cvrN/A:
dmi.product.name: SVS1311S9RB
dmi.product.version: C60ARA3P
dmi.sys.vendor: Sony Corporation

[Brad Figg (brad-figg)]

This change was made by a bot.

[Vladimir Rutsky (rutsky)]

Here is recent question describing this issue: http://askubuntu.com/questions/397249/system-freezes-unresponsive-unusable-when-copying-large-file-to-usb

I tried to reduce cache size up to

vm.dirty_background_ratio = 2
vm.dirty_ratio = 5

--- system became much more usable when files being copied, but still laggy (e.g. tabs switching in gnome-terminal hangs, text input freezes on several seconds in Firefox).

[Joseph Salisbury (jsalisbury)]

Did this issue occur in a previous version of Ubuntu, or is this a new issue?

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.13 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.13-rc7-trusty/

[Vladimir Rutsky (rutsky)]

I discovered two more requirements to reproduce this bug: free RAM should be low and swap must be enabled.

To "occupy" most of 8 GB RAM that I have after reboot I allocate it from python script:
python -c 'import sys; t = "X" * 1024 * 1024 * 1024 * 5; sys.stdin.read()'
after that only about 15-25% RAM is free.

With disabled swap all works smoothly even with large vm sysctl settings (vm.dirty_ratio = 60, vm.dirty_background_ratio = 40) and low free memory.

I tested with kernel version 3.13.0-031300rc7-generic installed from provided link and bug is reproducible.
I'm not tested yet with older Ubuntu releases, but I believe I experienced this bug more than year ago too.

[kdureidy (kdureidy-k)]

I have the same issue, i am not an expert, but I think this issue is because I dont have swap partition

[Joseph (joehms22)]

"I discovered two more requirements to reproduce this bug: free RAM should be low and swap must be enabled."

My system reacts this way even without RAM low, swap is on but the system sits at about 3gb/6 as far as RAM usage is concerned.

My system: Linux joseph-UX32A 3.11.0-17-generic #31-Ubuntu SMP Mon Feb 3 21:52:43 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Turning swap off mostly fixes the system stuttering.

[Simon P. (simpre)]

I have the same problem, running 14.04 (but the problem also occurred with the previous versions). My Ram is 4 Gb, Swap enabled.

Found another bug report describing a similar (if not the same?) issue: https://bugs.launchpad.net/ubuntu/+source/nautilus/+bug/1208993

[zzarko (zzarko-gmail)]

I have the exact same issue on 12.04 64bit when copying to/from USB 3.0 HD (ext4 and NTFS file systems on USB HD; same thing on 2.0 and 3.0 ports) for a few months. Everything regarding this worked OK about a half a year ago and earlier. Actually, same issue on two computers with 12.04. In the next week or so I'm switching to 14.04 and I will test this again. One machine has 8GB RAM, the other one 4GB. Turning swap off doesn't change anything.

[drscre (dr-scre)]

I have exactly the same problem with Lubuntu 13.10, 64 bit.

When i copy a big file from hard drive to USB stick, the system gets laggy and iotop shows that kswapd0 periodically eats 99.9% of IO.

Turing of hugepages (http://lwn.net/Articles/467328/) and adjusting dirty_background_bytes / dirty_bytes (http://lwn.net/Articles/572911/) did not solve the problem.

[John Mapp (spidermonkey2012)]

I have an ASUS N550JX running Mint 17.3, and am experiencing this same issue.
I applied the bug workaround, and that seems to have help a little. But still not 100%

--------------------------------
vm.dirty_ratio = 60
vm.dirty_background_ratio = 40

sudo swapoff -a

---------------------------------

[levien (levien)]

Part of the problem could be that that Nautilus eats up a lot of memory (even up to the point where it may crash) and may slow down or freeze when copying many files. This is due to issues with gvfs:
https://bugs.launchpad.net/ubuntu/+source/nautilus/+bug/1133477