unkillable process consuming 100% cpu time

[William Anderson]

creating disk images with either VirtualBox or qemu-img causes the process to stop responding and consume 100% of the cpu of one core of the dual core processor. Kill -9 does not effect the process. the only way to stop it is to reboot the computer. i also encountered this with transmission at one point as well.

on my system, to reproduce this, simply run the command:
$ qemu-img create -f raw Ubuntu-11.04-desktop.img 5G

this is supposed to create a five gigabyte raw disk image that i can install a copy of Ubuntu under qemu.

also happens every time i try to create a disk image in VirtualBox. Start VirtualBox, and create a new virtual machine. give it a name, select the OS and version, memory size, virtual hard disk, create new disk. this starts the "virtual disk creation wizard', i choose the VDI image type, and elect to have all of the disk space allocated by checking fixed size. next choose the location and size. lastly, a summary is displayed, clicking on create causes a status dialog to pop up and VirtualBox starts to make the disk. shortly, the VBoxSVC process appears and after a time, consumes 100% cpu time.

this bug is likely related to bug 913787 at https://bugs.launchpad.net/ubuntu/+source/linux/+bug/913787 . like Sampo, my home directory is encrypted. however, his dd command does not reproduce the bug for me.

in my case, this issue seemed to apear after upgrading the kernel to 3.0.0-15-generic, however while VirtualBox worked with 3.0.0-14 and older, this problem occures with qemu-img in 3.0.0-14 and -13 as well.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: qemu-kvm 0.14.1+noroms-0ubuntu6.2
ProcVersionSignature: Ubuntu 3.0.0-15.26-generic 3.0.13
Uname: Linux 3.0.0-15-generic i686
NonfreeKernelModules: wl
ApportVersion: 1.23-0ubuntu4
Architecture: i386
Date: Thu Feb 2 00:19:02 2012
EcryptfsInUse: Yes
ExecutablePath: /usr/bin/qemu-img
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release i386 (20111012)
KvmCmdLine: Error: command ['ps', '-C', 'kvm', '-F'] failed with exit code 1: UID PID PPID C SZ RSS PSR STIME TTY TIME CMD
MachineType: LENOVO 2958
ProcEnviron:
 SHELL=/bin/bash
 PATH=(custom, user)
 LANG=en_US.UTF-8
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.0.0-15-generic root=UUID=a6551c5d-43b4-4962-b132-0ddf59cb55ac ro crashkernel=384M-2G:64M,2G-:128M quiet splash vt.handoff=7
SourcePackage: qemu-kvm
UpgradeStatus: Upgraded to oneiric on 2012-01-14 (18 days ago)
dmi.bios.date: 05/05/2010
dmi.bios.vendor: LENOVO
dmi.bios.version: 18CN45WW(V2.54)
dmi.board.name: NITU1
dmi.board.vendor: LENOVO
dmi.board.version: REFERENCE
dmi.chassis.asset.tag: No Asset Tag
dmi.chassis.type: 10
dmi.chassis.vendor: No Enclosure
dmi.chassis.version: N/A
dmi.modalias: dmi:bvnLENOVO:bvr18CN45WW(V2.54):bd05/05/2010:svnLENOVO:pn2958:pvrLenovoG550:rvnLENOVO:rnNITU1:rvrREFERENCE:cvnNoEnclosure:ct10:cvrN/A:
dmi.product.name: 2958
dmi.product.version: Lenovo G550
dmi.sys.vendor: LENOVO

[William Anderson (william-anderson)]

Hi,

Just an added note, running the qemu-img command on a directory that is not encrypted results in a 5gb image file being created so fast that i thought there must have been an error of some sort, but there is a file created.

william@ishtar:/home/Books$ qemu-img create -f raw Ubuntu-11.04-desktop.img 5G
Formatting 'Ubuntu-11.04-desktop.img', fmt=raw size=5368709120
william@ishtar:/home/Books$ ls -al
total 8
drwxrwxrwx 2 root root 4096 2012-02-02 00:48 .
drwxr-xr-x 8 root root 4096 2012-01-20 00:39 ..
-rw-r--r-- 1 william william 5368709120 2012-02-02 00:51 Ubuntu-11.04-desktop.img
william@ishtar:/home/Books$ file Ubuntu-11.04-desktop.img
Ubuntu-11.04-desktop.img: data

VirtualBox also created an 8gb disk image file with no trouble.

both of these results were running the latest 3.0.0-15-generic kernel.

there seems to be a bad interaction with the kernel and the encrypted file system.

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream kernel? It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.3 kernel[1] (Not a kernel in the daily directory). Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag(Only that one tag, please leave the other tags). This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text.

If this bug is fixed by the mainline kernel, please add the following tag 'kernel-fixed-upstream-KERNEL-VERSION'. For example, if kernel version 3.3-rc2 fixed the issue, the tag would be: 'kernel-fixed-upstream-v3.3-rc2'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[1] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.3-rc2-precise/

[William Anderson (william-anderson)]

 today i caused this bug to come out when i tried moving one of the big image files into my home directory. same 100% cpu, same unkillable.

i downloaded and installed the latest mainline kernel ( 3.3.0-030300rc2-generic-pae), booted up into it with only one issue, my wireless network driver is not loaded, so i have no network connectivity.

after booting into the newly installed mainline kernel, i tried all of the failing tasks to try to reproduce the unkillable process that uses 100% of the CPU.

i started with the mv command, moving the 5Gb disk image file to my home directory. watching my mv command in top, the cpu percent in not 100%, but a more reasonable 30-60%, and data continues to be added to the destination directory. after a minute or so, the file finished copying, all 5Gb of it.

creating a disk image using qemu-img also works as expected. creating a 5Gb raw disk image file works. i am also able to stop the qemu-img process at any time in the terminal where i ran it from using Ctl-C.

creating a disk image using VirtualBox also works as expected. the VBoxSVC process spends a lot of time using 100% cpu, but the image file continues to have bytes allocated to it. the process creating an 8Gb disk image file completes. pressing the cancel icon on the progress dialog changes status message to "Canceling...", just like when the VBoxSVC process was unkillable; however, this time the process continues to allocate disk pace to the image file and, once all eight gigabytes is allocated, exits as expected.

i think that this bug does not exist in the latest mainline kernel

[Alejandro R. Mosteo (mosteo)]

I have too observed this behavior with disk-heavy processes, and also having home encrypted. The troublesome part is that it has come and gone since at least natty (see my report #665211), without any conditions (that I can identify) changing. After several months of happiness, I'm again intermittently getting it in another computer with the 'deluged' process.

My old report was marked invalid since it apparently went away and I couldn't really narrow down the cause. With this I mean that I wouldn't be totally confident after a few tests that the bug is fixed (although I hope so!), without having identified the cause.

[William Anderson (william-anderson)]

I noticed that there is a new kernel in the update manager (3.0.0-16-generic). I installed this and booted into it to see if this bug is still an issue with the updated kernel.

qemu-img is now running at 100%, i am unable to kill the process, closing the terminal window that the process was run under did nothing to it. the only difference this time is the memory allocated to the runaway process is not freed in response to the kill command. otherwise, this bug is still here.

[Alejandro R. Mosteo (mosteo)]

Bug 913787, which is in all likelihood the same one, is marked for expiry in less than two months. I think that report includes valuable information. It's marked as unconfirmed, this as confirmed. Might it be worth making it a duplicate of this one?

[William Anderson (william-anderson)]

I have no problem making bug 913787 a duplicate of this one.

 I think that the major issue regarding that bug is on the Linux Mint distribution which, while it is based on Ubuntu Oneric, is not actually Ubuntu and it is not unreasonable to expect that the Ubuntu team to not want to support it.

My system, where I am having the same issue, is running Ubuntu 11.10.

My current work around is to install the mainline 3.3-rc2 kernel described in message 3 above. To use that kernel, I had to change and re-configure the Broadcom wireless network card driver (changed driver to b43 from the wl driver the Ubuntu installer choose) and remove the Ubuntu Virtualbox packages and install the packages from Virtualbox.com which seems to successfully compile and install the vboxdrv kernel driver. With those two changes (and remembering that anything kernel related cannot be automatically installed or updated) my system runs with no trouble and this bug is not there.

[William Anderson (william-anderson)]

This morning I found that there is another kernel upgrade available (3.0.0-16.29, up from 3.0.0-16.28). I installed this kernel and booted up into it with no trouble:

william@ishtar:~$ uname -a
Linux ishtar 3.0.0-16-generic #29-Ubuntu SMP Tue Feb 14 12:49:42 UTC 2012 i686 i686 i386 GNU/Linux

Running this kernel, I executed the qemu-img command to create a disk image that reliably causes this problem to occur:

william@ishtar:~/test$ qemu-img create -f raw Ubuntu-11.04-desktop.img 5G
Formatting 'Ubuntu-11.04-desktop.img', fmt=raw size=5368709120

This is running on my encrypted home directory. Watching in top, I saw this command alternate between 20 and 60% cpu until the image file grew to around 1Gb in size, then the CPU percent stayed pegged at 100% and the file did not get any bigger. Up to this point this is behaving exactly like the bug report issue; however, with this kernel I am able to kill this process. Ctl-C in the terminal killes the process, as does kill <PID>.

The VirtualBox VBoxSVC process goes straight into the consuming 100% cpu state when creating a big disk image. However, as with the qemu-img command, I am now able to kill that process, too.

At this time, the 100% zombie process problem seems to be fixed with this latest kernel update; however, the issue of creating gigabyte sized files with both of these tools on an encrypted home directory still exists. Both of these applications run with no issues on a non-encrypted directory.

[William Anderson (william-anderson)]

This morning, when I checked the available updates in the update manager, I found a new kernel waiting to be installed. I upgraded to this kernel (version 3.0.0-17.30) and successfully booted into it with no problems.

william@ishtar:~$ uname -a
Linux ishtar 3.0.0-17-generic #30-Ubuntu SMP Thu Mar 8 17:34:21 UTC 2012 i686 i686 i386 GNU/Linux

After I booted into this kernel, I found that I was able to make big disk image files with both the qemu-img command and through the VirtualBox GUI. I was also able to interrupt the file creation process with ctl-c in the command window, or by pressing the cancel button in VirtualBox.

This issue seems to be fixed with this new kernel.

[William Anderson (william-anderson)]

today updated to new kernel version with the update manager. this new version (3.0.0-19.33) works creating big disk image files with no trouble.

[penalvch (penalvch)]

William Anderson, this bug report is being closed due to your last comment https://bugs.launchpad.net/ubuntu/+source/linux/+bug/925309/comments/11 regarding this being fixed with an update. For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status. Thank you again for taking the time to report this bug and helping to make Ubuntu better. Please submit any future bugs you may find.