vboxdrv occasionally fails to load on karmic 2.6.30 kernels due to active nmi_watchdog

[Niall Creech]

Binary package hint: virtualbox-ose

the vboxdrv module has an occasional load error on karmic 2.6.30 kernels,

Jul 12 07:13:17 crystaltree kernel: [ 28.053203] vboxdrv: Trying to deactivate the NMI watchdog permanently...
Jul 12 07:13:17 crystaltree kernel: [ 28.053207] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI
Jul 12 07:13:17 crystaltree kernel: [ 28.053209] vboxdrv: watchdog by specifying 'nmi_watchdog=0' at kernel command line.

This seems to affect virtualbox 2.2 and the 3. The error is due to vbox detecting the nmi_watchdog being active although it seems to be unpredictable for each boot and unaffected by adding nmi_watchdog=0 to the kernel. checking /proc/interrupts shows,

$ less /proc/interrupts |grep NMI
NMI: 0 0 Non-maskable interrupts

$ apt-show-versions virtualbox-2.2 linux-image-2.6.31-2-generic linux-image-2.6.31-3-generic
linux-image-2.6.31-2-generic/karmic uptodate 2.6.31-2.17
linux-image-2.6.31-3-generic/karmic uptodate 2.6.31-3.19
virtualbox-2.2/jaunty uptodate 2.2.4-47978_Ubuntu_jaunty

Forum thread here,
http://ubuntuforums.org/showthread.php?t=1209319

[max (mikhmv)]

Confirm on 2.6.31-3

dmesg | grep watchdog
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-2.6.31-3-generic root=UUID=b086dc0d-0fe7-4487-855b-494761765ca7 ro quiet acpi_osi=Linux nosplash [b]nmi_watchdog=0[/b]
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.31-3-generic root=UUID=b086dc0d-0fe7-4487-855b-494761765ca7 ro quiet acpi_osi=Linux nosplash[b] nmi_watchdog=0[/b]
[ 14.459621] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[ 14.459625] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI
[ 14.459626] vboxdrv: watchdog by specifying 'nmi_watchdog=0' at kernel command line.

$ cat /proc/interrupts |grep NMI
NMI: 0 0 Non-maskable interrupts

[max (mikhmv)]

$ dmesg |grep NMI
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
[ 14.459621] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[ 14.459625] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI

[max (mikhmv)]

dmesg | grep watchdog
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-2.6.31-3-generic root=UUID=b086dc0d-0fe7-4487-855b-494761765ca7 ro quiet acpi_osi=Linux nosplash [b]nmi_watchdog=0[/b]
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.31-3-generic root=UUID=b086dc0d-0fe7-4487-855b-494761765ca7 ro quiet acpi_osi=Linux nosplash[b] nmi_watchdog=0[/b]
[ 14.459621] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[ 14.459625] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI
[ 14.459626] vboxdrv: watchdog by specifying 'nmi_watchdog=0' at kernel command line.

Look like this bug can be fixed by virtualbox.
checking NMI could be done (in linux) by command:

    cat /proc/interrupts | grep NMI

in my case results:

    NMI: 0 0 Non-maskable interrupts

All 0 mean that NMI was initialized (because it present) but it is inactive.
This could be fixed by 2 ways:
First) Ask developers linux kernel to remove creation NMI records if NMI is inactive.
Second) Virtualbox developers could check value and if it is 0 assume than NMI is inactive.

[max (mikhmv)]

Related bug reports:
Bug #400227
http://www.virtualbox.org/ticket/4529

[Sergey Sedov (serg-sedov)]

I have this problem too

kernel 2 2.6.31-3-generic
VirtualBox 3.0.2 for Linux

dmesg | grep watchdog

[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.31-3-generic root=UUID=bf7e7382-637c-49b2-b010-a56276e27219 ro quiet splash nmi_watchdog=0
[ 20.440185] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[ 20.440190] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI
[ 20.440191] vboxdrv: watchdog by specifying 'nmi_watchdog=0' at kernel command line.
[ 586.110601] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[ 586.110606] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI
[ 586.110608] vboxdrv: watchdog by specifying 'nmi_watchdog=0' at kernel command line.
[ 1125.410335] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[ 1125.410341] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI
[ 1125.410342] vboxdrv: watchdog by specifying 'nmi_watchdog=0' at kernel command line.
[ 1641.876499] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[ 1641.876504] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI
[ 1641.876506] vboxdrv: watchdog by specifying 'nmi_watchdog=0' at kernel command line.

cat /proc/interrupts | grep NMI
NMI: 0 Non-maskable interrupts

[Arnaud Faucher (arnaud-faucher)]

Problen still present with kernel 2.6.31-4.

uname -a:
Linux green 2.6.31-4-generic #21-Ubuntu SMP Thu Jul 23 22:45:16 UTC 2009 i686 GNU/Linux

[Jordan Wilberding (diginux)]

I also have the error with a fresh upgrade to karmic with kernel 2.6.31-4

[MAKAPOH (makapoh)]

have this problem too

uname -a
Linux makapoh-desktop 2.6.31-4-generic #22-Ubuntu SMP Fri Jul 24 18:05:56 UTC 2009 x86_64 GNU/Linux

cat /proc/interrupts | grep NMI
NMI: 0 0 Non-maskable interrupts

dmesg | grep watchdog
[ 0.000000] Command line: root=UUID=f48887a2-bd9d-422f-9e92-39a08fe6ffe5 ro quiet splash nmi_watchdog=0
[ 0.000000] Kernel command line: root=UUID=f48887a2-bd9d-422f-9e92-39a08fe6ffe5 ro quiet splash nmi_watchdog=0
[ 523.770338] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[ 523.770349] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI
[ 523.770353] vboxdrv: watchdog by specifying 'nmi_watchdog=0' at kernel command line.

[Jeremy Wilkins (wjeremy)]

I am also suffering with this bug.
Using nmi_watchdog=0 has no effect. Still cannot load module. Some are reporting that the module loads sometimes, not others, but I have never had it load once for me. Same detail as previous post.

[max (mikhmv)]

For me some times helpful run several times this command:
 sudo /etc/init.d/vboxdrv setup

First attempt usually fail. but several repeats usually helpful.
If this command will finish successfully don't reboot and use virtualbox.

[dentaku65 (dentaku65)]

Last update from virtualbox repo fix the issue; see https://bugs.launchpad.net/ubuntu/+bug/408594

[Niall Creech (sevenmachines-deactivatedaccount)]

latest virtualbox 3.0.4-50677 add to SUPDrv-linux.c so that >= 2.6.31 kernels simply warn about nmi

/* 2.6.31+: The performance counter framework will initialize the LVTPC
* vector as NMI. We can't disable the framework but the kernel loader
* script will do 'echo 2 > /proc/sys/kernel/perf_counter_paranoid'
* which hopefilly prevents any usage of hardware performance counters
* and therefore triggering of NMIs. */

adding the attached patch vboxdrv dkms in virtualbox-ose-3.0.2-dfsg simply adds this warning too and allows the module to install instead of bail until package is upgraded

[Niall Creech (sevenmachines-deactivatedaccount)]

i've added the patched ose version that i'm using to my ppa if anyone else finds it useful. note it has a small fix for building against libpulse-dev/karmic uptodate 1:0.9.16~test4-0ubuntu1 so i dont know how it'll go with non up to date pulseaudio versions.
https://launchpad.net/~dinxter/+archive/ppa

virtualbox-ose (3.0.2-dfsg-1ubuntu2~dinxter3) karmic; urgency=low
  * 23-pulse_stubs_fix
  * 22-nmi_perf_counter patch (LP: #399732)
    - also adds echo 2 > perf_counter_paranoid to /etc/default/virtualbox-ose

[Oleksiy Krivoshey (oleksiyk)]

2.6.31-5-generic #24-Ubuntu SMP Sat Aug 1 12:48:18 UTC 2009 i686 GNU/Linux

Still experiencing this issue:

[ 308.523504] vboxdrv: Trying to deactivate the NMI watchdog permanently...
[ 308.523512] vboxdrv: NMI watchdog either active or at least initialized. Please disable the NMI
[ 308.523516] vboxdrv: watchdog by specifying 'nmi_watchdog=0' at kernel command line.

Doing this 3 times solves the problem for a session:

$ sudo modprobe vboxdrv
FATAL: Error inserting vboxdrv (/lib/modules/2.6.31-5-generic/updates/dkms/vboxdrv.ko): Invalid argument
$ sudo modprobe vboxdrv
FATAL: Error inserting vboxdrv (/lib/modules/2.6.31-5-generic/updates/dkms/vboxdrv.ko): Invalid argument
$ sudo modprobe vboxdrv
------------------

e.g. the mod inserts on 3rd time without any problems.

[Jeremy Wilkins (wjeremy)]

@Dinxter: Thank you for the ppa files. I am trying them tonight. I do not have the exact same version of libpulse so I will let you know if there are troubles, but I am running fully up to date.

[Jeremy Wilkins (wjeremy)]

After testing your ppa version, it would appear to be working just fine. I have other video glitches to deal with regarding the new ati drivers (inverted colors) that virtualbox seems to initiate faster than other programs, but that is not related. The pulseaudio sound is working great, and the VM started, shutdown and ran the windows updates just fine.

[Sergey Sedov (serg-sedov)]

Everything works fine after latest kernel update and witn latest build from http://www.virtualbox.org/

[Sense Egbert Hofstede (sense)]

Marking the bug as triaged and setting the status as they should be.
The provided patch indeed fixes the issue for me. However, the fix seems to be a workaround, rather than a fix for the problem. Also it only checks for 2.6.31, but this bug was initially reported by someone using a 2.6.30 kernel. Shouldn't that be included as well?

[Niall Creech (sevenmachines-deactivatedaccount)]

as the comment that goes with the patch says,

* The performance counter framework will initialize the LVTPC
* vector as NMI. We can't disable the framework but the kernel loader
* script will do 'echo 2 > /proc/sys/kernel/perf_counter_paranoid'
* which hopefilly prevents any usage of hardware performance counters
* and therefore triggering of NMIs.

a workaround that makes the problem hopefully unlikely to occur but isnt a guarantee that it won't. this is essentially the exact same patch the virtualbox team uses on the newer version, they've set it for 2.6.31 and greater so i've left it unchanged. i'll quickly test for 2.6.30

[Niall Creech (sevenmachines-deactivatedaccount)]

actually i'll not quickly test on 2.6.30, its been dropped from ubuntu pool for amd64. if someones still got it installed see if you can generate the problem. if it comes down to it i'll rebuild from source but obviously it would be easier if someone with it still installed tried. i think their may be be i386 packages available, if someones running that

[Felix Geyer (debfx)]

Should be fixed in 3.0.4

[Sense Egbert Hofstede (sense)]

I'm setting this back to Fix Released because there is nothing that seems to indicate that the bug has returned. Frankie: please don't change the status without providing a reason.