Disabling lock debugging due to kernel taint with no apparent cause

[Dave Gilbert]

Hi,
  I'm seeing the 'Disabling lock debugging due to kernel taint' message on both my raring host and on (this) raring kvm guest,
but I don't see anything in any logs saying why it's tainted.

/proc/sys/kernel/tainted has the value 2 which my reading is module forced, but I don't really see why.
(only possibility I can see is module signing it is causing it? I can see in the code it sets the flag but doesn't seem to moan, and we do have it enabled).

ProblemType: Bug
DistroRelease: Ubuntu 13.04
Package: linux-image-3.7.0-7-generic 3.7.0-7.15
ProcVersionSignature: Ubuntu 3.7.0-7.15-generic 3.7.0
Uname: Linux 3.7.0-7-generic x86_64
ApportVersion: 2.7-0ubuntu2
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: dg 1565 F.... pulseaudio
CRDA: Error: command ['iw', 'reg', 'get'] failed with exit code 1: nl80211 not found.
CurrentDmesg:
 [ 64.643842] ISO 9660 Extensions: Microsoft Joliet Level 3
 [ 64.670482] ISO 9660 Extensions: RRIP_1991A
 [ 64.820121] hda-intel: IRQ timing workaround is activated for card #0. Suggest a bigger bdl_pos_adj.
Date: Sun Jan 6 02:05:33 2013
HibernationDevice: RESUME=UUID=0755ab67-0c09-463d-b3f9-4efd03f72ced
InstallationDate: Installed on 2012-11-24 (42 days ago)
InstallationMedia: Kubuntu 13.04 "Raring Ringtail" - Alpha amd64 (20121123)
IwConfig:
 eth0 no wireless extensions.

 lo no wireless extensions.
Lsusb: Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: Bochs Bochs
MarkForUpload: True
ProcFB: 0 cirrusdrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.7.0-7-generic root=UUID=ced98a70-8ff2-4b6c-945e-8a2da664309b ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-3.7.0-7-generic N/A
 linux-backports-modules-3.7.0-7-generic N/A
 linux-firmware 1.99
RfKill:

SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 01/01/2011
dmi.bios.vendor: Bochs
dmi.bios.version: Bochs
dmi.chassis.type: 1
dmi.chassis.vendor: Bochs
dmi.modalias: dmi:bvnBochs:bvrBochs:bd01/01/2011:svnBochs:pnBochs:pvr:cvnBochs:ct1:cvr:
dmi.product.name: Bochs
dmi.sys.vendor: Bochs

[Brad Figg (brad-figg)]

This change was made by a bot.

[Joseph Salisbury (jsalisbury)]

I can reproduce this issue with Raring. It doesn't seem to happen with Quantal.

The value of 2 for /proc/sys/kernel/tainted may indicate:

#define TAINT_UNSAFE_SMP 2

Since I can reproduce the bug, I'll perform a kernel bisect to see what commit introduced this regression.

[Dave Gilbert (ubuntu-treblig)]

* Joseph Salisbury (<email address hidden>) wrote:
> I can reproduce this issue with Raring. It doesn't seem to happen with
> Quantal.
>
> The value of 2 for /proc/sys/kernel/tainted may indicate:
>
> #define TAINT_UNSAFE_SMP 2

Be careful - I think that's a value that's shifted; my reading
of it was that it's actually the TAINT_FORCED (sorry can't
remember exact name) that has a value of 1, and 1<<1 is 2.

> Since I can reproduce the bug, I'll perform a kernel bisect to see what
> commit introduced this regression.
>
> ** Tags added: regression-release

Dave

[Dave Gilbert (ubuntu-treblig)]

Joseph:
  It looks like it is the mod sign stuff; see this lkml thread:

https://lkml.org/lkml/2013/1/11/132

[Joseph Salisbury (jsalisbury)]

Thanks for finding that, Dave. Saves allot of time bisecting. I can build a test kernel with that patch applied. I'll post a link to the kernel here and test it myself as well.

[Joseph Salisbury (jsalisbury)]

It seems the proper fix is still being discussed upstream:
https://lkml.org/lkml/2013/1/12/19

I'll await a final version of the patch then build a test kernel.

[Chris Samuel (chris-csamuel)]

Hi there,

Thanks for finding my patch on LKML, two points:

1) The updated reporting has been incorporated upstream for 3.9 by Rusty in commit 64748a2c9062da0c32b59c1b368a86fc4613b1e1.

2) Rusty additionally changed the code so that loading an unsigned module no longer disables lockdep checking in commit 373d4d099761cb1f637bed488ab3871945882273.

The cause of the issue is that *after* doing the module install make-kpkg runs objcopy on the modules to copy out the debug sections for a debug package and then uses objcopy to remove the same debug sections (along with the crypto signatures) from the ones in the main package. :-( I've reported that as Ubuntu bug #1099371 (though my initial diagnosis was incorrect, I've updated it in the comments and will correct the description now).

So until that's fixed all Raring kernels will be tainted on boot due to loading unsigned kernel modules (unless Ubuntu disables CONFIG_MODULE_SIG).

[Joseph Salisbury (jsalisbury)]

@Chris, do you know if commit 64748a2c9062da0c32b59c1b368a86fc4613b1e1 will be sent to stable as well?

[Chris Samuel (chris-csamuel)]

@Joseph - sorry for the delay, been moving house! I don't think 64748a2c9062da0c32b59c1b368a86fc4613b1e1 was being sent to stable, it doesn't fix anything as it just gives you more info about it being tainted.

The real fix will have to be in make-kpkg to stop it altering modules after signing (bug #1099371).

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]

[Dave Gilbert (ubuntu-treblig)]

Hmm so I'm still seeing a taint on 3.9.0-7-generic #15-Ubuntu on Saucy; it seems to be being triggered by the crc_itu_t module for no apparent reason:

[ 0.865613] crc_itu_t: module verification failed: signature and/or required key missing - tmeainting kernel

Is this the same bug or more specific?

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]

[Chris Samuel (chris-csamuel)]

Will someone please stop this robot closing the bug - it's still there!

Ignoring problems doesn't make them go away.

[Dave Gilbert (ubuntu-treblig)]

Well, the question is, is it still there; on Saucy (3.11.0-4-generic #9):
# grep -i taint /var/log/kern.log
Aug 26 11:34:52 major vmunix: [ 0.903410] crc_itu_t: module verification failed: signature and/or required key missing - tainting kernel
Aug 27 11:55:01 major vmunix: [ 0.903210] crc_itu_t: module verification failed: signature and/or required key missing - tainting kernel
Aug 28 11:44:37 major vmunix: [ 0.903483] crc_itu_t: module verification failed: signature and/or required key missing - tainting kernel
Aug 29 11:47:56 major vmunix: [ 0.903025] crc_itu_t: module verification failed: signature and/or required key missing - tainting kernel

that does look like the first module loaded; so we're still getting the taint - at least it does give a clue to what's going on now.