Messages log filled with Intel IPS "ME failed to update for more than 1s, likely hung" messages

[Adam Drew]

Issue:
Messages log being continuously filled with the following messages:

Oct 11 00:21:18 adam-laptop kernel: [ 3625.690731] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:19 adam-laptop kernel: [ 3626.169839] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:19 adam-laptop kernel: [ 3626.373465] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:19 adam-laptop kernel: [ 3626.577043] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:20 adam-laptop kernel: [ 3627.109102] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:20 adam-laptop kernel: [ 3627.831785] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:21 adam-laptop kernel: [ 3628.187110] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:21 adam-laptop kernel: [ 3628.685203] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:23 adam-laptop kernel: [ 3630.489829] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:23 adam-laptop kernel: [ 3630.689944] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:24 adam-laptop kernel: [ 3631.180598] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:25 adam-laptop kernel: [ 3632.815064] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:26 adam-laptop kernel: [ 3633.176899] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:26 adam-laptop kernel: [ 3633.833168] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:27 adam-laptop kernel: [ 3634.755489] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:28 adam-laptop kernel: [ 3635.173216] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:29 adam-laptop kernel: [ 3636.288656] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:30 adam-laptop kernel: [ 3637.169538] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:30 adam-laptop kernel: [ 3637.413060] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:30 adam-laptop kernel: [ 3637.668569] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:30 adam-laptop kernel: [ 3638.003963] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:31 adam-laptop kernel: [ 3638.926266] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:32 adam-laptop kernel: [ 3639.664898] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:33 adam-laptop kernel: [ 3640.167989] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:33 adam-laptop kernel: [ 3640.990447] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:34 adam-laptop kernel: [ 3641.661202] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:36 adam-laptop kernel: [ 3643.657512] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:38 adam-laptop kernel: [ 3645.382263] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:38 adam-laptop kernel: [ 3645.653809] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:38 adam-laptop kernel: [ 3645.893304] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:39 adam-laptop kernel: [ 3646.516185] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:40 adam-laptop kernel: [ 3647.650096] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:42 adam-laptop kernel: [ 3649.386887] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:42 adam-laptop kernel: [ 3649.646407] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 11 00:21:44 adam-laptop kernel: [ 3651.067770] intel ips 0000:00:1f.6: ME failed to update for

These messages are constant, running multiple times per millisecond. Causing performance problems in other applications such as skipping audio.

Environment:
Ubuntu 10.10
adam@adam-laptop:~$ uname -a
Linux adam-laptop 2.6.35-22-generic #33-Ubuntu SMP Sun Sep 19 20:34:50 UTC 2010 i686 GNU/Linux
adam@adam-laptop:~$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i5 CPU M 430 @ 2.27GHz
stepping : 2
cpu MHz : 1199.000
cache size : 3072 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 4521.55
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i5 CPU M 430 @ 2.27GHz
stepping : 2
cpu MHz : 1199.000
cache size : 3072 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 2
apicid : 4
initial apicid : 4
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 4522.44
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i5 CPU M 430 @ 2.27GHz
stepping : 2
cpu MHz : 2267.000
cache size : 3072 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 1
initial apicid : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 4522.43
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 37
model name : Intel(R) Core(TM) i5 CPU M 430 @ 2.27GHz
stepping : 2
cpu MHz : 1199.000
cache size : 3072 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 2
apicid : 5
initial apicid : 5
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 11
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid
bogomips : 4522.44
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
adam@adam-laptop:~$ lspci
00:00.0 Host bridge: Intel Corporation Core Processor DRAM Controller (rev 12)
00:02.0 VGA compatible controller: Intel Corporation Core Processor Integrated Graphics Controller (rev 12)
00:16.0 Communication controller: Intel Corporation 5 Series/3400 Series Chipset HECI Controller (rev 06)
00:16.2 IDE interface: Intel Corporation 5 Series/3400 Series Chipset PT IDER Controller (rev 06)
00:16.3 Serial controller: Intel Corporation 5 Series/3400 Series Chipset KT Controller (rev 06)
00:1a.0 USB Controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 06)
00:1b.0 Audio device: Intel Corporation 5 Series/3400 Series Chipset High Definition Audio (rev 06)
00:1c.0 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 1 (rev 06)
00:1c.1 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 2 (rev 06)
00:1c.3 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 4 (rev 06)
00:1c.5 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 6 (rev 06)
00:1d.0 USB Controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 06)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev a6)
00:1f.0 ISA bridge: Intel Corporation Mobile 5 Series Chipset LPC Interface Controller (rev 06)
00:1f.2 SATA controller: Intel Corporation 5 Series/3400 Series Chipset 6 port SATA AHCI Controller (rev 06)
00:1f.3 SMBus: Intel Corporation 5 Series/3400 Series Chipset SMBus Controller (rev 06)
00:1f.6 Signal processing controller: Intel Corporation 5 Series/3400 Series Chipset Thermal Subsystem (rev 06)
03:00.0 Network controller: Intel Corporation Centrino Advanced-N 6200 (rev 35)
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03)
ff:00.0 Host bridge: Intel Corporation Core Processor QuickPath Architecture Generic Non-core Registers (rev 02)
ff:00.1 Host bridge: Intel Corporation Core Processor QuickPath Architecture System Address Decoder (rev 02)
ff:02.0 Host bridge: Intel Corporation Core Processor QPI Link 0 (rev 02)
ff:02.1 Host bridge: Intel Corporation Core Processor QPI Physical 0 (rev 02)
ff:02.2 Host bridge: Intel Corporation Core Processor Reserved (rev 02)
ff:02.3 Host bridge: Intel Corporation Core Processor Reserved (rev 02)

Switching to an older kernel such as 2.6.33-29 resolves the issue.

[Adam Drew (adamrdrew)]

Interesting from here:

http://<email address hidden>/msg00311.html

> > > + cur_seqno = (thm_readl(THM_ITV) &
> > > ITV_ME_SEQNO_MASK) >>
> > > + ITV_ME_SEQNO_SHIFT;
> > > + if (cur_seqno == last_seqno &&
> > > + time_after(jiffies, seqno_timestamp + HZ)) {
> > > + dev_warn(&ips->dev->dev, "ME failed to
> > > update for more than 1s, likely hung\n");
> > > + } else {
> > > + seqno_timestamp = get_jiffies_64();
> > > + last_seqno = cur_seqno;
> > > + }
> > > +
> > > + last_msecs = jiffies_to_msecs(jiffies);
>
> Once it triggers, this will print the "likely hung" message every
> second until the end of time, won't it?

[Adam Drew (adamrdrew)]

15 days and no activity on this bug. There's been a couple of kernel updates but the issue persists:

adam@adam-laptop:~$ uname -a
Linux adam-laptop 2.6.35-22-generic #35-Ubuntu SMP Sat Oct 16 20:36:48 UTC 2010 i686 GNU/Linux
adam@adam-laptop:~$ tail /var/log/messages
Oct 23 21:04:52 adam-laptop kernel: [ 1032.194988] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 23 21:04:52 adam-laptop kernel: [ 1032.478467] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 23 21:04:52 adam-laptop kernel: [ 1032.694066] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 23 21:04:53 adam-laptop kernel: [ 1033.093306] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 23 21:04:53 adam-laptop kernel: [ 1033.341838] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 23 21:04:53 adam-laptop kernel: [ 1033.620388] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 23 21:04:53 adam-laptop kernel: [ 1033.872902] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 23 21:04:54 adam-laptop kernel: [ 1034.088504] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 23 21:04:54 adam-laptop kernel: [ 1034.303136] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
Oct 23 21:04:54 adam-laptop kernel: [ 1034.563636] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung

If I run iotop rsyslogd is constantly at the top and I can hear my HDD constantly running. I have to continue to run on a much older 2.6.32 kernel I found in a PPA. Is this issue being addressed?

[Adam Drew (adamrdrew)]

The work around is to unload the intel_ips driver:

$ modprobe -r intel_ips

I put the in /etc/local. Hopefully this gets fixed and I can take advantage of the driver.

[Arie Skliarouk (skliarie)]

Same problem on the Dell Studio 1569: http://www.linlap.com/wiki/dell+studio+1569
System: ubuntu 10.10 amd64

Blacklisted the module using the following command:
echo blacklist intel_ips >> /etc/modprobe.d/blacklist

[Arie Skliarouk (skliarie)]

Sorry, the correct command was
echo blacklist intel_ips >> /etc/modprobe.d/blacklist.conf

[Murilo Opsfelder Araújo (mopsfelder)]

I believe this was fixed in kernel 2.6.38-10-generic #46 on Natty.
I'm not seeing those messages in /var/log/messages anymore.

Is this still present for someone?

[maoz (maozza)]

Yes, same issue here,
root@maoz-laptop:~# uname -a
Linux maoz-laptop 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:02:55 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

System Information
 Manufacturer: Hewlett-Packard
 Product Name: HP ProBook 4520s

[AlexWinner (scukonick)]

Same problem at Ubuntu 11.10 on Acer Aspire 5740:
$ uname -a
Linux acer 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream kernel? It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the release candidate kernel versus the daily build. Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

[Stephen A. Goss (postfuturist)]

I have the issue with Ubuntu 11.10 (amd64) on Lenovo Ideapad Z360.

I tested the upstream kernel, and it seems to be fixed, although my fan is running loudly for some reason:

$ uname -a
Linux z360-top 3.1.0-0301rc9-generic #201110050905 SMP Wed Oct 5 09:15:03 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

[Stephen A. Goss (postfuturist)]

The message eventually started appearing in the logs with the upstream kernel.

[AlexWinner (scukonick)]

$ uname -a
Linux acer 3.1.0-999-generic #201110171146 SMP Mon Oct 17 11:50:29 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

The message appears.

[Viktor Ilijasic (viktor.ilijasic)]

$ cat /etc/lsb-
lsb-base/ lsb-base-logging.sh lsb-release
viktor@tetra3:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04 LTS"
$ uname -a
Linux tetra3 3.2.0-24-generic #37-Ubuntu SMP Wed Apr 25 08:43:22 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
$ date
Sub Svi 5 18:56:46 CEST 2012

Affects me as well.

[Graeme Glass (graemeglass)]

Dell vostro 1540

uname -a
Linux carter 3.2.0-26-generic-pae #41-Ubuntu SMP Thu Jun 14 16:45:14 UTC 2012 i686 i686 i386 GNU/Linux

Last 8 entries from dmesg:

[29190.291567] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
[29190.518992] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
[29190.818344] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
[29191.017728] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
[29191.261094] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
[29191.580341] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
[29191.879564] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung
[29192.078998] intel ips 0000:00:1f.6: ME failed to update for more than 1s, likely hung

[znorris (znorris)]

Affects 3.5.1.

[Savio (abhijeet)]

affects me also on ubuntu 12.04 LTS
saviola@Savio-PC:~$ uname -a
Linux Savio-PC 3.2.0-31-generic #50-Ubuntu SMP Fri Sep 7 16:16:45 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

[Joseph Salisbury (jsalisbury)]

This bug was filed against a series that is no longer supported and so is being marked as Won't Fix. If this issue still exists in a supported series, please file a new bug.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.