Microcode SW error detected.

[Lukasz Marcinowski]

In dmesg logs I see

[81251.363118] iwlwifi 0000:04:00.0: regular scan timed out
[81251.363235] iwlwifi 0000:04:00.0: Microcode SW error detected. Restarting 0x2000000.
[81251.363241] iwlwifi 0000:04:00.0: CSR values:
[81251.363244] iwlwifi 0000:04:00.0: (2nd byte of CSR_INT_COALESCING is CSR_INT_PERIODIC_REG)
[81251.363250] iwlwifi 0000:04:00.0: CSR_HW_IF_CONFIG_REG: 0X00c89008
[81251.363256] iwlwifi 0000:04:00.0: CSR_INT_COALESCING: 0X00000040
[81251.363261] iwlwifi 0000:04:00.0: CSR_INT: 0X00000000
[81251.363266] iwlwifi 0000:04:00.0: CSR_INT_MASK: 0X00000000
[81251.363271] iwlwifi 0000:04:00.0: CSR_FH_INT_STATUS: 0X00000000
[81251.363276] iwlwifi 0000:04:00.0: CSR_GPIO_IN: 0X00000019
[81251.363281] iwlwifi 0000:04:00.0: CSR_RESET: 0X00000000
[81251.363286] iwlwifi 0000:04:00.0: CSR_GP_CNTRL: 0X08040005
[81251.363292] iwlwifi 0000:04:00.0: CSR_HW_REV: 0X00000201
[81251.363296] iwlwifi 0000:04:00.0: CSR_EEPROM_REG: 0Xd55555d5
[81251.363302] iwlwifi 0000:04:00.0: CSR_EEPROM_GP: 0Xd55555d5
[81251.363306] iwlwifi 0000:04:00.0: CSR_OTP_GP_REG: 0Xd55555d5
[81251.363311] iwlwifi 0000:04:00.0: CSR_GIO_REG: 0X001f0044
[81251.363316] iwlwifi 0000:04:00.0: CSR_GP_UCODE_REG: 0X00000000
[81251.363321] iwlwifi 0000:04:00.0: CSR_GP_DRIVER_REG: 0X00000000
[81251.363326] iwlwifi 0000:04:00.0: CSR_UCODE_DRV_GP1: 0X00000000
[81251.363331] iwlwifi 0000:04:00.0: CSR_UCODE_DRV_GP2: 0X00000000
[81251.363336] iwlwifi 0000:04:00.0: CSR_LED_REG: 0X00000060
[81251.363341] iwlwifi 0000:04:00.0: CSR_DRAM_INT_TBL_REG: 0X8884993d
[81251.363346] iwlwifi 0000:04:00.0: CSR_GIO_CHICKEN_BITS: 0X07800200
[81251.363351] iwlwifi 0000:04:00.0: CSR_ANA_PLL_CFG: 0Xd55555d5
[81251.363356] iwlwifi 0000:04:00.0: CSR_MONITOR_STATUS_REG: 0Xc03803c0
[81251.363361] iwlwifi 0000:04:00.0: CSR_HW_REV_WA_REG: 0X0001001a
[81251.363366] iwlwifi 0000:04:00.0: CSR_DBG_HPET_MEM_REG: 0Xffff0010
[81251.363369] iwlwifi 0000:04:00.0: FH register values:
[81251.363384] iwlwifi 0000:04:00.0: FH_RSCSR_CHNL0_STTS_WPTR_REG: 0X848c7800
[81251.363398] iwlwifi 0000:04:00.0: FH_RSCSR_CHNL0_RBDCB_BASE_REG: 0X0848c080
[81251.363413] iwlwifi 0000:04:00.0: FH_RSCSR_CHNL0_WPTR: 0X000000f0
[81251.363427] iwlwifi 0000:04:00.0: FH_MEM_RCSR_CHNL0_CONFIG_REG: 0X00801054
[81251.363442] iwlwifi 0000:04:00.0: FH_MEM_RSSR_SHARED_CTRL_REG: 0X000000fc
[81251.363456] iwlwifi 0000:04:00.0: FH_MEM_RSSR_RX_STATUS_REG: 0X03030000
[81251.363471] iwlwifi 0000:04:00.0: FH_MEM_RSSR_RX_ENABLE_ERR_IRQ2DRV: 0X00000000
[81251.363486] iwlwifi 0000:04:00.0: FH_TSSR_TX_STATUS_REG: 0X07ff0000
[81251.363500] iwlwifi 0000:04:00.0: FH_TSSR_TX_ERROR_REG: 0X00000000
[81251.363612] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[81251.363615] iwlwifi 0000:04:00.0: Status: 0x00000000, count: 6
[81251.363618] iwlwifi 0000:04:00.0: Loaded firmware version: 21.302800.0
[81251.363622] iwlwifi 0000:04:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
[81251.363624] iwlwifi 0000:04:00.0: 0x000006F4 | trm_hw_status0
[81251.363627] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
[81251.363630] iwlwifi 0000:04:00.0: 0x00000BD8 | branchlink2
[81251.363632] iwlwifi 0000:04:00.0: 0x00026AC4 | interruptlink1
[81251.363635] iwlwifi 0000:04:00.0: 0x00026AC4 | interruptlink2
[81251.363637] iwlwifi 0000:04:00.0: 0x00000000 | data1
[81251.363640] iwlwifi 0000:04:00.0: 0x00000080 | data2
[81251.363642] iwlwifi 0000:04:00.0: 0x07830000 | data3
[81251.363645] iwlwifi 0000:04:00.0: 0x7800010C | beacon time
[81251.363647] iwlwifi 0000:04:00.0: 0xF129DEEA | tsf low
[81251.363650] iwlwifi 0000:04:00.0: 0x00000077 | tsf hi
[81251.363653] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
[81251.363655] iwlwifi 0000:04:00.0: 0xEA60BCC9 | time gp2
[81251.363658] iwlwifi 0000:04:00.0: 0x00000000 | uCode revision type
[81251.363660] iwlwifi 0000:04:00.0: 0x00000015 | uCode version major
[81251.363663] iwlwifi 0000:04:00.0: 0x00049ED0 | uCode version minor
[81251.363665] iwlwifi 0000:04:00.0: 0x00000201 | hw version
[81251.363668] iwlwifi 0000:04:00.0: 0x00C89008 | board version
[81251.363671] iwlwifi 0000:04:00.0: 0x09270400 | hcmd
[81251.363673] iwlwifi 0000:04:00.0: 0x80022000 | isr0
[81251.363676] iwlwifi 0000:04:00.0: 0x00800000 | isr1
[81251.363678] iwlwifi 0000:04:00.0: 0x08001802 | isr2
[81251.363681] iwlwifi 0000:04:00.0: 0x00417CC4 | isr3
[81251.363683] iwlwifi 0000:04:00.0: 0x00000000 | isr4
[81251.363686] iwlwifi 0000:04:00.0: 0x00000110 | last cmd Id
[81251.363688] iwlwifi 0000:04:00.0: 0x00000000 | wait_event
[81251.363691] iwlwifi 0000:04:00.0: 0x0000B397 | l2p_control
[81251.363693] iwlwifi 0000:04:00.0: 0x00000020 | l2p_duration
[81251.363696] iwlwifi 0000:04:00.0: 0x00000000 | l2p_mhvalid
[81251.363698] iwlwifi 0000:04:00.0: 0x00000030 | l2p_addr_match
[81251.363701] iwlwifi 0000:04:00.0: 0x0000000D | lmpm_pmg_sel
[81251.363703] iwlwifi 0000:04:00.0: 0x09031905 | timestamp
[81251.363706] iwlwifi 0000:04:00.0: 0x0034F000 | flow_handler
[81251.363758] iwlwifi 0000:04:00.0: 0x00000000 | ADVANCED_SYSASSERT
[81251.363761] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
[81251.363763] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink2
[81251.363766] iwlwifi 0000:04:00.0: 0x00000000 | umac interruptlink1
[81251.363768] iwlwifi 0000:04:00.0: 0x00000000 | umac interruptlink2
[81251.363771] iwlwifi 0000:04:00.0: 0x00000000 | umac data1
[81251.363773] iwlwifi 0000:04:00.0: 0x00000000 | umac data2
[81251.363775] iwlwifi 0000:04:00.0: 0x00000000 | umac data3
[81251.363778] iwlwifi 0000:04:00.0: 0x00000000 | umac major
[81251.363780] iwlwifi 0000:04:00.0: 0x00000000 | umac minor
[81251.363783] iwlwifi 0000:04:00.0: 0xC0886298 | frame pointer
[81251.363785] iwlwifi 0000:04:00.0: 0xC0886298 | stack pointer
[81251.363788] iwlwifi 0000:04:00.0: 0x09270400 | last host cmd
[81251.363791] iwlwifi 0000:04:00.0: 0x00000000 | isr status reg
[81251.363801] ieee80211 phy0: Hardware restart was requested
[81251.841241] iwlwifi 0000:04:00.0: L1 Disabled - LTR Enabled
[81251.841489] iwlwifi 0000:04:00.0: L1 Disabled - LTR Enabled
[81251.979711] iwlwifi 0000:04:00.0: L1 Disabled - LTR Enabled
[81251.979969] iwlwifi 0000:04:00.0: L1 Disabled - LTR Enabled

Ubuntu 16.04.2 LTS
the linux-firmware version is: 1.157.10

network card (lspci -v)
04:00.0 Network controller: Intel Corporation Wireless 8260 (rev 3a)
 Subsystem: Intel Corporation Wireless 8260
 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
 Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
 Latency: 0, Cache Line Size: 64 bytes
 Interrupt: pin A routed to IRQ 322
 Region 0: Memory at df200000 (64-bit, non-prefetchable) [size=8K]
 Capabilities: <access denied>
 Kernel driver in use: iwlwifi
 Kernel modules: iwlwifi

[Joseph Salisbury (jsalisbury)]

Did this issue start happening after an update/upgrade? Was there a prior kernel version where you were not having this particular problem?

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.12 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.12-rc2

[Lukasz Marcinowski (nowymarluk)]

For kernel 4.12.0-041200rc2-generic, there is no microcode software error, but I'm not 100% sure that there is no bug in upstream. Instead of this there is a:

[19539.037240] BUG: scheduling while atomic: irq/323-iwlwifi/493/0x7ffffe00
[19539.037241] Modules linked in: ccm rfcomm option usb_wwan usbserial bnep snd_hda_codec_hdmi nls_iso8859_1 arc4 i2c_designware_platform 8250_dw i2c_designware_core intel_rapl x86_pkg_temp_thermal intel_powerclamp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc snd_hda_codec_realtek snd_hda_codec_generic iwlmvm snd_hda_intel aesni_intel snd_hda_codec snd_hda_core aes_x86_64 mac80211 snd_hwdep crypto_simd glue_helper cryptd intel_cstate intel_rapl_perf snd_pcm snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq serio_raw input_leds joydev snd_seq_device iwlwifi snd_timer btusb btrtl snd soundcore cfg80211 hci_uart btbcm serdev btqca btintel bluetooth idma64 shpchp ecdh_generic virt_dma mei_me intel_pch_thermal mei intel_lpss_pci mac_hid intel_lpss_acpi intel_lpss acpi_pad
[19539.037258] acpi_als kfifo_buf industrialio coretemp parport_pc ppdev lp parport autofs4 uas usb_storage hid_logitech_hidpp hid_logitech_dj hid_generic usbhid nouveau mxm_wmi i2c_algo_bit ttm drm_kms_helper r8169 syscopyarea mii sysfillrect sysimgblt fb_sys_fops ahci i2c_hid drm psmouse libahci hid wmi video pinctrl_sunrisepoint pinctrl_intel
[19539.037268] CPU: 1 PID: 493 Comm: irq/323-iwlwifi Tainted: G W 4.12.0-041200rc2-generic #201705212331
[19539.037269] Hardware name: MSI MS-7970/Z170A TOMAHAWK AC (MS-7970), BIOS 1.90 10/17/2016
[19539.037269] Call Trace:
[19539.037270] dump_stack+0x63/0x8d
[19539.037271] __schedule_bug+0x54/0x70
[19539.037273] __schedule+0x644/0x850
[19539.037274] ? irq_finalize_oneshot.part.35+0xf0/0xf0
[19539.037275] schedule+0x36/0x80
[19539.037276] irq_thread+0xac/0x1a0
[19539.037277] ? irq_forced_thread_fn+0x70/0x70
[19539.037278] kthread+0x109/0x140
[19539.037279] ? free_irq+0x70/0x70
[19539.037280] ? kthread_create_on_node+0x70/0x70
[19539.037281] ret_from_fork+0x2c/0x40

Also at the begining there is a problem loading firmware:

[ 8.607983] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8000C-30.ucode failed with error -2
[ 8.608666] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8000C-29.ucode failed with error -2
[ 8.608683] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8000C-28.ucode failed with error -2
[ 8.608954] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8000C-27.ucode failed with error -2
[ 8.608977] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8000C-26.ucode failed with error -2
[ 8.608983] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8000C-25.ucode failed with error -2
[ 8.609561] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8000C-24.ucode failed with error -2
[ 8.609978] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8000C-23.ucode failed with error -2
[ 8.610404] iwlwifi 0000:04:00.0: Direct firmware load for iwlwifi-8000C-22.ucode failed with error -2
[ 8.623739] iwlwifi 0000:04:00.0: loaded firmware version 21.302800.0 op_mode iwlmvm

a...

[Lukasz Marcinowski (nowymarluk)]

on kernel 4.12.0-041200rc3-generic #201705282131 the problem exist:

[ 8987.607601] iwlwifi 0000:04:00.0: regular scan timed out
[ 8987.607724] iwlwifi 0000:04:00.0: Microcode SW error detected. Restarting 0x2000000.
[ 8987.607841] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[ 8987.607846] iwlwifi 0000:04:00.0: Status: 0x00000000, count: 6
[ 8987.607849] iwlwifi 0000:04:00.0: Loaded firmware version: 21.302800.0
[ 8987.607853] iwlwifi 0000:04:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
[ 8987.607857] iwlwifi 0000:04:00.0: 0x000002B0 | trm_hw_status0
[ 8987.607860] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
[ 8987.607862] iwlwifi 0000:04:00.0: 0x00000BD8 | branchlink2
[ 8987.607866] iwlwifi 0000:04:00.0: 0x00026AC4 | interruptlink1
[ 8987.607869] iwlwifi 0000:04:00.0: 0x00026AC4 | interruptlink2
[ 8987.607871] iwlwifi 0000:04:00.0: 0x00000000 | data1
[ 8987.607874] iwlwifi 0000:04:00.0: 0x00000080 | data2
[ 8987.607877] iwlwifi 0000:04:00.0: 0x07830000 | data3
[ 8987.607880] iwlwifi 0000:04:00.0: 0x674164A9 | beacon time
[ 8987.607883] iwlwifi 0000:04:00.0: 0xE3A27B5E | tsf low
[ 8987.607886] iwlwifi 0000:04:00.0: 0x0000003D | tsf hi
[ 8987.607889] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
[ 8987.607892] iwlwifi 0000:04:00.0: 0x1722E819 | time gp2
[ 8987.607895] iwlwifi 0000:04:00.0: 0x00000000 | uCode revision type
[ 8987.607898] iwlwifi 0000:04:00.0: 0x00000015 | uCode version major
[ 8987.607901] iwlwifi 0000:04:00.0: 0x00049ED0 | uCode version minor
[ 8987.607904] iwlwifi 0000:04:00.0: 0x00000201 | hw version
[ 8987.607907] iwlwifi 0000:04:00.0: 0x00C89008 | board version
[ 8987.607910] iwlwifi 0000:04:00.0: 0x09D00400 | hcmd
[ 8987.607913] iwlwifi 0000:04:00.0: 0x00022000 | isr0
[ 8987.607916] iwlwifi 0000:04:00.0: 0x00800000 | isr1
[ 8987.607919] iwlwifi 0000:04:00.0: 0x0800180A | isr2
[ 8987.607922] iwlwifi 0000:04:00.0: 0x00417CC0 | isr3
[ 8987.607925] iwlwifi 0000:04:00.0: 0x00000000 | isr4
[ 8987.607928] iwlwifi 0000:04:00.0: 0x00000110 | last cmd Id
[ 8987.607930] iwlwifi 0000:04:00.0: 0x00000000 | wait_event
[ 8987.607933] iwlwifi 0000:04:00.0: 0x00000148 | l2p_control
[ 8987.607936] iwlwifi 0000:04:00.0: 0x00010020 | l2p_duration
[ 8987.607939] iwlwifi 0000:04:00.0: 0x0000003F | l2p_mhvalid
[ 8987.607942] iwlwifi 0000:04:00.0: 0x0000001A | l2p_addr_match
[ 8987.607945] iwlwifi 0000:04:00.0: 0x0000000F | lmpm_pmg_sel
[ 8987.607948] iwlwifi 0000:04:00.0: 0x09031905 | timestamp
[ 8987.607951] iwlwifi 0000:04:00.0: 0x0034F808 | flow_handler
[ 8987.608004] iwlwifi 0000:04:00.0: 0x00000000 | ADVANCED_SYSASSERT
[ 8987.608007] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
[ 8987.608010] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink2
[ 8987.608013] iwlwifi 0000:04:00.0: 0x00000000 | umac interruptlink1
[ 8987.608016] iwlwifi 0000:04:00.0: 0x00000000 | umac interruptlink2
[ 8987.608018] iwlwifi 0000:04:00.0: 0x00000000 | umac data1
[ 8987.608021] iwlwifi 0000:04:00.0: 0x00000000 | umac data2
[ 8987.608024] iwlwifi 0000:04:00.0: 0x00000000 | umac data3
[ 8987.608027] iwlwifi 0000:04:00.0: 0x00000000 | umac major
[ 8987.608030] iwlwifi 0000:04:00.0: 0x00000000 | umac minor
[ 8987.608033] iwlwifi 0000:04:00.0: 0x00...

[Kai-Heng Feng (kaihengfeng)]

Take a look at LP: #1694547. Probably same issue?

[Lukasz Marcinowski (nowymarluk)]

Same network controller, so probably the same issue.