log_check / kernel_tainted test from ubuntu_boot failed on Intel Cloud node grumman (failed with flag 2048)

[Po-Hsu Lin]

Issue found on node grumman, this is not a regression.

The kernel_tainted check reports a non-zero flag 2048, this means "workaround for bug in platform firmware applied"

log_check reports WARNINGs found in the dmesg:

10. 01/06 09:46:51 INFO |ubuntu_boo:0030| kernel: [ 7.852005] WARNING: CPU: 0 PID: 179 at /build/linux-0dlxvs/linux-4.4.0/drivers/dma/ioat/dca.c:342 ioat_dca_init+0x17c/0x1a0 [ioatdma]()
11. 01/06 09:46:51 INFO |ubuntu_boo:0030| kernel: [ 5.387292] WARNING: CPU: 0 PID: 181 at /build/linux-7Nk88i/linux-4.4.0/drivers/dma/ioat/dca.c:342 ioat_dca_init+0x17c/0x1a0 [ioatdma]()
12. 01/06 09:46:51 INFO |ubuntu_boo:0030| kernel: [ 6.039705] WARNING: CPU: 0 PID: 4 at /build/linux-7Nk88i/linux-4.4.0/drivers/dma/ioat/dca.c:342 ioat_dca_init+0x17c/0x1a0 [ioatdma]()

[Po-Hsu Lin (cypressyew)]

In Xenial source code, the warning was triggered here in ioat_dca_init() of drivers/dma/ioat/dca.c:

        if (dca3_tag_map_invalid(ioatdca->tag_map)) {
                WARN_TAINT_ONCE(1, TAINT_FIRMWARE_WORKAROUND,
                                "%s %s: APICID_TAG_MAP set incorrectly by BIOS, disabling DCA\n",
                                dev_driver_string(&pdev->dev),
                                dev_name(&pdev->dev));
                free_dca_provider(dca);
                return NULL;
        }

This is the WARNING in log_check, I think it generates the tainted flag as well.

The code was later replaced by: https://github.com/torvalds/linux/commit/036e9ef8becde736e693be4f4bef56d5b56fc298#diff-a6ce63e9c9f1a0aac9223c055a717e8fd34d3a4793a3166e4a9bb3b6d1c54c9f

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1912316

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[William Breathitt Gray (vilhelm-gray)]

Groovy generic-64k amd64 seems to be affected as well starting from 5.8.0-43.49.

[Po-Hsu Lin (cypressyew)]

Mark as affecting B/F/G based on comment in bug 1916049

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Kelsey Steele (kelsey-steele)]

Finding failure on Focal/hwe-5.8 5.8.0-45.51~20.04.1 generic-64k, hose: starmie-kernel

Though it's instead "ERROR: kernel tainted flag != 0: 512"

1. 03/11 04:40:30 INFO |ubuntu_boo:0042| Checking kernel tainted flags in /proc/sys/kernel/tainted
2. 03/11 04:40:30 INFO |ubuntu_boo:0046| ERROR: kernel tainted flag != 0: 512
3. 03/11 04:40:30 INFO |ubuntu_boo:0046|
4. 03/11 04:40:30 ERROR| test:0414| Exception escaping from test:
5. Traceback (most recent call last):
6. File "/home/ubuntu/autotest/client/shared/test.py", line 411, in _exec
7. _call_test_function(self.execute, *p_args, **p_dargs)
8. File "/home/ubuntu/autotest/client/shared/test.py", line 823, in _call_test_function
9. return func(*args, **dargs)
10. File "/home/ubuntu/autotest/client/shared/test.py", line 291, in execute
11. postprocess_profiled_run, args, dargs)
12. File "/home/ubuntu/autotest/client/shared/test.py", line 212, in _call_run_once
13. self.run_once(*args, **dargs)
14. File "/home/ubuntu/autotest/client/tests/ubuntu_boot/ubuntu_boot.py", line 59, in run_once
15. raise error.TestFail()
16. TestFail

[Po-Hsu Lin (cypressyew)]

Hi Kelsey,
I think the failure for starmie-kernel deserves a new bug (as it's not failing with code 2048).
Thus I opened bug 1918620
Let's track this there.

[Kelsey Steele (kelsey-steele)]

Good idea, thank you!

[Brian Murray (brian-murray)]

The Groovy Gorilla has reached end of life, so this bug will not be fixed for that release

[Kleber Sacilotto de Souza (kleber-souza)]

Found with trusty/linux-lts-xenial: 4.4.0-215.247~14.04.1 on node kili.

[Zachary Tahenakos (ztahenakos)]

Found with xenial/fips:4.4.0-1072.78 on node fili.

[Kleber Sacilotto de Souza (kleber-souza)]

The log_check WARNINGs with the 4.4 kernels is being addressed by bug 1965160 with commit 036e9ef8becd ("dmaengine: Replace WARN_TAINT_ONCE() with pr_warn_once()").

This commit has been applied upstream for v4.13-rc1, so from bionic+ we should have only the kernel tainted failure. However, that commit keeps the TAINT_FIRMWARE_WORKAROUND taint flag and this is still true for mainline, so probably this bug can't be fixed as this is a hardware issue.