ubuntu_ftrace_smoke_test failed on B-ARM64 with node wright

Bug #1851222 reported by Po-Hsu Lin on 2019-11-04
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-kernel-tests
Medium
Colin Ian King

Bug Description

Test failed with timeout.

Reproduce rate: 2 out of 2 attempts (with running the smoke-test test plan)

With bug 1829963 fixed, this issue seems still bothering the Bionic ARM64 ThunderX node wright (passed on ARM64 moonshot node)

 Running '/home/ubuntu/autotest/client/tests/ubuntu_ftrace_smoke_test/ubuntu_ftrace_smoke_test.sh'
 PASSED (CONFIG_FUNCTION_TRACER=y in /boot/config-4.15.0-68-generic)
 PASSED (CONFIG_FUNCTION_GRAPH_TRACER=y in /boot/config-4.15.0-68-generic)
 PASSED (CONFIG_STACK_TRACER=y in /boot/config-4.15.0-68-generic)
 PASSED (CONFIG_DYNAMIC_FTRACE=y in /boot/config-4.15.0-68-generic)
 PASSED all expected /sys/kernel/debug/tracing files exist
 PASSED (function_graph in /sys/kernel/debug/tracing/available_tracers)
 PASSED (function in /sys/kernel/debug/tracing/available_tracers)
 PASSED (nop in /sys/kernel/debug/tracing/available_tracers)
 PASSED (function can be enabled)
 PASSED (function_graph can be enabled)
 TIMER END Mon Nov 4 09:30:43 UTC 2019
 TIMEOUT
 FAILED: aborting, timeout, took way too long to complete

Po-Hsu Lin (cypressyew) on 2019-11-04
tags: added: 4.15 arm64 bionic sru-20191021 ubuntu-ft
tags: added: ubuntu-ftrace-smoke-test
removed: ubuntu-ft
description: updated
Colin Ian King (colin-king) wrote :

Hi Sam, if I can get access to the machine I'll try and debug and fix this.

Changed in ubuntu-kernel-tests:
importance: Undecided → Medium
assignee: nobody → Colin Ian King (colin-king)
status: New → In Progress
Po-Hsu Lin (cypressyew) wrote :

Will get the node ready once it's free.

Po-Hsu Lin (cypressyew) wrote :

BTW the third attempt on that node (wright-kernel) has passed.

And this issue could be found on i386 node pepe with 5.3 kernel on Bionic as well.

Colin Ian King (colin-king) wrote :

This is failing on a marginal timeout. I've bumped the timeouts significantly to try and overcome this kind of issue in the future.

Fix committed:
https://kernel.ubuntu.com/git/ubuntu/autotest-client-tests.git/commit/?id=94c77db7c0ec0b3e357f2a313d7dd7dea348a5c0

Colin Ian King (colin-king) wrote :

@Sam, do you mind re-testing to see if this is more reliable now?

Changed in ubuntu-kernel-tests:
status: In Progress → Fix Committed
Colin Ian King (colin-king) wrote :

I've run this remotely on the node multiple times today can I can't get it to fail :-(

17:55:19 INFO | PASSED (CONFIG_FUNCTION_TRACER=y in /boot/config-4.15.0-66-generic)
17:55:19 INFO | PASSED (CONFIG_FUNCTION_GRAPH_TRACER=y in /boot/config-4.15.0-66-generic)
17:55:19 INFO | PASSED (CONFIG_STACK_TRACER=y in /boot/config-4.15.0-66-generic)
17:55:19 INFO | PASSED (CONFIG_DYNAMIC_FTRACE=y in /boot/config-4.15.0-66-generic)
17:55:19 INFO | PASSED all expected /sys/kernel/debug/tracing files exist
17:55:19 INFO | PASSED (function_graph in /sys/kernel/debug/tracing/available_tracers)
17:55:19 INFO | PASSED (function in /sys/kernel/debug/tracing/available_tracers)
17:55:19 INFO | PASSED (nop in /sys/kernel/debug/tracing/available_tracers)
17:55:19 INFO | PASSED (function can be enabled)
17:55:19 INFO | PASSED (function_graph can be enabled)
17:55:19 INFO | PASSED (irq traces found must be > 16, got 10846)
17:55:19 INFO | PASSED (tracer hwlat can be enabled (got 11 lines of tracing output))
17:55:19 INFO | PASSED (tracer blk can be enabled (got 2 lines of tracing output))
17:55:19 INFO | PASSED (tracer function_graph can be enabled (got 1292 lines of tracing output))
17:55:19 INFO | PASSED (tracer function can be enabled (got 797 lines of tracing output))

Colin Ian King (colin-king) wrote :

I've push another change to autotest for ftrace, this uses 1 second timeout polls on the tracing output and more debug output so if anything locks up we *should* get a better idea of where and why it is hanging.

https://kernel.ubuntu.com/git/ubuntu/autotest-client-tests.git/commit/?id=b211b55fe30f5b16730cd61229096e1fa125e869

Do you mind re-testing?

Colin Ian King (colin-king) wrote :

Tests now pass, marking as fixed released.

<PHLin> cking, both pepe and wright have passed the test, thank you!

Changed in ubuntu-kernel-tests:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers