ubuntu_ftrace_smoke_test failed on B-ARM64 with node wright

Bug #1851222 reported by Po-Hsu Lin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-kernel-tests
Fix Released
Medium
Colin Ian King

Bug Description

Test failed with timeout.

Reproduce rate: 2 out of 2 attempts (with running the smoke-test test plan)

With bug 1829963 fixed, this issue seems still bothering the Bionic ARM64 ThunderX node wright (passed on ARM64 moonshot node)

 Running '/home/ubuntu/autotest/client/tests/ubuntu_ftrace_smoke_test/ubuntu_ftrace_smoke_test.sh'
 PASSED (CONFIG_FUNCTION_TRACER=y in /boot/config-4.15.0-68-generic)
 PASSED (CONFIG_FUNCTION_GRAPH_TRACER=y in /boot/config-4.15.0-68-generic)
 PASSED (CONFIG_STACK_TRACER=y in /boot/config-4.15.0-68-generic)
 PASSED (CONFIG_DYNAMIC_FTRACE=y in /boot/config-4.15.0-68-generic)
 PASSED all expected /sys/kernel/debug/tracing files exist
 PASSED (function_graph in /sys/kernel/debug/tracing/available_tracers)
 PASSED (function in /sys/kernel/debug/tracing/available_tracers)
 PASSED (nop in /sys/kernel/debug/tracing/available_tracers)
 PASSED (function can be enabled)
 PASSED (function_graph can be enabled)
 TIMER END Mon Nov 4 09:30:43 UTC 2019
 TIMEOUT
 FAILED: aborting, timeout, took way too long to complete

Po-Hsu Lin (cypressyew)
tags: added: 4.15 arm64 bionic sru-20191021 ubuntu-ft
tags: added: ubuntu-ftrace-smoke-test
removed: ubuntu-ft
description: updated
Revision history for this message
Colin Ian King (colin-king) wrote :

Hi Sam, if I can get access to the machine I'll try and debug and fix this.

Changed in ubuntu-kernel-tests:
importance: Undecided → Medium
assignee: nobody → Colin Ian King (colin-king)
status: New → In Progress
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Will get the node ready once it's free.

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

BTW the third attempt on that node (wright-kernel) has passed.

And this issue could be found on i386 node pepe with 5.3 kernel on Bionic as well.

Revision history for this message
Colin Ian King (colin-king) wrote :

This is failing on a marginal timeout. I've bumped the timeouts significantly to try and overcome this kind of issue in the future.

Fix committed:
https://kernel.ubuntu.com/git/ubuntu/autotest-client-tests.git/commit/?id=94c77db7c0ec0b3e357f2a313d7dd7dea348a5c0

Revision history for this message
Colin Ian King (colin-king) wrote :

@Sam, do you mind re-testing to see if this is more reliable now?

Changed in ubuntu-kernel-tests:
status: In Progress → Fix Committed
Revision history for this message
Colin Ian King (colin-king) wrote :

I've run this remotely on the node multiple times today can I can't get it to fail :-(

17:55:19 INFO | PASSED (CONFIG_FUNCTION_TRACER=y in /boot/config-4.15.0-66-generic)
17:55:19 INFO | PASSED (CONFIG_FUNCTION_GRAPH_TRACER=y in /boot/config-4.15.0-66-generic)
17:55:19 INFO | PASSED (CONFIG_STACK_TRACER=y in /boot/config-4.15.0-66-generic)
17:55:19 INFO | PASSED (CONFIG_DYNAMIC_FTRACE=y in /boot/config-4.15.0-66-generic)
17:55:19 INFO | PASSED all expected /sys/kernel/debug/tracing files exist
17:55:19 INFO | PASSED (function_graph in /sys/kernel/debug/tracing/available_tracers)
17:55:19 INFO | PASSED (function in /sys/kernel/debug/tracing/available_tracers)
17:55:19 INFO | PASSED (nop in /sys/kernel/debug/tracing/available_tracers)
17:55:19 INFO | PASSED (function can be enabled)
17:55:19 INFO | PASSED (function_graph can be enabled)
17:55:19 INFO | PASSED (irq traces found must be > 16, got 10846)
17:55:19 INFO | PASSED (tracer hwlat can be enabled (got 11 lines of tracing output))
17:55:19 INFO | PASSED (tracer blk can be enabled (got 2 lines of tracing output))
17:55:19 INFO | PASSED (tracer function_graph can be enabled (got 1292 lines of tracing output))
17:55:19 INFO | PASSED (tracer function can be enabled (got 797 lines of tracing output))

Revision history for this message
Colin Ian King (colin-king) wrote :

I've push another change to autotest for ftrace, this uses 1 second timeout polls on the tracing output and more debug output so if anything locks up we *should* get a better idea of where and why it is hanging.

https://kernel.ubuntu.com/git/ubuntu/autotest-client-tests.git/commit/?id=b211b55fe30f5b16730cd61229096e1fa125e869

Do you mind re-testing?

Revision history for this message
Colin Ian King (colin-king) wrote :

Tests now pass, marking as fixed released.

<PHLin> cking, both pepe and wright have passed the test, thank you!

Changed in ubuntu-kernel-tests:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.