| 2019-06-03 15:13:26 |
Dan Streetman |
bug |
|
|
added bug |
| 2020-07-23 12:43:57 |
Dan Streetman |
summary |
'testbed_script_replay_simple' test fails intermittently |
test-umockdev tests flaky on armhf |
|
| 2020-07-23 12:55:18 |
Dan Streetman |
description |
[impact]
this test fails intermittently, due to read failures when the test expects to read data, but the umockdev 'replay' script device doesn't provide the data to read.
this appears to just be a timing issue, as the test expects the umockdev device to provide specific data replayed at specific times, and those times are very short (ms). Looking at the autopkgtest results:
http://autopkgtest.ubuntu.com/packages/umockdev
it looks like the failures are mostly on armhf, presumably because that is run in a container and provides the least guarantee of accurate short usleep() timing, but it also does fail on other archs sometimes as well.
[test case]
check the previous autopkgtest logs, e.g.
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/armhf/u/umockdev/20190601_015323_8f795@/log.gz
[regression potential]
probably low, as it seems like this is really just a test case issue of expecting very specific ms-range timing. |
[impact]
these tests fail intermittently, due to read failures when the test expects to read data, but the umockdev 'replay' script device doesn't provide the data to read.
this appears to just be a timing issue, as the test expects the umockdev device to provide specific data replayed at specific times, and those times are very short (ms). Looking at the autopkgtest results:
http://autopkgtest.ubuntu.com/packages/umockdev
it looks like the failures are mostly on armhf.
[test case]
check the previous autopkgtest logs, e.g.
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/armhf/u/umockdev/20190601_015323_8f795@/log.gz
[regression potential]
the tests on armhf have been flaky ever since trusty, and people just retry and/or ignore them; any regression would likely result in (still) flaky tests and/or false positive or negative test results on armhf.
[scope]
as the tests are flaky on armhf all the way back to trusty, this is needed for all releases.
[other info]
the armhf testbeds are unfortunately "different" from all the other archs:
1) they don't run directly in a virtual instance, they run in an armhf lxd container inside an arm64 virtual instance.
2) all other archs get are run inside a small instance, with limited memory and 1 cpu, but the armhf tests are run in what appears to be a large instance, and the armhf container inherits the 4 cpus that the arm64 instance has. |
|
| 2020-07-29 12:41:38 |
Dan Streetman |
summary |
test-umockdev tests flaky on armhf |
test-umockdev tests flaky on armhf (and sometimes other archs) |
|
| 2020-07-29 12:55:56 |
Dan Streetman |
nominated for series |
|
Ubuntu Groovy |
|
| 2020-07-29 12:55:56 |
Dan Streetman |
bug task added |
|
umockdev (Ubuntu Groovy) |
|
| 2020-07-29 12:55:56 |
Dan Streetman |
nominated for series |
|
Ubuntu Xenial |
|
| 2020-07-29 12:55:56 |
Dan Streetman |
bug task added |
|
umockdev (Ubuntu Xenial) |
|
| 2020-07-29 12:55:56 |
Dan Streetman |
nominated for series |
|
Ubuntu Bionic |
|
| 2020-07-29 12:55:56 |
Dan Streetman |
bug task added |
|
umockdev (Ubuntu Bionic) |
|
| 2020-07-29 12:55:56 |
Dan Streetman |
nominated for series |
|
Ubuntu Focal |
|
| 2020-07-29 12:55:56 |
Dan Streetman |
bug task added |
|
umockdev (Ubuntu Focal) |
|
| 2020-07-29 12:56:02 |
Dan Streetman |
umockdev (Ubuntu Groovy): assignee |
|
Dan Streetman (ddstreet) |
|
| 2020-07-29 12:56:03 |
Dan Streetman |
umockdev (Ubuntu Focal): assignee |
|
Dan Streetman (ddstreet) |
|
| 2020-07-29 12:56:05 |
Dan Streetman |
umockdev (Ubuntu Bionic): assignee |
|
Dan Streetman (ddstreet) |
|
| 2020-07-29 12:56:07 |
Dan Streetman |
umockdev (Ubuntu Xenial): assignee |
|
Dan Streetman (ddstreet) |
|
| 2020-07-29 12:56:09 |
Dan Streetman |
umockdev (Ubuntu Groovy): importance |
Undecided |
Low |
|
| 2020-07-29 12:56:10 |
Dan Streetman |
umockdev (Ubuntu Focal): importance |
Undecided |
Low |
|
| 2020-07-29 12:56:12 |
Dan Streetman |
umockdev (Ubuntu Bionic): importance |
Undecided |
Low |
|
| 2020-07-29 12:56:13 |
Dan Streetman |
umockdev (Ubuntu Xenial): importance |
Undecided |
Low |
|
| 2020-07-29 12:56:15 |
Dan Streetman |
umockdev (Ubuntu Groovy): status |
New |
In Progress |
|
| 2020-07-29 12:56:17 |
Dan Streetman |
umockdev (Ubuntu Focal): status |
New |
In Progress |
|
| 2020-07-29 12:56:18 |
Dan Streetman |
umockdev (Ubuntu Bionic): status |
New |
In Progress |
|
| 2020-07-29 12:56:20 |
Dan Streetman |
umockdev (Ubuntu Xenial): status |
New |
In Progress |
|
| 2020-07-29 14:12:50 |
Dan Streetman |
description |
[impact]
these tests fail intermittently, due to read failures when the test expects to read data, but the umockdev 'replay' script device doesn't provide the data to read.
this appears to just be a timing issue, as the test expects the umockdev device to provide specific data replayed at specific times, and those times are very short (ms). Looking at the autopkgtest results:
http://autopkgtest.ubuntu.com/packages/umockdev
it looks like the failures are mostly on armhf.
[test case]
check the previous autopkgtest logs, e.g.
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/armhf/u/umockdev/20190601_015323_8f795@/log.gz
[regression potential]
the tests on armhf have been flaky ever since trusty, and people just retry and/or ignore them; any regression would likely result in (still) flaky tests and/or false positive or negative test results on armhf.
[scope]
as the tests are flaky on armhf all the way back to trusty, this is needed for all releases.
[other info]
the armhf testbeds are unfortunately "different" from all the other archs:
1) they don't run directly in a virtual instance, they run in an armhf lxd container inside an arm64 virtual instance.
2) all other archs get are run inside a small instance, with limited memory and 1 cpu, but the armhf tests are run in what appears to be a large instance, and the armhf container inherits the 4 cpus that the arm64 instance has. |
[impact]
these tests fail intermittently, due to various timing issues, as well as an actual code bug in how data fuzz is calculated.
it looks like the failures are mostly on armhf, but do happen less often on other archs.
[test case]
check the previous autopkgtest logs, e.g.
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/armhf/u/umockdev/20190601_015323_8f795@/log.gz
[regression potential]
any regression would likely result in incorrectly failing/passing autopkgtests, or in a incorrect pass or incorrect fail of the ScriptRunner
to verify the data's level of fuzz.
[scope]
as the tests are flaky on armhf all the way back to trusty, this is needed for all releases.
[other info]
Fixed upstream in PR https://github.com/martinpitt/umockdev/pull/103
Most of the changes are test case fixes, but there is one fix to source code to fix the ScriptRunning function that validates the level of fuzz in data, so this is not *only* a testcase fix. |
|
| 2020-07-29 16:13:13 |
Martin Pitt |
umockdev (Ubuntu Groovy): status |
In Progress |
Fix Committed |
|
| 2020-08-03 11:17:08 |
Launchpad Janitor |
umockdev (Ubuntu Groovy): status |
Fix Committed |
Fix Released |
|
| 2020-08-11 18:30:55 |
Brian Murray |
umockdev (Ubuntu Focal): status |
In Progress |
Fix Committed |
|
| 2020-08-11 18:30:58 |
Brian Murray |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2020-08-11 18:31:01 |
Brian Murray |
bug |
|
|
added subscriber SRU Verification |
| 2020-08-11 18:31:04 |
Brian Murray |
tags |
|
verification-needed verification-needed-focal |
|
| 2020-08-11 18:33:40 |
Brian Murray |
umockdev (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2020-08-11 18:33:46 |
Brian Murray |
tags |
verification-needed verification-needed-focal |
verification-needed verification-needed-bionic verification-needed-focal |
|
| 2020-08-21 21:38:40 |
Dan Streetman |
tags |
verification-needed verification-needed-bionic verification-needed-focal |
verification-done verification-done-bionic verification-done-focal |
|
| 2020-08-25 16:47:37 |
Launchpad Janitor |
umockdev (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2020-08-25 16:47:42 |
Brian Murray |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2020-08-25 16:48:03 |
Launchpad Janitor |
umockdev (Ubuntu Focal): status |
Fix Committed |
Fix Released |
|
| 2020-09-23 14:02:47 |
Dan Streetman |
umockdev (Ubuntu Xenial): assignee |
Dan Streetman (ddstreet) |
|
|
| 2020-09-23 14:02:53 |
Dan Streetman |
umockdev (Ubuntu Xenial): status |
In Progress |
Won't Fix |
|
