Activity log for bug #1853375

Date Who What changed Old value New value Message
2019-11-20 22:25:00 Connor Kuehl bug added bug
2020-01-10 07:39:59 Po-Hsu Lin tags s390x ubuntu-kernel-selftests xenial
2020-01-10 07:47:29 Po-Hsu Lin tags s390x ubuntu-kernel-selftests xenial bionic s390x ubuntu-kernel-selftests xenial
2020-01-10 08:13:08 Po-Hsu Lin bug task added linux (Ubuntu)
2020-01-10 08:13:17 Po-Hsu Lin nominated for series Ubuntu Bionic
2020-01-10 08:13:17 Po-Hsu Lin bug task added linux (Ubuntu Bionic)
2020-01-10 08:30:07 Ubuntu Kernel Bot linux (Ubuntu): status New Incomplete
2020-01-10 08:30:09 Ubuntu Kernel Bot linux (Ubuntu Bionic): status New Incomplete
2020-01-10 09:11:06 Po-Hsu Lin ubuntu-kernel-tests: assignee Po-Hsu Lin (cypressyew)
2020-01-10 09:11:08 Po-Hsu Lin linux (Ubuntu): assignee Po-Hsu Lin (cypressyew)
2020-01-10 09:11:10 Po-Hsu Lin linux (Ubuntu Bionic): assignee Po-Hsu Lin (cypressyew)
2020-01-10 09:32:56 Po-Hsu Lin linux (Ubuntu Bionic): status Incomplete In Progress
2020-01-10 11:25:17 Po-Hsu Lin nominated for series Ubuntu Xenial
2020-01-10 11:25:17 Po-Hsu Lin bug task added linux (Ubuntu Xenial)
2020-01-10 11:25:25 Po-Hsu Lin linux (Ubuntu Xenial): status New In Progress
2020-01-10 11:25:27 Po-Hsu Lin linux (Ubuntu Xenial): assignee Po-Hsu Lin (cypressyew)
2020-01-10 11:25:29 Po-Hsu Lin ubuntu-kernel-tests: status New In Progress
2020-01-10 23:40:07 Terry Rudd bug added subscriber Terry Rudd
2020-01-13 10:36:40 Po-Hsu Lin linux (Ubuntu): status Incomplete Fix Released
2020-01-13 10:36:54 Po-Hsu Lin linux (Ubuntu): assignee Po-Hsu Lin (cypressyew)
2020-01-16 04:23:11 Po-Hsu Lin description s390x: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/s390x/l/linux/20191114_180123_738a0@/log.gz Here's the relevant excerpt: 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 DEBUG| [stdout] running psock_fanout test 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 ERROR| [stderr] test: control single socket 17:34:27 ERROR| [stderr] test: control multiple sockets 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (4) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (3) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (2) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (1) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x1 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=10,10, expect=10,10 17:34:28 ERROR| [stderr] info: count=18,17, expect=18,17 17:34:28 ERROR| [stderr] test: datapath 0x3 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=15,5, expect=15,5 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x6 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x7 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 DEBUG| [stdout] [FAIL] == SRU Justification == The psock_fanout test is not very stable, stress tests shows a high fail rate here: * X - 44 failures out of 100, with 4.4 kernel on a KVM node * B - 33 failures out of 100, with 4.15 kernel on a KVM node This is because of the unrelated background traffic which should be filtered out. Also, the error message, "ERROR: incorrect queue lengths", is a bit misleading. It has been changed to "warning: incorrect queue lengths". == Fix == 1. 2e7a7217 (selftests/net: cleanup unused parameter in psock_fanout) 2. cc30c93f (selftests/net: ignore background traffic in psock_fanout) The first patch has already landed in Bionic, and it can be cherry-picked into Xenial. The second one needs to be backported to Xenial, but it can be applied to Bionic with git am -C2. == Test == Patch tested directly on 4.4 / 4.15 KVM nodes. Stress tested 100 times with: i=0 for idx in $(seq 1 100) do sudo ./psock_fanout &> /dev/null [ $? != 0 ] && i=$((i+1)) done echo $i All tests have passed. == Regression Potential == Low, change limited to testing tools, not affecting actual kernel functionality. == Original Bug Report == s390x: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/s390x/l/linux/20191114_180123_738a0@/log.gz Here's the relevant excerpt: 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 DEBUG| [stdout] running psock_fanout test 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 ERROR| [stderr] test: control single socket 17:34:27 ERROR| [stderr] test: control multiple sockets 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (4) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (3) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (2) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (1) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x1 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=10,10, expect=10,10 17:34:28 ERROR| [stderr] info: count=18,17, expect=18,17 17:34:28 ERROR| [stderr] test: datapath 0x3 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=15,5, expect=15,5 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x6 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x7 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 DEBUG| [stdout] [FAIL]
2020-01-16 04:31:49 Po-Hsu Lin description == SRU Justification == The psock_fanout test is not very stable, stress tests shows a high fail rate here: * X - 44 failures out of 100, with 4.4 kernel on a KVM node * B - 33 failures out of 100, with 4.15 kernel on a KVM node This is because of the unrelated background traffic which should be filtered out. Also, the error message, "ERROR: incorrect queue lengths", is a bit misleading. It has been changed to "warning: incorrect queue lengths". == Fix == 1. 2e7a7217 (selftests/net: cleanup unused parameter in psock_fanout) 2. cc30c93f (selftests/net: ignore background traffic in psock_fanout) The first patch has already landed in Bionic, and it can be cherry-picked into Xenial. The second one needs to be backported to Xenial, but it can be applied to Bionic with git am -C2. == Test == Patch tested directly on 4.4 / 4.15 KVM nodes. Stress tested 100 times with: i=0 for idx in $(seq 1 100) do sudo ./psock_fanout &> /dev/null [ $? != 0 ] && i=$((i+1)) done echo $i All tests have passed. == Regression Potential == Low, change limited to testing tools, not affecting actual kernel functionality. == Original Bug Report == s390x: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/s390x/l/linux/20191114_180123_738a0@/log.gz Here's the relevant excerpt: 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 DEBUG| [stdout] running psock_fanout test 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 ERROR| [stderr] test: control single socket 17:34:27 ERROR| [stderr] test: control multiple sockets 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (4) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (3) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (2) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (1) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x1 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=10,10, expect=10,10 17:34:28 ERROR| [stderr] info: count=18,17, expect=18,17 17:34:28 ERROR| [stderr] test: datapath 0x3 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=15,5, expect=15,5 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x6 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x7 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 DEBUG| [stdout] [FAIL] == SRU Justification == The psock_fanout test is not very stable, stress tests shows a high fail rate here:   * X - 44 failures out of 100, with 4.4 kernel on a KVM node   * B - 33 failures out of 100, with 4.15 kernel on a KVM node This is because of the unrelated background traffic which should be filtered out. Also, the error message, "ERROR: incorrect queue lengths", is a bit misleading. You can see this even the test has passed, it's not the real cause of the failure. This message has been changed to "warning: incorrect queue lengths" later. == Fix == 1. 2e7a7217 (selftests/net: cleanup unused parameter in psock_fanout) 2. cc30c93f (selftests/net: ignore background traffic in psock_fanout) The first patch has already landed in Bionic, and it can be cherry-picked into Xenial. The second one needs to be backported to Xenial, but it can be applied to Bionic with git am -C2. == Test == Patch tested directly on 4.4 / 4.15 KVM nodes. Stress tested 100 times with:     i=0     for idx in $(seq 1 100)     do         sudo ./psock_fanout &> /dev/null         [ $? != 0 ] && i=$((i+1))     done     echo $i All tests have passed. == Regression Potential == Low, change limited to testing tools, not affecting actual kernel functionality. == Original Bug Report == s390x: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/s390x/l/linux/20191114_180123_738a0@/log.gz Here's the relevant excerpt: 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 DEBUG| [stdout] running psock_fanout test 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 ERROR| [stderr] test: control single socket 17:34:27 ERROR| [stderr] test: control multiple sockets 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (4) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (3) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (2) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (1) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x1 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=10,10, expect=10,10 17:34:28 ERROR| [stderr] info: count=18,17, expect=18,17 17:34:28 ERROR| [stderr] test: datapath 0x3 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=15,5, expect=15,5 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x6 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x7 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 DEBUG| [stdout] [FAIL]
2020-01-16 04:35:37 Po-Hsu Lin description == SRU Justification == The psock_fanout test is not very stable, stress tests shows a high fail rate here:   * X - 44 failures out of 100, with 4.4 kernel on a KVM node   * B - 33 failures out of 100, with 4.15 kernel on a KVM node This is because of the unrelated background traffic which should be filtered out. Also, the error message, "ERROR: incorrect queue lengths", is a bit misleading. You can see this even the test has passed, it's not the real cause of the failure. This message has been changed to "warning: incorrect queue lengths" later. == Fix == 1. 2e7a7217 (selftests/net: cleanup unused parameter in psock_fanout) 2. cc30c93f (selftests/net: ignore background traffic in psock_fanout) The first patch has already landed in Bionic, and it can be cherry-picked into Xenial. The second one needs to be backported to Xenial, but it can be applied to Bionic with git am -C2. == Test == Patch tested directly on 4.4 / 4.15 KVM nodes. Stress tested 100 times with:     i=0     for idx in $(seq 1 100)     do         sudo ./psock_fanout &> /dev/null         [ $? != 0 ] && i=$((i+1))     done     echo $i All tests have passed. == Regression Potential == Low, change limited to testing tools, not affecting actual kernel functionality. == Original Bug Report == s390x: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/s390x/l/linux/20191114_180123_738a0@/log.gz Here's the relevant excerpt: 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 DEBUG| [stdout] running psock_fanout test 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 ERROR| [stderr] test: control single socket 17:34:27 ERROR| [stderr] test: control multiple sockets 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (4) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (3) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (2) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (1) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x1 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=10,10, expect=10,10 17:34:28 ERROR| [stderr] info: count=18,17, expect=18,17 17:34:28 ERROR| [stderr] test: datapath 0x3 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=15,5, expect=15,5 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x6 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x7 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 DEBUG| [stdout] [FAIL] == SRU Justification == The psock_fanout test is not very stable, stress tests shows a high fail rate here:   * X - 44 failures out of 100, with 4.4 kernel on a KVM node   * B - 33 failures out of 100, with 4.15 kernel on a KVM node This is because of the unrelated background traffic which should be filtered out. Also, the error message, "ERROR: incorrect queue lengths", is a bit misleading. You can see this even the test has passed, it's not the real cause of the failure. This message has been changed to "warning: incorrect queue lengths" later. == Fix == 1. 2e7a7217 (selftests/net: cleanup unused parameter in psock_fanout) 2. cc30c93f (selftests/net: ignore background traffic in psock_fanout) The first patch has already landed in Bionic, and it can be cherry-picked into Xenial. The second one needs to be backported to Xenial, but it can be applied to Bionic with git am -C2. == Test == Patch tested directly on 4.4 / 4.15 KVM nodes. Stress tested 100 times with:     i=0     for idx in $(seq 1 100)     do         sudo ./psock_fanout &> /dev/null         [ $? != 0 ] && i=$((i+1))     done     echo $i All tests have passed without any failure. == Regression Potential == Low, change limited to testing tools, not affecting actual kernel functionality. == Original Bug Report == s390x: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/s390x/l/linux/20191114_180123_738a0@/log.gz Here's the relevant excerpt: 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 DEBUG| [stdout] running psock_fanout test 17:34:26 DEBUG| [stdout] -------------------- 17:34:26 ERROR| [stderr] test: control single socket 17:34:27 ERROR| [stderr] test: control multiple sockets 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (4) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (3) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (2) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:27 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:27 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:27 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:27 ERROR| [stderr] info: trying alternate ports (1) 17:34:27 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: trying alternate ports (0) 17:34:28 ERROR| [stderr] test: datapath 0x0 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,0, expect=20,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] test: datapath 0x1000 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=20,0, expect=15,5 17:34:28 ERROR| [stderr] ERROR: incorrect queue lengths 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x1 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=10,10, expect=10,10 17:34:28 ERROR| [stderr] info: count=18,17, expect=18,17 17:34:28 ERROR| [stderr] test: datapath 0x3 17:34:28 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:28 ERROR| [stderr] info: count=15,5, expect=15,5 17:34:28 ERROR| [stderr] info: count=20,15, expect=20,15 17:34:28 ERROR| [stderr] test: datapath 0x6 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x7 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=5,15, expect=15,5 17:34:29 ERROR| [stderr] info: count=20,15, expect=15,20 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] info: count=20,0, expect=20,0 17:34:29 ERROR| [stderr] test: datapath 0x2 17:34:29 ERROR| [stderr] info: count=0,0, expect=0,0 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 ERROR| [stderr] info: count=0,20, expect=0,20 17:34:29 DEBUG| [stdout] [FAIL]
2020-01-16 15:02:01 Stefan Bader linux (Ubuntu Xenial): importance Undecided Medium
2020-01-16 15:02:04 Stefan Bader linux (Ubuntu Bionic): importance Undecided Medium
2020-01-29 02:38:26 Khaled El Mously linux (Ubuntu Xenial): status In Progress Fix Committed
2020-01-29 02:38:28 Khaled El Mously linux (Ubuntu Bionic): status In Progress Fix Committed
2020-01-30 14:52:47 Ubuntu Kernel Bot tags bionic s390x ubuntu-kernel-selftests xenial bionic s390x ubuntu-kernel-selftests verification-needed-xenial xenial
2020-02-03 22:38:43 Ubuntu Kernel Bot tags bionic s390x ubuntu-kernel-selftests verification-needed-xenial xenial bionic s390x ubuntu-kernel-selftests verification-needed-bionic verification-needed-xenial xenial
2020-02-05 07:54:21 Po-Hsu Lin tags bionic s390x ubuntu-kernel-selftests verification-needed-bionic verification-needed-xenial xenial bionic s390x ubuntu-kernel-selftests verification-done-bionic verification-needed-xenial xenial
2020-02-06 07:18:34 Po-Hsu Lin tags bionic s390x ubuntu-kernel-selftests verification-done-bionic verification-needed-xenial xenial bionic s390x ubuntu-kernel-selftests verification-done-bionic verification-done-xenial xenial
2020-02-06 07:18:38 Po-Hsu Lin ubuntu-kernel-tests: status In Progress Fix Released
2020-02-17 10:36:02 Launchpad Janitor linux (Ubuntu Bionic): status Fix Committed Fix Released
2020-02-17 10:36:02 Launchpad Janitor cve linked 2019-18885
2020-02-17 10:36:02 Launchpad Janitor cve linked 2019-19078
2020-02-17 10:36:02 Launchpad Janitor cve linked 2019-19082
2020-02-17 10:36:02 Launchpad Janitor cve linked 2019-19332
2020-02-17 10:36:02 Launchpad Janitor cve linked 2019-20096
2020-02-17 10:36:02 Launchpad Janitor cve linked 2019-5108
2020-02-17 14:18:22 Launchpad Janitor linux (Ubuntu Xenial): status Fix Committed Fix Released