port80 test in ubuntu_kvm_unit_tests failed with timeout

Bug #1748105 reported by Po-Hsu Lin
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
ubuntu-kernel-tests
Invalid
Undecided
Po-Hsu Lin
linux (Ubuntu)
Invalid
Undecided
Unassigned
linux-azure (Ubuntu)
Invalid
Undecided
Unassigned
linux-azure-edge (Ubuntu)
Invalid
Undecided
Unassigned
linux-kvm (Ubuntu)
Invalid
Undecided
Unassigned
linux-oracle-5.0 (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

With Joshua's comment in bug 1719524: "Nested KVM can only be tried on instance sizes with nested Hypervisor support: Ev3 and Dv3.", although the instance name is E4v3 here but I can start a KVM on it.

Test port80 test will timeout on it.

Steps:
1. git clone --depth=1 https://git.kernel.org/pub/scm/virt/kvm/kvm-unit-tests.git
2. cd kvm-unit-tests; ./configure; make
3. Run the port80 test as root:

# TESTNAME=port80 TIMEOUT=90s ACCEL= ./x86/run x86/port80.flat -smp 1
timeout -k 1s --foreground 90s /usr/bin/qemu-system-x86_64 -nodefaults -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -machine accel=kvm -kernel x86/port80.flat -smp 1 # -initrd /tmp/tmp.3p9PWc2SRi
enabling apic
begining port 0x80 write test
qemu-system-x86_64: terminating on signal 15 from pid 7790

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.14.0-1004-azure-edge 4.14.0-1004.4
ProcVersionSignature: User Name 4.14.0-1004.4-username-edge 4.14.14
Uname: Linux 4.14.0-1004-azure-edge x86_64
ApportVersion: 2.20.1-0ubuntu2.15
Architecture: amd64
Date: Thu Feb 8 06:13:18 2018
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: linux-azure-edge
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :
Po-Hsu Lin (cypressyew)
summary: - port80 test in kvm-unit-test failed on E4v3 azure node with 4.14 kernel
+ port80 test in kvm-unit-test failed on E4v3 azure node with 4.13/4.14
+ kernel
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1748105

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
Po-Hsu Lin (cypressyew) wrote : Re: port80 test in kvm-unit-test failed on E4v3 azure node with 4.13/4.14 kernel

Spotted on X-KVM.

Po-Hsu Lin (cypressyew)
tags: added: linux-kvm sru-20190603 ubuntu-kvm-unit-tests
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in linux-azure (Ubuntu):
status: New → Confirmed
Changed in linux-azure-edge (Ubuntu):
status: New → Confirmed
Changed in linux-kvm (Ubuntu):
status: New → Confirmed
Po-Hsu Lin (cypressyew)
summary: - port80 test in kvm-unit-test failed on E4v3 azure node with 4.13/4.14
- kernel
+ port80 test in kvm-unit-test failed with timeout
summary: - port80 test in kvm-unit-test failed with timeout
+ port80 test in ubuntu_kvm_unit_tests failed with timeout
tags: added: sru-20191111
tags: added: 5.3 bionic
Sean Feole (sfeole)
Changed in ubuntu-kernel-tests:
status: New → Triaged
tags: added: eoan sru-20191202
Sean Feole (sfeole)
Changed in linux-oracle-5.0 (Ubuntu):
status: New → Confirmed
tags: added: aws azure oracle
tags: added: sru-20200217
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

It needs about 90 second to run on Azure Standard_D48_v3

$ time sudo ./tests/port80
BUILD_HEAD=5c052c90
timeout -k 1s --foreground 180 /usr/bin/qemu-system-x86_64 --no-reboot -nodefaults -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -machine accel=kvm -kernel /tmp/tmp.cKEDR6Z0v0 -smp 1 # -initrd /tmp/tmp.o2ZRnk8tmj
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2]
enabling apic
begining port 0x80 write test
done
PASS port80

real 1m20.377s
user 0m42.160s
sys 0m38.299s

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

3 more tests shows it will finish around 80 ~ 110 seconds on that azure instance (with Bionic 4.15 Azure)

However, it will take about 330 seconds with the same instance on Xenial 4.15 Azure (3 attempts,

And on Oracle Bionic 4.15, it will take up to 620 second to run (5 attempts, 610 ~ 620 seconds):
$ time sudo ./tests/port80
BUILD_HEAD=4671e4ba
timeout -k 1s --foreground 1800 /usr/bin/qemu-system-x86_64 -nodefaults -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -machine accel=kvm -kernel /tmp/tmp.iqkRAFLRW3 -smp 1 # -initrd /tmp/tmp.WiUqj082JM
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2]
enabling apic
begining port 0x80 write test
done
PASS port80

real 10m17.146s
user 2m50.722s
sys 7m26.440s

So perhaps a better timeout threshold for now is 500 seconds or something.

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Typo in #8, recommended to bump it up to 900 seconds

Po-Hsu Lin (cypressyew)
tags: added: sru-20200831
Revision history for this message
Kelsey Steele (kelsey-steele) wrote :

Failed on Bionic/oracle : 5.4.0-1026.26~18.04.1 : amd64

tags: added: 5.4 sru-20200921
Revision history for this message
Francis Ginther (fginther) wrote :

Seen with linux-oracle 4.15.0-1065.73~16.04.1.

tags: added: sru-20210125
Revision history for this message
Kleber Sacilotto de Souza (kleber-souza) wrote :

Issue found with xenial/linux-kvm 4.4.0-1094.103.

tags: added: 4.4 kvm sru-20210412
Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote :

Found on Bionic/linux-kvm, cycle sru-20210412

tags: added: 4.15
Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote :

Issue observed in B/KVM, cycle sru-20210510.

tags: added: sru-20210510
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

This test will be removed in the future. (Removed in upstream.)

Changed in ubuntu-kernel-tests:
status: Triaged → In Progress
assignee: nobody → Po-Hsu Lin (cypressyew)
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/commit/2805ff600e44936ce4d2c024f3ae81078c59658d

Note that even with this test removed, we might need to investigate if we need to run this for older kernels.

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Test has long gone, let's close this.

Changed in ubuntu-kernel-tests:
status: In Progress → Invalid
Changed in linux (Ubuntu):
status: Incomplete → Invalid
Changed in linux-azure (Ubuntu):
status: Confirmed → Invalid
Changed in linux-azure-edge (Ubuntu):
status: Confirmed → Invalid
Changed in linux-kvm (Ubuntu):
status: Confirmed → Invalid
Changed in linux-oracle-5.0 (Ubuntu):
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.