Ubuntu
linux-kvm package

vfork in ubuntu_stress_smoke test failed on KVM node with B-KVM kernel

Bug #1788385 reported by Po-Hsu Lin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Stress-ng
Fix Released
High
Colin Ian King
ubuntu-kernel-tests
Fix Released
Undecided
Unassigned
linux-kvm (Ubuntu)
Invalid
Undecided
Unassigned
Nominated for Bionic by Po-Hsu Lin

Bug Description

Fail rate 2 out of 2, failed on a kvm node "gummo", with 1G ram

uname -vr : 4.15.0-1020-kvm #20-Ubuntu SMP Thu Aug 16 16:46:26 UTC 2018
Bionic 18.04
arch: amd64

DEBUG - 11:24:24 DEBUG| [stdout] vfork STARTING
DEBUG - 11:24:24 DEBUG| [stdout] vfork STARTING
DEBUG - 11:24:26 DEBUG| [stdout] vfork RETURNED 2
DEBUG - 11:24:26 DEBUG| [stdout] vfork RETURNED 2
DEBUG - 11:24:26 DEBUG| [stdout] vfork FAILED
DEBUG - 11:24:26 DEBUG| [stdout] vfork FAILED
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] 2 processors online, 2 processors configured
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] 2 processors online, 2 processors configured
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: info: [1437] dispatching hogs: 4 vfork
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: info: [1437] dispatching hogs: 4 vfork
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] cache allocate: reducing cache level from L3 (too high) to L2
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] cache allocate: reducing cache level from L3 (too high) to L2
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] cache allocate: default cache size: 4096K
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] cache allocate: default cache size: 4096K
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] starting stressors
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] starting stressors
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1438] stress-ng-vfork: started [1438] (instance 0)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1438] stress-ng-vfork: started [1438] (instance 0)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] 4 stressors spawned
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] 4 stressors spawned
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1439] stress-ng-vfork: started [1439] (instance 1)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1439] stress-ng-vfork: started [1439] (instance 1)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1440] stress-ng-vfork: started [1440] (instance 2)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1440] stress-ng-vfork: started [1440] (instance 2)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1441] stress-ng-vfork: started [1441] (instance 3)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1441] stress-ng-vfork: started [1441] (instance 3)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1441] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] info: 5 failures reached, aborting stress process
DEBUG - 11:24:26 DEBUG| [stdout] info: 5 failures reached, aborting stress process
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1441] stress-ng-vfork: exited [1441] (instance 3)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1441] stress-ng-vfork: exited [1441] (instance 3)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1440] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] info: 5 failures reached, aborting stress process
DEBUG - 11:24:26 DEBUG| [stdout] info: 5 failures reached, aborting stress process
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1440] stress-ng-vfork: exited [1440] (instance 2)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1440] stress-ng-vfork: exited [1440] (instance 2)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1438] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] info: 5 failures reached, aborting stress process
DEBUG - 11:24:26 DEBUG| [stdout] info: 5 failures reached, aborting stress process
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1438] stress-ng-vfork: exited [1438] (instance 0)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1438] stress-ng-vfork: exited [1438] (instance 0)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: fail: [1439] stress-ng-vfork: fork failed
DEBUG - 11:24:26 DEBUG| [stdout] info: 5 failures reached, aborting stress process
DEBUG - 11:24:26 DEBUG| [stdout] info: 5 failures reached, aborting stress process
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: error: [1437] process 1438 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: error: [1437] process 1438 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] process [1438] terminated
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] process [1438] terminated
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1439] stress-ng-vfork: exited [1439] (instance 1)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1439] stress-ng-vfork: exited [1439] (instance 1)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: error: [1437] process 1439 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: error: [1437] process 1439 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] process [1439] terminated
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] process [1439] terminated
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: error: [1437] process 1440 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: error: [1437] process 1440 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] process [1440] terminated
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] process [1440] terminated
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: error: [1437] process 1441 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: error: [1437] process 1441 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] process [1441] terminated
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: debug: [1437] process [1441] terminated
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: info: [1437] unsuccessful run completed in 1.49s
DEBUG - 11:24:26 DEBUG| [stdout] stress-ng: info: [1437] unsuccessful run completed in 1.49s

See original description
Revision history for this message
Colin Ian King (colin-king) wrote :

Do you mind providing information on the architecture and kernel version + release that is being tested.

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Sorry was in a hurry, info updated in the bug description.

description: updated
description: updated
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Tested multiple time on the same node with both Bionic generic / kvm kernel:

4.15.0-32.35: failed 0/3
4.15.0.1019.19: failed 2/3

Test was executed 3 times consecutively without reboot, output from syslog when it fails:
Aug 23 06:18:59 gummo kernel: Adding 1478652k swap on /swap.img. Priority:-2 extents:3 across:1830908k
Aug 23 06:18:59 gummo kernel: Adding 1048572k swap on /home/ubuntu/autotest/client/tmp/ubuntu_stress_smoke_test/src/stress-ng/swap.img. Priority:-3 extents:3 across:7340032k
Aug 23 06:18:59 gummo stress-ng: invoked with './stress-n' by user 0
Aug 23 06:18:59 gummo stress-ng: system: 'gummo' Linux 4.15.0-1019-kvm #19-Ubuntu SMP Fri Aug 10 10:24:21 UTC 2018 x86_64
Aug 23 06:18:59 gummo stress-ng: memory (MB): total 996.66, free 7.01, shared 0.50, buffer 11.73, swap 2467.99, free swap 2467.99
Aug 23 06:18:59 gummo stress-ng: info: [823] dispatching hogs: 4 vfork
Aug 23 06:19:01 gummo stress-ng: fail: [827] stress-ng-vfork: fork failed
Aug 23 06:19:01 gummo stress-ng: fail: [825] stress-ng-vfork: fork failed
Aug 23 06:19:01 gummo stress-ng: message repeated 4 times: [ fail: [825] stress-ng-vfork: fork failed]
Aug 23 06:19:01 gummo stress-ng: fail: [827] stress-ng-vfork: fork failed
Aug 23 06:19:01 gummo stress-ng: message repeated 3 times: [ fail: [827] stress-ng-vfork: fork failed]
Aug 23 06:19:02 gummo stress-ng: fail: [824] stress-ng-vfork: fork failed
Aug 23 06:19:02 gummo stress-ng: message repeated 4 times: [ fail: [824] stress-ng-vfork: fork failed]
Aug 23 06:19:02 gummo stress-ng: fail: [826] stress-ng-vfork: fork failed
Aug 23 06:19:02 gummo stress-ng: error: [823] process 824 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
Aug 23 06:19:02 gummo stress-ng: error: [823] process 825 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
Aug 23 06:19:04 gummo stress-ng: fail: [826] stress-ng-vfork: fork failed
Aug 23 06:19:04 gummo stress-ng: message repeated 3 times: [ fail: [826] stress-ng-vfork: fork failed]
Aug 23 06:19:04 gummo stress-ng: error: [823] process 826 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
Aug 23 06:19:04 gummo stress-ng: error: [823] process 827 (stress-ng-vfork) terminated with an error, exit status=1 (stress-ng core failure)
Aug 23 06:19:04 gummo stress-ng: info: [823] unsuccessful run completed in 4.51s

Revision history for this message
Colin Ian King (colin-king) wrote :

This is not a kernel error; it's a bug in stress-ng when running with the --verify option on systems where memory is low and EAGAIN or ENOMEM occurs. I've fixed this in stress-ng with commit:

http://kernel.ubuntu.com/git/cking/stress-ng.git/commit/?id=8ab0ac7ce995517c0cb10dddddb60bee5a19b7b8

Do you mind running a re-test to see if this fixes the issue. Apologies for this stress-ng bug. My bad.

Changed in stress-ng:
importance: Undecided → High
assignee: nobody → Colin Ian King (colin-king)
status: New → In Progress
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Passed with 3 run on the same node with the updated repo.
Thank you!

Changed in ubuntu-kernel-tests:
status: New → Fix Released
Changed in linux-kvm (Ubuntu):
status: New → Invalid
Po-Hsu Lin (cypressyew)
Changed in stress-ng:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.