[18.10] Timed out message while taking dump using virsh dumpxml command & fails with 'held by remoteDispatchDomainCoreDump' error
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
The Ubuntu-power-systems project |
Fix Released
|
Medium
|
David Britton | ||
libvirt (Ubuntu) |
Fix Released
|
Undecided
|
Ubuntu on IBM Power Systems Bug Triage | ||
Bionic |
Won't Fix
|
Undecided
|
Unassigned |
Bug Description
Problem Description:
=======
Tried to take dump using virsh dumpxml command and it fails with Timed out 'held by remoteDispatchD
Steps to re-create:
=======
1. boslcp3g4 is installed with 4.15.0-15-generic kernel.
2. LTP & memory map tests were running inside guest.
3. After some time guest in hung state.
4. Tried to take dump using virsh dumpxml.
root@boslcp3:~# virsh dump boslcp3g4 boslcp3g4_mmap_ltp --memory-only
error: Failed to core dump domain boslcp3g4 to boslcp3g4_mmap_ltp
error: Disconnected from qemu:///system due to keepalive timeout
error: Timed out during operation: cannot acquire state change lock (held by remoteDispatchD
root@boslcp3:~# virsh list --all
Id Name State
1 boslcp3g3 running
2 boslcp3g4 paused
4 boslcp3g1 running
5. It fails with Timed out during opearation & with held by remoteDispatchD
6. /var/log/syslog dumps
Apr 18 03:29:14 boslcp3 libvirtd[5538]: 2018-04-18 08:29:13.956+0000: 5576: warning : qemuDomainObjBe
Apr 18 03:29:14 boslcp3 libvirtd[5538]: 2018-04-18 08:29:13.958+0000: 5576: error : qemuDomainObjBe
Apr 18 03:29:44 boslcp3 libvirtd[5538]: 2018-04-18 08:29:44.492+0000: 5573: warning : qemuDomainObjBe
7. Attached syslog & sosreport
== Comment: #3 - Application Cdeadmin <email address hidden> - 2018-04-18 08:11:01 ==
When i tried for second time same command it was successful but syslog dumps below warnings continuously
warning : :4863 : Cannot start job (query, none) for domain boslcp3g4; current job is (async nested, dump) owned by (5574 remoteDispatchD
root@boslcp3:~# virsh dump boslcp3g4 boslcp3g4_mmapltp --memory-only
Domain boslcp3g4 dumped to boslcp3g4_mmapltp
vmcore located at:
vmcore at kte111:
Access kte111 using debug@9.3.111.155 (don2rry)
== Comment: #8 - Application Cdeadmin <email address hidden> - 2018-04-19 05:26:32 ==
Tried to start the guest boslcp3g1 guest which has qlogic disk as boot & IO disk
root@boslcp3:~# virsh list --all
Id Name State
1 boslcp3g4 running
3 boslcp3g3 running
- boslcp3g1 shut off
root@boslcp3:~# echo 10240 > /proc/sys/
root@boslcp3:~# virsh start --console boslcp3g1
--> Than saw guest went to paused state.
root@boslcp3:/home# virsh list --all
Id Name State
1 boslcp3g4 running
3 boslcp3g3 running
5 boslcp3g1 paused
Then tried to destroy the guest and its fails with Timed out during operation: cannot acquire state change lock. Even resume command also failing as below
Corresponding syslog from /var/log:
Apr 19 05:17:09 boslcp3 libvirtd[5576]: 2018-04-19 10:17:09.056+0000: 5635: error : virProcessKillP
== Comment: #26 - Shivaprasad G. Bhat <email address hidden> - 2018-05-17 08:57:25 ==
Got to test the patches independently. The below commits from upstream fix the false alarms and allows the dump to go through clean.
a5bc7130f3
e712579200
150930e309
9a1755b7fe
501e3c3c96
88c2360753
3455a7359c
fd1a9e5c56
2a4d847e77
9d73df98c2
93412bb827
a8ef7b69dc
5870f95a7a
3f99bb06d1
Changed in ubuntu-power-systems: | |
assignee: | nobody → David Britton (davidpbritton) |
importance: | Undecided → Medium |
tags: | added: triage-g |
Changed in ubuntu-power-systems: | |
status: | New → Triaged |
summary: |
- Timed out message while taking dump using virsh dumpxml command & fails - with 'held by remoteDispatchDomainCoreDump' error + [18.10] Timed out message while taking dump using virsh dumpxml command + & fails with 'held by remoteDispatchDomainCoreDump' error |
Changed in ubuntu-power-systems: | |
status: | Triaged → Fix Released |
tags: |
added: targetmilestone-inin1810 removed: targetmilestone-inin1804 |
Default Comment by Bridge