5. change to terminal 3. check block stat info, it will hang for a few seconds, then it will report an error.
# virsh domblkstat r72
(hang a few seconds)
error: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainGetBlockIoTune)
log as following :
......
2016-05-24 01:50:46.698+0000: 1309: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fe7d0a47970
2016-05-24 01:50:46.712+0000: 2512: warning : qemuDomainObjBeginJobInternal:2180 : Cannot start job (query, none) for domain r72; current job is (query, none) owned by (2513 remoteDispatchDomainGetBlockIoTune, 0 <null>) for (41s, 0s)
2016-05-24 01:50:46.712+0000: 2512: error : qemuDomainObjBeginJobInternal:2192 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainGetBlockIoTune)
......
6. It seems that it will hang in the terminal 1:
# virsh blkdeviotune r72 vda
...... hang here, nothing output.
but virsh list or other checkinfo except block related works well.
7.If I Ctrl+c terminal this command and re-try it again:
# virsh blkdeviotune r72 vda
^C
# virsh blkdeviotune r72 vda
error: Unable to get number of block I/O throttle parameters
error: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainGetBlockIoTune)
As above, Do these steps can be used to reproduce this issue? It will hang in step6 and step7.
Try to reproduce it via libvirt. 1.3.4-1. el7.x86_ 64 rhev-2. 6.0-2.el7. x86_64
Version:
libvirt-
qemu-kvm-
Step: ------- ------- ------- ------- ------- ------- ---
1. Start a guest with image on NFS storage
# virsh list
Id Name State
-------
2 r72 running
2. disconnect with the NFS storage server.
# iptables -A OUTPUT -d $IP -p tcp --dport 2049 -j DROP
I want to know when it will hang on virDomainGetCon trolInfo( ). So I tried this :
#!/bin/bash
while :
do
virsh domcontrol r72
sleep 30
done
3. on terminal 1, check IO throughttling with the image. ------- ------- ------- ------- ------- ------
# virsh domblklist r72
Target Source
-------
hda -
vda /tmp/zp/r7.2.img
# virsh blkdeviotune r72 vda
......hang here
log as following : ginJobInternal: 2097 : Starting job: query (vm=0x7fe7501f2d40 name=r72, current job=query async=none) ginJobInternal: 2120 : Waiting for job (vm=0x7fe7501f2d40 name=r72)
......
2016-05-24 01:50:16.712+0000: 2512: debug : qemuDomainObjBe
2016-05-24 01:50:16.712+0000: 2512: debug : qemuDomainObjBe
2016-05-24 01:50:17.666+0000: 1309: info : virObjectRef:296 : OBJECT_REF: obj=0x7fe7d0a34300
......
4.change to termianl 2, check other info, it works well ------- ------- ------- ------- ------- ------- ---
# virsh list
Id Name State
-------
2 r72 running
# virsh schedinfo r72
Scheduler : posix
cpu_shares : 1024
vcpu_period : 100000
vcpu_quota : -1
emulator_period: 100000
emulator_quota : -1
global_period : 100000
global_quota : -1
5. change to terminal 3. check block stat info, it will hang for a few seconds, then it will report an error. omainGetBlockIo Tune)
# virsh domblkstat r72
(hang a few seconds)
error: Timed out during operation: cannot acquire state change lock (held by remoteDispatchD
log as following : ginJobInternal: 2180 : Cannot start job (query, none) for domain r72; current job is (query, none) owned by (2513 remoteDispatchD omainGetBlockIo Tune, 0 <null>) for (41s, 0s) ginJobInternal: 2192 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchD omainGetBlockIo Tune)
......
2016-05-24 01:50:46.698+0000: 1309: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fe7d0a47970
2016-05-24 01:50:46.712+0000: 2512: warning : qemuDomainObjBe
2016-05-24 01:50:46.712+0000: 2512: error : qemuDomainObjBe
......
2016-05-24 01:45:32.325+0000: 2514: debug : virThreadJobSet:96 : Thread 2514 (virNetServerHa ndleJob) is now running job remoteDispatchD omainGetInfo ar:121 : Thread 2514 (virNetServerHa ndleJob) finished job remoteDispatchD omainGetInfo with ret=0
2016-05-24 01:45:32.325+0000: 2514: info : virObjectNew:202 : OBJECT_NEW: obj=0x7fe7a403d340 classname=virDomain
......
2016-05-24 01:45:32.325+0000: 2514: debug : virThreadJobCle
......
6. It seems that it will hang in the terminal 1:
# virsh blkdeviotune r72 vda
...... hang here, nothing output.
but virsh list or other checkinfo except block related works well.
7.If I Ctrl+c terminal this command and re-try it again: omainGetBlockIo Tune)
# virsh blkdeviotune r72 vda
^C
# virsh blkdeviotune r72 vda
error: Unable to get number of block I/O throttle parameters
error: Timed out during operation: cannot acquire state change lock (held by remoteDispatchD
As above, Do these steps can be used to reproduce this issue? It will hang in step6 and step7.